JP3231105B2 - Data encoding method and data restoration method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a data encoding method for compressing data by universal encoding using a Jib-Lempel code and a data restoring method.

[0002]

2. Description of the Related Art In recent years, with the development of OA (Official Automation), it has become possible to incorporate various media such as characters, figures, and images in a single document. Mixed information such as character codes and black-and-white binary images is now handled as document data in G4 facsimile and optical disk file systems, etc., along with their layout information. The amount of such information is also rapidly increasing. It is increasing. When document information composed of such multimedia is used as digital data, the data amount of image information is generally ten times to several tens times larger than the data amount of character codes. For this reason, when handling image information in data storage, data transmission, etc., in order to efficiently perform such processing, the data amount is compressed by omitting redundant portions in the data,
It aims to reduce storage capacity and increase transmission efficiency.

However, in a large-capacity file system or a document database, the character code information in the document data becomes large as a whole, so that not only image information but also character code information must be compressed.

A universal encoding method is known as a method capable of compressing various data such as character codes and image data by one method. As a typical method, a Jib Lempel code (Seiji Munakata, "Ziv -Lempel Data Compression Method ", Information Processing, Vol. 26, No. 1, Jan. 1985).

There are two algorithms for the Jib-Lempel code, a universal type and an incremental paring type.

Further, as an improvement of the universal algorithm, there is an LZW code (TC Bell, "Better OPM / LT
ext Compression ", IEEE Trans. on Commun., Vol.COM-3
4, No. 12, Dec. 1986).

An LZW code is also an improved version of the incremental decomposition type algorithm (TA Welch, "A Techn.
ique for High-Performance Data Compression ", Comput
er, June 1984).

[0008] Among these encoding methods, LZW codes have recently been used for compression of files stored in storage devices because of their high speed processing and simple algorithms. I have.

Here, two algorithms of a universal type and an incremental decomposition type of the Jib Lempel code, which are typical methods of the universal coding, will be described. 1. Universal type algorithm This algorithm requires a large amount of computation, but can provide a high compression rate. The maximum length sequence (subsequence) that matches data to be encoded from an arbitrary position in the past data sequence
And encodes it as a copy of a past series.

FIG. 15 (a) shows the basic concept of the encoding of such a universal type Jib Lempel code. Figure (a)
.. Abc... Stored in the P buffer shown in FIG. On the other hand, data (character string) “abcdef” to be encoded is input and stored in the Q buffer.

In such a state, when encoding the data in the Q buffer, the data sequence in the P buffer is scanned using the data sequence in the Q buffer as a key, and the data sequence in the Q buffer is scanned in the P buffer. Is obtained ("abc" in the example of FIG. 3A). Then, in order to specify the substring having the maximum length in the P buffer, a set of information in the format shown in FIG. This set of information includes "the start position of the maximum matching sequence in the P buffer" (the address of "a" in the example of FIG. 3A), and "length of matching" (in the example of FIG. 3 "), and" next symbol "(" d "in the example of FIG. 3A).

Subsequently, the encoded sequence (in this case, "abc") in the Q buffer is moved and stored in the P buffer to obtain a new past data sequence. Hereinafter, the same operation is repeated for the remaining data series “def” in the Q buffer, and the remaining data series in the Q buffer is
The data sequence is decomposed into partial strings already stored in the buffer, encoded as described above, and the data series in the P buffer is updated.

2. Incremental decomposition type algorithm Although this algorithm is inferior in compression ratio to the universal type, the algorithm is simple and the calculation is easy, so that high-speed processing can be performed.

L which is a typical method of this algorithm
The ZW encoding method will be described with reference to a flowchart shown in FIG. 16, a dictionary (learning dictionary) shown in FIG. 17, and a schematic diagram of data conversion shown in FIG.

The LZW encoding has one rewritable dictionary (learning dictionary), divides an input character string into different character strings (substrings), and assigns reference numbers in the order in which these character strings appear. And registering the character string currently input in the dictionary with a reference number assigned to a matching character string of the maximum length already registered in the dictionary. In the following description, one word unit of data is referred to as a character, and data connected with an arbitrary word is referred to as a character string, following the name used in information theory.

In the LZW encoding process, first, at step S
1, the pre dictionary D _C, is initialized to register a string of one character per total characters. That is, for example, when encoding character with 8-bit code, a string of one character per maximum 256 all characters are initially registered in the address 0 to 255 address of the dictionary D _C. Thus, for example, as shown in FIG. 17, the address of the dictionary D _C 0, 1,
, 255, the alphabet "a",
“B”, “c”,..., Hiragana, katakana, numerals, and the like are registered. It should be noted that the character string table B1 shown on the left side of the figure is supplementarily shown for easy explanation.

In the following description, an example in which an input character string as shown in FIG. 18 is input will be described for easy understanding. First, at step S1, the leading address n for writing dictionary D _C, which is the next address of the storage address of the last character string the initial registration "2
56 "is set as the storage address to the dictionary D _C of the character string to be newly registered.

[0018] Then, similarly in step S1, searches the dictionary D _c the first character K which is input as the key data (index), reference numeral omega (reference number of a character K which is registered in the dictionary D _C) And assign this to the initial string (prefi
x string). As a result, the input character string is, for example, “ababcbababaaaa” as shown in FIG.
If aaaa ", dictionary D _C is searched for, which is the first letter K" a "as an index," obtained as a reference number "0" of a "reference number ω, the reference number" 0 "is the word This is the initial character string (see the output code column in FIG. 18).

Next, in step S2, the next character K of the input character string is read. Thus, the character “b” next to the first input character “a” is read. Subsequently, in a step S3, it is determined whether or not the character K is present. this is,
This is a process for determining whether or not the input character string has not been completed yet.

In the case of the input character string shown in FIG. 18, the character string "b" next to "a" is read in step S2, and the character string has not been finished yet. determination, and then in step S4, a character string "ωK" searches whether are registered in the dictionary D _C.

Thus, the initial character string ω (here, reference number “0”) obtained in step S1 is added to step S1.
(In this case "b") read in two characters K is the string "0b" plus, whether or not it is registered in the dictionary D _C is examined.

If the result of this search is No, the process proceeds to step S6, where the code "code (ω)" of the reference number ω of the character K obtained in step S1 is output, and the character string "ωK" It is registered in the address n of the dictionary D _C to impart a new reference number n in.

As a result, in the case of the input character string shown in FIG. 18, first, the code of "0" which is the reference number ω of "a" is output, and the character string "0b" which has not been detected is replaced with the reference number. been granted the "256", the address of the dictionary D _C 2
56 is registered.

[0024] Then, similarly in step S6, along with replacing the input character K read in step S2 to the reference numbers omega, the address n of the dictionary D _C is incremented "1", the process returns to the step S2 next character K Read.

[0025] Thus, in the example of the input string in FIG. 18, reference numeral ω is replaced by "b" is a reference number "1", the dictionary D in _C of a character string next newly registered Registration address n is incremented by "257"
Changes to

On the other hand, if the character string "ωK" is registered in the dictionary D _C in step S4, in this case, step S5
The process proceeds to step S2 to the replace the string "ωK" at reference numeral omega, not find in step S4 string "ωK" from the dictionary D _C returns to step S2 again
Steps S5 to S5 are repeated to continue searching for the character string having the maximum matching length.

The processing of LZW encoding performed by such a method is described by the input character string "ababcbabab" shown in FIG.
To be specific addresses the aaaaaaa ", firstly, when typing the first letter" a ", there is no character string that matches the other" a "in the dictionary D _C, granted to" a " The code code (0) of the reference number “0” is output. Then, registered in the dictionary D _C by giving the reference number "256" to the extended character string "ab". Figure 13 shows the actual dictionary registration.
Is registered in the form of a character string “0b” as shown on the right side of “.

Subsequently, the second character "b" becomes the head of a new search character string. In this case, the dictionary D _C outputs the sign of the reference numbers "1" since there is no character that matches the other characters "b" which are assigned to the letter "b" code (1),
Is not registered in the extended character string "ba" is also still Dictionary D _C simultaneously represents a character string "ba" in "1a",
Given the reference number "257" is registered in the dictionary D _C is. Then, next, the third character “a” is replaced with the next search character string “ω
K ”. Similarly, by continuing such processing, the input character string “abab” shown in FIG.
“cbababaaaaaaa” is “0, 1, 256, 2, 257, 260, 0, 2” shown in the output code column of FIG.
62, 263 ", and as a result, the input character string is compressed.

Next, an algorithm for restoring LZW encoded code data as described above will be described with reference to the flowchart of FIG. As a specific example of this restoration, the output code string “0,
1, 256, 257, 260, 0, 262, 263 "
Is again shown in FIG. 20 (a) as an input code string to assist the explanation.

First, in step S11, as in the case of the LZW encoding, a character string consisting of one character for every character is initially registered in the dictionary Dd. In the specific example described below, each of the characters “a”, “b”,
"C",... Are represented by reference numbers "0", "1",
Are registered in the dictionary Dd by adding "2",..., And "2" which is the next address of the storage address of the last character string initially registered is added to the writing start address n of the dictionary Dd.
56 "is set as the storage address n of the newly registered character string in the dictionary Dd.

Next, in step S11, the first code CODE is read, and a reference number corresponding to the code CODE is set to OLDω. As a result, FIG.
In the example of the input code string shown in (a), the code code (0) of the reference number “0”, which is the first input code, is read, converted to the reference number “0”, and set to OLDω.

Subsequently, in step S11, the character K corresponding to the reference number "OLDω" is restored. In this process, since the first input code CODE corresponds to one of the one-character reference numbers initially registered in the dictionary Dd as described above, the code that matches the input code CODE is used.
The code (K) is searched from the dictionary Dd, and the corresponding character "K" is output. Note that the output character "K" is also set in FINchar in preparation for an exception process performed as necessary later.

Thus, in the example of the input code string shown in FIG. 20A, first, the character "a" corresponding to the reference number "0"
Is restored and output, and is also set in FINchar.

Subsequently, in step S12, the next input code CODE is read. That is, in the example of the input code string shown in FIG. 20A, the code code (1) of "1" is read. Then, in a step S13, it is determined whether or not there is a code CODE newly read, that is, whether or not the code input is completed. In the example of the input code string shown in FIG.
In step S12, the code code (1) having the reference number "1" is read as a new input code CODE.

As described above, if there is a new input code CODE, the process proceeds to step S14, where the input code CODE is set.
Is set to INω. Thus, in the example of the input code shown in FIG. 20A, the reference number “1” is set to INω.

Next, in step S15, it is determined whether or not the reference number "ω" has already been registered in the dictionary Dd (ω ≧
n) is determined. In this process, normally, the read code CODE is already registered in the dictionary Dd in the previous process, so that ω <n, and the process proceeds to step S16.
The dictionary Dd is searched, and the character string ω′K corresponding to the reference number “ω” is read from the dictionary Dd, and the character string corresponding to the reference number “ω” is a two-character character string “ω′K”. Is determined. If it is a two-character string "ω'K", the character "K" is temporarily stacked in step S17, and the reference number "ω '" is set as a new reference number ω, and the process returns to step S16 again. This step S16, S17
Is repeated recursively until the character string corresponding to the reference number ω becomes one character "K", and the process proceeds to the last step S18, where the last restored character K is output, and then all of the characters stacked in the step S17 are output. LIFO (Last
(In First Out) pop-up and output (restoration and output of code CODE read in step S12). Further, in step S18, after setting the first character K of the restored character string to FINchar, the first character K of the restored character string is represented by a set (OLDω, K) from the reference number OLDω of the previously restored processing and the first character K of the currently restored character string. The character string is registered at the address n of the dictionary Dd with a new reference number "n". Subsequently, the address n is incremented by "1", and "n + 1" is set as a registration address n of a character string to be registered next in the dictionary Dd.
The reference number “ω” corresponding to the code CODE restored this time, which has been set to
Return to 12.

Thus, in the case of the input code shown in FIG. 20A, as shown in FIG. 20B, the code CODE (= code (1)) of the reference number "1" read second. The character "b" is restored and output from
ar and the code COD that was previously restored
A character string “0b” consisting of a series of a reference number “0” corresponding to E (= code (0)) and one character “b” restored this time
Is given a new reference number "256" and registered in the dictionary Dd.

The registered address n of the dictionary Dd is "2
After being updated to “57”, the reference number “1” corresponding to the code CODE (= code (1)) restored this time is set in OLDω, and the third code co is set in step S12.
de (2 56) is read.

Then, the character string "0b" obtained by searching the dictionary Dd is replaced with the character string "ab", and the character string "ab" is output. At the same time, a character string “1a” (= “ba”) obtained by combining the reference number “1” corresponding to the code code (1) restored last time and the first character “a” of the character string restored this time is new. Reference number "25
"7" is added and registered in the address "257" of the dictionary Dd.

On the other hand, if it is determined in step S15 that the read code (ω) is not registered in the dictionary Dd in the previous process (ω ≧ n), the process proceeds to step S19 to perform an exception process. . In this exception processing, first, after outputting the first character "FINchar" of the previously restored character string, the reference number "OLDω" corresponding to the code CODE restored last time is set as the reference number ω, and then the previously restored character string is restored. A character string “OLDω, FINchar” obtained by adding the first character “FINchar” of the character string is obtained, a reference number corresponding to the new character string is set to INω, and the process proceeds to step S16.

Thus, for example, in the case of the input code string shown in FIG.
Reference number "260" corresponding to code (260) of
Are not defined in the dictionary Dd at this time. In this case, first, in step S19, the previously restored code co
After the first character (FINchar) of the character string “bab” corresponding to de (257) is output, the previously restored character string “ba” is added to the reference number “257” corresponding to the code code (257) previously restored. Is obtained by adding the first character "b" of ".", A reference number "260" is given to this character string, and this reference number is set to INω. Then, by repeating the processing of steps S16 → S17, the characters are stacked one by one in the order of “a” and “b”. In step S18, the character string "ab" is output by a pop-up operation, and finally the code
de (2 6 0) as well as restore and output the character string "bab", registered in the dictionary Dd by applying a reference number "260" of the character string "257b" (FIG. (b) ~ (e )).

Thereafter, by repeating the same processing sequentially, the input code string shown in FIG. 20A is restored to the character string shown in FIG.

[0043]

The data compression by the Jib-Lempel coding described above is not a method of performing compression by assuming in advance the statistical properties and stationarity of the target data as seen in other systems. Since it is an information-storing type data compression method that is completely restored to the original information when encoded, it is possible to use data that requires complete restoration, such as character codes, program source codes or object codes. Suitable for compression.

Since the Jib-Lempel code can be directly applied to an arbitrary symbol string, if image data is divided into a certain amount of data and the data is handled in the same manner as a character code,
Image data can also be compressed by Jib-Lempel coding. Therefore, it is possible to compress information in which a plurality of types of data having different properties, such as a character code and image data, are mixed by Jib-Lempel coding.

However, the conventional data compression by Jib Lempel coding is performed using only one dictionary, and this dictionary is created and updated while encoding the input data, so that the capacity of the dictionary becomes full. Data is coded by clearing (initializing) immediately as soon as possible, or clearing when the compression rate has deteriorated after the capacity is full, and restarting the dictionary registration from the beginning. I have. For this reason, if the capacity of the dictionary is small, local characteristics of the input data can be promoted, but sufficient learning cannot be performed, and the compression ratio of the input data does not increase much.

On the other hand, if the capacity of the dictionary is too large, the general nature of the input data is promoted, but the response to local changes in the input data becomes slow, and the data compression ratio deteriorates in this aspect. There was a problem of doing.

The present invention has been made in view of such a conventional problem. By performing Jib-Lempel encoding using a plurality of rewritable dictionaries having different registration capacities, the general scope of input data can be improved. An object of the present invention is to perform efficient Jib-Lempel encoding corresponding to various characteristics and local changes, and to improve the data compression ratio by Jib-Lempel encoding.

[0048]

FIG. 1 is a principle diagram of the present invention (first invention). According to the first invention, in a data encoding method for compressing data by universal encoding using a Jib Lempel code, an initial setting is performed every time input data is Jib Lempel encoded over a certain specific section. A first dictionary 1; a second dictionary 2 capable of registering all dictionary data generated in a Jib-Lempel encoding process over a plurality of continuous sections of the input data; A first encoding means 3 for performing Jib-Lempel encoding using the dictionary 1 and registering new dictionary data generated in the Jib-Lempel encoding process in the first dictionary 1; The data input to the encoding means 1 is subjected to Jib-Lempel encoding using the second dictionary 2, and new dictionary data generated in this Jib-Lempel encoding process is A second encoding unit 4 to be registered in the second dictionary 2, a jib-Lempel code string obtained by the first encoding unit 3 and the second encoding unit 4 for each of the predetermined sections. Compressed data output means 5 for comparing the data amount of the obtained Jib-Lempel code sequence and outputting the smaller data amount of the Jib-Lempel code sequence together with a flag indicating the dictionary used for this encoding. It is characterized by having.

In the first aspect, the certain section is
For example, after the first dictionary 1 is initialized as described in claim 2, the number of data subjected to the Jib-Lempel encoding by the second encoding unit 4 becomes from "0" to a specific number. May be defined as the period.

Further, the fixed section may be, for example, a period from when the first dictionary 1 is initialized to when its registered capacity becomes full, as described in claim 3. May be defined.

Further, the fixed section is provided with a jib output from the first encoding means 3 after the registered capacity of the first dictionary 1 becomes full, for example. It may be defined as a period until the compression ratio of the input data of the Lempel code string decreases to a certain lower limit.

Further, the fixed section may be, for example, a period during which the first encoding means 3 and the second encoding means 4 output one Jib-Lempel code. May be set.

In the various configurations as described above, the first encoding means 3 and the second encoding means 4
May be configured to perform Jib-Lempel encoding of the same input data in parallel.

Next, FIG. 2 is a principle diagram of another present invention (second invention). The second invention is a decompression method for decompressing Jib-Lempel encoded data by the data encoding method of the first invention. The first dictionary 11 to be initialized, the second dictionary 12 capable of registering the dictionary data generated in the decompression process over the plurality of continuous sections of the compressed data, and the decompression with reference to the flag It is determined whether the dictionary used at the time of compressing the compressed data to be compressed is one of the first dictionary 1 or the second dictionary 2 of the first aspect of the present invention.
Restoring means for selecting either the dictionary 11 or the second dictionary 12 and restoring the compressed data using this dictionary and, if necessary, registering the dictionary data obtained by this restoration in the dictionary. 13 is provided.

According to the second aspect of the present invention, in the above-mentioned configuration, for example, in the specific section, the data length of the restored data restored by the restoring unit 13 after the first dictionary 11 is initialized is a specific length. It may be defined to be a period until it becomes equal to the value.

Further, the certain section is, for example,
As described in claim 9, the period may be defined to be a period from when the first dictionary 11 is initialized to when its registered capacity becomes full.

Further, for example, the fixed section is a period from when the restoration means 13 inputs one Jib Lempel code to when the restoration means 13 restores the Jib Lempel code. May be defined as

In the various configurations described above, the decompression means 13 performs the decompression of the compressed data using the first dictionary 12 and the decompression of the compressed data using the second dictionary 12 in parallel. It may be configured to be performed at the same time.

[0059]

First, in the first invention shown in FIG. 1, for example, a Jib-Lempel code in which one character such as an alphabet, a kana, or an alphanumeric character corresponds to the first dictionary 1 and the second dictionary 2 in advance. Are registered in association with each other (initial setting).

When the data is input, the first encoding means 3 refers to the first dictionary 1 and the second encoding means 4 refers to the second dictionary 2 to correspond to the input data. It is checked whether the Jib Lempel code to be registered is registered in each dictionary. If it has been registered, the next data is input, and it is checked whether a character string obtained by adding the input data to the previous input data is registered in each dictionary. The first encoding unit 1 and the second encoding unit 2 repeat such search processing of the dictionaries 1 and 2 until it is found that the input character string is not registered in each dictionary. When the first encoding unit 1 and the second encoding unit 2 find that the character string input so far is not registered in the respective dictionaries 1 and 2, the first encoding unit 1 and the second encoding unit 2 A corresponding Jib-Lempel code is output, an unused Jib-Lempel code is assigned to the input character string up to this time, and registered in each of the dictionaries 1 and 2. Input characters) then jib
The above processing is repeated as the first character of the character string to be Lempel encoded.

When such processing is repeated many times, new character strings are registered in the first and second dictionaries 1 and 2 more and more. Eventually, the registered capacity of the first dictionary 1 having a small capacity becomes full.

When the first dictionary 1 is in such a state,
The first encoding unit 1 initializes the first dictionary 1 at a predetermined timing as described above. Therefore, the first
The character string corresponding to the data to be input from now on is registered again in the dictionary 1.

On the other hand, in the second dictionary 2 having a large capacity, unregistered characters appearing in a character string from the first input data to the latest input data are registered more and more. For this reason,
In the second dictionary 2, a large number of character strings indicating the global properties of the input data are accumulated.

On the other hand, the first dictionary 1 is initialized for each fixed section of the input data and starts a new registration.
The registered character string reflects the local properties of the input data. Therefore, the first dictionary 1 can easily cope with a change in the property (type) of the input data. When the property of the input data changes, the first dictionary 1 uses the second dictionary 2. The compression ratio is higher than in the case.

For this reason, the data amount of the compressed data (Jib-Lempel code string) of the input data output from the first encoding means 3 and the second encoding means 4 is compared for each of the above-mentioned fixed sections. By selectively outputting compressed data having a smaller data amount, the data compression ratio can be improved as compared with the conventional case. The compressed data output means 5 is such that, when outputting the compressed data, the compressed data is subjected to Jib-Lempel encoding using either the first dictionary 1 or the second dictionary 2. Also outputs a flag indicating whether or not. Thus, the decompression side of the compressed data can know which dictionary should be used at the time of decompression before the decompression, so that the input compressed data can be correctly decompressed.

Next, in the second invention shown in FIG. 2, first, the first dictionary 11 and the second dictionary 12 are initialized similarly to the dictionaries 11 and 12 of the first invention. . When this initialization is completed, the decompression means 13 starts decompression of the compressed data generated by the first invention. The compressed data is composed of a series of a plurality of sets of (flag, compressed data).
Then, it is determined whether the input compressed data is compressed using one of the first dictionary 1 and the second dictionary 2, and in the case of the first dictionary 1, the first dictionary 11 To
In the case of the second dictionary 2, with reference to the second dictionary 12, the compressed data to be input thereafter is restored in units of the fixed section. Further, in this restoration process, the restoration means 13 stores the first dictionary 1 and the second dictionary 2 in the same manner as the first encoding means 3 and the second encoding means 4 of the first invention. ,
The unregistered restored data is registered in association with the Jib-Lempel code. Further, the restoring unit 13 initializes the first dictionary 11 every time data restoration in a certain section is completed, as in the case of the first invention. Therefore, the first dictionary 11 is registered and initialized similarly to the first dictionary 1 of the first invention. Therefore, the decompression means 3 can correctly decompress the compressed data output from the first invention.

[0067]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 3 is a basic configuration diagram of a data encoding system according to one embodiment of the present invention.

In the figure, original data 110 is data in which a plurality of types of information such as character codes and image information are mixed and stored in one file. Also, LZW
The encoding A system 120 includes an LZW encoder A for performing LZW encoding of the original data 110 and a global dictionary 121. The LZW encoder A learns the global properties of the original data 110 using the global dictionary 121. LZW encoding of the original data 110 is performed. The global dictionary 121 is a rewritable dictionary having a sufficiently large dictionary capacity. In the process of LZW encoding of the original data 110 by the LZW encoder A, a dictionary in which the global properties of the original data 110 are reflected. Become.

On the other hand, the LZW encoding B system 130 is composed of an LZW encoder B for performing LZW encoding of the original data 110 and a local dictionary 131, and localizes changes (local changes) in the properties of each part of the original data 110. The LZW encoding of the original data 110 is performed while grasping and learning by the dictionary 131. The local dictionary 131 is a rewritable dictionary having a dictionary capacity smaller than that of the global dictionary 121, and is an LZW encoder B
In the process of LZW encoding of the original data 110, the new registration is repeated many times (initial setting) in units of a specific number of input data (hereinafter, referred to as a block), so that the original data 110 Becomes a dictionary corresponding to the local change of.

The LZW encoding A system 120 and the LZW encoding B system 130 perform LZW encoding of the original data 110 in parallel by the LZW encoder A and the LZW encoder B on a block-by-block basis. The compressed data A and compressed data B of each block of the original data 110 obtained by the encoding are stored in the buffer A150 and the buffer B160, respectively. Compression ratio comparator 1 4 0, based on the data amount of the LZW code data stored in the LZW code data of the data amount and a buffer B160 stored in the buffer A150, each of the original data 110 according LZW encoder A The compression ratio A of the block data and the compression ratio B of the data of each block of the original data 110 by the LZW encoder B are compared between the same blocks, and the compressed data A
The selection signal a instructs the MPX (multiplexer) 170 to select the compressed data having the higher compression rate from the compressed data B.

[0071] MPX170 is the compression ratio comparator 140
In accordance with the instruction of the selection signal a added from the above, the smaller compressed data stored in either the buffer A 150 or the buffer B 160 is selectively output.

The MPX 170 performs the selective output of the compressed data for each block of the original data 110.
The compressed data output from the MPX 170 is discrete time-series data. Further, when the compressed data is selectively output, the MPX 170 adds a dictionary flag indicating which of the global dictionary 121 and the local dictionary 131 is used at the head of the compressed data to output the compressed data. I do.

FIG. 4 shows the configuration of compressed data sequence 180 output from MPX 170. The set of (dictionary flag 181, compressed data 182) shown in the figure is output from the MPX 170 in LZW coding block units (coding block units) of the original data 110 for comparing the compression ratios A and B.

Next, an algorithm of a data encoding (data compression) system according to an embodiment of the present invention will be described with reference to a flowchart of FIG. In this example, LZW
Encoding A system 120 LZW coder A and LZW coding B system 1 3 0 of LZW coder B of has both an input counter (not shown). This input counter is a subtraction counter used to count the number of data in one block (for example, 100 Kbytes), which is a coding unit of the original data 110 as input data. The data input from the original data 110 is performed in units of 1 byte. Even if the 1-byte data is data other than the character code (for example, image information), it is treated as one character, and the input data including a plurality of characters is processed. Expressed as a character string.

First, both the LZW encoder A and the LZW encoder B set the number of data of one block (block size), which is the coding unit of the original data 110, as an initial value in each input counter (step S1). S701).

Subsequently, the LZW encoder A and the LZW encoder B
Inputs the first data of the original data 110 from the corresponding input file (step S702). Next, the LZW encoder A and the LZW encoder B determine whether or not the input data is “EOF” (End of File) indicating the end of the input file (step S703). If there is, the processing ends immediately because the LZW encoding of all the original data 110 is completed, but the input data is “EOF”
If not, the input counter is decremented by "1" (step S704), and it is determined whether or not the value of the decremented input counter has become "0" (CT = 0) (step S705). The determination process of step S705 is a process of determining whether or not LZW encoding of input data of a certain block in the original data 110 has been completed.

In this determination, if CT = 0 is not satisfied, LZ
W encoder A and LZW encoder B judges that the LZW encoding of one block data of the original data 110 is not yet finished at the same time in parallel LZW coding of data input in step S70 2 And start (S70
6).

In this parallel processing, the LZW encoder A performs LZW encoding on the input data using the global dictionary 121 (step S707A), and the LZW encoder B uses the local dictionary 131 to perform LZW encoding on the input data. (Step S707B).

If the LZW code corresponding to the input data is registered in the global dictionary A (S708A, YES), the LZW encoder A attempts LZW encoding of the character string including the input data. Again at step S70
Return to step 2 and enter the next data.

Therefore, the LZW encoder A determines that the input data (one character or character string) is not registered in the general dictionary 121 in the step S708A, and the steps S702 to S706 and the steps S707A to
S709A is repeated.

When the LZW encoder A determines in step S708A that the data string (character string) input so far is not registered in the global dictionary 121 (step S708A).
7081A, NO), the buffer A1 5 the LZW code read from global dictionary 121 in the previous step S707A
0, and a new LZW code is assigned to the input data (one character or character string) found not to be registered in the global dictionary 121 in step S707A, and a set of these (input data, LZW code) is set. Is registered in the general dictionary 121 using the input data as an index (step S709A), and the process returns to step S702 again to start inputting new data of the original data 110 to be subjected to the next LZW encoding from the input file.

The LZW encoder A performs the above operation,
In step S705, the process is performed until the input counter value is determined to be “0”, that is, it is determined that all LZW encoding of the first block of the original data is completed. The compressed data A (LZW code string) of all data of the first block of the original data 110 obtained by the LZW encoding is stored in the buffer A 150 .

On the other hand, LZW encoder B also has LZW encoder A
In parallel with the above, the same processing as in the above-described LZW encoder A is performed using the local dictionary 131, all data of the first block of the original data 110 are LZW-encoded, and the original data 110 obtained by this LZW encoding is obtained. Is stored in the buffer B160 (steps S702-S706, S707B-
S709B).

As described above, LZW encoder A and LZW encoder A
When the W encoder B completes LZW encoding of the first block of the original data 110 by parallel processing (S705, YE
S), the compression ratio comparator 140 compares the compressed data A generated by the LZW encoder A stored in the buffer A150 with the compressed data B generated by the LZW encoder B stored in the buffer B160. The sizes of the capacities are compared (step S710), and if the capacity of the compressed data A is smaller than the capacity of the compressed data B (step S71).
0, YES), and instructs the MPX 170 to select the compressed data A by the selection signal a.

Upon receiving the selection signal a, the MPX 170 first outputs a flag A indicating that the data is compressed data using the general dictionary 121 to an output file (step S712), and then outputs the general data stored in the buffer A150. The compressed data A using the dictionary 121 is selectively output to the output file (step S713).

[0086] On the other hand, the compression ratio comparator 140, at step S710, the capacity of the compressed data B is determined to be less than capacity <br/> amount of compressed data A (step S710, N
O), the selection signal a instructs the MPX 170 to selectively output the compressed data B.

Upon receiving the selection signal a, the MPX 170 first outputs a flag B indicating that the data is compressed data using the local dictionary 131 to the output file (S
713), the compressed data B obtained by using the local dictionary 131 stored in the buffer B 160 is selectively output to the output file (S714).

Thus, (flag, compressed data)
Is output as compressed data of the first block consisting of The above processing is performed for each block after the second block of the original data 110 in units of blocks, and the LZW encoder A and the LZW encoder B perform LZW encoding on all data of the original data 110 in the input file. Is determined to be completed (step S7).
03, YES), the LZW encoding of all data of the original data 110 ends.

As described above, for each block of the original data, the compressed data A by the LZW encoder A using the global dictionary 121 and the compressed data B by the LZW encoder B using the local dictionary 131 are created in parallel. For each block, the compressed data having the higher compression ratio of either the compressed data A or the compressed data B is stored in the output file together with a flag (flag A or flag B) indicating the dictionary used for the compression. Is stored.

Accordingly, the original data 110 is converted into an optimal LZW for each block according to the data properties of each block.
By encoding, the original data 110 can be efficiently compressed at a very high compression ratio.

[0091] In the above embodiment, compressed data A stored in the buffer A150 and the buffer B1 6 0, but by comparing the capacity of B so that to determine the relative merits of the compression ratio, LZW coding A System 120 and LZW coded B system 1
A means for calculating the compression ratio may be provided in the unit 30, and this means may be configured to directly output the compression ratios A and B of the compressed data A and B to the compression ratio comparator 160.

Further, in the above case, the data compression ratio is compared in units of blocks composed of a large amount of input data. However, the input data string (search matching character string) encoded for each LZW encoding is performed. The data length of each column may be compared, and the LZW code having the shorter data length may be sequentially output as compressed data.

FIG. 6 is a diagram showing an example of a method of using the global dictionary 121 and the local dictionary 131 when the original data 110 is subjected to LZW encoding in block units by the data encoding method.

FIG. 11A is a diagram showing a change in the number of registered general dictionaries 121 and local dictionaries 131 in the process of LZW-encoding the original data 110 in block units according to the data encoding method. the axis LZW encoded number of data (number of input data), and the vertical axis corresponds to the registration number of the global dictionary 121 and local dictionary 1 31. Also, L on the vertical axis
Each scale of _T and L _K indicates the registration capacity of the global dictionary 121 and the local dictionary 131, respectively.

FIG. 13B shows the data compression ratio of each block when the global dictionary 121 and the local dictionary 131 are used in the same data encoding process as in FIG. FIG. 7 is a diagram showing a comparison, in which the horizontal axis corresponds to the same number of input data as in FIG. 7A and the vertical axis corresponds to the data compression ratio in each block.

In the example shown in FIG. 6, the LZW encoder B
Indicates that the number of registrations in the local dictionary 131 is LZW of each block.
If it becomes full during encoding, dictionary registration is immediately stopped. Also, the LZW encoder B calculates the L of each block.
Before starting the ZW encoding, the local dictionary 131 is always cleared (initial setting).

The local dictionary 131 can be cleared, for example, by
This is an initialization process for registering all characters consisting of one character as initial dictionary information. As shown in the figure, in the LZW encoding of the first block, the local dictionary 131 and the global dictionary 1
21 are the same, and the number of registrations in the local dictionary 131 becomes full during the LZW encoding process of the first block of the original data 110. Therefore, the compression ratio of the first block is determined by using the global dictionary 121. The compressed data A output from the LZW encoder A obtained by using the local dictionary 131
The compression ratio is slightly higher than the obtained compressed data B output from the LZW encoder B.

In the LZW encoding of the second block and the third block of the original data 110, the compression rate of the compressed data B obtained by using the global dictionary 121 is larger than that of the local dictionary 1
The compression ratio is higher than that of the compressed data A obtained by using No. 31. However, in the LZW encoding of the fourth block of the original data 110, the nature of the input data has changed, so that the compressed data B obtained using the local dictionary 131 is better than the compressed data A obtained using the global dictionary 121. However, the compression ratio is slightly higher.

Therefore, the compressed data sequence output from the MPX 170 includes (flag A, compressed data A ₁ , flag A, compressed data A ₂ , flag A, compressed data A ₃ , flag B, compressed data B ₄ ... ). Note that a subscript i (i = 1, 2, 2) added to the compressed data A and the compressed data B
3, 4,...) Are numbers indicating the compressed data of the block i.

As described above, in the example shown in FIG. 6, the number of registered local dictionaries 131 is set to such a small amount that the registered capacity is saturated in the latter half of LZW encoding of each block. Is set to a large capacity such that the registered capacity does not become saturated even when LZW encoding is performed over a plurality of blocks. Here, the local dictionary 13
FIGS. 7 and 9 show another method of coping with the case where the registered capacity of No. 1 becomes full during the LZW encoding process of the block.

In the example shown in FIG. 7, when the registration in the local dictionary 131 is full (if it is saturated), the lowest level (lowest level) of the decomposition component tree (tree) for registering the registration character is registered. Level) and characters attached to the branches connected to the (section) (in the case of the level shown in FIG. 8, each component of X ₃ , X ₉ , X ₈ belonging to level 3 and Pruning deletion processing to delete the characters attached to the branches connected to those components) or processing to preferentially delete the registered characters with a small number of references based on the reference frequency of each registered character, and register An empty space is reserved, and new characters are registered in the empty space. The structure of the decomposition component tree is described in the above-mentioned Munakata “Ziv-
This is described in detail in 3.1, Incremental Decomposition of Input Sequence, in Lempel's Data Compression Method.

On the other hand, in the example shown in FIG.
LRU (Leastly)
According to the Recent Used method, one of the oldest registered character strings is deleted, and a new character string is registered instead.

Further, the example shown in FIG.
In the process of LZW encoding of each block, the registration capacity of the local dictionary 131 is ensured to be sufficiently large in advance so that the registration capacity does not become full (so as not to be saturated). In such a case, the data compression rate of the LZW encoder B is generally higher than that of each of the other system local dictionaries 131.

Further, in the example shown in FIG. 11, even when LZW encoding has progressed only halfway through the block, the local dictionary 131 is cleared immediately when the registration capacity of the local dictionary 131 becomes full, and then a new local dictionary is cleared. This is to restart dictionary registration.

Further, in the example shown in FIG. 12, when the registration capacity of the local dictionary 131 becomes full (saturated), the registration of a new dictionary is stopped, and thereafter, the data compression ratio is continuously monitored. When the data compression ratio becomes lower than a predetermined reference value, it is determined that the data compression ratio has deteriorated, the local dictionary 131 is cleared at this time, and then dictionary registration is restarted.

Next, FIG. 13 shows a basic configuration of a data decompression system for decompressing the compressed data sequence 182 output from the data encoding system and having the configuration shown in FIG. 4 to original data.
Shown in The LZW decompression A system 220 includes a global dictionary 221 having a large registration capacity and an LZW decompressor A for decompressing the compressed data 182 using the global dictionary 221.

[0107] Further, LZW recovered of B system 230 includes a small local dictionary 231 registration capacity, using the local dictionary 231 consists LZW decompressor B for restoring the compressed data 1 8 2.

The LZW decompression system 220 and the LZW decompression system 230 simultaneously decompress the compressed data 182 . The LZW reconstructor A and the LZW reconstructor B
Is recovered data A restored respectively, using global dictionary 221, the local dictionary 23 1, and outputs the restored data B to the MPX (multiplexer) 250.

The dictionary flag discriminator 240 receives the dictionary flag 181 of the compressed data sequence 180, and uses the dictionary flag 181 to determine whether the LZW reconstruction A system 220 and the LZW
The decompression B system 230 determines the dictionary (global dictionary 121 or local dictionary 131) used to create the compressed data 182 which is decompressed in parallel, and performs LZW encoding of the compressed data 182 currently being decompressed. Is output to the MPX 250 described above.

The MPX 250 includes a dictionary flag discriminator 240
In accordance with the instruction of the selection signal d input from the
If the instruction 1 is designated, the restoration data A outputted from the LZW restoration unit A is selected, and if the local dictionary 131 is designated, the restoration data B outputted from the LZW restoration unit B is selectively outputted.

Next, a method of decompressing the compressed data sequence 180 (see FIG. 4) output from the data encoding system shown in FIG. 3 performed by the data decompression system having the above configuration will be described with reference to the flowchart of FIG. I do.

First, the LZW decompressor A and the LZW decompressor B
Is the original data 110 in the data encoding system.
Is set in the input counter incorporated therein (S8).
01).

Subsequently, the dictionary flag discriminator 240 outputs data (first input data) added from the file (the output file generated by the data encoding system) to the head of the compressed data 182. That is, the dictionary flag 181 (see FIG. 4) is read (step S802).

Next, the LZW decompressor A and the LZW decompressor B
Simultaneously inputs the first data of the compressed data 182 following the dictionary flag 181 from the file (step S8).
03), it is determined whether or not the input data is “EOF” (End of File) indicating the end of the file (step S804).

Then, the LZW decompressor A and the LZW decompressor B
Determines that the input data is not "EOF" (step S804, NO), the dictionary flag discriminator 240 determines whether the input data is the dictionary of the global dictionary 121 or the local dictionary 131 based on the dictionary flag 181 previously input. Is determined using LZW encoding, and a selection signal d indicating the corresponding dictionary used for LZW encoding is set to M
Output to PX250. During this time, the LZW decompressors A and LZ
The W restorer B restores the input data in parallel using the global dictionary 221 and the local dictionary 321, respectively, and outputs the restored data A and B to the MPX 250, respectively.

The MPX 250 has the dictionary flag discriminator 24.
According to the selection signal d added from 0, the corresponding data of the restored data A or the restored data B is selectively output. By the way, LZW decompressor A and LZW decompressor B
If the restored data has not been registered in the respective dictionaries 221 and 231, the set of (the restored data and the LZW code corresponding to the restored data) is stored in the dictionary 221, respectively.
1 and 231 (step S805).

Subsequently, the LZW decompressor A and the LZW decompressor B
Decrements the input counter by the number of characters of the restored data, and sets the number of characters of the remaining restored data of the block currently being restored (current block) to each of the input counters (S807).

Then, the LZW decompressor A and the LZW decompressor B determine whether the value of each input counter is "0", that is, whether or not the restoration of the current block has been completed (step S808).

Then, the LZW decompressor A and the LZW decompressor B
Determines that the value of each input counter is not “0” and that the restoration of the current block has not been completed yet,
(Step S808, NO), the process returns to step S803, and repeats the process of inputting the next data from the file and restoring the remaining data of the current block (repetition of steps S803 to S808).

Then, the LZW decompressor A and the LZW decompressor B
Is the compressed data 1 of the current block in step S808.
If it is determined that the data restoration has been completed for all of the 82 (step S808, YES), the LZW reconstructor A,
B, the dictionary flag discriminator 240, and the MPX 250 perform the processing of steps S801 to S808 again, and the LZW
Until the decompressor A and the LZW decompressor B read "EOF" from the file in step S804, the compressed data of all the remaining blocks after the next block stored in the file is decompressed.

Then, the LZW decompressor A and the LXW decompressor B
Determines that the decompression of the compressed data 182 of all blocks stored in the file is completed (step S8).
04, YES), the decompression process of the compressed data 182 ends.

In this manner, the data that is LZW-encoded and compressed in block units by the data encoding method shown in the flowchart of FIG. 5 described above is added to the dictionary added to the head of the compressed data 182 of each block. Flag 1
The image data is sequentially restored in block units with reference to 81.

[0123]

According to the first aspect of the present invention, there are provided two dictionaries in which the capacity of the first dictionary having a small capacity and the capacity of the second dictionary having a large capacity are different from each other. Since the first dictionary is initially set for each fixed section of the input data, dictionary information corresponding to a local change in the property of the input data, which is difficult to handle with the second dictionary alone, can be registered in the first dictionary. as a result, the perform the LZW coding adapted to the global nature of the input data by the second dictionary, it is possible to perform the LZW coding corresponding to the local properties of the input data by the first dictionary, Input data in which a plurality of types of data having different properties such as character codes and black and white image data are mixed can be subjected to data compression at a higher compression than that of the conventional LZW encoding.

The second aspect of the present invention is the second aspect of the present invention.
According to the invention, since the first and second dictionaries having the same functions as those of the first invention are provided, the decompression means is provided for each compressed data which the first invention adds to each compressed data and outputs the compressed data. By referring to the flag indicating the dictionary used to create the data, the data compressed according to the first aspect can be
The original data can be completely restored by referring to, updating, and registering the corresponding dictionary.

[Brief description of the drawings]

FIG. 1 is a principle diagram (part 1) of the present invention.

FIG. 2 is a principle diagram (part 2) of the present invention.

FIG. 3 is a basic configuration diagram of a data encoding system according to an embodiment of the present invention.

FIG. 4 is a diagram illustrating a configuration of a compressed data sequence according to the present invention.

FIG. 5 is a data encoding method (data compression method) of the present invention.
5 is a flowchart illustrating an algorithm according to one embodiment.

FIG. 6 is a diagram illustrating a first example of a method of using a global dictionary and a local dictionary.

FIG. 7 is a diagram (part 1) illustrating another example of how to use the local dictionary.

FIG. 8 is a diagram illustrating a configuration of a decomposition component tree for registering a registration character.

FIG. 9 is a diagram (part 2) illustrating another example of how to use the local dictionary.

FIG. 10 is a diagram (part 3) illustrating another example of how to use the local dictionary.

FIG. 11 is a diagram (part 4) illustrating another example of how to use the local dictionary.

FIG. 12 is a diagram (part 5) illustrating another example of how to use the local dictionary.

FIG. 13 is a basic configuration diagram of a data restoration system according to an embodiment of the present invention.

FIG. 14 is a flowchart illustrating an algorithm of a data restoration method performed by the data restoration system.

FIG. 15 is a diagram illustrating a basic concept of encoding of a universal type Jib Lempel code.

FIG. 16 is a flowchart illustrating an algorithm of LZW encoding.

FIG. 17 is a diagram illustrating a configuration of a dictionary used for LZW encoding.

FIG. 18 is a schematic diagram illustrating an LZW encoding method.

FIG. 19 is a flowchart illustrating an algorithm for restoring an LZW code.

FIG. 20 is a schematic diagram for explaining a specific example of restoration of an LZW code.

[Explanation of symbols]

 1, 11 First dictionary 2, 12 Second dictionary 3 First encoding means 4 Second encoding means 5 Compressed data output means 13 Decompression means

────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Hirotaka Chiba 1015 Kamiodanaka, Nakahara-ku, Kawasaki City, Kanagawa Prefecture Inside Fujitsu Limited (56) References JP-A-5-66917 (JP, A) JP-A-4-43720 (JP, A) JP-A-2-34038 (JP, A) JP-A-1-158825 (JP, A) JP-A-62-209948 (JP, A) JP-A-2-73876 (JP, U) ( 58) Investigated field (Int.Cl. ⁷ , DB name) G06F 5/00 G06T 9/00 H03M ^7/ 30-7/46 H04N 1/41 H04N 1/413

Claims (11)

(57) [Claims] In a data encoding method for compressing data by universal encoding using Jib-Lempel code, an initial setting is made each time input data is subjected to Jib-Lempel encoding over a specific fixed section. And a second dictionary (2) capable of registering all dictionary data generated in a Jib-Lempel encoding process over a plurality of continuous intervals of the input data. The first dictionary (1) performs Jib-Lempel encoding on the data to be processed, and new dictionary data generated in the Jib-Lempel encoding process is registered in the first dictionary (1). And Jib-Lempel encoding of data input to the first encoding means (1) using the second dictionary (2). Excessive Second encoding means (4) for registering new dictionary data generated in the process in the second dictionary (2), and the first encoding means (3) obtained for each of the predetermined sections. The data amount of the jib-Lempel code string obtained by the second encoding means (4) is compared with the data amount of the jib-Lempel code string, and the jib-Lempel code string with the smaller data amount is used for this encoding. And a compressed data output means (5) for outputting the compressed data together with a flag indicating the dictionary. 2. The method according to claim 1, wherein the certain section includes the first dictionary.
2 is a period from the initial setting to the time when the number of data subjected to the Jib Lempel encoding by the second encoding means (4) becomes from “0” to a specific number. Data encoding method. 3. The method according to claim 1, wherein the certain section is the first dictionary.
2. The data encoding method according to claim 1, wherein the period is from the initial setting until the registered capacity becomes full. 4. The method according to claim 1, wherein the certain section is the first dictionary.
Is a period from the time when the registered capacity becomes full to the time when the compression ratio for the input data of the Jib-Lempel code string output from the first encoding means (3) decreases to a certain lower limit value. The data encoding expression according to claim 1. 5. The fixed section is a period in which the first encoding means (3) and the second encoding means (4) output one Jib Lempel code. 1
The described data encoding method. 6. The first encoding means (3) and the second encoding means (3)
The encoding means (4) performs Jib-Lempel encoding of the same input data in parallel.
2. A data encoding method according to 2, 3, 4 or 5. 7. A data restoration method for restoring Jib-Lempel-encoded compressed data according to the data encoding method according to claim 1, wherein initialization is performed each time the compressed data is restored over a specific fixed section. A first dictionary (11) to be registered; a second dictionary (12) capable of registering dictionary data generated in a decompression process over a plurality of continuous sections of the compressed data; and the flag, It is determined whether the dictionary used for compressing the compressed data to be decompressed is either the first dictionary (1) or the second dictionary (2), and the first dictionary (11 ) Or the second dictionary (12) is selected, and the compressed data is restored using the selected dictionary, and the dictionary data obtained by this restoration is added to the dictionary if necessary. Climb Recording means (13) for recording data. 8. The first dictionary (1)
8. The data restoration method according to claim 7, wherein 1) is a period from the initial setting to the time when the data length of the restoration data restored by the restoration means (13) becomes equal to a specific value. 9. The first dictionary (1)
8. The data restoration method according to claim 7, wherein 1) is a period from the initial setting to the time when the registered capacity becomes full. 10. The method according to claim 1, wherein the certain section is provided with the restoration means (1).
8. The data restoration method according to claim 7, wherein 3) is a period from inputting one Jib Lempel code to restoring the Jib Lempel code. 11. The decompression means (13) performs decompression of the compressed data using the first dictionary (12) and the second dictionary (12).
11. The data decompression method according to claim 7, wherein the decompression of the compressed data using the dictionary (12) is performed in parallel and simultaneously.