JPH07288475A - Device for managing a plurality of dictionaries in data compression based on associative memory - Google Patents

Abstract

PURPOSE: To suppress the loss of data compression, which is generated when the dictionary of data compression system is reset in a dictionary base, to a minimum by efficiently switching an active dictionary and a reserve dictionary. CONSTITUTION: A data compressor/decompressor circuit 50 enables data transfer between one of dictionaries and a data compression engine 52 during ordinary operation. Besides, the reserve dictionary directly receives data from the data compression engine 52 or from the active dictionary, and a partial data entry cluster can be prepared. While the circuit 50 is operated, a switch controller 76 selects a dictionary 1 or 2 as the active dictionary. When the dictionary 1 is selected, next, the compression engine starts the compression of data and the read/write of encoded data to a data entry field 94 in the dictionary 1. Based on this compression algorithm, a certain encoded data string is discriminated as a candidate to be written in the reserve dictionary 2.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates generally to data compression and decompression methods and apparatus, and more particularly to lossless data compression algorithms using dictionaries for storing compression and decompression information. Concerning the implementation of.

[0002]

BACKGROUND OF THE INVENTION In most compression methods, binary sequences or "codewords" are used to encode multiple character strings. Binary sequences or codewords are not used to encode individual characters. Each string consists of an "alphabet" or a single character string. This alphabet represents the smallest unique information processed by the compressor. Therefore, an algorithm that uses 8 bits to represent that character has 256 unique characters in the alphabet. The compression is so effective that a multi-character string represented by an encoding method can be seen in a file with a data stream. In analogy with the bilingual dictionaries used for translating human languages, the device that performs the mapping between uncompressed code and compressed code is commonly called a "dictionary".

In general, the effectiveness of dictionary-based compression methods is
Depends on the frequency of use of dictionary entries for multi-character strings. Optimizing a fixed dictionary for one file type is less likely to be optimal for other file types. For example, dictionaries that have many combinations of characters, such as those found in newspaper text files, are less likely to efficiently compress database files, spreadsheet files, bitmapped graphics files, computer aided design files, etc.

The adaptive compression method is well known. In this method, the dictionary used for compressing the input data is expanded while compressing the input data. Codewords are placed in this dictionary to represent each single character that may be present in the uncompressed input data. When a multi-character string appears in that file, an additional entry is added to the dictionary. This additional dictionary entry is used to encode such a multi-character string when it occurs subsequently. For example, the matching of the current input pattern is done only for the phrases currently in the dictionary. Each time a match fails, a new phrase is added to the dictionary. This new phrase is formed by expanding the matched phrase by one symbol (e.g., an input symbol that "breaks" the match). Compression is done to the extent that the most frequently occurring multi-character strings in this file appear during dictionary expansion.

In decompression, the dictionary is similarly constructed. Thus, when a codeword of a character string present in the compressed file appears, the dictionary has the information necessary to reconstruct the corresponding character string. Widely used compression algorithms that use dictionaries to store compression and decompression information are LZ1, respectively.
Lempel and Jib (Z) called LZ2
iv) first and second methods. These methods are disclosed in US Pat. No. 4,464,650 to Eastman et al., And various improvements to these algorithms have been made by Welc.
h) U.S. Pat. No. 4,558,302, Mirror (Mi
Ller) et al., U.S. Pat. No. 4,814,746. These references further describe the use of dictionaries.

In practice, the amount of memory available for compression / decompression is finite. Therefore, the number of entries in the dictionary is finite, and the length of the codeword used to encode each entry is limited. Typically,
This length is between 12 and 16 bits. If the input data sequence is long, the dictionary will eventually “fill”. At this time, it can be dealt with in several ways. For example, the dictionary can be frozen in its current state and used for the rest of the input sequence. In the second method, the dictionary is reset and a new dictionary is created from scratch. In the third method, the dictionary is temporarily frozen until the compression ratio drops, and then reset.

The first method has the disadvantage that the learning ability of the basic compression algorithm is lost. When the statistics of the input data change, the dictionary cannot keep up with such changes,
A rapid reduction in compression ratio occurs. The dictionary reset method preserves the learning ability of the algorithm, but causes a temporary loss of compression ratio when switching to an empty dictionary (eg, loses knowledge of all sources accumulated so far). .

One of the methods for reducing the number of times the dictionary is reset is to increase the memory size of the dictionary. However, increasing the memory size increases the cost and may increase the time required to search the dictionary data entry. Much research has been done on hashing algorithms that quickly discover data in memory that can be accessed sequentially. For example, Welch, US Pat. No. 4,558,
No. 302, etc.

Another obstacle to compression / decompression performance is the amount of time it takes to search the dictionary for a previously appearing character string. Hashing algorithms are commonly used to search previously stored dictionary entries and to find storage locations available for new character strings. A typical configuration is a Random Access Memory (RAM) having two to four memory locations for each dictionary entry, as disclosed in US Pat. No. 4,558,302 to Welch (LZW). Is used.

The hashing algorithm maps each unique dictionary entry into a RAM random access memory space at an address based on a simple arithmetic function of the contents of a data word. Since such algorithms use all words or fields within a word to calculate the mapping address, two or more data words may be mapped to the same location in memory, resulting in hashing collisions. In this case, you have to find an alternative location for this data. Inevitably, when the RAM fills up, the second dictionary entry will mix in with the previously used location. This situation must be resolved in order to continue compression. Hashing circuit,
Hashing collisions in particular significantly complicate the compression / decompression system logic and reduce system throughput.

Usually, a dictionary based on compressed data is
It is a small subset of all possible data entries. Therefore, one way to reduce hashing collisions is to increase the number of dictionary locations. However, this approach increases system complexity and cost,
The memory and compression / decompression control logic cannot be integrated. In addition, the increased memory will increase the search time required to determine if a character string has been previously loaded into memory.

Another obstacle to data compression / decompression is the amount of time and circuit complexity required to encode and decode data character strings. For example, when compressing data, if one finds that a character string does not match any of the data phrases previously stored in memory, it must be stored in an unused data storage location. A codeword must be generated that identifies the stored character string and the subphrase in the character string that previously matched the dictionary data entry. This codeword must be stored so that it can be combined with additional characters during subsequent data compression operations.

During decompression of data, the compressed data codewords are converted to uncompressed data characters, such as the Hewlett-Packard Journal (Hewl).
ett-Packard Journal), 1989.
It may represent an additional codeword, such as a link to the rest of the uncompressed data string as described in the June issue, pages 27-31. The HP-DC method described therein sequentially encodes codewords and stores the codeword (OMEGA) associated with the next byte (K) at the dictionary address location determined by the compressed code. Therefore, this dictionary must be read several times before the actual decompressed data string is generated. Because this compression and decompression method is iterative, additional clock cycles other than the clock cycles used to access the dictionary significantly increase the overall compression and decompression times. However, this encoding, decoding and dictionary search method requires two or more clock cycles to compress or decompress each input character. Moreover, these encoding and decoding algorithms require complex compression and decompression hardware. Therefore, the performance improvement of the dictionary-based data compression system and the dictionary-based data compression /
There is a need for improved encoding and decoding in decompression systems.

[0014]

SUMMARY OF THE INVENTION It is therefore an object of the present invention to minimize the loss in data compression that occurs when the dictionary of a dictionary-based data compression system is reset.

A second object of the invention is to increase the adaptability of a data compression system for input data sequences with varying statistical properties.

Another object of the present invention is to reduce the time required to encode and decode character strings in a dictionary-based data compression / decompression system.

Another object of the present invention is to simplify the search of the data compression / decompression dictionary for previously appearing character strings and prevent hashing collisions.

Yet another object of the present invention is to maximize the data compression capability of a dictionary-based data compression / decompression system with a minimum amount of memory.

Yet another object of the present invention is to efficiently switch between multiple dictionaries in a dictionary-based data compression / decompression system.

[0020]

One of the features of the present invention is a data compression / decompression system for simultaneously constructing a current dictionary and a standby dictionary. Current dictionaries serve the same purpose as dictionaries in standard data compression engines. The standby dictionary is constructed in parallel with the current dictionary to have a subset of the phrases in the current dictionary. This subset is chosen to best describe the patterns that occur in the source data. When the current dictionary is full, it is replaced by the standby dictionary, and a new standby dictionary is built "from scratch" while continuing to build and use the new current dictionary. Therefore, the compressor will never switch to an empty dictionary, reducing the degradation of data compression that occurs due to the limited memory size of the dictionary.

The current dictionary initially has enough empty space to add new data entries, so that it is possible to adapt continuously to the source data. This feature is especially important for source data compression with varying statistics. Some information is lost when switching to a smaller number of data entries in the standby dictionary, but the time to rebuild this dictionary with maximum efficiency is still better than the time to completely reset the dictionary. short. Therefore, a smaller dictionary memory can be used with less adverse effect on the data compression ratio.

The criteria for selecting a subset of the current dictionary to enter the standby dictionary may vary from application to application. For example, an encoded data string is duplicated if it has matched a data entry in the current dictionary at least once. Also, the entries in the wait dictionary can be selected according to the length of the string, the most recent match with the data entry, or a criterion that identifies the entry that maximizes compression in a given application.

Further, the criteria for switching (resetting) from the current dictionary to the standby dictionary can be changed depending on the type of data or the application. For example,
The current dictionary can be reset when the valid data entries are full. Alternatively, the current dictionary is Kato et al., U.S. Pat. No. 4,847,6.
As described in No. 19, it can be reset if it falls below a certain performance threshold when used for compression.

In the second application of the standby dictionary, this compressor makes two passes through the data, primarily in situations where the data characteristics are unchanged. In the first pass, the compressor scans large data samples. This sample is sized to fill the current dictionary multiple times, thereby causing the standby dictionary to replace the current dictionary a corresponding number of times. Each time a dictionary switch occurs, the current dictionary is "refined" after a few iterations until the algorithm builds a strongly customized dictionary for this data sample. This customized dictionary is then used by the second compression engine to compress the input data.
Set as the only dictionary to use during the pass. This customized dictionary thereby works much better than a single dynamic dictionary for the same data.

A second feature of the present invention is a dictionary-based compression / decompression system architecture that simplifies encoding and decoding circuits using address values of stored data entries in the compression / decompression system dictionary. And that way. The system preferably has associative memory (C) with additional logic including local feedback circuitry to speed up access to the memory and provide special features to simplify external compression / decompression logic.
AM) is used. This memory structure has the unique ability to provide lossless data compression or decompression at a constant rate of one character per clock cycle without hashing or hashing collisions.

That is, the system preferably comprises an associative memory which encodes a character string according to the address location of the data entry held in the memory. A combination of input character strings that has never occurred before in the input data string is stored in the dictionary as a new data entry. The CAM (Associative Memory) is organized into "words" that each store a unique character string data entry. This memory performs an associative parallel search with an input character string having selected bits in a "word" for every word previously stored in the dictionary. If there is a match, the match line associated with that data entry is activated. All matching lines are then encoded into a single codeword representing a character string. This codeword is combined with the next input character and again compared with the data entry previously stored in memory. In this way, character strings are assigned codewords according to their addresses in memory. If the search fails, a codeword (OMEGA) that represents the last matching character string.
The (e.g. its address) is output and the search begins again with a new character string starting with the character (K) that caused the match failure. This compressed data character (codeword) is a pointer to a data entry in the dictionary. Therefore, the character string is encoded by using this compressed data character as an address to the decompression dictionary. For example, initially an external compressed character is used as the address to this dictionary. Next, the data entry at the encoded address position is read. If the output of the data entry from memory does not require further decompression (for example,
If the memory output is the "root" of the linked list) then this data entry is output. If this data entry contains another codeword (eg, an encoded link to another dictionary address location), this codeword is fed back to memory as the next dictionary address.

An internal address generator is used for both compression and decompression and resets at the same time the memory is reset. Any write to memory (either an explicit write or a failed match) will cause the address to be incremented to the next address. The increment need not be sequential, as long as both the compressor address generator and the decompressor address generator are in the same state and increment in the same way, so that the compression and decompression dictionaries are identical, for example pseudo-random. May be This logic eliminates the need for address generation / storage in external control logic and can improve compression / decompression performance (eg, fewer clock cycles and faster data compression).

To further reduce the time required to compress the data, a special update circuit allows the memory to be searched and the data to be written during the same clock cycle. When a character string is compared with a data entry in memory, if the search fails, this string must be stored as a new data entry.
The next available address location is already known from the address generator and this character string already exists in the memory data input. Therefore, the control logic can be used to automatically write this character string to memory if no match occurs during the search.
Therefore, the memory is automatically updated during the memory search clock cycle. If a match is found during the search operation, this update circuit prevents this character string from being loaded into memory as valid data.

The system and method summarized above provide a simple, inexpensive and versatile high speed data compression / decompression system. The system and method may be implemented in software on a general purpose computer or hardware using custom or semi-custom integrated circuits. The system and method can be used to perform storage / retrieval of linked list data structures. And, it can be easily applied to various adaptive dictionary-based encoders.

The third feature of the present invention is that the minimum memory / high compression capacity of the standby dictionary system is combined with the high-speed 1-cycle encoding / decoding per character of the CAM (associative memory) circuit. This circuit uses multiple dictionaries in the memory locations of the CAM circuit. The CAM circuit receives the compressed and uncompressed character strings and stores them as data entries into one of the dictionaries. A codeword representing each data character string is then generated according to the address of the dictionary data entry that matches this character string.

To support multiple dictionaries, each memory location in the CAM includes a status field and a data field. The data field stores the data entry,
The status field indicates which dictionary is assigned to that data entry. During a search operation, this circuit can mask certain bits in both the status and data fields. This allows the system to determine which dictionary is assigned to a data entry and whether a particular storage location is not currently assigned to the dictionary.

The assignment of the dictionary to each data entry can be easily switched by changing the state of the compression / decompression circuit. At least one dictionary is reset by changing the state of the circuit.
This causes the memory location previously assigned to that dictionary to be an empty memory location not assigned to any dictionary. A new character string can be stored in this empty storage location. Such state changes can be triggered by different events, maximizing the compression ratio and the adaptability of the system to different types of data. For example, the compression / decompression circuit changes state automatically when one of the dictionaries is full, or when the compression ratio drops below a predetermined performance level. To provide single clock cycle search capability, this compression / decompression circuit builds a standby dictionary in parallel with the current dictionary and searches multiple dictionaries simultaneously. The above objects, features and advantages of the present invention, as well as other objects, features and advantages, will be more apparent from the following detailed description of the embodiments with reference to the drawings.

[0033]

【Example】

I. Data compression / decompression system using standby dictionary FIG. 1 is a data flow diagram of a data compression / decompression system having a current dictionary and a standby dictionary. The method shown in FIG. 1 uses the current dictionary (CD) and the standby dictionary (S) in block 8.
It starts from the initial setting of D). For example, a codeword is entered into the dictionary to represent each single character that may be present in the uncompressed input data. Alternatively, the initial dictionary can be left empty. The encoding of the character string from the data sequence is performed in any desired encoding scheme.

In block 10, the input data is compared to the previously encoded data entry of the current dictionary to determine if this character string matches any of the dictionary's data entries. At block 12, the unmatched character string is stored as a new encoded data entry in the current dictionary. When the match cannot be expanded further, the code of the longest matched string is output at block 13.

In block 14, a subset of the data entries encoded before the current dictionary (CD) is stored in the standby dictionary (S
It is stored in D). The subset selection process in block 14 can be modified depending on the input data to obtain the highest compression ratio using a given number of data entries in the standby dictionary, as described above. .
For example, a data entry in the standby dictionary can be selected based on the number of times an input character string matches a data entry in the dictionary. Alternatively, the standby dictionary subset can be selected according to the number of input characters represented by the encoded character string. In general, any method can be used at this stage.

Decision block 16 determines if a dictionary reset is needed. For example, a reset is needed when the current dictionary reaches a predetermined number of encoded character string entries, or when the compression ratio falls below a certain performance threshold. When the current dictionary does not require a reset, the compression engine reads the new character string and returns to block 10. When the current dictionary is reset, block 18 replaces the current dictionary with entries in the standby dictionary, initializes a new standby dictionary, reads a new character string, and then returns to block 10.

The dictionary-based compression / decompression method of FIG. 1 can be used to generate a static, customized current dictionary used to compress data. For example, a data sample of the input data sequence is selected. The current dictionary is then customized by repeatedly replacing the current dictionary with the standby dictionary. The customized current dictionary is locked to a read-only function and is used exclusively by the compression engine to compress or decompress this data sequence.

FIG. 2 is a detailed data flow diagram showing an example of a data compression algorithm using the current dictionary and the standby dictionary. FIG. 2 illustrates how the data string is copied to the standby dictionary when the input data string matches an entry in the current dictionary. This method ensures that this data string is "seen" at least twice during input. The current and standby dictionaries are switched when the current dictionary is full (eg, when a predetermined number of valid data entries has been reached). As described above, the dictionary switching criteria and the standby dictionary data entry selection criteria can be easily implemented according to specific application conditions.

The input data string is compared at block 20 with the data entry in the current dictionary. Decision block 2
2 is divided into block 23 and block 24 when there is no match between the input data and the entry of the current dictionary. The longest matched data string is then encoded in block 23 and output.

Decision block 24 determines if the current dictionary is full. If the current dictionary is not full, block 28 stores the data string as a data entry in the current dictionary. Alternatively, block 2
In any case except 2, the flag can be set. If the current dictionary is full, block 26 switches the current dictionary to the standby dictionary. The data string is then stored in this new current dictionary. The current dictionary has been replaced by a smaller subset of data entries (eg, the data entry of the standby dictionary), thus providing space to store the new data string. At block 30, the address counter of the current dictionary is incremented. At block 34, the new input character is read from the input data and the comparison process of block 20 is repeated.

When decision block 22 determines that the input data matches an entry in the current dictionary, a check is made at decision block 36 to determine if this data string was previously stored in the standby dictionary. To be judged. If this data string has not been previously copied to the standby dictionary, a flag is set in the status field in the current dictionary. Alternatively, this flag can be set at any time except block 22. This flag is bound to the current dictionary data entry that matched the data string. This flag indicates to the compression engine that this data entry has been previously copied to the standby dictionary. This prevents the same data entry from being copied multiple times into the standby dictionary. Block 40 writes the data string to the standby dictionary and block 42 increments the standby dictionary address counter. Since this data string matched a data entry in the current dictionary, block 44 adds the next input character to the current data string and returns to block 20. If decision block 36 determines that this data string was previously stored in the standby dictionary, the process directly enters block 44 and continues as described above.

FIG. 3 shows a data compression / decompression integrated circuit (I
C) An example of carrying out the invention in 50 is shown, which is the presently preferred embodiment. The data compressor / decompressor (DCD) integrated circuit (IC) 50 includes a data compression / decompression engine 52 in combination with a data compressor interface circuit 54. A data compressor / decompressor (DCD) integrated circuit (IC) 50 includes a dictionary 1 (D1) and R arranged in a random access memory (RAM) 88.
It is used in combination with the dictionary 2 (D2) configured in the AM 90. The circuit shown here is a single integrated circuit (IC).
Integrated circuits (ICs) 50, 88, 9 in or in separate
It is preferable to implement it as 0. Although dictionary 1 (D1) and dictionary 2 (D2) are shown as RAM, they can be implemented publicly in an associative memory or any other memory structure. This RAM is conventional.
RA of each of the dictionary 1 (D1) and the dictionary 2 (D2)
The M dictionary storage location is stored in the data entry field (dat
a_entry) 94 and 98 and wait state fields (stdby_stat) 92 and 96.

Input data in the data entry field
Contains the unique data string that occurs during the sequence
To be done. The wait state field is the data entry in the current dictionary.
A wait indicating whether the bird was previously stored in the wait dictionary
Contains the machine dictionary status flag. Valid data in the waiting status field
Dict identifying the data entry valid feel
Can be included. In data compression system
Multi-bit dict About using the valid field
“Dictionary,” filed on September 25, 1991
Reset Performance Enhancement For
・ Data compression application (DI
CTIONARY RESET PERFORMANCE
E ENHANCEMENT FOR DATA CO
MPRESSION APPLICATION) "
Commonly assigned US patent application Ser. No. 07 / 776,47
No. 5, which is incorporated herein by reference.

The data compression engine 52 is preferably LZ2.
Alternatively, it is designed to implement the LZ1 compression algorithm, but can be designed to implement any suitable dictionary-based compression scheme. Also, if desired, the compression engine may incorporate or be used in conjunction with automatic means for controlling dictionary reset as described in US Pat. No. 4,847,619. Otherwise, the data compression engine algorithm and architecture need not be described further, as it is conventional.

The data compressor interface circuit 54 consists of two main sub-circuits. Switch controller subcircuit 76 monitors dictionary reset request signal 72 and data string match signal 70 output from data compression engine 52 or other circuitry coupled to the data compressor / decompressor integrated circuit (IC). Sub-circuit 76 controls which RAM acts as the current dictionary and the standby dictionary. This subcircuit is in the standby state (st
Read the dby_stat) field to determine if the current encoded data signal has been previously copied to the standby dictionary.

The address generator circuit 68 uses the data entry (data_ent) of the dictionary operating in the standby mode.
ry) The ordering is done through the binary value of the field. A typical implementation of this subcircuit is a binary counter,
Other forms of sequencer can also be used. Multiplexers 56 and 58 and transceivers 84 and 86 are coupled to this subcircuit. Multiplexer 56
Selects the read / write signals 60 and 62 from the data compression engine 52 and the switch controller circuit 76. The multiplexer 58 is the data compression engine 5
2 and the address signal 64 from the address generator 68
And 66 are selected. The transceiver 86 operates as a bus controller that selects either the data entry (data_entry) field 94 or 98 to make a connection to the data bus 87 connected to the data compression engine 52. The transceiver 84 is in a standby state (st
Select the dby_stat) field 92 or 96 to connect to the switch controller 76. These multiplexers and transceivers are controlled by control signal 78 and address generator 68 controls signal 8
Controlled by two. Both of these control signals come from the switch controller circuit 76.

The data compressor / decompressor (DCD) circuit 50 transfers conventional data between one of the dictionaries (the current dictionary) and the data compression engine 52 during normal compression / decompression operations. To enable. The system also allows the standby dictionary to receive data from the data compression engine 52 or directly from the current dictionary to create a data entry subset according to the present invention.

During operation of the circuit 50, the switch controller 76 uses the dictionary 1 (D1) or the dictionary 2 (D2) as the current dictionary.
Select. For example, the dictionary 1 (D1) is selected. The compression engine then begins compressing the data, reading and writing the encoded data into the data entry (data_entry) field 94 of dictionary 1 (D1).
This compression algorithm determines that some encoded data string is a candidate for writing to the standby dictionary dictionary 2 (D2), for example, some encoded data string is some data in the current dictionary dictionary 1 (D1). When the entry matches, the match signal 70 is generated. The switch controller 76 thereby checks the wait state (stdby_stat) field 92 from the current dictionary to determine if this data entry has been previously copied to the wait dictionary. If not copied, this data string is dict of dictionary 2 (D2).
Written in the location provided by the address generator 68 in the _entry field 98. further,
The standby state (stdby_stat) field 92 of the current dictionary is set by the switch controller 76 to prevent the same encoded data string from being duplicated in the standby dictionary. When the data compression engine 52 generates the reset signal 72, the switch controller 76 changes the value of the control signal 78. This new control signal changes the connection to the multiplexer and transceiver so that Dictionary 2 (D2) acts as the current dictionary and Dictionary 1 (D1) acts as the standby dictionary. The subset of data entries loaded into dictionary 2 (D2) is used as the initial data entry set for compression engine 52. Thus, when the data compression engine shown in FIG. 3 is reset, the dict_entry field of the new dictionary contains the high compression ratio subset of the previously encoded input data.

The switch controller 76 can be stopped by the data compression engine generating a particular combination of the match signal 70 and the reset signal 72. This allows the data compression engine to read / write the encoded data with respect to only one dictionary. It is used for the operation of the customized data dictionary described above and also for compatibility with the single dictionary scheme. The method described above has proven effective. For example, the 550 file containing the user manual is a standard UNIX "compres"
Compressed using the s "command (conventional implementation of LZ2). The customized dictionary was then constructed using the current / standby dictionary method described above, and the file was created from the data sample. It was compressed using. The result is summarized as follows.

Original file size: 6,602,30
0 byte Unix (UNIX) compress: 2,78
1,686 bytes Customized dictionary: 2,025,742 bytes Compression improvement: 37%

Thus, this customized dictionary offers significant compression improvements over conventional compression methods.

This aspect of the invention may be modified in structure and detail without departing from its basic principles. For example, both the current dictionary and the standby dictionary can be implemented in the same RAM by providing a field that indicates whether the entry is in the standby dictionary. When a reset occurs, all non-waiting dictionary entries are cleared. The address generation circuit is more complicated. that is,
This is because each entry is not in a continuous position after reset. This method is based on the associative memory (CA
It is suitable for the embodiment using M).

II. Memory Circuit for Lossless Data Compression / Decompression Dictionary Storage FIG. 4 is a block diagram showing the overall configuration of a circuit 136 for a CAM (associative memory) compression / decompression system according to the second aspect of the invention. Is. The circuit 136 has a data compression / decompression (CD) engine 142, an uncompressed data interface 138, a compressed data interface 148, and a processor interface 152. The CD engine 142 comprises a string table memory 144 and control logic 146. The uncompressed data interface 138 stores uncompressed data (RAW
DATA) on the data bus 154, and the compressed data interface 148 transmits compressed data (COMP).
DATA) on the data bus 158. External control signals for interfaces 138 and 148 are received via control buses 156 and 160. Each interface has a first-in first-out data buffer (FIF
O) 140, 150 and another conventional interface circuit (not shown).

The circuit 136 can be used in either a compression or decompression mode or state, and is mode switchable for bidirectional communication.
Alternatively, the circuit 136 could be used as a dedicated compressor with a simple dedicated decompression circuit having the RAM replacement blocks 182, 184, 192 of FIG. The following description assumes that circuit 136 is used for both compression and decompression.

In the compressed mode, the uncompressed data interface 138 receives uncompressed data characters from the data bus 154 and supplies them to the compression / decompression engine 142 via the data buffer 140. Data compression /
The string table memory 144 and control logic 146 in the decompression (CD) engine 142 compresses these characters into codewords which are output to the data bus 158 via the data buffer 150. In the decompressed mode, the compressed data interface 148 receives compressed data codewords from the data bus 158 and provides them via the data buffer 150 to the CD engine 142. The string table 144 works in conjunction with the control logic 146 to decompress this data codeword into a character string and output the result on the data bus 154 via the data buffer 140. A microprocessor (not shown) controls the registers for setting the direction of data flow and the compression / decompression mode, as well as various other functions via the processor interface 152.

FIG. 5 shows the string table memory 1 of FIG.
4 is a detailed block diagram of 44 and control logic 146. FIG. This string table memory is a CAM (associative RAM) with additional internal logic to reduce compression / decompression processing time.
It consists of 188. The CAM 188 is organized into "words" (eg 3832 x 20 bits), whereby each word stores a separate character string entry. Data is written to the memory 188 on the data bus 190 (DATA_IN). The data bus 190 is the bus 180
Receives the external character string (K) entered above,
Character string (OMEGA) encoded on data bus 202 through multiplexer 192 (MUX3)
To receive. External characters on bus 180 come from the uncompressed data stream on uncompressed (RAWDATA) bus 154 (FIG. 4), and codewords come from the output of encoder 194. The data input selection logic circuit 182 determines which bit of the data bus 190 (DATA_IN) is on the bus 180 via the multiplexer 192.
Controls whether it comes from the external character string above and the codeword on bus 202. Data input selection logic circuit 182
Is the search signal input 17 from the control logic 146 (FIG. 4).
8, driven by the read / write signal input 164 and the match signal input 168 from the encoder 194.

The CAM 188 provides a set of coincidence signals via the match line 206 (eg, 264 to 4095). A match signal is associated with each word in CAM 188. When a character string on bus 190 matches one of the data entries in CAM 188, the match signal associated with that location is activated. Encoder 194 encodes all match signals from CAM 188 to produce the codeword provided on bus 202. This codeword is CA
Equal to the address location of the matched data entry in M188. Encoder 194 is also activated when any data entry in CAM 188 matches a character string on data bus 190.

Address decoder 184 uses external compressed characters on external address bus 177, CAM on data bus 186.
Either the internal character string output from 188 or the internal address from address generator 170 is selectively received and the associative array is accessed via word select lines 204 (eg, 264 to 4095). Externally compressed characters on bus 177 come from the compressed data stream on compressed data (COMPDATA) bus 158 (FIG. 4). Internal address generator 1
70 is controlled by the match signal 168, the search signal 178, the read / write signal 164, and the reset signal 162 from the encoder 194. Read / write and reset signals are control logic 146 (FIG. 4).
come from. The address generator is reset (eg, to 264) after initialization, and then the dictionary is built and incremented.

The source of the address supplied to address decoder 184 is controlled by read select logic 172 and read / write signal 164 via multiplexers 176 and 176. Read selection logic 17
2 is controlled by the reset signal 162 and the compressed state of the data entry output 162 from the memory 188. The data entry compression state can be determined by the value of the data entry character. For example, values greater than 256 can be assigned to encoded character strings and values less than 256 can consist of a single data character. The multiplexer 176 (MUX1) selects an input from either the bus 177 or the bus 186, and the multiplexer 174 (MUX2) selects the multiplexer (MUX1).
UX) 176 output and address generator 170 output. Decoder 184 also has an automatic update function, described below, that allows searching for data and updating memory to occur during the same memory access cycle.

FIG. 6 is a logic diagram of the preferred embodiment of the automatic update function of the address decoder 184 of FIG. Each address (ADDRN [19: 0]) input from the multiplexer 174 (MUX2) to the address decoder 184 is supplied to two AND gates. AND gates 208 and 214 represent a single address line. Further, the AND gate 208 has a search signal 178 (FIG. 5) and a coincidence signal 168.
The inverted value (NOMATCH) of (FIG. 5) is supplied. This search signal is also inverted and provided to AND gate 214 with the "qualified" write signal. O
The R gate 212 receives the outputs of these two AND gates and generates a word selection signal (WORDN). Equivalent functions can be provided by multiplexers or other combinational logic.

This update circuit is activated during the data search during the data compression operation. If the search during data compression fails, the character string must be placed at the next available address in memory. To eliminate the extra clock cycle required to write a data word to memory after a data search, gate 208 goes high when no match occurs. This character string is already on the data bus 190 and the next available address has already been set by the address generator 170 so that it can be written to as soon as a match indication occurs. Therefore, the inverted match signal (NOMATCH) activates the gate 208 and causes the word line (WORD) coupled to the next available memory location.
N) is started.

If a match is found during the search operation, the word select line is disabled and the write operation does not occur.
The modified write signal is used, for example, during an external microprocessor write operation to force a write of data even when no match occurs in memory. This update function provides true 1 cycle per byte performance. Because the writing of the dictionary is "transparent" and does not require extra memory access.

Further, the circuit of FIG. 6 has "data" in the memory.
The "_valid" field can be set. For example, the system of Figure 5 can copy each new character string into memory prior to checking for a match in memory. If a match occurs, the word The line (WORDN) signal is associated with this newly stored date and ring "data--
Used to activate the "valid" field.

In the operation of data compression circuit 144, for compression, a microprocessor (not shown) initializes the system for compression and resets CAM 188. This microprocessor receives signals (search signal 17) coming from the uncompressed data interface 152 via control logic 146 (FIG. 4).
8, read / write signal 164, reset signal 16
2) is controlled. The reset line can be used for various initialization operations. For example, the reset line is coupled to the CAM 188 to reset the "data_valid" associated with each memory location. further,
The reset line initializes the address generator to the starting memory location for character string storage.

Several techniques can be used to initialize a single input character. For example, a single input character can be algorithmically encoded as part of a compressed data stream. Alternatively, a set of encoded values, each representing any single input data character, can be loaded into memory.

Read / write line 164 directs multiplexer 174 to connect the address provided by address generator 174 to address decoder 184. Uncompressed data interface 138
The external character string from (FIG. 4) is provided to the byte field (DATA_IN [7: 0]) and codeword field (DATA_IN [19: 8]) of bus 190. The search signal 178 is then activated and the CAM
188 CAM18 codeword / byte string
Compare with each place in 8. Initially no match occurs. This is because nothing has been written in the CAM 188 before that. Therefore, the codeword / byte string on bus 190 is the first available address location in CAM 188 (eg, the initialized address generated by address generator 170).
Written in. The address generator 170 is then incremented and the new input character from bus 180 is read into the byte field of the data input of memory. This process is repeated to continue writing unmatched codeword / byte strings to CAM 188.

If there is a match, the input data selection logic 182
Instruct the multiplexer 192 to put the codeword generated from the encoder 194 into the codeword field of the data bus 190 (DATA_IN [1
9: 8]). The new external character from bus 180 is then provided to the byte field of data bus 190. This codeword thus represents the previously matched character string. Since the codeword assigned to this character string is obtained directly from the matching data entry address, the control logic required to encode the input character can be significantly reduced. Further, by feeding back the codeword to the multiplexer 192 (MUX3) and combining this codeword with the next input character, the input character can be processed every clock cycle.

This new codeword / byte string is then compared to the data entry in CAM 188. This process is repeated until no match is found. At this time, the compressor outputs the codeword from the last match, and the new codeword / byte string is C
Write to AM188. The last input character (K) supplied to the byte field is compared to the dictionary updated by searching the memory of byte K paired with the blank codeword (initially to contain the "root" codeword. (For configured dictionaries), it creates a route to start a new string. The new external character (K) from bus 180 is then supplied to the byte field and the matching process is repeated to build a new string (by LZW). Alternatively, the last character (K) can be output following the encoded character string (OMEGA) (as in the case of LZ2). Alternatively, the K address can be output as the K codeword following the OMEGA. When the dictionary is full, the address generator 170 issues a table full signal 196 to the rest of the compression system (FIG. 4) indicating that no more character strings can be written to memory. Any additional input data is C
Compressed according to the current entry stored in AM 188.

Decompression of Data Regarding the decompression of data, in the circuit 144, C
This operation is initiated by resetting the AM 188 and initializing this circuit for compression of input data.
Decompression considers the decompression of linked lists. For example, a compressed data address may, on the contrary, refer to an address where there is an uncompressed data string in memory (eg, the "root" codeword of a linked list). However, this address may have a "non-root" codeword (eg, this codeword is a link to the next address needed to further decompress the encoded character string). As mentioned above, "root" and "non-root" codewords can be determined in various ways. For example, it can be determined by the value of the codeword or by using the identifier bit in memory.

When the compressed data interface 148 (FIG. 4) has compressed data, it is written to the decoder 184 on the external address bus 177. “Non-root”
After receiving the codeword, the memory is read,
The byte field (DATA_OUT [7: 0]) of bus 186 (assuming a valid location) is the control logic 146.
It is extruded to the LIFO stack inside (Fig. 4). Bus 186
Codeword field (DATA_OUT [19:
8]) is a non-root codeword, multiplexer (MUX1) and multiplexer (MUX2).
To be fed back to the address decoder 184,
Another memory read is performed. Prior to the feedback of the non-root codeword, the last byte of the data entry read from memory is put into FIFO 140. This process ends when the "root" codeword is read from memory, at which time a new codeword is read from external address bus 177.

After the root codeword is identified, the last encoded character output is combined with the previous outer encoded character and read into the next available address in CAM 188. Read select logic 172 checks the "root" codeword and directs multiplexer 176 to read external address bus 177 or data bus (DA
TA_BUS) 186 to the address decoder 184. Read select logic 172 also provides encoded element signal 198 to control logic 146 to indicate a fully decompressed codeword. FIFO 140 dumps this decompressed decoded character onto bus 154.

The system of FIG. 5 simplifies the decompression operation. Since decompression considers the linked list, the embedded logic provides feedback of this memory output data to the address decoder decompression logic without additional interaction with the external decompression logic (FIG. 4). provide. Therefore, each decompression cycle may be of shorter duration, simplifying the decompression logic. There are many implementations for "validating" valid words and codewords in the CAM 188. One method is to use a comparator, and another method is to use an additional resettable bit for each word.
Which technology to use depends on the application requirements. In a one-way system (eg CDROM), the decompression circuit has a conventional R with a feedback circuit as described above for considering the linked list.
Further simplification is possible using AM.

FIG. 7 is a dataflow diagram illustrating a general method of data compression / decompression or linked list storage / retrieval in a system in accordance with the above-described features of the present invention. The method described below is adaptive such that the dictionary is embedded in the codeword and therefore does not need to be transferred separately with the compressed data. Other methods, such as a dictionary being transferred with compressed data, can also be implemented with this system.

The broken line block 232 is the compression process of this system, and the broken line block 234 is the decompression process. The uncompressed data (K) at input 224 is block 22
Coded character string (OME
GA) to decision block 226. As described above, the encoded character string (OMEG
A) represents the address of a data entry that encodes a character string. Encoded character string (O
MEGA) and the uncompressed data (K) are combined and compared at decision block 226 with the entries in the dictionary.
When the OMEGA-K input matches an entry in memory, block 228 encodes this input with the address of the matched data entry. This encoded value (new OMEGA) is fed back, combined with the next external character K and input to decision block 226.
This process is repeated until the OMEGA-K string no longer matches any entry in memory. Block 230 then updates the string table memory with this OMEGA-K string, outputs OMEGA, and supplies the letter K to encoding block 228. K is encoded in block 228 and combined with the next external data character K before being fed back to decision block 226.

This encoded data OMEGA is sent to block 236 for decompression. An encoded input character (OMEGA (i)) is used as an address to access the string table memory. Address OMEGA in decision block 238
It is determined whether the data entry of (i) is a root character. If it is a root character, there are no more encoded characters in the data entry output from memory (eg, OMEGA (j) no longer exists). K
Of the memory data entry is then output on line 246 as the decompressed output character. Decision block 23
8 jumps to block 240, where the previous encoded data character (OMEGA (i-1)) is combined with K and written to the next available address location. Next, block 242 commands block 236 to use the next encoded character (OMEGA (i + 1)) in the input stream as the address location of the next data entry read from memory.

If the output from the string table memory is not root (eg, if this output consists of the encoded character OMEGA (j) and the decoded character K), then K is output on line 246 and the decision block Two
38 jumps to block 244. Block 244
Uses this encoded character OMEGA (j) as the address of the next data entry output from the memory. The data entry in storage location OMEGA (j) is then processed as described above. This process is repeated until all encoded input characters have been decompressed.

FIG. 8 is a detailed data flow diagram of the dashed line block 232 of FIG. This data decompression process
It is started when a start signal or a reset signal is generated in block 248. The memory circuit (discussed next) is initialized at block 250 and operates, for example, in compressed or decompressed mode to reset the dictionary. Any dictionary valid bits should preferably be initialized in parallel. The dictionary is a single letter codeword or Welch U.S. Pat. No. 4,558,302.
Externally generated according to a selected encoding algorithm such as DCLZ disclosed in the ECMA-151 standard paired with a blank codeword for identifying the LZW or entry disclosed as a single character or a "root" codeword. Can be initialized with a set of codewords. Alternatively, instead of pre-storing a set of codewords, they may be shared, for example, in commonly assigned US Pat. No. 5,142,282, "Data Compression Dictionary Access Minimization (Data Compression Di
motionary Access Minimizat
Ion) ”, it can also be generated in real time each time a match fails. Other initialization methods can be used, including empty dictionaries.

The first character in an input data stream is read at block 252 and either stored directly in the OMEGA field or encoded (eg, CODE (CHAR)) and then stored in the OMEGA field. . Next, the next input character (K) of the input data stream is read at block 256. Block 258 combines OMEGA and K as a character string (ie, combines OMEGA and K) and searches the dictionary for a data entry that matches this OMEGA-K string. Decision block 260 indicates that there is no match because no data string has been stored in the dictionary yet. The OMEGA-K string is not currently represented and is stored in memory if decision block 266 determines that there is storage space available. If the memory is not full, the operation of block 268 automatically loads the OMEGA-K string into the next available memory storage location (ADDR (N)). Clock 270 then increments the address counter to the next available storage location (ADDR) in memory.
(N + 1)) is identified. The encoded value OMEGA (address) of the first input character is output as the first character of this encoded data string at block 272, if available.

When the memory is full, the compression system can simply prevent writing further character strings to the system. For example, when decision block 266 determines that the memory is full, the steps of loading the character string in block 268 and incrementing the address counter in block 270 are skipped, and the process is encoded in block 272 described below. And jump to output processing.

After OMEGA is output, block 27
The step 4 replaces the first input character (OMEGA) with the second input character (K) or code (K). The next input character from the input data stream is read (K), thereby providing the next OMEGA-K string. The process then loops to block 258 where the memory is allocated to this new OMEGA.
-Search using K string.

If a decision block 260 indicates a match,
The process jumps to block 164, where OME
OMEGA-K with GA field equal to match address
Replaced with an encoded value that represents the string.
The next input character from the data stream is copied into the K field. A new OMEG that combines the OMEGA and K fields to represent three input characters
An AK string is formed. Processing then returns to block 258 where the dictionary data entry is compared to this new character string. More input characters are added to the character string as long as the previous character string is located at some data entry in memory. If the new character string does not match the data entry, decision block 260 jumps to block 266, where block 266 as described above in block 266,
The 268 and 270 memory update procedure is performed. Block 272 outputs the value OMEGA (eg, the encoded character string from the last input character string and the data entry match). Block 274 takes the last character of the character string (eg, the character that caused this character string to no longer match the following data entry in the string table) and copies it to the OMEGA field. Next, block 2
At 74, copy the next input character from the input data stream into the K field and the process loops to block 258. This character string is compressed by this. This is because the single encoded value of the OMEGA output from this compression process represents multiple input characters.

FIG. 9 is a detailed flow chart of the decompression circuit 246 of FIG. Block 276 initializes the string table memory to perform decompression. Block 278 is the first encoded word (OLDWORD)
To get The process exits when there is no more data during this input read step or any subsequent input read step. The first encoded word is decoded at block 280 algorithmically or by reading a preloaded entry in the string table memory. This first encoded word is the root character and is therefore decoded and output.

Block 282 gets the next encoded word (INCODE) and block 284 INCODE
Use E as the address of the data entry output by the string table. First, in one embodiment, the string table consists of only single character bytes, so block 284 outputs byte K.
Byte K is then output at block 286. In a later cycle, block 284 returns OMEGA-K as described next.

Decision block 288 determines if this byte is the end of the string (eg, the root character), and if so, jumps to block 292. Block 292 builds a new data entry at the next available address in the string table. This is the first encoded input word (OLDCODE)
And the last byte output (K) combined. Block 294 points to an unused address location and block 296 the first encoded word (OLDWORD).
Is replaced with the last encoded input word (INCODE) and processing returns to block 282.

The block 282 reads the next encoded input word (INCODE), and the block 284 outputs the data entry of the address INCODE. If the data entry output at address INCODE is not root, this is the decoded byte K and a codeword field (OM) indicating the next address to be further decoded.
EGA) is included. Output K at block 286 and decision block 288 jumps to block 290. Block 290 uses the codeword field (OMEGA) as the address of the next data entry output from the string table and then loops to block 284. This process is repeated until the data entry with the root character is output from the string table (ie, the end of the string). Decision block 288 proceeds to block 292 where the previously read encoded word (OLDCODE) is combined with the last output byte (K). Block 294 and Block 296
Function is then performed and processing returns to block 282.
Therefore, this decompression process reproduces the original data stream compressed by the compression process of FIG.

FIG. 10 illustrates the compression and decompression algorithms of FIGS. 8 and 9. Raw data stream 300 consists of uncompressed character strings input to the data compression / decompression process shown in FIG.
In this example, the single letters R, I, N, and T are stored at locations ADDR0, ADDR1, AD in memory 302A during initialization.
Loaded into DR2 and ADDR3. Input characters are encoded by assigning each character a value at its address location, but to speed up compression, a single input character is encoded algorithmically before starting the process described below. be able to. Memory 302
A indicates the dictionary immediately after the initial setting, and the memory 302B indicates the dictionary after the compression is completed.

The first input character R from the data stream 300 matches the data entry at address location ADDR1. Because of the match, the compression system combines the encoded value of R (Addr0 = 0) with the next input character "I" and searches memory 302A for a match with "0I". Since there is no match with "0I" in the memory, "0I" is written to the next available storage location (ADDR4) as shown in memory 302B. Codeword of the largest matched sequence (ie,
“R” = 0 codeword) is the compressed character stream 30
4 is output as the first encoded character. The compression system now searches memory 302B for a string of encoded values of "I" (ie, ADDR1 = 1) combined with the next input character "N". Since the string "1N" is not in the dictionary, it is written to the next available storage location (ADDR5) as shown at 302B. The value 1 (eg, the last matched character string = “I”) is output as the second encoded character of compressed character stream 304.

The process of encoding the input characters in a manner similar to the construction of memory 302B continues until the second "I" in uncompressed character stream 300 is processed (eg, character 306). This compression system encodes "I" with the value 1. This is because "I" is in the address position ADDR1. This encoded value 1 is combined with the next input character N and this string 1N is compared with the data entry in memory 302B. Since the sequence "1N" has occurred earlier in the character stream 300, the string "1N" will have an entry in memory (eg,
Data entry of Addr5). Therefore, the string "1N" is encoded as "5" and combined with the next input character "T". The string "5T" does not match any entry in memory 302B, so
"5T" is the next usable address position (ADDR8)
, And the codeword of the last matched character string “5” is output in the character stream 304. Then, the encoded value of the input character "T" (ADDR
3 = 3) is combined with the next input character "I" and the process is repeated. Memory 302A is a character stream 300
Shows all the characters built for the dictionary from. Character stream 304 is a fully compressed character stream of raw data stream 300. Note that only 6 encoded characters are needed to represent the 9 characters in character stream 300.

The decompression dictionary is reinitialized to perform decompression as shown in FIG. 302C. Therefore,
The first four address locations have the decoded values of the single input characters R, I, N and T. Again, single character decoding can be performed algorithmically. First
The encoded input character "0" of is used as an address to the memory 302C. The decompression system determines that this value "0" is the root codeword, for example by checking if this value is less than 4. Thereby, the data entry for ADDR0 (eg, "R") is output as the first character in the decompressed data stream 308. The decompression system then reads the next encoded input character "1". This value is also the root codeword, so the data entry for ADDR1 is output as the second character "I" in the decompressed data stream 308.

At this time, a new dictionary entry is constructed using the last decompressed character "I" combined with the previous codeword "0". This string "0I" is then written to the next available address location (ADDR4) as shown in memory 302D. The next codeword "2" is input and the process is repeated. At this time, the data entry (for example, N) at the address position ADDR2 is output, and the string "1N" is written at the address position ADDR5 of the memory.

This process is similarly repeated until the input character "5" is read by the decompression system.
The decompression engine uses this codeword for AD
Refer to the DR5 data entry. The encoded character "5" is not root because it is greater than 3, so the decompression system outputs the last byte (eg, N) of the data entry at address location ADDR5. The rest (eg 1) of this data entry is used as the next address. Since the code word "1" is the root, the data entry of ADDR1 (for example, "I") is output, and further decompression is unnecessary. The decompressed character "IN" is then placed in the character stream 308. A new data entry in memory is written to address location ADDR7 with the last decompressed output character "I" and the previous encoded input character "3". This process is repeated until all the characters in character stream 304 have been decompressed. It should be noted that the dictionaries constructed by the HP-DC method using hashing are different from each other. In contrast, compression dictionary 302B and decompression dictionary 302D built using this system and method have the same address / entry.

III. Use of Multiple Dictionaries in a CAM Compression / Decompression System In order to further reduce the amount of memory required to compress data using CAM, the CAM data compression system previously shown in FIG. 5 uses a standby dictionary (see FIG. 3). ) Is used in conjunction with. The CAM has the ability to process one character per clock cycle, but it can compress the data with minimal memory. In addition, the data compression ratio can be increased by maintaining a useful character set in the current dictionary after reset. The method described next is adaptive so that the dictionary is embedded in the codeword so that there is no need to transfer another dictionary before each decompression process.

FIG. 11 is a high level block diagram of a combined CAM multiple dictionary compression / decompression system. For purposes of description, like 2 ^b × and the system as shown in FIG. 5 (b
+ M + 2) implemented with CAM 312. CAM
312 comprises a control bus 314 coupled to a control processor (not shown). The address bus 316 (b-bit width) and the data bus (n-bit width) are CAM 312.
Is bound to. DATA_MASK of n-bit width
A zero bit on bus 320 disables the corresponding bit during the CAM search. For example, the first mask bit (DA
"0" signal on TA_MASK [0]) is CAM 31
The first DATA_IN bit (DATA
_IN [0]) is disabled. DATA_ was made impossible
The IN bit is not taken into account when searching for a data entry in CAM 312 that matches the signal on line 318. Data masking circuits are well known in the art. Therefore, details of the masking circuitry used in CAM 312 are not shown here. The data on the bus 318 is CAM in the successful match line 322.
Activated when it matches a previously stored entry in 312. MATCH_ADDRESS bus 326
Contains the value of the matching profit A or the address of the entry, and DATA
The _OUT line 324 is used to output the data entry previously stored in the CAM 312.

FIG. 12 shows different entries contained in each dictionary entry of the CAM. Each CAM data entry is
It has three fields. An m-bit wide character field (CHAR) for storing the suffix K, a b-bit wide code field (CODE) for storing the encoded character value OMEGA, and the associated CODE and CHAR fields. 2-bit wide status field (S
T). The status field (ST) takes one of four possible values:

FREE: The CAM memory location is currently unused in the current dictionary. The CD: CAM location contains data entries that belong to the current dictionary and not the standby dictionary. The SD: CAM location contains data entries that belong to both the current dictionary and the standby dictionary. INV: Invalid value. It does not occur in normal operation.

The binary values corresponding to FREE, CD, SD and INV are not fixed. The compressor and decompressor operate as a state machine that can be in one of four possible states (S) (0≤S≤3). The particular binary value of the state field (ST) is the function FR of the state (S)
EE (S), CD (S), SD (S) and INV
(S), which is defined in FIG. For example, state S
= 0, if bits [0: 0] are present in the status field of the CAM data entry, the storage location is FRE.
E, which is considered to be currently unused in the current dictionary. However, if the compressor / decompressor system is in state S = 2, the CAM location with the bit value [0: 0] of its state field is considered to be the data entry assigned to the standby dictionary.

Initially, the system is in state S = 0 and all ST fields are [0: 0] (eg
ST = FREE (S)) is set. Only then is global initialization necessary to minimize initialization time delays that occur during dictionary reset, as described next. The compressor is initially in state S = 0, but begins reading input characters, compressing the input string, and building the current dictionary (CD) and the standby dictionary (SD) in parallel. When the CD is full, a dictionary switch occurs which causes the data entry in SD to become a new data entry in the CD. SD is basically emptied and all valid data entries are removed.

This dictionary switching occurs when the system undergoes a state transition from S = 0 to S = 1. In FIG. 13, in the state S = 1, the empty entry is ST = [1:
0], which is the same as the CD value in state S = 0. In state S = 1, the CD entry is S
T = [1: 1], which is the same as the SD value in state S = 0. State transitions only occur when the CD is full, so all entries in the CAM are marked as CD or SD (ie,
There is no entry in the status field with a value of FREE and the value INV is never written. ) Therefore,
Immediately after the state transition from S = 0 to S = 1. All entries have a value of FREE or CD (but
Excludes initial single character strings that are not actually held in dictionary memory as described below. The new SD starts with an empty state, since no entry has the value SD. The same situation applies for S = 1 to S = 2, S = 2 to S = 3, S = 3
To state transition from S to S = 0. FIG. 14 shows the state transition of the compression / decompression system described above.

FIG. 15 shows a simple hardware implementation for changing the state of the compressor / decompressor. The initial bit value of the state register 28 is shown as state S = 0. For each state transition, the bits of the state register are cyclically shifted as FREE->INV->SD->CD-> FREE. Therefore, state control can be easily implemented using an 8-bit circular shift register and shifting register 328 two bits to the left for each state change.

To describe a CAM-based standby dictionary compressor, the contents of the CAM memory location are 3 bit bytes (ST, C
ODE, CHAR), code (A) represents the encoded value of the single character string "A". For purposes of explanation, codewords are assigned values corresponding to memory address locations. However, the codeword value can also be easily obtained as a simple function of memory address location and can be easily implemented by those skilled in the art. It is assumed that the code (code (A)) in a given address space (eg, addresses 0 to 2 ^m -1) is available immediately without accessing the dictionary. As mentioned above (see Figure 5), the storage locations corresponding to these codes need not be physically present in the CAM. Therefore, we assume that all CAM searches exclude these locations. For simplicity of explanation, the "end of file" condition is also ignored.

Implementation of a CAM-based Multiple Dictionary System FIG. 16 is a detailed circuit diagram of a CAM-based multiple dictionary compression / decompression system. The circuit diagram of FIG. 16 illustrates the additional functional components required to provide multiple dictionary compression / decompression. The CAM compression / decompression circuit 312 is the same as that shown in FIG. 11, and the status register 328 is the same as that shown previously in FIG. DATA_IN register 342
And MASK register 350 for ST, CODE and CH
The AR field is provided via the CAM 312 DATA_IN and MASK ports. The ST field of each data entry in the CAM is controlled directly via the status register or indirectly via the ST pattern generator 338. The ST pattern generator is shown in detail in FIG.

The CD and SD lines supplying the DATA_IN port are connected to the multiplexer 340 (MUX).
It is controlled by operating the control bus 314 via M1). The signals on control bus 314 come from the system processor (not shown) and the control compression / decompression functions in CAM 312. Control bus 314 contains the same read, write, search and reset signals as previously shown in FIG. Internal compressor in CAM 312 /
The decompressor control logic is also similar to that shown in FIG. Minor changes to this logic may be required to perform some of the functions described below. These circuit modifications can be easily carried out by those skilled in the art and will therefore not be described in detail.

Line 326 is the CAM 312 MATC
H_ADDRESS port to DATA_ of CAM312
Connect to IN port. External data bus 344 is ADD
Directly coupled to the RESS_IN port, register 342
To the DATA_IN port via. ST
A pattern generator line 336 and a data input line 348 supply to the ST field of MASK register 350 via multiplexer 346 (MUX M2). The search-type signal on line 349 and various other control signals from control generation circuit 352 are controlled by the MATCH_SUCCESS signal on line 322.
The DATA_OUT signal on line 324 outputs the compressed or decompressed data to the data interface shown in FIG. The internal address pointer 354 (NEXT_CODE) can write data to the second address pointer 356 (SAVE_CODE) or receive data from the CAM DATA_IN port.

FIG. 17 is a detailed circuit diagram of the ST pattern generator 338 of FIG. The first bit from the CD and SD fields of status register 328 (FIG. 16) is AND gate 358 and EXCLUSI.
It is input to the VE-NOR gate 362. The second bit from the CD and SD fields is AND gate 360 and EXCLUSIVE-NOR gate 3
64. AND gate is CAM DATA_
Supply to ST field of IN port, EXCL
The USIVE-NOR gate feeds the ST field of the CAM MASK port (FIG. 16).

The compression / decompression system must be able to search both CD and SD dictionary entries at the same time (as described in detail below). This is done by manipulating the bits in the status register 328 (FIG. 16). One of the bits in the CD and SD is always the same and the second bit is always different (see Figure 13). Therefore, the match bit is used to search for a valid SD or CD dictionary entry and the second bit is masked. For example, in state S = 0, the bit value of the current dictionary CD is [1: 0] and the bit value of the standby dictionary is [1: 1]. This allows AN
D gate 358 and EXCLUSIVE-NOR gate 3
62 output goes high, AND gate 360 and EXC
The output of LUIVE-NOR gate 364 goes low. Therefore, a C having a "1" at its first bit position
Any ST field in the AM dictionary (eg CD
(S) SD (S)) is identified as a valid dictionary entry for either CD or SD.

Data Compression The system of FIG. 16 compresses data as follows. This system has a status register 328.
The state S = 0 is set by loading the bit value as shown in FIG. All ST fields in the CAM dictionary are ST = FREE (S) (ie
[0: 0]) and address pointer NEXT
_CODE is set to the first available address in the CAM. As described above for the CAM shown in FIG. 5, a single input character can be algorithmically encoded during data compression, in which case all CAM addresses can be used to store a character string. But,
When a single data character is stored in the CAM, the first address available for writing the code character string is usually the address location after the last single character.

If necessary, the first input character is encoded by reading the first data character from input data line 344 and generating an address on line 326 for the input character / data entry match. To be done. This encoded first character (OMEGA) is then transferred to register 34
In 2, it is combined with the second input character (K) from the input data line 344 to produce the OMEGA, K character string. The CAM is searched for a data entry that matches the OMEGA, K string. At the same time, S
A CD or SD value (eg, OMEGA, K string already stored as a CD or SD entry) that matches the value generated by the ST pattern generator for the T field is searched for. SE
All bits of the ARCH_TYPE signal 349 have a value of "1" during a match search, which enables the CODE and CHAR fields of the CAM mask. MUX M1 and MUX M2 select the ST fields of the MASK and DATA_IN ports from the ST pattern generator 338 previously shown in FIG.

This is the first OMEG supplied to the CAM
MAT on line 322 because it is an A, K string
The CH_SUCCESS signal indicates that there is no match. OMEGA is then output on line 324 and the character strings CD (S), OMEGA, K are in CAM dictionary position N.
It is written in the ST field, CODE field and CHAR field in EXT_CODE. O
The letter K of the MEGA, K string is then encoded (C
ODE (K)), used as a new value of OMEGA. The CD (S) value written in the ST field is supplied to the ST field of the register 342 by MUX 3
Sourced directly from register 328 by changing 40 inputs.

Next, the system is ready for the next available CA.
Search for M dictionary entries (eg, ST = FREE (S)). Therefore, SEARCH_TYPE signal 3
49 takes the value of "0", and the CODE field and CHA
Mask R field, [1: 1] on line 348
Enable the ST field via the bit value. At the same time, control line 314 coupled to MUX 340 selects the value FREE from register 328 as the value searched for in the ST field. The match address from line 326 is used as NEXT_CODE to store the next OMEGA, K string. This process extracts the next character from the input data string on line 344 and combines it with OMEGA to generate an O for the next search.
Generate MEGA, K. If the next search finds a match, the address location of that match is M for the next match.
DATA_ on ATCH_ADDRESS line 326
It is fed back to the IN port. This address is a new OMEGA representing the previous OMEGA, K string
Used as a value. At the same time, S from register 328
The D (S) value is written in the ST field at this match address.

As described above, after the new OMEGA, K string is written to the CAM location, a search is made to find the next FREE value in the status field. If the search fails, this indicates that the current dictionary is full, causing the system to switch to state S = 1. This is done by rotating the contents of register 328 left by two bit positions. SD before
The status field position that had the (S) value is now CD
(S) value. All state fields in the CAM were set to CD (S) or SD (S) at state S = 0 (eg, FRE just before state change).
Since there was no E state field value), state S =
All FREE storage locations of 1 become the previous CD (S) entries from state S = 0. Furthermore, the standby dictionary is in state S
Empty except for the initial single character string of = 1. This is because the INV value is never written in state S = 0. Compression continues as above with system state S = 1. This process continues to generate compressed data characters and switch states until all the input data is compressed.

Decompression of Data Decompression of data using the system of FIG. 16 is performed as follows. The CAM is initialized by resetting the bits in register 328 to state S = 0. F
The REE bit value is written into the status field of each available memory dictionary location. Internal address pointer 354 (NEXT_CODE) is the first available storage location of the CAM (eg, NEXT_CODE =
2 ^m ) and the internal address pointer 356 (S
AVE_CODE) is set to zero.

Decompression is done in the same way as shown in FIG. For example, the first encoded character (OMEGA) from the encoded character string is line 344.
Read on. OMEGA is next CAM ADDR
Used as an address supplied to the ESS_IN port. If the value of the CODE field output on line 324 is not "root", the CHAR field is output on line 324 and the CODE field is fed back to the CAM as the next address location. This process is repeated until the "root" CODE field is read from the CAM. Compressed input characters (OM
EGA) is decompressed and the decompressed character string is output on line 324, after which the ST field at address location OMEGA is set to SD (S). This is done by writing the SD (S) value from register 328 into the ST field of register 342. The first character (K) from the decompressed data string is then placed in the address location SA of register 342.
The dictionary is built by feeding back to VE_CODE. CD (S) from status register 328
Value, the OMEG initially read via line 344
The A value and the first character from the decompressed OMEGA output string (K) are written to the CAM dictionary at ST, CODE, and CHAR field address locations NEXT_CODE.

The value of the address pointer NEXT_CODE is then written to the address pointer SAVE_CODE. A value of "0" is placed on line 349, line 3
The [1: 1] bit value on 48 allows for "state field only" searches. The next dictionary entry in the CAM with the FREE state field is found by searching the ST field with the FREE value.
The address value of this FREE status field is line 32.
It is written to the address pointer NEXT_CODE via 6. Then, the next encoded character OMEGA is read from line 344.

When the current dictionary is full (eg F
When the REE state field value is not present), the system switches to state S = 1 by shifting the bits in register 328 as described above, resetting the value of address pointer NEXT_CODE. Therefore, the current dictionary contains only entries from the previous standby dictionary. The system then reads the next encoded character (OMEGA) from line 344 and the data decompression process continues.

FIG. 18 is a data flow diagram showing a general method of data compression using a CAM having a standby dictionary. Block 376 is an initialization process that sets the system state and state conditions. That is, the system is set to state S = 0, all state registers in the CAM dictionary are set to ST = FREE (S), and the address pointer is set to the next available address in the CAM (eg,
2 ^m → NEXT CODE).

The first character (CHAR-K) from the input data stream is read at block 378 and encoded (eg, code (K)) to provide the value OMEGA. The next input character (K) of this input data stream is read at block 380. Block 38
2 combines OMEGA and K as a character string (eg, joins like OMEGA, K). Then a search is performed. It not only looks for a data entry that matches the OMEGA, K string, but also matches one of the two alternate status register patterns (ST = CD (S) or ST = SD (S)). This search must be done for both current and wait values. That is because either value indicates that a valid character string (eg, OMEGA, K) was written to the CAM. For example, the status register value ST = CD
(S) indicates that the associated CODE and CHAR fields were loaded with the OMEGA, K character string during the current processing state. Status register value ST = S
D (S) indicates that the associated CODE and CHAR fields are loaded with the OMEGA, K character string and have matched the second OMEGA, K character string at least once in the current processor state. Therefore, both status register values (CD (S) and SD (S)) must not be overwritten by a valid CAM.
Indicates a data entry.

If the CAM does not yet contain the data string, decision block 384 indicates that there is no match. The encoded value OMEGA is output as the first character in the encoded data string at block 388. This OMEGA, K string is the first available CAM address location (NEXT_CODE)
Written in. A value CD (S) indicating a valid data entry in the CAM is written in the status field (ST) of the address position NEXT_CODE. Next, block 388 replaces OMEGA with the encoded value of the second input character (code (K) → OMEGA).

Block 390 indicates ST = F for CAM.
Search for the next available address location which is REE (S). If no status register with FREE (S) is found, the current dictionary in the CAM is full. Decision block 392 sends the CAM to the next state S = S + 1.
Replace the current dictionary (CD) with the standby dictionary (SD) by changing to mod 4. During a state change, the value of each state register is reallocated as previously described (see Figure 13). ST field value is FREE → INV → S
It is reassigned as D → CD → FREE. Processing returns to block 380, where the next input character (K) is read. This matching process is repeated as described above.
If the current dictionary is not full, decision block 390 jumps to block 394.

Block 394 determines the next address in the CAM with the FREE status register value and assigns that address to NEXT_CODE (eg, ma.
tch address → NEXT CODE). Processing returns to block 380, where the next input character (K) is read.

If a match is indicated by decision block 384, processing jumps to block 386, where OM occurs.
The EGA field is replaced with the address of the dictionary entry that matched the OMEGA, K string. The matched character string (represented by the CODE and CHAR fields of the match address) is automatically assigned to the standby dictionary by setting the match address status field ST to SD (S). The process then returns to block 380, where the next input character (K) is read from the data stream. Matching address (OMEG
A) and K are combined to form a new OMEGA, K string, which represents the three input characters. Block 382 then searches the current dictionary and the standby dictionary for a character string match.

The actual construction of the standby dictionary is OMEGA, K
Block 38 when the ST field at the address of the character string match is set to SD (S).
Done at 6. However, since it is not necessary to check the ST field, it is possible to implement by simpler hardware.

FIG. 19 is a data flow diagram showing a general method for decompressing data using a CAM with a standby dictionary. Block 398 puts this system into a state (S = 0)
Default to ST for all available dictionary entries
Initialize the field to the value ST = FREE (0).
The address pointer NEXT_CODE is set to the first empty address position (NEXT_CODE = 2 ^m +1) and the second address pointer SAVE_CODE is set to zero. The first encoded character (OMEGA) from the compressed data string is block 400.
Is read by.

Block 401 decompresses OMEGA into a decompressed character string W as described in FIG. For example, by using OMEGA as the address, the CHAR field at storage location OMEGA is output by the CAM. Address OMEG
If the CODE field from A is not "root",
It is used as the next address supplied to the CAM. The CHAR field at the next address is output as the next decompressed character K. When the CODE field of the address OMEGA is “root”, the CHAR field of the address OMEGA is output, and the CODE field and the CHAR field of the address OMEGA are assigned to the standby dictionary (for example, SD (S) →
ST). Block 402 assigns the first character of character string W to register C.

If the address pointer SAVE_CODE is non-zero, decision block 403 jumps to block 404 where the character string CD
The dictionary is constructed by writing (S), SAVE_CODE, C to the address location (NEXT_CODE) of the CAM dictionary. If SAVE_CODE is equal to zero or the character string is written at block 404, then at block 405 the address location OMEGA
State field of is assigned to the standby dictionary (SD (S) →
(OMEGA)), OM the current SAVE_CODE value
Replace with the value of EGA. The value ST = F in block 406
The next state field with REE (S) is searched. If the FREE ST field is found, block 408 jumps to block 410 and block 41
In 0, the matching address is the address pointer NEXT_CO.
Assigned to DE (eg MATCH ADD
→ NEXT CODE). Next, processing continues at block 400.
Returning to, the next encoded character from the compressed data stream (OMEGA) is read and decompressed.

If no ST field has a value of FREE (S), decision block 408 jumps to block 412. This process is now the next state where the current dictionary is switched to the standby dictionary (ie, S = S + 1 mo).
Change to d). This also changes the entry in the current dictionary from its previous state to the FREE position. At block 413, the next empty position with ST = FREE (S) is searched, the address pointer SAVE_CODE is reset to zero, and jumps to block 410. Block 4
At 10, the address value of the FREE position found at block 413 is assigned to the address pointer NEXT_CODE. Block 410 then returns to block 400,
Processing then continues until all data from the compressed data stream is decompressed.

FIG. 20 illustrates the compression and decompression algorithms of FIGS. 18 and 19. Raw data stream 414 consists of an uncompressed character string input to the CAM compression process shown in FIG. In this example,
The single letters R, I, N and T are stored in memory 41 during initialization.
6 positions ADDR0, ADDR1, ADDR2 and A
It is loaded into DDR3. Single-character input is encoded by assigning each address a value at its address location, but for faster compression, single-input characters should be encoded algorithmically before starting the process described below. You can The memory 416 is in the state S immediately after the initial setting.
= 0, and memory 418 shows the dictionary in the S = 0 state immediately before the replacement of the current dictionary with the standby dictionary (eg, change from state S = 0 to S = 1). Memory 420
Indicates a dictionary in the state of S = 2 after compression of the raw data stream 414.

Each memory location in the dictionary has a status field (S
T), code field (CODE), and character field (CHAR). For purposes of explanation, CA
There are 5 dictionary positions available in M for storing character strings.
Assume that there is only one (eg, ADDR4-ADDR8). Address locations ADDR0-ADDR3 are assigned to a single character and are not searched for available dictionary locations. The bits in each status field have the value FREE =
Initialized to [0: 0] (eg, FREE (S) = 0), the address pointer NEXT_CODE is the first available CAM memory location (NEXT_CODE = AD).
Initially set to DR4).

The first input character "R" from the raw data stream 414 matches the data entry at address location ADDR0 and has a first value in OMEGA (eg, OM).
It is used as EGA = 0). This compression system is O
Combine MEGA with the next input character "I" (OMEGA,
K), CODE field and CH in memory 416
Search for a match with "0I" in the AR field. At the same time, the corresponding ST field in memory 416 is searched for a bit combination of "1: 0" or "1: 1" (eg CD (S) or SD (S) in state S = 0).
Ask for. All memory locations were FREE and there was no "0I" string previously written to memory. Therefore, no match is found. Therefore, OMEG
The value of A (“0”) is the first of the compressed stream 422.
Character string (CD (S),
OMEGA, K) is the first FREE memory location (ADDR
4) is written. Then the letter "I" is encoded,
The next value of OMEGA (eg, OMEGA = 1) is generated.

The CAM dictionary is searched for the next ST field with the FREE value, and its address location is the address pointer NEXT_CODE (eg NE
XT_CODE = 5). The next character (K = "N") from the raw data stream is then OMEGA
Is combined with (OMEGA = "1"), the CAM is searched for a "1N" character string. CAM here too
No match occurs in the character string (CD (S),
1, N) is written to address location ADDR5. This process is repeated in the same way, after writing a character string that does not match any of the previous entries, the next available address is written to the ST, CODE and CHAR fields.

The first character string / data entry match is the character 424 ("I") from the raw data stream.
N "). The character" IN "consists of the encoded OMEGA, K character string (" IN "). The CODE field at address location ADDR5 and C
Since the HAR fields are "I" and "N" respectively, and the status field was previously set to ST = CD (S), a character string match occurs. The matching address is used as a new OMEGA value (OMEGA = 5) and the data entry in ADDR5 is assigned to the standby dictionary (when S = 0, ST = SD (S) = [1:
1]). The next letter T is read from the raw data stream 414 and combined with OMEGA. This new OME
The GA, K string ("5T") represents three characters, which are searched for as described above. There is no OMEGA, K string with the value "5T" in the CAM. Therefore, it is written to the next available address location (ADDR8). The encoded character "5" is output to the compressed character stream 422 and is "T".
The encoded value of is the next OMEGA value (OMEGA =
Used as 3).

The memory 418 shows the state of the CAM immediately after the writing of the character string "5T" to the address position ADDR8. This process searches the memory 418 and searches for the next FR.
Find the EE state field. Assuming ADDR8 is the last available location in the CAM current dictionary, FR
No EE status field is found. This is because the current dictionary is full and the system will accordingly state S = 1.
Means to be changed to. In state S = 1, the state field bit value [1: 0] constitutes a FREE memory location and the bit value [1: 1] constitutes the current dictionary entry (see FIG. 13). Therefore, all dictionary positions in the current dictionary in the S = 1 state can be used to store character strings except the character string at address ADDR5. When the status changes, the address pointer N
EXT_CODE is reset to the first FREE memory location (NEXT_CODE = 4).

Explaining the memory 420 in the S = 1 state, the next input character 426 ("I") is extracted from the raw character stream 414 and combined with OMEGA for the next OMEGA, K search. ("3I"). The string "3I" is in storage location ADDR7, but the status field for this location is now FREE. Therefore, no match is found and the encoded value "3" is output in the compressed character stream 422 as character 438.
This character string (CD (S), 3, I) is written to memory location ADDR4 and the character "I" is the next OMEGA.
It is encoded as a value (OMEGA = 1). Next FRE
The address location of the E state field is assigned to the address pointer (NEXT_CODE = 6). Note that address location ADDR5 is skipped because its state field represents the current dictionary entry after switching from state S = 0 to state S = 1.

The next input character 428 from the raw data stream 414 is combined with OMEGA consisting of a new character string ("IN"). Address location ADDR5
A match occurs, so OMEGA is assigned the match address value and the status field at address location ADDR5 is assigned to the standby dictionary. The bit allocation of the standby dictionary in the state of S = 1 is [0: 1] (see FIG. 13). The next input character from raw data stream 414 is combined with OMEGA and the search process is repeated. This process keeps changing the state of the system each time the current dictionary is "filled" until all characters from the raw data stream 414 have been compressed.

Memory 432 represents memory ready for decompression immediately after initialization for decompression. The memory 434 stores S immediately before the change from the state S = 0 to the state S = 1.
Shows the system in the = 0 state. Memory 436 shows the data entry in the S = 1 state after decompression of the compressed character stream 422. The dictionary in memory 432 has a single input character R for the first four address positions,
Initialized to have I, N and T encoded values. Again, single character decoding can be done algorithmically. The system is set to state S = 0 and all dictionary state registers are set to FREE (S). The address pointer NEXT_CODE is the first
Address dictionary SAVE_CODE is set to zero, and the address pointer SAVE_CODE is set to zero.

Decompression is performed as described above and OME
GA is used as an address pointer to the memory 432. The first input code from the compressed character stream 422 has an OMEGA value (OMEGA = 0). The decompression system determines the value "0" as the root codeword, for example by checking that the value is less than 4. The ADDR0 data entry (eg, "R") is thereby output as the first character in the decompressed character stream 430. The status field at address location OMEGA is set to SD (S).

The dictionary stores the first character K (eg, "R") from the decompressed codeword at address location S.
It is reconstructed by writing back to the CHAR_field of AVE_CODE. In this case, "R" is rewritten in the CHAR field of ADDR0. Next, the character string (CD (S), 0, R) is assigned to the address location NEXT_
Written to CODE (eg, ADDR4), SA
If VE_CODE is NEXT_CODE (for example, NE
XT_CODE = 5). Next, the address pointer NEXT_CODE is set to the value of the next empty address in the memory 434 (for example, NEXT_CODE =
5) is assigned.

The character "1" is read from the compressed character stream 422 and serves as the next value for OMEGA. O
The MEGA is decompressed and the decoded character "I" is output to the decompressed character stream 430 as the next character. The ST field of the address ADDR1 is SD (S)
Is set to (eg, [1: 1]) and the first character (“I”) from the decompressed OMEGA value is CHAR.
Field address position SAVE_CODE (ADD
It is written in R4). Then the character string (CD
(S), 1, I are ST fields of the memory 434, C
Address NE of ODE field and CHAR field
Written to XT_CODE (eg, ADDR5). The value of NEXT_CODE is used as the new value of SAVE_CODE. The next FREE status register is found and NEXT_CODE is set to that address (NEXT_CODE = 5).

Processing continues similarly for encoded characters "2" and "3" from compressed character stream 422. First "5" from compressed character stream 422
Is the first non-root codeword and has the address ADD
The data entry for R5 is the character string "IN". Therefore, the CODE field ("1") of ADDR5 is fed back as the next storage location read by the CAM. The output of ADDR1 (“I”) is C along with the previous CHAR field “N”.
Output by AM, ST field of ADDR5 is set to SD (0). The first character ("I") from the compressed codeword is the CHAR field at memory location ADDR7 (eg, SAVE_CODE =
7) written in the character string (CD (S),
5, I) is the CAM position NEXT_CODE (eg,
ADDR8) and the value of SAVE_CODE is the value of NEXT_CODE (eg, SAVE_CODE
E = 8) is set. Memory 434 shows the state of the current dictionary immediately after writing this character string to memory.

The next search shows that there is no status field with a FREE value. Therefore, the system switches to state S = 1 and the state register values are reallocated as shown in FIG. To explain the memory 436, the first F is stored in the address pointer NEXT_CODE.
The REE storage location (ADDR4) is allocated. Address locations ADDR0-ADDR3 and ADDR5 are currently entries in the current dictionary and address locations ADDR
4 and ADDR6-ADDR8 constitute the FREE position in the S = 1 state. Character 438 from compressed character stream 422 is set to OMEGA (OMEGA
= 3), it is decompressed in the new decompression state S = 1. The decoded character "T" is output to the decompressed character stream 430, and the ST field of ADDR3 has the value SD (S) for the state S = 1 (eg, [1:
0]) is assigned. SAVE_CODE is ADD
R8 is designated, and the character "T" is ADDR in the memory 436.
8 CHAR fields. The character string (CD (S), 3, T) is written to address location ADDR4 and SAVE_CODE has the value NEXT_CO
DE is assigned. Next FREE dictionary position is ADD
R6 and therefore address pointer NEXT_C
Assigned to ODE. This process continues similarly until all the characters in the compressed character stream 422 have been decompressed.

In the conventional LZ2 implementation, the codes are assigned sequentially, C ₀ , C ₀ + for a single character string.
Codes are assigned in the order of 1, C ₀ +2, ..., C ₀ + (2 ^m −1), where C ₀ is a small constant (eg, C ₀
= 0). This new for multiple character strings codes ^{_{C 0 +2 m, C 0 +}} (2 m +1), ..., 2 b -2,2
^b- 1 are assigned in this order, and each character string has a sequential address value in the CAM. Therefore, assigning a code to a string can be done simply by initializing the counter to C ₀ +2 ^m and incrementing it each time a new dictionary string is created. This causes the compressor to use a code of length m + 1 after the dictionary reset, and then the length of this output code to be the next 2 entries in the dictionary.
A variable length output code can be used by lengthening it by one bit each time the power of is reached. Therefore, the length of the output code varies between (m + 1) and b,
Here, 2 ^b is the maximum size of the dictionary. This can slightly improve the compression ratio. This is because the compressor uses a shorter output code if the dictionary address code is shorter. The decompressor can build its dictionary in steps synchronized with the compressor to track the expected length of the compressed code.

In the process shown in FIG. 16, the encoded value of the new character string is the address of the first FREE dictionary location. Immediately after the dictionary switch, the CD consists of a character string from the previous standby dictionary with positions in the CAM that are not necessarily contiguous. These strings retain their old address and hence code after switching. Therefore, the addresses (codes) returned by the FREE search do not form a continuous sequence. Also, each encoded character string C in the range 0 ≦ C ≦ 2 ^b −1 is potentially available immediately after dictionary reset.

As a result, the output stream must use a fixed length code. However, the adverse effect of this on the compression ratio is not so great. Since the CD starts in a partially full state after a dictionary reset, the number of bits required to represent the code is not much different from the maximum bit (b), even if the order of the codes in the CD changes. For example, experiments have shown that current dictionaries typically start at between 1/4 and 1/2 of maximum capacity. This means that even if the codes are arranged in consecutive order, b-1 bits are required after switching. However, variable length codes can be used during construction of the first current / standby dictionary (CD, SD). Alternatively, the current dictionary can be reordered after each reset.

Compression Results The compression and decompression process of the CAM multiple dictionary system is
Source code, executable object code, ASC
II has been applied to various types of data including data files, test files, bitmap image files, etc. The same file is a conventional L using a variable length output code
Compressed by ZW method. Figure 2 shows the overall result of this compression.
Shown in 1. Line 440 is a graphical representation of the compression ratio for this CAM multiple dictionary system and line 442 shows the compression ratio for the standard LZW algorithm. Lines 440 and 442 are compression ratios (original file size / compressed file size)
As b, a function of the maximum number of bits in the output code (ie, dictionary size log ₂ ).

To emphasize the advantages of this CAM / standby dictionary, dashed line 444 in the figure shows the compression ratio achieved by the 12-bit LZW compressor. The value of b for CAM multiple dictionary processing that achieves the same compression ratio is shown below. As shown in FIG. 21, this CAM multiple dictionary system provides the same compression ratio at 1/2 to 1/4 the number of dictionary entries in a standard LZW compressor (eg, 1 bit less requires less memory space than 1). / 2). This compression ratio is achieved using CAM dictionary entries that are only 1 or 2 bits longer than conventional LZW compressor data entries.

For simplicity, a minimal implementation of the standby dictionary has been described. Many changes can be made to increase the compression ratio. For example, the compression / decompression process shown in FIGS. 18 and 19 is premised on initializing the dictionary with a set of all single character strings in the input alphabet. Instead of this, as mentioned above,
Blanks or intermediate defaults can also be used. Even in processing based on a combination of intermediate initialization and standby dictionary, a high compression ratio can be obtained using a very small dictionary. Another implementation of this system does not switch dictionaries immediately after the current dictionary is full. Instead, the current dictionary is frozen and dictionary switching is based on compression ratio (ie, compression ratio is below a certain level).
It is also possible to freeze the standby dictionary while freezing the current dictionary. Alternatively, it can continue building until the next dictionary switch.

Another modification specific to the standby dictionary method is shown in FIG.
The state field value INV shown in 3 is used. Currently,
INV is not used for dictionary entries. INV can be used to define a second level standby dictionary denoted SD2. Entries already labeled SD are renamed to SD2 (new name of the current INV value) when referenced more than once. CD when switching dictionaries
The entry becomes FREE, the SD entry becomes a CD entry, and the SD2 entry becomes an SD entry. First
The standby dictionary (SD) of the SD2 starts from the set of character strings in SD2 and the new SD2 starts from the empty state. This modification can be easily implemented in the system shown in FIG. 16 by those skilled in the art.

[0147]

As described above, the modification of the Lempel-Ziv data compression algorithm for constructing the standby dictionary in parallel with the current compression dictionary has been described. When the current dictionary is full, the standby dictionary replaces it and a new standby dictionary is started. The standby dictionary contains a selected subset of the character strings of the main dictionary, allowing both dictionaries to be implemented on the same memory buffer. In a preferred embodiment of this system,
An associative memory module is used. To reduce processing time and circuit complexity, dictionary switching is based on a simple state transition scheme that eliminates the need for dictionary initialization after power up. Therefore, the CAM multiple dictionary compressor / decompressor system uses less memory and achieves a compression ratio comparable to conventional data compression methods while reducing the increase in control circuit complexity.

Although the principle of the present invention has been described and illustrated with the embodiments, it is obvious that the present invention can be modified in its configuration and details without departing from the principle. We claim all such modifications that fall within the spirit and scope of the claims.

[Brief description of drawings]

FIG. 1 is a data flow diagram of a data compression system having a current dictionary and a standby dictionary according to the present invention.

FIG. 2 is a detailed data flow diagram showing an example of a data compression algorithm using the standby dictionary of FIG.

FIG. 3 is a block diagram of one embodiment of a data compression / decompression system of the present invention.

FIG. 4 is a block diagram of an embodiment using an associative memory of the data compression / decompression system of the present invention.

5 is a detailed block diagram of the memory and logic circuit shown in FIG. 4. FIG.

6 is a logic diagram of an automatic update circuit of the address decompressor shown in FIG. 5. FIG.

FIG. 7 is a data flow diagram illustrating a general method of data compression / decompression storage / retrieval using the associative memory of the present invention.

8 is a detailed data flow diagram of the broken line block of FIG. 7. FIG.

9 is a detailed flow diagram of the decompression circuit of FIG. 7.

FIG. 10 illustrates the compression and decompression algorithms of FIGS. 8 and 9.

FIG. 11 is a block diagram of an associative memory multiple dictionary compression / decompression system of the present invention.

FIG. 12 is a diagram showing different entries included in each dictionary entry of the associative memory shown in FIG. 11.

FIG. 13 is a diagram showing each compressed / decompressed state of the ST field shown in FIG. 11.

FIG. 14 is a diagram showing a modified state of the associative memory multiple dictionary compression / decompression system.

FIG. 15 is a simple hardware logic diagram for changing the state of the compressor / decompressor.

16 is a detailed circuit diagram of the associative memory multiple dictionary compression / decompression system shown in FIG. 11. FIG.

17 is an ST pattern generator 33 shown in FIG.
9 is a detailed circuit diagram of FIG.

FIG. 18 is a data flow diagram showing a general method of data compression using an associative memory having a standby dictionary.

FIG. 19 is a data flow diagram showing a general method for decompressing data using an associative memory with a standby dictionary.

FIG. 20 is a diagram showing the compression and decompression algorithms of FIGS. 18 and 19;

FIG. 21: Associative memory multiple dictionary system and standard LZ
It is a graph figure which shows the compression result of a W compression system.

[Explanation of symbols]

 50 data compression / decompression integrated circuit 52 data compression / decompression engine 54 interface circuit 56,58 multiplexer 68 address generator 76 switch controller 84,86 transceiver 88,90 memory 92,96 standby state field 94,98 data entry field 136 associative Memory compression / decompression circuit 138, 148 interface 142 Compression / decompression engine 144 String table memory 146 Control logic 150, 140 Data buffer 152 Processor interface

 ─────────────────────────────────────────────────── ———————————————————————————————————————— Inventor Karl Bee Lantz Corvallis, Oregon, USA North East Thirty Force Street 640

Claims (1)

[Claims] 1. A memory device (88,90) including first and second dictionaries each having a finite number of memory locations (94,98) for storing data from a data sequence and a predetermined compression. A data compression engine (50) for writing to and reading from the first dictionary according to an algorithm and means for compressing / decompressing the input data to create an encoded data string and a first. Means (52) for reading and writing the data entry of the memory location in one dictionary to the second dictionary by a subset of the data entries written to the first dictionary by the data compression and decompression engine. A dictionary-based compression / decompression device comprising writing logic means (54).