WO1995019662A1 - Procede et appareil de compression de donnees - Google Patents

Procede et appareil de compression de donnees Download PDF

Info

Publication number
WO1995019662A1
WO1995019662A1 PCT/US1994/014823 US9414823W WO9519662A1 WO 1995019662 A1 WO1995019662 A1 WO 1995019662A1 US 9414823 W US9414823 W US 9414823W WO 9519662 A1 WO9519662 A1 WO 9519662A1
Authority
WO
WIPO (PCT)
Prior art keywords
buffer
history buffer
digital sequence
match
responsive
Prior art date
Application number
PCT/US1994/014823
Other languages
English (en)
Other versions
WO1995019662B1 (fr
Inventor
Jeffrey A. Weiss
Douglas H. Schremp
Aaron T. Smith
Original Assignee
Telco Systems, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Telco Systems, Inc. filed Critical Telco Systems, Inc.
Publication of WO1995019662A1 publication Critical patent/WO1995019662A1/fr
Publication of WO1995019662B1 publication Critical patent/WO1995019662B1/fr

Links

Classifications

    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
    • H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
    • H03M7/3084—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction using adaptive string matching, e.g. the Lempel-Ziv method
    • H03M7/3086—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction using adaptive string matching, e.g. the Lempel-Ziv method employing a sliding window, e.g. LZ77
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • G06T9/005—Statistical coding, e.g. Huffman, run length coding
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
    • H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
    • H03M7/40—Conversion to or from variable length codes, e.g. Shannon-Fano code, Huffman code, Morse code
    • H03M7/42—Conversion to or from variable length codes, e.g. Shannon-Fano code, Huffman code, Morse code using table look-up for the coding or decoding process, e.g. using read-only memory

Definitions

  • the present invention relates to methods and apparatus for compression of digital sequences.
  • Lossy compression does not include all of the original information in its compressed output. This type of compression can be used in video and audio applications where approximations are tolerated by the human eye or human ear. Lossless compression methods produce compressed output that can be decoded to restore the original sequence exactly. These methods are thus generally more appropriate for digital computer applications, such as program files, where loss of information usually cannot be tolerated.
  • This type of encoding method may either use predefined statistical information (e.g., about the English language) or it may adaptively develop its statistical information during operation.
  • One known method for developing such statistical information is to monitor symbols passing through a compression system and develop a tree representing the statistical distribution of the symbols within the sequence. This method requires some computational overhead, but allows the encoding method to adapt to changes in the statistical content of the data.
  • the invention features compressing a digital sequence by encoding the sequence according to a compression procedure.
  • the compressed sequence is buffered and the compression procedure is altered depending on buffer usage. This altering may include changing the depth of searching for matches performed in linked lists depending on the amount of the sequence being buffered.
  • the invention features compressing a digital sequence by maintaining a linked list of entries, which each include a pointer. An adjacent element can be found in this linked list using one of the pointers, and a match can also be found in a history buffer using the same pointer.
  • the invention features compressing a digital sequence by sampling a subset of received symbols, and updating frequency values for the received symbols based on the sampled subset.
  • a token is derived for each of the received symbols from the updated frequency values. Huffman encoding may be used to derive the tokens.
  • the invention features detecting, in a digital sequence, a first match between a first and a second subsequence and a second match between a third and fourth- subsequence. The fourth subsequence is encoded using an offset referenced to the first match.
  • the invention features decompressing a digital sequence by receiving a first match token, and thereafter receiving a second match token including an offset. The position of a subsequence in a history buffer is determined based on the offset and the first match token, and the subsequence is substituted for the second match token in the sequence.
  • the invention features compressing a digital sequence by maintaining an insert pointer and a check pointer into a history buffer. Characters are inserted at the insert pointer and the history buffer is accessed to search for matches in the history buffer. A determination is made as to whether the step of accessing the history buffer is valid based on the insert and check pointers. The maximum amount of the history buffer in which accessing is valid may be set.
  • FIG. 1 is a block diagram of a compression system according to the invention
  • FIG. 2 is a diagram illustrating the storage of data in connection with the first (LZ77) encoding stage of the embodiment of FIG. 1;
  • FIG. 3 is a physical data storage diagram illustrating physical storage of the data for the first (LZ77) encoding stage of the decoder of FIG. 1;
  • FIG. 4 is a diagram of an exemplary random access memory structure for use in implementing the embodiment of FIG. 1;
  • FIG. 5 is a diagram illustrating tokens produced by the segmenter of FIG. l;
  • FIG. 6 is a composite graph illustrating an exemplary probability distribution of the likelihood of matching a given string for an increasing offset within the history buffer, and corresponding segment sizes;
  • FIG. 7 is a diagram illustrating an exemplary output stream from the compression system of FIG. 1;
  • FIG. 8 is a block diagram of the last (Huffman) encoding stage of the embodiment of FIG. l;
  • FIG. 9 is a block diagram of a decompression system according to the invention.
  • FIG. 10 is an illustrative flow chart presenting principles of operation of the first (LZ77) encoding stage of the embodiment of FIG. 1;
  • FIG. 11 is an illustrative flowchart presenting principles of operation of the adaptive linked list searching of the first (LZ77) encoding stage of the embodiment of FIG l;
  • FIG. 12 is an illustrative flowchart presenting principles of operation of the last (Huffman) encoding stage of the embodiment of FIG. 1;
  • FIG. 13 is an illustrative flowchart presenting principles of operation of the segmenter of the embodiment of FIG. 1.
  • an exemplary encoding system 10 includes an LZ77-variant encoder 12, a segmenter 14, a coupling first-in-first-out (FIFO) buffer 16, a Huffman encoder 18, and a byte converter 19.
  • One or more coded indications of fill level (i.e., half full, one-quarter full, etc.) of the FIFO buffer are fed back to the LZ77 encoder over one or more feedback lines 17.
  • An offset data path 15 is provided between the segmenter and the byte converter via the FIFO buffer.
  • the system 10 may be implemented with digital hardware and/or software.
  • the exemplary encoding system 10 receives an input digital sequence 20, compresses it, and provides it as a compressed output sequence to a communications channel 22.
  • the encoding system uses the LZ77 encoder 12 to encode repeated sequences of characters each as an offset and a length.
  • the segmenter 14 expresses these using a segmenting scheme, and provides the resulting segment-based offset and the resulting segment-based length to the coupling FIFO buffer.
  • the coupling FIFO buffer receives both the segment-referenced lengths and any uncompressed literal data, and provides them to the Huffman encoder 18.
  • the Huffman encoder encodes these segment-referenced lengths and literal data and supplies them to the byte converter in the form of tokens.
  • the FIFO buffer supplies the buffered segment-based offset to the byte converter.
  • the byte converter provides the tokens to the communications channel in a byte-delineated format.
  • Control of the system is performed by control logic 11, which may be located within the individual modules, or may be centralized, as shown in FIG. 1.
  • control logic 11 may be located within the individual modules, or may be centralized, as shown in FIG. 1.
  • the LZ77 encoder 12 includes a circular history buffer 24, hashing logic 27, and an open hashing data structure 26.
  • the hashing data structure includes a hash table 28 and one or more linked lists 30, which lists are each linked to separate entries 36 in the hash table.
  • Each linked list includes one or more list elements 31, and each element 31 in the linked lists includes a pointer 38 to its predecessor and a pointer 40 to is successor.
  • the history buffer 24 is supplied with eight-bit binary characters, and the hashing logic 27 and hashing data structure 26 are used to search for three-byte matches in the history buffer. While eight-bit characters are described in the present embodiment, the invention is, of course, applicable to other formats. There is also no requirement that matches be performed on three bytes to reap the benefits of the invention. Further parameter changes, minor changes in the sequencing of operations, and other variations are also possible without losing the benefits of the invention.
  • the circular history buffer 24 is initialized by making a check pointer 32 equal to an insert pointer 34 (step 200 in FIG. 10). This can be done by setting both pointers equal to zero. At this point, the entire history buffer is deemed to include invalid data. Data to be compressed are then added to the history buffer at the location pointed to by the insert pointer (step 202) . As these data are added, the insert pointer is separated from the check pointer (step 204),
  • the check pointer 32 marks the end of the history buffer. As the history buffer is filled, the check pointer does not change. Once a predetermined number of characters M (e.g., 32K) have been inserted into the history buffer (see step 206), the check pointer begins to follow the insert pointer (step 208).
  • This number of characters M may be a user-settable parameter, allowing the user to effectively set the size of the history buffer depending on the anticipated application. For example, where it is important that a high compression ratio be obtained, the user may set the history buffer to its maximum size. Th.s feature may also allow systems using different amounts of physical memory to be operated together, with a system having more memory operating in a compatibility mode which sets a reduced history buffer size.
  • the hashing logic 27 applies a hashing function (step 207) to these three most recently inserted characters each time a new character is inserted.
  • An exemplary hashing function is:
  • the resulting hash value is used to address the hash table 28 (step 210). If there is no entry 36 in the hash table that corresponds to this value (branch 211), this indicates that there is no match in the history buffer. A pointer to the location of the three-character sequence in the history buffer is then stored at that location (step 234), a character of literal data is provided to the segmenter (step 232), and the next character sequence is processed.
  • the location addressed by the hash value contains an entry 36 (branch 213), this indicates that there could be a match in the history buffer 24.
  • This entry is first checked to see if it points to a valid location by determining whether it is between the insert and check pointers (step 212). To check for a match, the three bytes pointed to in the history buffer by the pointer stored in the hash entry are then compared with the current three byte sequence which was provided to the hashing logic 27 (step 214). If these do not match (branch 215), the linked list that is linked to the addressed hash table entry will be searched for a match.
  • a detected match (branch 221) is shorter than 64 bytes long (branch 231), but longer than any previous match, it is saved in a current best match register as the current longest match (step 223). Then, once any searching in the hashing data structure is complete (branch 227), the current longest match is encoded as an offset and a length and provided to the segmenter 14 (step 230). The offsets can be expressed as the number of characters separating the beginning of the encoded match and the end of the earlier sequence that it matches. If no match has been detected (step 228), one character is provided to the segmenter as literal data (step 232).
  • FIFO fill level information may be used at the encoder to determine the search depth parameter. For example, a "FIFO nearing empty" signal may be provided on the feedback lines 17 from the FIFO when it has fewer than a certain number of entries (step 240 of FIG. 11). This signal will then cause the encoder to reduce its search depth (step 242) and provide data to the FIFO 16 via the segmenter 14 more rapidly. In this mode, there is a chance that shorter than optimal matches will be found, but encoded data will be provided to the FIFO at a higher rate.
  • a "FIFO adequately filled” signal may be provided to the encoder 12 on the feedback lines 17. This signal will cause the encoder to increase its search depth again (step 246) .
  • the encoder may be implemented using a series of four memory arrays having equal numbers of entries.
  • the first array is the history buffer 24 (e.g., 32K x 8 bits)
  • the second array is the hash table 28 (e.g., 32K x 16 bits)
  • the third array is a predecessor array 42 (e.g., 32K x 16 bits)
  • the fourth array is a successor array 44 (e.g., 32K x 16 bits) .
  • the predecessor array and the successor array make up the storage area for the linked lists 30.
  • a new three-byte combination in the history buffer 24 causes the hashing logic 27 to address a particular location 46 in the hash table 28.
  • This location contains a pointer entry to a buffer entry 48 in the history buffer.
  • the three bytes stored beginning at this buffer entry can now be tested to determine whether there is a match, by comparing them with the three-byte combination. If there is no match, and there is no element of a linked list associated with the table entry, a match is not found.
  • a copy of the insert pointer is now added to the structure as the first element (not shown) of the linked list associated with that entry 46.
  • the hashing logic addresses another particular hash table entry, for example table entry 50, which has a linked list associated with it, the buffer entry 52 pointed to by the contents of the table entry is similarly checked for a match. If no match is found, the entry 50 of the hash table is used to point to a predecessor entry 54 in the predecessor array 42, and its corresponding successor entry 56 in the successor array 44. Together, these two entries make up a first element of the linked list.
  • the predecessor entry 54 in the predecessor array points back to the hash table entry 50.
  • the entry 56 in the successor array points to another location 58 in the history buffer which may be checked for another match.
  • this successor entry also points to the next predecessor entry 60 and successor entry 62 (i.e., the second element in the linked list) . Because entries in the linked list point to other entries in the linked list and to entries in the history buffer at the same time, each element of the linked list need only contain a predecessor and successor entry, and need not contain a separate pointer to the history buffer. This type of structure occupies less memory than might otherwise be required. The fact that the arrays are all the same size facilitates the implementation of this feature.
  • the predecessor entry 60 of the second element in the linked list points to the first element in the linked list, which includes its predecessor entry 54 and its successor entry 56. Note that it also points (not shown) to the history buffer entry 52 pointed to by the second hash table entry 50.
  • the successor entry 62 of the second element in the linked list points to yet another buffer entry 64 in the history buffer 24, which may be checked for a match.
  • This successor entry 62 also points to a predecessor entry 66 and a successor entry 68 of a third linked list element.
  • This third predecessor entry points back to the predecessor element 60 and the successor element 62 of the second linked list element.
  • the successor entry of the third linked list element in this instance is an end marker, which indicates the end of the linked list.
  • RAM random-access memory
  • the conventional memory structure 70 includes a bank 72 of storage elements, addressing logic 74, an address input 76, and a data output 78.
  • an address is placed on the address input 76, and the addressing logic 74 selects one of the entries to be presented at the data output 78.
  • This type of random access memory structure can be further controlled by control lines, such as READ lines, WRITE lines, ENABLE lines, and the like, the use of which is well known and will not be discussed further.
  • control lines such as READ lines, WRITE lines, ENABLE lines, and the like, the use of which is well known and will not be discussed further.
  • 16 least significant address bits are used to address locations within the arrays, while more significant address bits select between the arrays.
  • the segmenter 14 receives (step 260) the output of the LZ77 encoder 12 in a parallel format, and determines whether it represents a match or literal data (step 262), by testing for a match length value of zero. If literal data is detected, a literal token 80 is provided to the Huffman encoder 18 (step 264).
  • a literal token 80 may include a literal token indicator bit sequence 82 followed by literal data 84.
  • the segmenter selects either an absolute or difference encoding mode (step 266). This selection is made based on the determination of which mode results in the shortest encoded sequence.' The mode chosen affects the point from which the offset is measured, but does not affect the length.
  • a match token is then provided to the Huffman encoder (step 270).
  • a match token is made up of a segment number 92 (e.g., 4 bits), and a length 94 (e.g., 6 bits)
  • the segments 96, 98, 100, 110, 112, 114 used by the segmenter have different lengths, and therefore specifying a position (or segment-based offset) in each of the segments requires a different number of bits.
  • the segments have been arranged such that the shortest segments are most recent and the longest segments are oldest. This takes advantage of the fact that most matches are generally found to occur in the relatively recent past of the history buffer, as illustrated by the exemplary probability distribution curve 116.
  • the first segment 96 will thus have the highest density of matches. Because this is also the shortest segment, these matches will have the shortest segment-based r ⁇ .fset indicators. This segmented method of specifying of. :ets therefore provides added compression over the LZ77 variant.
  • the offset is encoded as an offset from the most recent match, even if it is not a match of the same subsequence, and this offset is provided to the byte converter (step 272).
  • a match token is then provided to the Huffman encoder (step 270). The format of this match token is somewhat different from the match token used in absolute encoding.
  • a special segment number is used in difference coding mode.
  • offsets are encoded to be interpreted as a difference offset 111 from the last match 109 to the current match 113, rather than from the beginning of a segment boundary.
  • This mode can be particularly useful for compressing in a packet-based environment, where packets which contain different types of data are interleaved onto the communications stream. In such an environment, it is likely that data in a packet will tend to match data in an earlier occurrence of that data type in an earlier packet. These related packets may be separated by a number of intervening packets, however, which could result in a succession of long offsets if the difference coding mode were not used.
  • the offsets in the difference coding mode are not segmented.
  • the tokens are encoded by the Huffman encoder 18 to produce Huffman encoded tokens 118 for literal tokens, and 119 for match tokens.
  • Huffman encoded tokens are provided to the byte converter 19, which adds the appropriate offsets 120 to the Huffman encoded match tokens 119.
  • the offsets are not Huffman encoded, because it has been found that these numbers tend to be quite random and therefore result in little or no compression, while taxing the Huffman encoder.
  • the Huffman encoded tokens 118, 119 will be of variable length, by definition.
  • the offsets 120 will also be of variable length, due to the segmenting.
  • the byte converter breaks the encoded data stream 121 up into bytes for transmission.
  • the Huffman encoder 18 is made up of a window or buffer 122, which is responsive to the tokens from the coupling FIFO 16, and Huffman tree storage 124.
  • the tree storage is responsive to tokens from the coupling FIFO and the output of the buffer 122.
  • the Huffman encoder also includes control logic 126 for controlling which symbols are supplied to the Huffman tree storage, and Huffman substitution logic 128, which is responsive to the tokens from the coupling FIFO and the statistical information stored in the Huffman tree storage.
  • Tokens are received from the segmenter 14, via the coupling FIFO 16, into the buffer 122.
  • each token is substituted for with its current Huffman representation by the Huffman substitution logic 128 (step 250 in FIG. 12) .
  • the frequency of occurrence of that particular token may be updated in the Huffman tree, which is stored in the Huffman tree storage 124 (step 254). This generally has the effect of updating the Huffman code, so that the next occurrence of the token will usually be encoded with a shorter Huffman code.
  • tokens move through the buffer 122 in a FIFO manner, eventually they reach the output of the buffer. At this point, they may be provided to the Huffman tree storage and the frequency for this particular token may be decremented.
  • the manner in which the tree is maintained and updated is known and is discussed further in "Variations On A Theme By Huffman” by Robert G. Gallager, IEEE. Transactions on Information Theory, Vol. IT-24, 6, November 1978, and "Dynamic Huffman Coding" by Donald E. Knuth, Journal of Algorithms 6, 163-180 (1985).
  • the control logic 126 controls the admission of the tokens from the input and the output of the buffer 122 to the Huffman tree storage.
  • the control logic may admit less than all of the tokens. For example, it may admit only a fraction of the tokens (step 252), such as every four or eight tokens. This sampling may be performed by decrementing a counter each time a token is received, until it reaches zero, at which point a sample is taken. The counter is then reloaded with the sampling ratio. For sufficient data passing through the buffer, the statistical content of the Huffman tree will be representative of the data encoded, but the standard Huffman updating operations will need to be performed less frequently. This will reduce required processing time significantly, which can result in improved throughput.
  • the sampling ratio used by the control logic may be a user-settable parameter.
  • segment numbers 0 through 3 indicate literal data.
  • the relationship between subsequent segment numbers, the range of offsets for each segment, and the numbers of bits required to offset into the segments for one embodiment are shown in Table I.
  • a further segment number can be used for difference encoding.
  • a non-adaptive fixed code may also be substituted for the above adaptive Huffman code, in cases where the adaptive approach would result in degraded performance.
  • This non-adaptive fixed code can shorten the literal data indicator 82 in each literal token 80 to a single bit (see FIG. 6).
  • These modified literal tokens are provided as the output of the encoder along with the match tokens 88 and the variable-length offsets 120.
  • an exemplary decoding system 130 includes a Huffman decoder 132 responsive to a stream of compressed data on the communications channel 22, an unsegmenter 134 responsive to the stream of compressed data and the Huffman decoder, and a string substitution module 136 responsive to the unsegmenter.
  • the Huffman decoder 132 decodes Huffman encoded tokens and provides decoded tokens to the unsegmenter 134.
  • the Huffman decoder maintains its frequency information in the same manner as does the Huffman encoder. If the Huffman encoded token represents a literal, the unsegmenter adds the literal data from that token to an output stream 138. If the decoded token represents a match, the unsegmenter reads the segment number so that it can interpret the offset that follows a match token. The unsegmenter then converts the segment number and the segment-based offset into an absolute offset and provides it, along with the length, to the string substitution module 136.
  • the string substitution module 136 maintains a history buffer, and uses the length and offset to copy matched strings from the history buffer into the output stream.
  • a compression system and a decompression system can be provided in the same hardware implementation for bidirectional operation. In software embodiments, these can take the form of different software modules.
  • the encoding and decoding circuits can be separate (full-duplex), or they can share circuitry (half-duplex) . Of course, individual compression and decompression systems can each be constructed separately.
  • software and hardware embodiments can communicate with each other. The two encoding stages can also each be used separately.
  • a compression system and decompression system share circuitry in an 0.7 micron CMOS gate array, which uses external memory for the history buffer, hash table, and predecessor and successor arrays.
  • the integrated circuit includes control circuitry, registers, hashing circuitry, and a look-ahead buffer. This embodiment has a microprocessor interface and is constructed to interact via DMA with bus-based computer hardware that has an eight or sixteen-bit bus.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)

Abstract

Appareil et procédé de compression d'un séquence numérique. On peut mettre en mémoire tampon la séquence comprimée et modifier une procédure de compression selon l'utilisation du tampon. Cette modification peut consister à changer la profondeur de recherche de correspondances réalisée dans des listes enchaînées en fonction du volume de la séquence mise en mémoire tampon. On peut trouver un élément adjacent dans une liste enchaînée au moyen d'un pointeur, ainsi qu'une correspondance dans un tampon historique à l'aide du même pointeur. On peut également mettre à jour les valeurs de fréquence pour les symboles en fonction d'un sous-ensemble échantillonné desdits symboles, et dériver un jeton, par exemple, par codage Huffman, pour chaque symbole provenant des valeurs de fréquence mises à jour. On peut coder une sous-séquence au moyen d'un décalage référencé par rapport à une correspondance antérieure. On peut conserver un pointeur d'insertion et un pointeur de vérification dans un tampon historique, et déterminer si l'accès au tampon historique est correct ou non en fonction de ces pointeur de vérification. Le volume maximum du tampon historique auquel l'accès est correct peut être fixé. L'invention porte également sur la décompression de séquences.
PCT/US1994/014823 1994-01-13 1994-12-21 Procede et appareil de compression de donnees WO1995019662A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US18118794A 1994-01-13 1994-01-13
US08/181,187 1994-01-13

Publications (2)

Publication Number Publication Date
WO1995019662A1 true WO1995019662A1 (fr) 1995-07-20
WO1995019662B1 WO1995019662B1 (fr) 1995-08-10

Family

ID=22663256

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US1994/014823 WO1995019662A1 (fr) 1994-01-13 1994-12-21 Procede et appareil de compression de donnees

Country Status (1)

Country Link
WO (1) WO1995019662A1 (fr)

Cited By (27)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0789326A2 (fr) * 1996-02-09 1997-08-13 Fujitsu Limited Système de compression et de décompression de données
GB2311635A (en) * 1996-03-27 1997-10-01 Memory Corp Plc Compression of data for storage using two CAM dictionaries in parallel
GB2323754A (en) * 1997-01-30 1998-09-30 Peter Graham Craven Lossless data compression and buffering of audio signals for DVD
EP0987823A2 (fr) * 1998-09-17 2000-03-22 Navigation Technologies Corporation Méthode et système de compression de données géographiques
WO2000035098A1 (fr) * 1998-12-07 2000-06-15 Marconi Communications Israel Ltd. Dispositif et procedes de compression en temps reel sans perte
WO2000045516A1 (fr) * 1999-01-29 2000-08-03 Interactive Silicon, Inc. Systeme et procede pour la compression et la decompression de donnees en parallele
EP1064727A2 (fr) * 1998-01-27 2001-01-03 Expand Networks Ltd. Systeme et procede adaptatifs de compression de paquets
EP1134901A1 (fr) * 2000-03-16 2001-09-19 Lucent Technologies Inc. Méthode et dispositif de compression des paquets de données dans un réseau utilisant des tables de hash à base de paquets
WO2002052731A2 (fr) * 2000-12-22 2002-07-04 Cilys 53 Inc. Systeme et procede de compression et de decompression de donnees en temps reel
US6523102B1 (en) 2000-04-14 2003-02-18 Interactive Silicon, Inc. Parallel compression/decompression system and method for implementation of in-memory compressed cache improving storage density and access speed for industry standard memory subsystems and in-line memory modules
US6657565B2 (en) * 2002-03-21 2003-12-02 International Business Machines Corporation Method and system for improving lossless compression efficiency
EP1437713A1 (fr) * 2001-10-03 2004-07-14 Sony Corporation Appareil et procede de codage, appareil et procede de decodage et appareil et procede d'enregistrement de support d'enregistrement
US6819271B2 (en) 1999-01-29 2004-11-16 Quickshift, Inc. Parallel compression and decompression system and method having multiple parallel compression and decompression engines
US6822589B1 (en) 1999-01-29 2004-11-23 Quickshift, Inc. System and method for performing scalable embedded parallel data decompression
US6879266B1 (en) 1997-08-08 2005-04-12 Quickshift, Inc. Memory module including scalable embedded parallel data compression and decompression engines
US6885319B2 (en) 1999-01-29 2005-04-26 Quickshift, Inc. System and method for generating optimally compressed data from a plurality of data compression/decompression engines implementing different data compression algorithms
US7129860B2 (en) 1999-01-29 2006-10-31 Quickshift, Inc. System and method for performing scalable embedded parallel data decompression
US7190284B1 (en) 1994-11-16 2007-03-13 Dye Thomas A Selective lossless, lossy, or no compression of data based on address range, data type, and/or requesting agent
US7538694B2 (en) 1999-01-29 2009-05-26 Mossman Holdings Llc Network device with improved storage density and access speed using compression techniques
WO2011100336A1 (fr) * 2010-02-11 2011-08-18 Sycamore Networks, Inc. Procédé de réduction de données de motifs redondants sans perte
US8514697B2 (en) 2010-01-08 2013-08-20 Sycamore Networks, Inc. Mobile broadband packet switched traffic optimization
US8775776B2 (en) 2012-01-18 2014-07-08 International Business Machines Corporation Hash table using hash table banks
US9325625B2 (en) 2010-01-08 2016-04-26 Citrix Systems, Inc. Mobile broadband packet switched traffic optimization
CN108233942A (zh) * 2018-01-08 2018-06-29 武汉斗鱼网络科技有限公司 一种用于数据存储的方法、装置及计算机设备
US10432216B1 (en) * 2017-06-05 2019-10-01 Amazon Technologies, Inc. Configurable compression circuit
CN116886104A (zh) * 2023-09-08 2023-10-13 西安小草植物科技有限责任公司 一种基于人工智能的智慧医疗数据分析方法
CN117294397A (zh) * 2023-11-27 2023-12-26 深圳市正东兴通讯设备有限公司 一种无线通信数据存储方法及系统

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1987005178A1 (fr) * 1986-02-18 1987-08-27 Deutsche Thomson-Brandt Gmbh Procede numerique de telecommunications
EP0309280A1 (fr) * 1987-09-23 1989-03-29 BRITISH TELECOMMUNICATIONS public limited company Codeur vidéo
US5140321A (en) * 1991-09-04 1992-08-18 Prime Computer, Inc. Data compression/decompression method and apparatus
US5159336A (en) * 1991-08-13 1992-10-27 Iomega Corporation Tape controller with data compression and error correction sharing a common buffer

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1987005178A1 (fr) * 1986-02-18 1987-08-27 Deutsche Thomson-Brandt Gmbh Procede numerique de telecommunications
EP0309280A1 (fr) * 1987-09-23 1989-03-29 BRITISH TELECOMMUNICATIONS public limited company Codeur vidéo
US5159336A (en) * 1991-08-13 1992-10-27 Iomega Corporation Tape controller with data compression and error correction sharing a common buffer
US5140321A (en) * 1991-09-04 1992-08-18 Prime Computer, Inc. Data compression/decompression method and apparatus

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
BELL, CLEARY AND WITTEN: "text compression", PRENTICE HALL, ENGLEWOOD CLIFFS NJ (US) *

Cited By (41)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7190284B1 (en) 1994-11-16 2007-03-13 Dye Thomas A Selective lossless, lossy, or no compression of data based on address range, data type, and/or requesting agent
EP1339022A3 (fr) * 1996-02-09 2008-12-31 Fujitsu Limited Système de compression et de décompression de données
EP0789326A3 (fr) * 1996-02-09 1999-07-21 Fujitsu Limited Système de compression et de décompression de données
EP0789326A2 (fr) * 1996-02-09 1997-08-13 Fujitsu Limited Système de compression et de décompression de données
EP1339022A2 (fr) * 1996-02-09 2003-08-27 Fujitsu Limited Système de compression et de décompression de données
GB2311635A (en) * 1996-03-27 1997-10-01 Memory Corp Plc Compression of data for storage using two CAM dictionaries in parallel
GB2323754A (en) * 1997-01-30 1998-09-30 Peter Graham Craven Lossless data compression and buffering of audio signals for DVD
GB2323754B (en) * 1997-01-30 2002-03-20 Peter Graham Craven Lossless compression using iir prediction filters
US6879266B1 (en) 1997-08-08 2005-04-12 Quickshift, Inc. Memory module including scalable embedded parallel data compression and decompression engines
EP1064727A2 (fr) * 1998-01-27 2001-01-03 Expand Networks Ltd. Systeme et procede adaptatifs de compression de paquets
EP1064727A4 (fr) * 1998-01-27 2004-09-15 Expand Networks Ltd Systeme et procede adaptatifs de compression de paquets
EP0987823A2 (fr) * 1998-09-17 2000-03-22 Navigation Technologies Corporation Méthode et système de compression de données géographiques
EP0987823A3 (fr) * 1998-09-17 2003-07-09 Navigation Technologies Corporation Méthode et système de compression de données géographiques
WO2000035098A1 (fr) * 1998-12-07 2000-06-15 Marconi Communications Israel Ltd. Dispositif et procedes de compression en temps reel sans perte
US6822589B1 (en) 1999-01-29 2004-11-23 Quickshift, Inc. System and method for performing scalable embedded parallel data decompression
US6885319B2 (en) 1999-01-29 2005-04-26 Quickshift, Inc. System and method for generating optimally compressed data from a plurality of data compression/decompression engines implementing different data compression algorithms
US7538694B2 (en) 1999-01-29 2009-05-26 Mossman Holdings Llc Network device with improved storage density and access speed using compression techniques
WO2000045516A1 (fr) * 1999-01-29 2000-08-03 Interactive Silicon, Inc. Systeme et procede pour la compression et la decompression de donnees en parallele
US6208273B1 (en) 1999-01-29 2001-03-27 Interactive Silicon, Inc. System and method for performing scalable embedded parallel data compression
US6819271B2 (en) 1999-01-29 2004-11-16 Quickshift, Inc. Parallel compression and decompression system and method having multiple parallel compression and decompression engines
US7129860B2 (en) 1999-01-29 2006-10-31 Quickshift, Inc. System and method for performing scalable embedded parallel data decompression
EP1134901A1 (fr) * 2000-03-16 2001-09-19 Lucent Technologies Inc. Méthode et dispositif de compression des paquets de données dans un réseau utilisant des tables de hash à base de paquets
US6523102B1 (en) 2000-04-14 2003-02-18 Interactive Silicon, Inc. Parallel compression/decompression system and method for implementation of in-memory compressed cache improving storage density and access speed for industry standard memory subsystems and in-line memory modules
WO2002052731A3 (fr) * 2000-12-22 2003-05-22 Cilys 53 Inc Systeme et procede de compression et de decompression de donnees en temps reel
WO2002052731A2 (fr) * 2000-12-22 2002-07-04 Cilys 53 Inc. Systeme et procede de compression et de decompression de donnees en temps reel
EP1437713A4 (fr) * 2001-10-03 2006-07-26 Sony Corp Appareil et procede de codage, appareil et procede de decodage et appareil et procede d'enregistrement de support d'enregistrement
US7318026B2 (en) 2001-10-03 2008-01-08 Sony Corporation Encoding apparatus and method, decoding apparatus and method, and recording medium recording apparatus and method
EP1437713A1 (fr) * 2001-10-03 2004-07-14 Sony Corporation Appareil et procede de codage, appareil et procede de decodage et appareil et procede d'enregistrement de support d'enregistrement
US6657565B2 (en) * 2002-03-21 2003-12-02 International Business Machines Corporation Method and system for improving lossless compression efficiency
US8514697B2 (en) 2010-01-08 2013-08-20 Sycamore Networks, Inc. Mobile broadband packet switched traffic optimization
US8560552B2 (en) 2010-01-08 2013-10-15 Sycamore Networks, Inc. Method for lossless data reduction of redundant patterns
US9325625B2 (en) 2010-01-08 2016-04-26 Citrix Systems, Inc. Mobile broadband packet switched traffic optimization
WO2011100336A1 (fr) * 2010-02-11 2011-08-18 Sycamore Networks, Inc. Procédé de réduction de données de motifs redondants sans perte
US8775776B2 (en) 2012-01-18 2014-07-08 International Business Machines Corporation Hash table using hash table banks
US10432216B1 (en) * 2017-06-05 2019-10-01 Amazon Technologies, Inc. Configurable compression circuit
CN108233942A (zh) * 2018-01-08 2018-06-29 武汉斗鱼网络科技有限公司 一种用于数据存储的方法、装置及计算机设备
CN108233942B (zh) * 2018-01-08 2022-02-22 武汉斗鱼网络科技有限公司 一种用于数据存储的方法、装置及计算机设备
CN116886104A (zh) * 2023-09-08 2023-10-13 西安小草植物科技有限责任公司 一种基于人工智能的智慧医疗数据分析方法
CN116886104B (zh) * 2023-09-08 2023-11-21 西安小草植物科技有限责任公司 一种基于人工智能的智慧医疗数据分析方法
CN117294397A (zh) * 2023-11-27 2023-12-26 深圳市正东兴通讯设备有限公司 一种无线通信数据存储方法及系统
CN117294397B (zh) * 2023-11-27 2024-02-20 深圳市正东兴通讯设备有限公司 一种无线通信数据存储方法及系统

Similar Documents

Publication Publication Date Title
WO1995019662A1 (fr) Procede et appareil de compression de donnees
JP2713369B2 (ja) データ圧縮装置及び方法
US5710562A (en) Method and apparatus for compressing arbitrary data
CA2263453C (fr) Technique de compression de donnees de type lempel-ziv utilisant un dictionnaire contenant deja des combinaisons de lettres frequentes, des mots frequents et/ou des locutions frequentes
US4906991A (en) Textual substitution data compression with finite length search windows
KR100300789B1 (ko) 적응형 데이터 압축기의 데이터 압축 효율을 개선하는 방법및 장치
US5058144A (en) Search tree data structure encoding for textual substitution data compression systems
US5870036A (en) Adaptive multiple dictionary data compression
US6657565B2 (en) Method and system for improving lossless compression efficiency
KR100353171B1 (ko) 적응형데이터압축을수행하는방법및장치
JPH07104971A (ja) ネットワークパケットに適用される小型辞書を用いた圧縮方法
US5874908A (en) Method and apparatus for encoding Lempel-Ziv 1 variants
WO1998006028A9 (fr) Technique de compression de donnees de type lempel-ziv utilisant un dictionnaire contenant deja des combinaisons de lettres frequentes, des mots frequents et/ou des locutions frequentes
US8106799B1 (en) Data compression and decompression using parallel processing
KR0178201B1 (ko) 가변 길이 복호화 장치
US6094151A (en) Apparatus and method for finite state machine coding of information selecting most probable state subintervals
US5555323A (en) System for bi-level symbol coding/decoding with saved storage and method for same
US5841953A (en) Method for compressing and decompressing data files
US6621428B1 (en) Entropy codec for fast data compression and decompression
EP0340039B1 (fr) Codage de structures de données d'arborescence de recherche pour des systèmes de compression de données par substitution de chaînes
JP3061278B2 (ja) 可変ビット長コード語のビット長通信方法
US7612692B2 (en) Bidirectional context model for adaptive compression
GB2360915A (en) Run length compression encoding of selected bits of data words
GB2311705A (en) Compression of arbitrary data using a plurality of context models and entropy coding
GB2360917A (en) Run length compression encoding of runs of similar data words

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): CA JP

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): AT BE CH DE DK ES FR GB GR IE IT LU MC NL PT SE

DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
121 Ep: the epo has been informed by wipo that ep was designated in this application
NENP Non-entry into the national phase

Ref country code: CA

122 Ep: pct application non-entry in european phase