CN103023509A - Hardware LZ77 compression implementation system and implementation method thereof - Google Patents
Hardware LZ77 compression implementation system and implementation method thereof Download PDFInfo
- Publication number
- CN103023509A CN103023509A CN2012104553279A CN201210455327A CN103023509A CN 103023509 A CN103023509 A CN 103023509A CN 2012104553279 A CN2012104553279 A CN 2012104553279A CN 201210455327 A CN201210455327 A CN 201210455327A CN 103023509 A CN103023509 A CN 103023509A
- Authority
- CN
- China
- Prior art keywords
- data
- module
- compression
- dictionary
- data storage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Abstract
The invention discloses a hardware LZ77 compression implementation system and a hardware LZ77 compression implementation method. The hardware LZ77 compression implementation system comprises a PCIE (Peripheral Component Interconnect Express) interface module, a data direct access module, a data packing/unpacking module, a data write-in buffer module, a compression algorithm module, a dictionary module, an unfixed length code element joining module and a data read-out buffer module. The hardware LZ77 compression implementation method comprises the following steps of: 1. caching data to be compressed; 2. performing compressed encoding on character string data; 3. splicing unfixed length data; and 4. caching the compressed data. According to the system and the method provided by the invention, LZ77 compression is realized through hardware; and therefore, the efficiency of the LZ77 compression algorithm can be effectively improved and CPU (Central Processing Unit) is released from mass data compression.
Description
Technical field
The present invention relates to data compression, particularly adopt hardware LZ77 compression realization system and its implementation of ping-pong operation.
Background technology
Along with the fast development of information and communication technology (ICT), the exchanges data amount increases day by day, and large-scale data are processed also become more numerous and diverse.Must effectively compress for huge data like this, can effectively reduce the data storage amount, rationally utilize to greatest extent limited data transfer bandwidth.In addition, the data after the compression are the encryption to initial data to a certain extent, better protected data.
Data compression can be divided into two types, a kind of Lossless Compression that is called, and another kind is called lossy compression method.
Lossless Compression refers to use the data after the compression to be reconstructed (perhaps be called reduction, decompress), and the data after the reconstruct and original data are identical; Lossless Compression is used for requiring signal and the on all four occasion of primary signal of reconstruct.A very common example is the compression of disk file.According to present technical merit, lossless compression algorithm generally can be the data compression of ordinary file to original 1/2～1/4.Most of condensing routines use the LZ algorithm based on self-adapting dictionary to dwindle file." LZ " refers to inventor Lempel and the Ziv of this algorithm, and " dictionary " refers to the method that data block is sorted out.
At present, the compression of the overwhelming majority still has software to realize, data compression is processed operation to remain and is finished by central processing unit (CPU), when in the face of mass data processing, will inevitably take a large amount of cpu resources, become a difficult problem so that when carrying out data compression, carry out other operations.In addition, Software Compression is a kind of in sequence operation, can't obtain high efficiency concurrent operation.
Therefore, how effectively to improve the efficient of compression algorithm, alleviate the subject matter that pressure that Data compression brings to CPU becomes existing Software Compression decompression technique.
Summary of the invention
The technical problem that the present invention will solve provides a kind of hardware LZ77 compression and realizes system and method, can effectively improve the efficient of LZ77 compression algorithm, and CPU is freed from Data compression.
The present invention adopts following technical scheme for achieving the above object:
A kind of hardware LZ77 compresses the realization system, it is characterized in that this system comprises:
The PCIE interface module be used for to realize and the communicating by letter of host computer;
Data access directly module DMA is used for realizing the direct access of data;
Data packing parse module, the group bag to data when being used for realizing data communication is conciliate package operation;
Data write cache module, are used for buffer memory data to be compressed;
The compression algorithm module is used for realizing searching of repeat character string, calculates the matching length of repeat character string, carries out the LZ77 coding;
Dictionary module is used for storing historical character string;
Non-fixed length code element concatenation module is for the data block that random length code high speed is spliced into fixed length;
The data reading cache module is used for the data after buffer memory compresses.
It is further characterized in that described data write cache module and comprise:
Two data storage devices such as random access memory ram or pushup storage FIFO are used for storing data to be compressed;
MUX is used for one of them data storage device that selection is stored in data to be compressed two data storage devices;
Realize the ping-pong operation that data write by MUX and two data storage devices.
Described compression algorithm module comprises:
Data are kept in module, are used for interim temporary a certain amount of data;
The dictionary read module is for generation of read control signal and the data that read in the dictionary of dictionary;
The maximum length matching module is used for calculating the maximum repeat length when repeat character string occurring;
The LZ77 coding module is used for treating packed data and encodes accordingly;
The dictionary updating module is for generation of the write control signal of dictionary and the data in the renewal dictionary.
Hardware LZ77 according to claim 1 compresses the realization system, it is characterized in that described dictionary module comprises:
The Hash table module is used for storing up-to-date historical dictionary information;
Dictionary chained list module, the historical dictionary information that is used for storing other;
The index initialization module is used for initialization Hash table module;
The chained list initialization module is used for initialization dictionary chained list module.
Described non-fixed length code element concatenation module comprises:
Non-fixed length code element is used for random length data encoding is spliced into the data of fixed length to fixed length code element modular converter;
The file size computing module is used for the data amount check after the calculation document compression, and with compression before the file data number relatively;
Compact model is selected module, according to before the compressing file and the data amount check after the compression how much choose compact model;
The direct memory module of data is used for when the direct store compressed pattern of data selection the operation to data.
Described data reading cache module comprises:
Two data storage devices such as random access memory ram or pushup storage FIFO are for the data after the store compressed;
The data writing MUX, the data after being used for selecting to compress write one of them data storage device of two data storage devices;
The sense data MUX be used for to select to read the data in one of them data storage devices of two data storage devices.
Further, the Hash table module of above-mentioned dictionary module comprises:
Two data storage devices such as random access memory ram or Content Addressable Memory CAM are used for storing up-to-date historical dictionary information;
Write MUX, be used for the up-to-date historical dictionary information of one of them data storage device stores of two data storage devices of choice for use;
Read MUX, be used for to select to read the dictionary information in one of them data storage devices of two data storage devices;
The initialization MUX, for a data storage device selecting two data storage devices of initialization, alternately two data storage devices of initialization improve the efficient of compression.
The dictionary chained list module of above-mentioned dictionary module comprises:
Two data storage devices such as random access memory ram or Content Addressable Memory CAM, the historical dictionary information that is used for storing other;
Write MUX, be used for the up-to-date historical dictionary information of one of them data storage device stores of two data storage devices of choice for use;
Read MUX, be used for to select to read the dictionary information in one of them data storage devices of two data storage devices;
The initialization MUX, for a data storage device selecting two data storage devices of initialization, alternately two data storage devices of initialization improve the efficient of compression.
The non-fixed length code element of above-mentioned non-fixed length code element concatenation module comprises to fixed length code element modular converter:
Non-fixed length code element is spliced into shorter fixed-length data module, is used for the non-block code of compression algorithm module output is spliced into short fixed-length data;
Be spliced into than the long data module than short data, be used for being spliced into long data with what non-fixed length code element was spliced into shorter fixed-length data module output than short data, be spliced into the parallel running of shorter fixed-length data module with non-fixed length code element, accelerated the process of splicing.
A kind of hardware LZ77 compression implementation method comprises the steps:
(1) buffer memory data to be compressed;
(2) string data is carried out compressed encoding;
(3) splice random length data;
(4) data after the buffer memory compression.
It is further characterized in that described step (1) buffer memory data to be compressed use MUX control data to be compressed to store in two data storage devices.
Described step (2) is carried out compression encoding process to string data and is comprised:
During packed data, get a certain amount of data and carry out hash conversion, in Hash table, search, if do not find, a certain amount of data of then getting are fresh characters, with the non-block code of fresh character coding output, if find, a certain amount of data of then getting are repeat character string, seek maximum matching length as data head, export non-block code with the repeat character (RPT) string encoding;
In the compression process, the signal according to Hash table and dictionary chained list feedback carries out corresponding read operation to data to be compressed, and the data in Hash table and the dictionary chained list are upgraded accordingly;
Alternately initialization and two Hash tables of use and two dictionary chained lists in the compression process.
The random length data procedures of splicing of described step (3) comprising:
Utilize shift register that random length data are spliced into fixed-length data;
Calculate the data amount check after compressing, compare with data amount check before the compression, choose suitable compact model, if the data amount check after the compression is more than the front data amount check of compression, choose the mode of direct storage, from buffer memory data to be compressed, read, if the data amount check before the no more than compression of data amount check after the compression is chosen the mode of compression storage;
Add the data block head after compressing;
Data trailer after the compression is carried out byte-aligned to be processed;
Process with the packed data parallel work-flow, improved the efficient of compression.
Data procedures after the compression of described step (4) buffer memory comprises:
Use the data writing MUX, the data after selection will be compressed write one of them data storage device in two data storage devices;
If a data block has been finished compression, then utilize the sense data MUX, select to read the data in the corresponding data storage device.
The present invention realizes the LZ77 compression by a kind of hardware, can effectively improve the efficient of LZ77 compression algorithm, and CPU is freed from Data compression.
Description of drawings
Fig. 1 is the structural representation that hardware LZ77 provided by the invention compresses the realization system;
Fig. 2 is the structural representation that hardware LZ77 provided by the invention compresses an embodiment of realization system;
Fig. 3 is the structural representation that data write the embodiment of cache module in the implementation example of hardware LZ77 provided by the invention compression realization system;
Fig. 4 is the structural representation that hardware LZ77 provided by the invention compresses another embodiment of realization system;
Fig. 5 is the structural representation of the embodiment of compression algorithm module in the implementation example of hardware LZ77 provided by the invention compression realization system;
Fig. 6 is the structural representation that hardware LZ77 provided by the invention compresses another embodiment of realization system;
Fig. 7 is the structural representation of the embodiment of dictionary module in the implementation example of hardware LZ77 provided by the invention compression realization system;
Fig. 8 is the structural representation that hardware LZ77 provided by the invention compresses another embodiment of realization system;
Fig. 9 is the structural representation of the embodiment of non-fixed length code element concatenation module in the implementation example of hardware LZ77 provided by the invention compression realization system;
Figure 10 is the structural representation that hardware LZ77 provided by the invention compresses another embodiment of realization system;
Figure 11 is the embodiment structural representation of data reading cache module in the implementation example of hardware LZ77 provided by the invention compression realization system;
Figure 12 is the structural representation that hardware LZ77 provided by the invention compresses another embodiment of realization system;
Figure 13 is the structural representation of the embodiment of Hash table module or dictionary chained list module in the implementation example of hardware LZ77 provided by the invention compression realization system;
Figure 14 is the structural representation that hardware LZ77 provided by the invention compresses another embodiment of realization system;
Figure 15 be in the implementation example of hardware LZ77 provided by the invention compression realization system non-fixed length code element to the structural representation of the embodiment of fixed length code element modular converter;
Figure 16 is the flow chart of hardware LZ77 compression implementation method provided by the invention;
Figure 17 is the flow chart of another embodiment of hardware LZ77 compression implementation method provided by the invention;
Figure 18 is the flow chart of another embodiment of hardware LZ77 compression implementation method provided by the invention;
Figure 19 is the flow chart of another embodiment of hardware LZ77 compression implementation method provided by the invention;
Figure 20 is the flow chart of another embodiment of hardware LZ77 compression implementation method provided by the invention;
The hardware LZ77 that Figure 21 provides for bold and unrestrained name compresses the flow chart of a concrete application example of implementation method,
Embodiment
With an exemplary embodiment of the present invention the present invention is carried out comprehensively careful description and explanation with reference to the accompanying drawings.
Fig. 1 is the structural representation that hardware LZ77 provided by the invention compresses the realization system.
As shown in Figure 1, hardware LZ77 compression realization system 100 comprises: PCIE interface module 101, data access directly module DMA102, data packing parse module 103, data write cache module 104, compression algorithm module 105, dictionary module 106, non-fixed length code element concatenation module 107, data reading cache module 108.
Wherein, PCIE interface module 101 be used for to realize and the communicating by letter of host computer.For example, the register that host computer PCI allocation E is corresponding realizes that host computer and hardware LZ77 compress communicating by letter of realization inter-system data.
Data access directly module DMA102 is used for realizing the direct access of data.For example, according to address and the data length of host computer configuration, from disk or data storage device, from the data of the address read fetching measured length of appointment, perhaps think to write in the address of appointment the data of designated length.
Data packing parse module 103, the group bag to packet when being used for realizing data communication is conciliate package operation.For example, during from the host computer reading out data, analytical propagation to data write the packet of cache module, remove data packet head, packet tail and data check information; During to the host computer data writing, the data that read are added data packet head, packet tail and check information from the data reading cache module.
Data write cache module 104, are used for buffer memory data to be compressed.For example, write data to be compressed from the interface of module, write cache module 104 through data, can read data to be compressed with ping-pong, reduce reading the time of data to be compressed.
Non-fixed length code element concatenation module 107 is used for the data block with random length code high speed splicing Cheng Dingchang.For example, become random length LZ77 coding after a certain amount of string data process compression algorithm module 105, non-fixed length code element concatenation module 107 can be spliced into these random length LZ77 code high speed the data of fixed length.
Data reading cache module 108 is used for the data after buffer memory compresses.For example, a data block has been finished data compression, utilize the sense data MUX, the data in the corresponding data storage device are read in selection, simultaneously, utilize the data writing MUX, the data after selection will be compressed are written in the another one data storage device, utilize two storage devices alternately implementing reading and writing read the ping-pong operation of packed data.
Fig. 2 is the structural representation that hardware LZ77 provided by the invention compresses an embodiment of realization system.
As shown in Figure 2, hardware LZ77 compression realization system 200 comprises: PCIE interface module 201, data access directly module (DMA) 202, data packing parse module 203, data write cache module 204, compression algorithm module 205, dictionary module 206, non-fixed length code element concatenation module 207, data reading cache module 208; Wherein the PCIE interface module 201, data access directly module (DMA) 202, data packing parse module 203, compression algorithm module 205, dictionary module 206, non-fixed length code element concatenation module 207, data reading cache module 208 can be respectively and the PCIE interface module 101 shown in Fig. 1, data access directly module (DMA) 102, data packing parse module 103, compression algorithm module 105, dictionary module 106, non-fixed length code element concatenation module 107, data reading cache module 108 has identical structure, its concrete technology contents of no longer too much elaboration here.
As shown in Figure 2, data write cache module 204 and further comprise: data storage device 2041, data storage device 2042 and MUX 2043.
Wherein, data storage device 2041 and data storage device 2042 are used for storing data to be compressed.For example, can adopt the known data storage device of these professional recognitions such as random-access memory (ram) or pushup storage (FIFO) to realize that the data width of data storage device (Width) and data depth (Depth) can dispose according to data demand.
Fig. 3 is the embodiment structural representation that data write cache module in the implementation example of hardware LZ77 provided by the invention compression realization system.
As shown in Figure 3, data write buffer memory 300 modules and comprise: data storage device 301, data storage device 302 and MUX 303.Concrete being operating as in embodiment, after the initialization of hardware LZ77 compressibility, system will begin to write cache module 300 interfaces to data and write data to be compressed, select data to be compressed are written in two data storage devices one by MUX (MUX) 303, when a data storage device data write full, simultaneously another data storage device is when data have been read sky, in another data storage device, write new data to be compressed, alternate cycles data writing in two data storage devices so, with the ping-pong operation of realizing that data write, this process will last till till the whole end of transmissions of all data blocks.
From above-mentioned narration, as seen, write the height continuity that buffer memory 300 can realize that data write by data, improved in the efficient of data compression hour hands to the data read operation.
Fig. 4 is the structural representation that hardware LZ77 provided by the invention compresses another embodiment of realization system.
As shown in Figure 4, hardware LZ77 compression realization system 400 comprises: PCIE interface module 401, data access directly module (DMA) 402, data packing parse module 403, data write cache module 404, compression algorithm module 405, dictionary module 406, non-fixed length code element concatenation module 407, data reading cache module 408; Wherein the PCIE interface module 401, data access directly module (DMA) 402, data packing parse module 403, data write cache module 404, dictionary module 406, non-fixed length code element concatenation module 407, data reading cache module 408 can be respectively and the PCIE interface module 101 shown in Fig. 1, data access directly module (DMA) 102, data packing parse module 103, data write cache module 104, dictionary module 106, non-fixed length code element concatenation module 107, data reading cache module 108 has identical structure, its concrete technology contents of no longer too much elaboration here.
As shown in Figure 4, compression algorithm module 405 further comprises: data are kept in module 4051, dictionary read module 4052, maximum length matching module 4053, LZ77 coding module 4054 and dictionary updating module 4055.
Wherein, data are kept in module 4051, are used for interim temporary a certain amount of data.For example, the data that read m Byte are temporarily stored in the temporary module 4051 of data, are convenient to the efficient hash conversion of compression algorithm module 405.
Dictionary read module 4052 is for generation of read control signal and the data that read in the dictionary of dictionary.For example, after data in the temporary module 4051 of data are carried out hash conversion, produce corresponding control signal and address signal, read the information of assigned address in the dictionary, judge according to the feedback information of dictionary whether the data in the temporary module 4051 of data are repeat character string, if then compress processing; If not, then do not compress processing.
Maximum length matching module 4053 is used for calculating the maximum repeat length when character string occurring.For example, " ab " and " abcde " arranged in the data of having compressed, data to be compressed are " abcde ", then can calculate the maximum repeat length of repeat character string by the maximum length matching module, have improved to a certain extent the performance of compression.
Fig. 5 is the structural representation of the embodiment of compression algorithm module in the implementation example of hardware LZ77 provided by the invention compression realization system.
As shown in Figure 5, compression algorithm module 500 comprises: data are kept in module 501, dictionary read module 502, maximum length matching module 503, LZ77 coding module 504 and dictionary updating module 505.
A certain amount of data of storage in the temporary module 501 of data, after carrying out hash conversion by dictionary read module 502 for data wherein, produce corresponding control signal reading out data from dictionary, whether the signal determining data according to the dictionary feedback are repeat character string, if, then start maximum length matching module 503, calculate the maximum repeat length of repeat character string, start LZ77 coding module 504 and carry out the repeat character (RPT) string encoding, if not, then start LZ77 coding module 504 and carry out fresh character coding, simultaneously, write the more new data of requirement in the dictionary according to the signal of dictionary feedback.
By above-mentioned as seen in compression algorithm module 500, to the read-write operation of data and frequent, the temporary module 501 of data has reduced the read-write operation of partial data in certain degree, improved the efficient of hash conversion, simultaneously compression algorithm module 500 be one comparatively independently module can with other module parallel running, therefore with the Function of data writing buffer memory 300 to a more satisfactory state.
Fig. 6 is the structural representation that hardware LZ77 provided by the invention compresses another embodiment of realization system.
As shown in Figure 6, hardware LZ77 compression realization system 600 comprises: PCIE interface module 601, data access directly module (DMA) 602, data packing parse module 603, data write cache module 604, compression algorithm module 605, dictionary module 606, non-fixed length code element concatenation module 607, data reading cache module 608; Wherein the PCIE interface module 601, data access directly module (DMA) 602, data packing parse module 603, data write cache module 604, compression algorithm module 605, non-fixed length code element concatenation module 607, data reading cache module 608 can be respectively and the PCIE interface module 101 shown in Fig. 1, data access directly module (DMA) 102, data packing parse module 103, data write cache module 104, compression algorithm module 105, non-fixed length code element concatenation module 107, data reading cache module 108 has identical structure, its concrete technology contents of no longer too much elaboration here.
As shown in Figure 6, dictionary module 606 further comprises: Hash table module 6061, dictionary chained list module 6062, index initialization module 6063 and chained list initialization module 6064.
Wherein, Hash table module 6061 is used for storing up-to-date historical dictionary information.For example, data that " ab " [1] was compressed before being, and " ab " [2] are just good data of compression, the information that then Hash table module 6061 can up-to-date ab are the assigned address of information updating prescribed storage means in the Hash table module 6061 of " ab " [2].
Dictionary chained list module 6062, the historical dictionary information that is used for storing other.For example, the data that " ab " [1] was compressed before being, and " ab " [2] are the good data of just compression, the Hash table module 6061 i.e. information of " ab " [1] of information that can abandon older ab then, this moment, the information of the ab that dictionary chained list module 6062 reception Hash table modules 6061 abandon was the information of " ab " [1], was stored to the assigned address of prescribed storage means in the dictionary chained list module 6062.
Chained list initialization module 6064 is used for initialization dictionary chained list module 6062.For example, when the compression of finishing a data block, when needing another data block of compression, need to empty operation to the dictionary chained list, chained list initialization module 6064 empties initialization operation to dictionary chained list module 6062 accordingly at this moment.
Fig. 7 is the structural representation of the embodiment of dictionary module in the implementation example of hardware LZ77 provided by the invention compression realization system.
As shown in Figure 7, dictionary module 700 comprises: Hash table module 701, dictionary chained list module 702, index initialization module 703 and chained list initialization module 704.
During packed data, Hash table module 701 feedback data, fresh character or repeat character string when judging data to be compressed, in case after determining, Hash table module 701 receives up-to-date dictionary information, dictionary information that will be older abandons; Dictionary chained list module 702 receives the data that Hash table module 701 abandons, and with the historical dictionary information of its storage as other; When hardware LZ77 compressibility initialization or when finishing a data block compression, index initialization module 703 and chained list initialization module 704 start, and the data in Hash table module 701 and the dictionary chained list module 702 are emptied.
Fig. 8 is the structural representation that hardware LZ77 provided by the invention compresses another embodiment of realization system.
As shown in Figure 8, hardware LZ77 compression realization system 800 comprises: PCIE interface module 801, data access directly module (DMA) 802, data packing parse module 803, data write cache module 804, compression algorithm module 805, dictionary module 806, non-fixed length code element concatenation module 807, data reading cache module 808; Wherein PCIE interface module 801, data access directly module (DMA) 802, data packing parse module 803, data write cache module 804, compression algorithm module 805, dictionary module 806, data reading cache module 808 and can be respectively write cache module 104, compression algorithm module 105, dictionary module 106, data reading cache module 108 with the PCIE interface module 101 shown in Fig. 1, data access directly module (DMA) 102, data packing parse module 103, data and have identical structure, its concrete technology contents of no longer too much elaboration here.
As shown in Figure 8, non-fixed length code element concatenation module 800 further comprises: non-fixed length code element is selected module 8073 and the direct memory module 8074 of data to fixed length code element modular converter 8071, file size computing module 8072, compact model.
Wherein, non-fixed length code element is used for the data with random length data encoding splicing Cheng Dingchang to fixed length code element modular converter 8071.For example, random length data encoding A and B are arranged, wherein the A code length is 16, the B code length is 20, then become data and the remaining coding of fixed length through non-fixed length code element to fixed length code element modular converter 8071, the data of fixed length can be 8,32 or 64 etc., add data block head before the data block of while after compression, and polishing when the data block trail byte does not line up makes the data byte alignment.
File size computing module 8072 is used for the data amount check after the calculation document compression, and with compression before the file data number relatively.For example, the data amount check before the file data compression is 32K, and the data amount check after the compression is 20K.Two above information are compared, feed back corresponding state and signal and select module 8073. to compact model
Compact model is selected module 8073, be used for according to before the compressing file and the data amount check after the compression how much choose compact model.For example, if the data amount check before the data compression is 32K, the data amount check after the compression is 20K, then chooses the compression memory module; If the data amount check before the compression is 1K, the data amount check after the compression is 2K, then chooses direct memory module.
The direct memory module 8074 of data, when being used for selecting direct store compressed mode compression data to the operation of data.For example, be 1K if get the data number before the compression, the data amount check after the compression is 2K, compact model selects module 8073 to select direct memory module, the direct memory module 8074 of data starts, and directly writes from data and reads corresponding data the cache module, and add corresponding data block head.
Fig. 9 is the structural representation of the embodiment of non-fixed length code element concatenation module in the implementation example of hardware LZ77 provided by the invention compression realization system.
As shown in Figure 9, non-fixed length code element concatenation module 900 comprises: non-fixed length code element is selected module 903 and the direct memory module 904 of data to fixed length code element modular converter 901, file size computing module 902, compact model.
When having non-fixed length code element to input from the interface of non-fixed length code element concatenation module 900, non-fixed length code element to fixed length code element modular converter 901 non-fixed length code element is kept in and concatenation until export can form the fixed length code element time, simultaneously file size computing module 902 calculates the number through the data of non-fixed length code element after 901 processing of fixed length code element modular converter, and with the compression before data amount check compare, tell the result compact model to select module 903, choose suitable compact model, if what compact model selection module 903 was chosen is direct memory module, the direct memory module 904 of log-on data.
By above-mentioned content as seen, be the pipeline organization of a parallel running fully between non-fixed length code element concatenation module 900 and the compression algorithm module, accelerated so greatly the efficient of data compression.
Figure 10 is the structural representation that hardware LZ77 provided by the invention compresses another embodiment of realization system.
As shown in figure 10, hardware LZ77 compression realization system 1000 comprises: PCIE interface module 1001, data access directly module (DMA) 1002, data packing parse module 1003, data write cache module 1004, compression algorithm module 1005, dictionary module 1006, non-fixed length code element concatenation module 1007, data reading cache module 1008; Wherein the PCIE interface module 1001, data access directly module (DMA) 1002, data packing parse module 1003, data write cache module 1004, compression algorithm module 1005, dictionary module 1006, non-fixed length code element concatenation module 1007 can be respectively and the PCIE interface module 101 shown in Fig. 1, data access directly module (DMA) 102, data packing parse module 103, data write cache module 104, compression algorithm module 105, dictionary module 106, non-fixed length code element concatenation module 107 has identical structure, its concrete technology contents of no longer too much elaboration here.
As shown in figure 10, data reading cache module 1008 comprises: data storage device 10081, data storage device 10082, data writing MUX 10083 and sense data MUX 10084.
Wherein, data storage device 10081 and data storage device 10082 for the data after the store compressed, can adopt the known storage device of these professional persons such as random-access memory (ram) or pushup storage (FIFO) to realize.
Data writing MUX 10083, the data after being used for selecting to compress write one of them data storage device of two data storage devices.For example, to deposit data after the compression in data storage device 10081, signal and the data path between data writing MUX 10083 gated data interfaces and the data storage device 10081 then, thus can write data after the compression in the data storage device 10081.
Figure 11 is the structural representation of the embodiment of data reading cache module in the implementation example of hardware LZ77 provided by the invention compression realization system.
As shown in figure 11, data reading cache module 1100 comprises: data storage device 1101, data storage device 1102, write MUX 1103 and read MUX 1104.
When just when executing data compresses, select data storage devices 1101 or data storage device 1102 to be used for data after the store compressed by writing MUX 1103, if there is the data reading signal to produce simultaneously, then select the another one data storage device by reading MUX 1104, and read the data in this data storage device.For example, writing MUX 1103 selects data storage device 1101 for the data after the store compressed, if there is this moment read output signal to produce, then select data storage device 1102 by reading MUX 1104, and general's data reading wherein, data in data storage device 1102 are all read, wait is finished a data block and must be compressed, be when the data in 1101 have been ready to read in the data storage device, writing MUX 1103 transfers to select data storage 1102 for the data after the store compressed, so repeatedly, realized the table tennis of data reading, the efficient when having improved data reading.
By said process as seen, data reading cache module 1100 also is parallel running with compression algorithm module and non-fixed length code element concatenation module, and this has also accelerated the efficient of compression to a certain extent.
Figure 12 is the structural representation that hardware LZ77 provided by the invention compresses another embodiment of realization system.
As shown in figure 12, hardware LZ77 compression realization system 1200 comprises: PCIE interface module 1201, data access directly module (DMA) 1202, data packing parse module 1203, data write cache module 1204, compression algorithm module 1205, Hash table module 12061, dictionary chained list module 12062, index initialization module 12063, chained list initialization module 12064, non-fixed length code element concatenation module 1207, data reading cache module 1208; Wherein the PCIE interface module 1201, data access directly module (DMA) 1202, data packing parse module 1203, data write cache module 1204, compression algorithm module 1205, index initialization module 12063, chained list initialization module 12064, non-fixed length code element concatenation module 1207, data reading cache module 1208 can be respectively and the PCIE interface module 601 shown in Fig. 6, data access directly module (DMA) 602, data packing parse module 603, data write cache module 604, compression algorithm module 605, index initialization module 6063, chained list initialization module 6064, non-fixed length code element concatenation module 607, data reading cache module 608 has identical structure, its concrete technology contents of no longer too much elaboration here.
As shown in figure 12, Hash table module 12061 further comprises: data storage device 120611, data storage device 120612, MUX 120613 and initialization MUX 120614; Dictionary chained list module 12062 further comprises: data storage device 120621, data storage device 120622, MUX 120623 and initialization MUX 120624.
Wherein data storage device 120611 and data storage device 120612 are used for storing up-to-date historical dictionary information.Data storage device 120621 and data storage device 120622, other historical dictionary information that are used for storing dictionary.
MUX 120613 is for the up-to-date historical dictionary information of one of them data storage device stores of two data storage devices of choice for use; MUX 120623 is used for other the historical dictionary information of one of them data storage device stores of two data storage devices of choice for use.
Initialization multifinder 120614 is used for the data storage device that two data storage devices are initially changed in selection, and alternately two data storage devices of initialization improve the efficient of compression; Same initialization MUX 120624 also is to play this effect.
Figure 13 is the structural representation of the embodiment of Hash table module or dictionary chained list module in the implementation example of hardware LZ77 provided by the invention compression realization system.
As shown in figure 13, Hash table module or dictionary chained list module 1300 comprise: data storage device 1301, data storage device 1302, MUX 1303 and initialization institute road selector 1304.
During packed data, MUX 1303 need to select data storage device 1301 or the data storage device 1302 of use, with the data storage device stores dictionary information of selecting, simultaneously initialization MUX 1304 selects another data storage device that the dictionary information in the data storage device is emptied operation.
By above-mentioned process as seen, in the process of compression by being used alternatingly and initialization of two pairs of dictionary datas, the efficient of compression that added piece.
Figure 14 is the structural representation that hardware LZ77 provided by the invention compresses another embodiment of realization system.
As shown in figure 14, hardware LZ77 compression realization system 1400 comprises: PCIE interface module 1401, data access directly module (DMA) 1402, data packing parse module 1403, data write cache module 1404, compression algorithm module 1405, dictionary module 1406, non-fixed length code element to fixed length code element modular converter 14071, file size computing module 14072, compact model selection module 14073, the direct memory module 14074 of data, data reading cache module 1408; Wherein the PCIE interface module 1401, data access directly module (DMA) 1402, data packing parse module 1403, data write cache module 1404, compression algorithm module 1405, dictionary module 1406, file size computing module 14072, compact model is selected module 14073, the direct memory module 14074 of data, data reading cache module 1408 can be respectively and the PCIE interface module 801 shown in Fig. 8, data access directly module (DMA) 802, data packing parse module 803, data write cache module 804, compression algorithm module 805, dictionary module 806, file size computing module 8072, compact model is selected module 8073, the direct memory module 8074 of data, data reading cache module 808 has identical structure, its concrete technology contents of no longer too much elaboration here.
As shown in figure 14, non-fixed length code element further comprises to fixed length code element modular converter 14071: non-fixed length code element is spliced into shorter fixed-length data module 140711 and is spliced into than long data module 140712 than short data.
Wherein, non-fixed length code element is spliced into shorter fixed-length data module 140711, is used for the non-block code of compression algorithm module output is spliced into short fixed-length data.For example, have respectively in the compression algorithm module output encoder: 1 bit, 3 bits, 5 bits, 7 bits, 16bit need to be spliced into these data the whole font data of 8 bits.
Be spliced into than long data module 140712 than short data, be spliced into long data for the shorter fixed-length data that non-fixed length code element is spliced into shorter fixed-length data module 140711 outputs.For example, the data of 8 bits are spliced into the data of 32 bits or 64 bits.
Figure 15 be in the implementation example of hardware LZ77 provided by the invention compression realization system non-fixed length code element to the structural representation of the embodiment of fixed length code element modular converter.
As shown in figure 15, non-fixed length code element comprises to fixed length code element modular converter 1500: non-fixed length code element is spliced into shorter fixed-length data module 1501 and is spliced into than long data module 1502 than short data.
When carrying out non-fixed length code element to the splicing of fixed length code element, non-fixed-length data reaches non-fixed length code element to the Data Input Interface of fixed length code element modular converter 1500, be spliced into shorter fixed-length data module 1501 by non-fixed length code element first random length coding is spliced into short fixed-length data, have again and be spliced into the fixed-length data that to lack than long data module 1502 than short data and be spliced into the long fixed-length data output that data transmission format requires that meets, in this process, the splicing of data is streamlineds, have the at a high speed function of splicing, meanwhile also reduced greatly the consumption of hardware resource.
Figure 16 is the flow chart of hardware LZ77 compression implementation method provided by the invention.
The flow process 1600 of hardware LZ77 compression implementation method comprises as shown in figure 16: step 1601, step 1602, step 1603 and step 1604.
Wherein, step 1601, the data that buffer memory is to be compressed.For example, utilize MUX 303, data storage device 301 and data storage device 302 to realize the operation that data table tennis writes.
Step 1602 is encoded to string data.For example, from temporal data module 501, fetch data and send into dictionary and read and carry out hash conversion in the mould 502, judge institute's fresh character that fetches data or repeat character string according to the feedback signal of Hash table module 701, calculate the pointer of maximum matching length and maximum matched character string in conjunction with maximum length matching module 503 and dictionary chained list module 702 after, send into and carry out the non-block code output of LZ77 in the LZ77 coding module 504; Simultaneously, the data message that dictionary updating module 505 will have been compressed is updated in Hash table module 701 and the dictionary linked list data module 702.
Step 1603 is spliced random length data.For example, non-fixed-length data is delivered to non-fixed length code element and is spliced into the shorter fixed-length data of shorter fixed-length data module 1501 outputs, again by the data length that is spliced into than short data than the 1502 output the transmission of data appointments of long data module, select data after module 903 and the direct memory module 904 of data are selected suitable compact model output squeezing in conjunction with file size computing module 902, compact model, simultaneously, by adding data block head information in the data of file size computing module 902 after the compression, select module 903 that the data block tail is carried out byte-aligned by compact model and process.
Step 1604, the data after the buffer memory compression.For example, the data after the compression are delivered to the sense data cache module, are stored in data storage device 1101 or the data storage device 1102 by the data that write after MUX 1103 is selected to compress; Simultaneously, select the data reading after the compression in data storage device 1101 or the data storage device 1102 by reading MUX 1104.
Figure 17 is the flow chart of another embodiment of hardware LZ77 compression implementation method provided by the invention.
As shown in figure 17, the flow process of hardware LZ77 compression implementation method comprises: step 17011, step 1702, step 1703 and step 1704, wherein step 1702, step 1703 and step 1704 are carried out identical or similar operation with step 1602, step 1603 and step 1604 shown in Figure 16 respectively, no longer carry out too much narration at this.
Figure 18 is the flow chart of another embodiment of hardware LZ77 compression implementation method provided by the invention.
As shown in figure 18, the flow process 1800 of hardware LZ77 compression implementation method comprises: step 1801, step 18021, step 18022, step 18023, step 1803 and step 1804, wherein step 1801, step 1803 and step 1804 respectively step 1601, step 1603 and the step 1604 among Figure 16 carry out identical or similar operation, no longer carry out too much narration at this.
Seen by said process, by alternately initialization and two Hash tables of use and dictionary chained list can realize that the ping-pong operation of dictionary uses, and has improved the efficient of compression.
Figure 19 is the flow chart of another embodiment of a kind of hardware LZ77 compression implementation method provided by the invention.
The flow process 1900 of hardware LZ77 compression implementation method comprises as shown in figure 19: step 1901, step 1902, step 19031, step 19032, step 19033, step 19034 and step 1904, wherein step 1901, step 1902 and step 1904 respectively with Figure 16 in step 1601, step 1602 and step 1604 carry out identical or similar operation, no longer carry out too much narration at this.
Above-mentioned process and the step 1602 among Figure 16 are executed in parallel, and as seen, such structure can improve the efficient of compression greatly.
Figure 20 is the flow chart of another embodiment of hardware LZ77 compression implementation method provided by the invention.
As shown in figure 20, the flow process 2000 of hardware LZ77 compression implementation method comprises: step 2001, step 2002, step 2003, step 20041 and step 20042, wherein step 2001, step 2002 and step 2003 respectively with Figure 16 in step 1601, step 1602 and step 1603 carry out identical or similar operation, no longer carry out too much elaboration at this.
Figure 21 is the flow chart of a concrete application example of hardware LZ77 compression implementation method provided by the invention.
As shown in figure 21, the flow process 2100 of hardware LZ77 compression implementation method comprises: step 2101, step 2102, step 2103, step 2104, step 2105, step 2106, step 2107, step 2108, step 2109 and step 2010.
With reference to the exemplary description of the invention described above, those skilled in the art can understand the present invention and have following several superiority:
The invention provides hardware LZ77 compression and realize system and method, adopt FPGA(Field Programmable Gate Arry) realization LZ77 compression function, realize that by the method that adopts data to write cache module, data reading cache module number removes the ping-pong operation that writes and read, improved the efficient of LZ77 compression.
The invention provides hardware LZ77 compression and realize system and method, adopt to be used alternatingly and initialization Hash table module and dictionary chained list module, realized the parallel work-flow of dictionary initialization and compression, improved to a certain extent the efficient of LZ77 compression.
The invention provides hardware LZ77 compression and realize system and method, adopt LZ77 coding module and non-fixed length code element concatenation module to realize the parallel work-flow that LZ77 compressed encoding and non-fixed length code element are spliced at a high speed, further promoted the efficient of LZ77 compression.
The invention provides hardware LZ77 compression and realize system and method, adopt compact model to select module and the direct memory module of data to realize, data are directly stored and are compressed two kinds of patterns of storage, have guaranteed the compression ratio of LZ77 compression, have guaranteed the performance of compression.
Illustrate and describe although the present invention specializes some specific examples herein, yet the present invention is not restricted to the details of institute's crucial point, because not departing from spirit of the present invention and the scope and equivalency range in claim, can make multiple improvement and structural change.Therefore, wide region and as illustrated in the claim more in some sense the explanation consistent with scope of the present invention additional what is claimed is suitable.
Claims (13)
1. a hardware LZ77 compresses the realization system, it is characterized in that this system comprises:
The PCIE interface module be used for to realize and the communicating by letter of host computer;
Data access directly module DMA is used for realizing the direct access of data;
Data packing parse module, the group bag to data when being used for realizing data communication is conciliate package operation;
Data write cache module, are used for buffer memory data to be compressed;
The compression algorithm module is used for realizing searching of repeat character string, calculates the matching length of repeat character string, carries out the LZ77 coding;
Dictionary module is used for storing historical character string;
Non-fixed length code element concatenation module is for the data block that random length code high speed is spliced into fixed length;
The data reading cache module is used for the data after buffer memory compresses.
2. hardware LZ77 according to claim 1 compresses the realization system, it is characterized in that described data write cache module and comprise:
Two data storage devices such as random access memory ram or pushup storage FIFO are used for storing data to be compressed;
MUX is used for one of them data storage device that selection is stored in data to be compressed two data storage devices;
Realize the ping-pong operation that data write by MUX and two data storage devices.
3. hardware LZ77 according to claim 1 compresses the realization system, it is characterized in that described compression algorithm module comprises:
Data are kept in module, are used for interim temporary a certain amount of data;
The dictionary read module is for generation of read control signal and the data that read in the dictionary of dictionary;
The maximum length matching module is used for calculating the maximum repeat length when repeat character string occurring;
The LZ77 coding module is used for treating packed data and encodes accordingly;
The dictionary updating module is for generation of the write control signal of dictionary and the data in the renewal dictionary.
4. hardware LZ77 according to claim 1 compresses the realization system, it is characterized in that described dictionary module comprises:
The Hash table module is used for storing up-to-date historical dictionary information;
Dictionary chained list module, the historical dictionary information that is used for storing other;
The index initialization module is used for initialization Hash table module;
The chained list initialization module is used for initialization dictionary chained list module.
5. hardware LZ77 according to claim 1 compresses the realization system, it is characterized in that described non-fixed length code element concatenation module comprises:
Non-fixed length code element is used for random length data encoding is spliced into the data of fixed length to fixed length code element modular converter;
The file size computing module is used for the data amount check after the calculation document compression, and with compression before the file data number relatively;
Compact model is selected module, according to before the compressing file and the data amount check after the compression how much choose compact model;
The direct memory module of data is used for when the direct store compressed pattern of data selection the operation to data.
6. hardware LZ77 according to claim 1 compresses the realization system, it is characterized in that described data reading cache module comprises:
Two data storage devices such as random access memory ram or pushup storage FIFO are for the data after the store compressed;
The data writing MUX, the data after being used for selecting to compress write one of them data storage device of two data storage devices;
The sense data MUX be used for to select to read the data in one of them data storage devices of two data storage devices.
7. hardware LZ77 according to claim 4 compresses the realization system, it is characterized in that described Hash table module comprises:
Two data storage devices such as random access memory ram or Content Addressable Memory CAM are used for storing up-to-date historical dictionary information;
Write MUX, be used for the up-to-date historical dictionary information of one of them data storage device stores of two data storage devices of choice for use;
Read MUX, be used for to select to read the dictionary information in one of them data storage devices of two data storage devices;
The initialization MUX, for a data storage device selecting two data storage devices of initialization, alternately two data storage devices of initialization improve the efficient of compression;
Described dictionary chained list module comprises:
Two data storage devices such as random access memory ram or Content Addressable Memory CAM, the historical dictionary information that is used for storing other;
Write MUX, be used for the up-to-date historical dictionary information of one of them data storage device stores of two data storage devices of choice for use;
Read MUX, be used for to select to read the dictionary information in one of them data storage devices of two data storage devices;
The initialization MUX, for a data storage device selecting two data storage devices of initialization, alternately two data storage devices of initialization improve the efficient of compression.
8. hardware LZ77 according to claim 5 compresses the realization system, it is characterized in that described non-fixed length code element comprises to fixed length code element modular converter:
Non-fixed length code element is spliced into shorter fixed-length data module, is used for the non-block code of compression algorithm module output is spliced into short fixed-length data;
Be spliced into than the long data module than short data, be used for being spliced into long data with what non-fixed length code element was spliced into shorter fixed-length data module output than short data, be spliced into the parallel running of shorter fixed-length data module with non-fixed length code element, accelerated the process of splicing.
9. a hardware LZ77 compression implementation method comprises the steps:
(1) buffer memory data to be compressed;
(2) string data is carried out compressed encoding;
(3) splice random length data;
(4) data after the buffer memory compression.
10. hardware LZ77 compression implementation method according to claim 9 is characterized in that, described step (1) buffer memory data to be compressed use MUX control data to be compressed to store in two data storage devices.
11. hardware LZ77 compression implementation method according to claim 9 is characterized in that described step (2) is carried out compression encoding process to string data and comprised:
During packed data, get a certain amount of data and carry out hash conversion, in Hash table, search, if do not find, a certain amount of data of then getting are fresh characters, with the non-block code of fresh character coding output, if find, a certain amount of data of then getting are repeat character string, seek maximum matching length as data head, export non-block code with the repeat character (RPT) string encoding;
In the compression process, the signal according to Hash table and dictionary chained list feedback carries out corresponding read operation to data to be compressed, and the data in Hash table and the dictionary chained list are upgraded accordingly;
Alternately initialization and two Hash tables of use and two dictionary chained lists in the compression process.
12. hardware LZ77 compression implementation method according to claim 9 is characterized in that the random length data procedures of splicing of described step (3) comprising:
Utilize shift register that random length data are spliced into fixed-length data;
Calculate the data amount check after compressing, compare with data amount check before the compression, choose suitable compact model, if the data amount check after the compression is more than the front data amount check of compression, choose the mode of direct storage, from buffer memory data to be compressed, read, if the data amount check before the no more than compression of data amount check after the compression is chosen the mode of compression storage;
Add the data block head after compressing;
Data trailer after the compression is carried out byte-aligned to be processed;
Process with the packed data parallel work-flow, improved the efficient of compression.
13. hardware LZ77 compression implementation method according to claim 9 is characterized in that the data procedures after the compression of described step (4) buffer memory comprises:
Use the data writing MUX, the data after selection will be compressed write one of them data storage device in two data storage devices;
If a data block has been finished compression, then utilize the sense data MUX, select to read the data in the corresponding data storage device.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2012104553279A CN103023509A (en) | 2012-11-14 | 2012-11-14 | Hardware LZ77 compression implementation system and implementation method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2012104553279A CN103023509A (en) | 2012-11-14 | 2012-11-14 | Hardware LZ77 compression implementation system and implementation method thereof |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN103023509A true CN103023509A (en) | 2013-04-03 |
Family
ID=47971688
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2012104553279A Pending CN103023509A (en) | 2012-11-14 | 2012-11-14 | Hardware LZ77 compression implementation system and implementation method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103023509A (en) |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103095305A (en) * | 2013-01-06 | 2013-05-08 | 中国科学院计算技术研究所 | System and method for hardware LZ77 compression implementation |
| CN103368579A (en) * | 2013-07-09 | 2013-10-23 | 杭州东方通信软件技术有限公司 | Method and system for compressing WLAN (Wireless Local Area Network) management equipment data |
| CN103475375A (en) * | 2013-09-04 | 2013-12-25 | 东南大学 | LZ77 compression algorithm hardware acceleration system and acceleration method |
| CN104202054A (en) * | 2014-09-16 | 2014-12-10 | 东南大学 | Hardware LZMA (Lempel-Ziv-Markov chain-Algorithm) compression system and method |
| CN104749429A (en) * | 2015-04-08 | 2015-07-01 | 四川拓普测控科技有限公司 | Lightning overvoltage signal losslessly compressing and recording system |
| CN105007083A (en) * | 2015-08-13 | 2015-10-28 | 东南大学 | Method for storing output result of LZ77 compression algorithm |
| CN105183557A (en) * | 2015-08-26 | 2015-12-23 | 东南大学 | Configurable data compression system based on hardware |
| CN105207678A (en) * | 2015-09-29 | 2015-12-30 | 东南大学 | Hardware realizing system for improved LZ4 compression algorithm |
| CN105553483A (en) * | 2015-12-09 | 2016-05-04 | 广东顺德中山大学卡内基梅隆大学国际联合研究院 | Method and device for generating LZ77 |
| CN105610447A (en) * | 2015-10-29 | 2016-05-25 | 吴均 | LZ77 algorithm based zonal coding and compression method |
| CN105677686A (en) * | 2014-11-21 | 2016-06-15 | 高德软件有限公司 | Road coding method and device |
| CN108141225A (en) * | 2016-07-14 | 2018-06-08 | 华为技术有限公司 | Use the generic data compression of SIMD engines |
| CN109361398A (en) * | 2018-10-11 | 2019-02-19 | 南威软件股份有限公司 | It is a kind of based on parallel and the pipeline design LZ process hardware compression method and system |
| CN111178490A (en) * | 2019-12-31 | 2020-05-19 | 北京百度网讯科技有限公司 | Data output method, data acquisition method, data output device, data acquisition device and electronic equipment |
| CN111382855A (en) * | 2018-12-28 | 2020-07-07 | 上海寒武纪信息科技有限公司 | Data processing device, method, chip and electronic equipment |
| CN112199343A (en) * | 2020-12-03 | 2021-01-08 | 南京烽火星空通信发展有限公司 | Data compression/decompression acceleration method for small data block scene |
| CN117119120A (en) * | 2023-10-25 | 2023-11-24 | 上海伯镭智能科技有限公司 | Cooperative control method based on multiple unmanned mine cars |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1786939A (en) * | 2005-11-10 | 2006-06-14 | 浙江中控技术有限公司 | Real-time data compression method |
| CN101572552A (en) * | 2009-06-11 | 2009-11-04 | 哈尔滨工业大学 | High-speed lossless data compression system based on content addressable memory |
| US20100058002A1 (en) * | 2008-08-27 | 2010-03-04 | Netapp, Inc. | System and method for file system level compression using compression group descriptors |
| CN202931289U (en) * | 2012-11-14 | 2013-05-08 | 无锡芯响电子科技有限公司 | Hardware LZ 77 compression implement system |
-
2012
- 2012-11-14 CN CN2012104553279A patent/CN103023509A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1786939A (en) * | 2005-11-10 | 2006-06-14 | 浙江中控技术有限公司 | Real-time data compression method |
| US20100058002A1 (en) * | 2008-08-27 | 2010-03-04 | Netapp, Inc. | System and method for file system level compression using compression group descriptors |
| CN101572552A (en) * | 2009-06-11 | 2009-11-04 | 哈尔滨工业大学 | High-speed lossless data compression system based on content addressable memory |
| CN202931289U (en) * | 2012-11-14 | 2013-05-08 | 无锡芯响电子科技有限公司 | Hardware LZ 77 compression implement system |
Non-Patent Citations (1)
| Title |
|---|
| 李雷定: "基于FPGA的数据实时无损压缩系统设计", 《中国优秀硕士学位论文全文数据库 信息科技辑》, no. 11, 15 November 2009 (2009-11-15) * |
Cited By (28)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103095305A (en) * | 2013-01-06 | 2013-05-08 | 中国科学院计算技术研究所 | System and method for hardware LZ77 compression implementation |
| CN103368579B (en) * | 2013-07-09 | 2016-08-17 | 杭州东方通信软件技术有限公司 | The compression method of a kind of WLAN Network Management Equipment data and system |
| CN103368579A (en) * | 2013-07-09 | 2013-10-23 | 杭州东方通信软件技术有限公司 | Method and system for compressing WLAN (Wireless Local Area Network) management equipment data |
| CN103475375A (en) * | 2013-09-04 | 2013-12-25 | 东南大学 | LZ77 compression algorithm hardware acceleration system and acceleration method |
| CN104202054A (en) * | 2014-09-16 | 2014-12-10 | 东南大学 | Hardware LZMA (Lempel-Ziv-Markov chain-Algorithm) compression system and method |
| CN105677686B (en) * | 2014-11-21 | 2019-06-21 | 高德软件有限公司 | A kind of road codes method and device |
| CN105677686A (en) * | 2014-11-21 | 2016-06-15 | 高德软件有限公司 | Road coding method and device |
| CN104749429A (en) * | 2015-04-08 | 2015-07-01 | 四川拓普测控科技有限公司 | Lightning overvoltage signal losslessly compressing and recording system |
| CN105007083A (en) * | 2015-08-13 | 2015-10-28 | 东南大学 | Method for storing output result of LZ77 compression algorithm |
| CN105183557A (en) * | 2015-08-26 | 2015-12-23 | 东南大学 | Configurable data compression system based on hardware |
| CN105183557B (en) * | 2015-08-26 | 2018-11-20 | 东南大学 | A kind of hardware based configurable data compression system |
| CN105207678B (en) * | 2015-09-29 | 2018-10-26 | 东南大学 | A kind of system for implementing hardware of modified LZ4 compression algorithms |
| CN105207678A (en) * | 2015-09-29 | 2015-12-30 | 东南大学 | Hardware realizing system for improved LZ4 compression algorithm |
| CN105610447B (en) * | 2015-10-29 | 2018-06-19 | 吴均 | Zonal coding compression method based on LZ77 algorithms |
| CN105610447A (en) * | 2015-10-29 | 2016-05-25 | 吴均 | LZ77 algorithm based zonal coding and compression method |
| CN105553483A (en) * | 2015-12-09 | 2016-05-04 | 广东顺德中山大学卡内基梅隆大学国际联合研究院 | Method and device for generating LZ77 |
| CN105553483B (en) * | 2015-12-09 | 2018-12-21 | 广东顺德中山大学卡内基梅隆大学国际联合研究院 | A kind of method and device generating LZ77 |
| CN108141225B (en) * | 2016-07-14 | 2020-10-27 | 华为技术有限公司 | General purpose data compression using SIMD engines |
| CN108141225A (en) * | 2016-07-14 | 2018-06-08 | 华为技术有限公司 | Use the generic data compression of SIMD engines |
| CN109361398B (en) * | 2018-10-11 | 2022-12-30 | 南威软件股份有限公司 | LZ process hardware compression method and system based on parallel and pipeline design |
| CN109361398A (en) * | 2018-10-11 | 2019-02-19 | 南威软件股份有限公司 | It is a kind of based on parallel and the pipeline design LZ process hardware compression method and system |
| CN111382855A (en) * | 2018-12-28 | 2020-07-07 | 上海寒武纪信息科技有限公司 | Data processing device, method, chip and electronic equipment |
| CN111382855B (en) * | 2018-12-28 | 2022-12-09 | 上海寒武纪信息科技有限公司 | Data processing device, method, chip and electronic equipment |
| CN111178490A (en) * | 2019-12-31 | 2020-05-19 | 北京百度网讯科技有限公司 | Data output method, data acquisition method, data output device, data acquisition device and electronic equipment |
| US11562241B2 (en) | 2019-12-31 | 2023-01-24 | Beijing Baidu Netcom Science and Technology Co., Ltd | Data output method, data acquisition method, device, and electronic apparatus |
| CN112199343A (en) * | 2020-12-03 | 2021-01-08 | 南京烽火星空通信发展有限公司 | Data compression/decompression acceleration method for small data block scene |
| CN117119120A (en) * | 2023-10-25 | 2023-11-24 | 上海伯镭智能科技有限公司 | Cooperative control method based on multiple unmanned mine cars |
| CN117119120B (en) * | 2023-10-25 | 2023-12-22 | 上海伯镭智能科技有限公司 | Cooperative control method based on multiple unmanned mine cars |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103023509A (en) | Hardware LZ77 compression implementation system and implementation method thereof | |
| CN202931289U (en) | Hardware LZ 77 compression implement system | |
| CN103095305A (en) | System and method for hardware LZ77 compression implementation | |
| US6829695B1 (en) | Enhanced boolean processor with parallel input | |
| CN102754078B (en) | Enhanced multi-processor waveform data exchange using compression and decompression | |
| CN102457283B (en) | A kind of data compression, decompression method and equipment | |
| CN102970043B (en) | A kind of compression hardware system based on GZIP and accelerated method thereof | |
| WO2013175909A1 (en) | Data compression/decompression device | |
| EP4030628A1 (en) | Near-storage acceleration of dictionary decoding | |
| CN102244518A (en) | System and method for realizing parallel decompression of hardware | |
| CN104202054A (en) | Hardware LZMA (Lempel-Ziv-Markov chain-Algorithm) compression system and method | |
| JP6009676B2 (en) | Data compression device and data decompression device | |
| CN108287877B (en) | FPGA (field programmable Gate array) compression/decompression system and hardware decompression method for RIB (run in Box) rendering compressed file | |
| WO2013048529A1 (en) | Method and apparatus for high bandwidth dictionary compression technique using delayed dictionary update | |
| CN101478311B (en) | Hardware accelerated implementation process for bzip2 compression algorithm | |
| CN111723059A (en) | Data compression method and device, terminal equipment and storage medium | |
| CN103268299B (en) | A kind of generic data compression IP kernel being applied to PXI Express bus testing system | |
| CN108932315A (en) | A kind of method and relevant apparatus of data decompression | |
| WO2000046925A1 (en) | Predictive data compression system and methods | |
| CN111832257B (en) | Conditional transcoding of encoded data | |
| US20210157485A1 (en) | Pattern-based cache block compression | |
| CN116318171B (en) | LZ4 decompression hardware acceleration realization/compression method, device, medium and chip | |
| CN101458679A (en) | Unification inverse discrete cosine transform (IDCT) microcode processor engine | |
| Kadayif et al. | Instruction compression and encoding for low-power systems | |
| RU2450441C1 (en) | Data compression method and apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C53 | Correction of patent of invention or patent application | ||
| CB02 | Change of applicant information |
Address after: Room E701 No. 20 building science and Technology Park Liye sensor network university 214135 Jiangsu province Wuxi City District Qingyuan Road Applicant after: Wuxi Xinxiang Electronic Technology Co., Ltd. Address before: 214135 Jiangsu Province, Wuxi City District Qingyuan Road Branch Park 530 building A room 512 Applicant before: Wuxi Xinxiang Electronic Technology Co., Ltd. |
|
| C12 | Rejection of a patent application after its publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20130403 |