CN106407285A - RLE and LZW-based optimized bit file compression and decompression method - Google Patents
RLE and LZW-based optimized bit file compression and decompression method Download PDFInfo
- Publication number
- CN106407285A CN106407285A CN201610752157.9A CN201610752157A CN106407285A CN 106407285 A CN106407285 A CN 106407285A CN 201610752157 A CN201610752157 A CN 201610752157A CN 106407285 A CN106407285 A CN 106407285A
- Authority
- CN
- China
- Prior art keywords
- string
- char
- character
- data
- current
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/174—Redundancy elimination performed by the file system
- G06F16/1744—Redundancy elimination performed by the file system using compression, e.g. sparse files
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
Abstract
The invention discloses an RLE and LZW-based optimized bit file compression and decompression method. By performing data format analysis on an FPGA configuration bit file, a head control word of the bit file is extracted; preliminary compression is performed by adopting an RLE code with the run length of 4 from real configuration data; and then LZW compression is performed, so that the compression rate is further increased. The decompression is an inverse process of compression; and LZW decompression is performed to recover intermediate data first, and then a data part which does not contain the head control word is subjected to RLE decompression to recover the original FPGA configuration bit file. The method comprehensively considers compression/decompression time and compression rate; compared with a compression tool of Xilinx, single use of an RLE algorithm and single use of an LZW algorithm, the method has the advantages that the win-win purpose of the compression rate and the compression speed is achieved; and the problem of an excessively large FPGA configuration bit file of advanced type of Xilinx is solved, the overhead of a storage chip is reduced, and a key technical support is provided for an FPGA in-orbit reconstruction technology.
Description
Technical field
The present invention relates to a kind of optimization bit file compression ＆ decompression method based on RLE and LZW, belong to ratio in communication
Special compressing file and decompression technique field.
Background technology
With on star apply complexity increase, the also more and more higher of the requirement to data-handling capacity on star, employing
The device that FPGA model is had to using greater degree also with the lifting of application complexity.The FPGA of these advanced models by
More powerful in function, corresponding bit file also becomes much larger, and configures the bit file of FPGA of a piece of V5130T relatively
V4SX55 increases about 3 times about, and this means that the storage resource applying a piece of V5FPGA to need 3 times in the past is joined to store
Put bit, this is undoubtedly a problem it is necessary to take certain compression method to reduce storage for limited resource on star
Resource.And it must is fulfilled for claimed below：1) necessary lossless reduction bit file, bit stream is carried out with decompression requirement can
Recover former data exactly；2) compression ratio is high, and the compressed and decompressed time is short.Want for this problem and above 2 points
Ask it is proposed that a kind of optimization bit file compression ＆ decompression method based on RLE and LZW, to realize high speed and high pressure shrinkage
Bit file is compressed and decompressed.
Data compression generally includes lossy compression method and two kinds of lossless compress.Lossy compression method is to lose really to compress, irreversible
, information can incur loss.Lossless compress is undistorted compression, removes the redundancy in data, after decompression in compression process
Do not lose any information, with compression before completely the same.Lossless compress is generally used for the compression of text, program, significant data, its energy
Ensure fully to recover initial data.Bit stream is carried out with compression requirement and can recover former data exactly it is therefore desirable to select
With lossless compressiong, bit file is compressed and decompresses the transmission to solve configuration bit data and storage problem.
Classical lossless compression algorithm mainly has：Based on the algorithm of statistical model, such as Shannon coding, huffman coding, calculation
Art coding etc.；Based on the algorithm of dictionary mode, such as LZ serial algorithm；Also have some other algorithms, such as Run- Length Coding, JPEG-
LS algorithm, the DEM algorithm of JPEG2000, BWT algorithm etc..Consider design complexities and compression efficiency, the calculation of dictionary mode
Method is particularly suited for bit file compression compared with the algorithm of statistical model.Both at home and abroad currently for the main employing of bit file compression
Compression method is lexicographic compression or adopts Run- Length Coding technique compresses.The tool of compression of Xilinx oneself adopts lexicographic
LZ77 method carries out bit file compression.Can be seen that the bit file compression of Xilinx compression by experimental data below
Rate is not high.
Content of the invention
Present invention solves the technical problem that being：Overcome prior art not enough, a kind of ratio of the optimization based on RLE and LZW is provided
Special compressing file and decompression method, under the requirement of Fast Compression and high compression rate, the present invention is by bit file data
Structure is analyzed, adaptive Optimal improvements RLE algorithm, and lifts compression ratio further with reference to LZW method, devises one
Plant the optimization bit file compressing/decompressing method based on RLE and LZW, the method is by dividing to bit file data structure
Analysis, creative optimizes RLE algorithm according to bit file feature；On the basis of optimizing application RLE algorithm, in order to further
Lifting compression ratio, is creatively combined by the RLE algorithm after optimizing and with lzw algorithm, is applied to bit file compressed and decompressed.
This method save storage resource, improve computational efficiency, the design of compression and decompression is realized, and demonstrates the correct of method
Property, effectiveness, high efficiency and feasibility.
The present invention solve technical scheme be：A kind of optimization bit file compression method based on RLE and LZW, including step
Suddenly as follows：
Step one：FPGA configuration bit file is made up of many partial datas, comprises head control word, data and command word,
Wherein, data and order word format are all with nybble for a unit, scratch the head control word except FPGA configuration bit file；
Brigade commander's enumerator in initialization RLE；By the run length of RLE according to step one bit file data characteristicses
It is set to 4；
The compression dictionary of initialization LZW, makes dictionary comprise all possible, and described is single entry；
Setting prefix pre_char, makes current prefix pre_char be sky；
Step 2：Setup string pre_string；The FPGA configuration bit literary composition except head control word will be scratched in step one
The first character string of part is assigned to character string pre_string；Described scratch except head control word FPGA configuration bit file be by
Compressed file；
Step 3：To be compressed by compressed file, step is as follows：
(1) judge whether have character string to need to compress in compressed file, need to compress without character string, then first defeated
Go out character string pre_string, then export the value of brigade commander's enumerator, enter step 4；
(2) need to compress if there are character string, current string current_string is set, makes current string
Current_string is equal to the character late string in character stream；
(3) judge whether pre_string is consistent with current_string：If pre_string and current_
String is consistent, and brigade commander's enumerator adds 1, judges whether brigade commander's Counter Value counts 255；If brigade commander's Counter Value meter is to 255,
First export pre_string, then export brigade commander's Counter Value, make brigade commander's Counter Value be 1,
The unpressed character string reading file to be compressed is to pre_string, return to step three
If brigade commander's Counter Value does not count 255, return to step three；
If pre_string is inconsistent with current_string, export pre_string, export brigade commander's Counter Value,
Now pre_string, return to step three will be assigned to by current_string；
Step 4：Set current character current_char, according to step 3 (1) output order, output is arranged in order
The character stream of Cheng Xin, the first character reading in this character stream is assigned to current_char；
Step 5：Judge that pre_char and current_char is arranged in order the entry of composition whether in dictionary：If
In dictionary, the entry that pre_char and current_char is arranged in order composition is assigned to pre_char；If not in dictionary
In, then export the code word of current prefix pre_char, add the entry that pre_char and current_char is arranged in order composition
To in dictionary, the value of current_char is assigned to pre_char；
Step 6：Judge in new character stream whether to also have character to need to compress, need compression if there are character it would be desirable to
The first character of compression is assigned to current_char, return to step five；Need to compress without character, export current prefix
The code word of pre_char, completes to compress.
The compression dictionary of LZW is used for storing the entry producing in compression process.
Brigade commander's enumerator, starts counting up from 1, that is, count 1 when reading first character string, can continuously repeat appearance running into
Character string when Jia 1 every time, when different character strings occur, the value of brigade commander's enumerator can be exported, reduce initial value 1；Trip
The value of long counter is 1-255, reaches and reduces 1 when 255.
The index value of the compression dictionary of initialization LZW is 1-4096.
A kind of optimization bit file decompression method based on RLE and LZW, as follows including step：
Step one：Initial decompression dictionary, makes dictionary comprise all possible, and described is single entry；
Step 2：Setting variable pw, reads first code stream in data flow to be decompressed, is assigned to pw, with pw for index
Inquiry decompression dictionary, reads the corresponding entry of this index, setup string variable pre_char, and the index that pw is represented corresponds to
Entry be assigned to pre_char, and export pre_char；
Step 3：Setting variable cw, continues to read the next code stream in data to be decompressed, is assigned to cw, judges
Whether there is the corresponding entry of the index that cw represents in dictionary, the corresponding entry of index representing if there are cw, output cw represents
Index corresponding entry, the corresponding string data of index that is, cw represents, variable current_char is set, cw is represented
Index corresponding string data and be assigned to current_char, character variable pc is set, pre_char is assigned to pc, setting
Character variable cc, the first character of corresponding for current_char entry is assigned to cc, pc and cc is arranged in order composition
Character string is added in decompression dictionary；The corresponding entry of index representing without cw, the value of pre_char is assigned to pc,
The first character of current_char is assigned to cc, output pc and cc is arranged in order the string data of composition, and adds pc
It is arranged in order the string data of composition with cc in dictionary；
Step 4：Judge in data flow to be decompressed, whether also have data to need to decompress, if also data needs to solve
Compression, return to step three；Need to decompress without data, the output of step 3 is arranged in new character by output order
String data, identifies the head control word in new string data；Setup string head, and head control word is assigned to
head；
Step 5：Setup string variable current_string, from the new string data removing head control word
According to brigade commander read character string be assigned to current_string, then read a character after this character string again, that is, swim
The value of long counter, restores original compression data according to the value of the character string reading out and brigade commander's enumerator；
Step 6：Judge the data whether needing in the new string data remove head control word to be decompressed, if
Have, then replace removing the new string data of head control word, return to step five by data to be decompressed；If, no, will
The data that head is exported with step 5 is arranged in order and obtains initial data, completes to decompress.
Present invention advantage compared with prior art is：
(1) under the requirement of Fast Compression and high compression rate, the present invention by being analyzed to bit file data structure,
Adaptive Optimal improvements RLE algorithm, and lift compression ratio further with reference to LZW method, devise a kind of based on RLE and
The optimization bit file compressing/decompressing method of LZW.The method is by the analysis to bit file data structure, creative
RLE algorithm is optimized according to bit file feature；On the basis of optimizing application RLE algorithm, in order to lift compression ratio further,
Creatively be combined by the RLE algorithm after optimizing and with lzw algorithm, be applied to bit file compressed and decompressed.The method is saved
Storage resource, improves computational efficiency, and the design of compression and decompression is realized, and demonstrates the correctness of method, effectiveness, height
Effect property and feasibility.
(2) present invention is by bit file structure and RLE and lzw algorithm analysis, devising one for bit file
Plant the compression of optimization bit file and the decompression technique method based on RLE and LZW, creative is excellent according to bit file feature
Change RLE algorithm, in order to lift compression ratio further, be creatively combined by the RLE algorithm after optimizing and with lzw algorithm, should
Compressed and decompressed for bit file, save compression time and substantial amounts of storage money on the basis of ensureing certain compression ratio
Source, the correctness of compression method, high efficiency and feasibility by corresponding gunzip design verification.
Brief description
Fig. 1 is the substantially digital structure of the present invention；
Fig. 2 is a kind of optimization bit file compression algorithm flow chart based on RLE and LZW of the present invention；
Fig. 3 is a kind of optimization bit file decompression algorithm flow chart based on RLE and LZW of the present invention；
Fig. 4 is different compression algorithm compression ratio comparison schematic diagram.
Specific embodiment
The basic ideas of the present invention are：A kind of optimization bit file compression ＆ decompression method based on RLE and LZW, leads to
Cross and data form analysis is carried out to FPGA configuration bit file, pluck out the head control word of bit file, from real deployment data
Start, the RLE coding being 4 using brigade commander carries out primary compression, then carry out LZW compression to lift compression ratio further.During decompression
For the inverse process of compression, first carry out LZW decompression and restore intermediate data, then the data division not comprising head control word is entered
Row RLE decompresses, and restores original FPGA configuration bit file.The method considered compression/de-compression time and
Compression ratio, is compared with the tool of compression that Xilinx carries, and with simple application RLE algorithm, simple application lzw algorithm compares, and realizes
The doulbe-sides' victory of compression ratio and compression speed.Solve the problems, such as that Xilinx advanced model FPGA configuration bit file is excessive, save
The expense of storage chip, is that FPGA in-orbit reconfiguration technique provides key technology and supports.
With specific embodiment, the present invention is described in further detail below in conjunction with the accompanying drawings.
Run- Length Coding (RLE) is also called run-length encoding, is a kind of relatively simple data compression technique.The distance of swimming refer to by
In the data flow that source symbol or signal sample are constituted, each character continuously repeats and forms the length of character string, long also known as the distance of swimming
Degree or brigade commander.Run- Length Coding is exactly the length of character string and the mark sequence of the position gone here and there that this character string maps bunchiness
Row, when the character given stroke string, the position of string and length are it becomes possible to recover original data stream.The basic number of data flow
Code structure as shown in figure 1, wherein identification code according to depending on specifically used it is also possible to omit.
According to the data flow basic structure of Run- Length Coding, when only run length is more than 3, Run- Length Coding just has number
Function according to compression.The efficiency of Run- Length Coding depends on the repetitive rate of source symbol, and repetitive rate is higher, and brigade commander is longer, its compression
Effect is more obvious.Distance of swimming compressed logic and hardware are realized simply, and speed is quickly.In Practical Project sometimes by Run- Length Coding and
Other coded methods mix together, and can obtain more preferable compression effectiveness.
By the principle of Run- Length Coding it is recognised that only when by compression samples repetitive rate Run- Length Coding when higher
Compression effectiveness just can be preferably.The framework of analysis bit file, each bit file first has the header information of oneself, this part
The repetitive rate of message sample is very low, and for whole bit file, the shared bit of header information is considerably less, therefore
When being compressed to bit file, pluck out header information position, only the data after header information is compressed, so certain
Compression speed and compression ratio are improved on degree.After plucking out header information, remaining bit file is still to have certain rule, that
No matter being command word or data being all to be a unit with 4 bytes, so application Run- Length Coding is to bit file
It is most suitable that the brigade commander of compression is chosen for 4 bytes, chooses and can increase more brigade commander's information upon compression less than if 4 bytes
File in, compression ratio can be reduced；Choose more than if 4 bytes, the repetitive rate of sample can reduce, meeting in file upon compression
Increase identification code information, reduce compression ratio.After achieving optimization RLE compression algorithm, analysis result be also demonstrate that by experiment.
Improving Run- Length Coding does not need to store due to compression process, does not calculate, so compression speed is quickly, but limited
In bit file complexity and compression algorithm itself, compression ratio is not especially high.Therefore here is entered on the basis of optimizing RLE algorithm
One step is further processed to lift compression ratio to the data after compression with reference to lzw algorithm.
Lzw algorithm is proposed in 1984 by Terry Welch, and it is the innovatory algorithm of LZ78, compared with LZ78 algorithm, goes
Fall second field of mark.Lzw algorithm is the compression algorithm based on dictionary mode, does not rely on the probability distribution of information source,
Be a kind of towards conventional data, the lossless data compression algorithms that are easily achieved.Lzw algorithm does not rely on any data form, tool
There is very big range of application, and coding rate is fast, logic is simple and has adaptive function, be particularly conducive to hardware real
Now there is very high real-time simultaneously.
Lzw algorithm first all characters in alphabet is initialised in dictionary, 8 characters is usually used, therefore exists
First 256 of dictionary have just been occupied before input data, that is, 0～255.Because dictionary has been initialised, input
Character late can find in dictionary.Dictionary encoding is exactly the expansion to 8 character set in practice, for representing data
The character string of middle appearance, newly-increased dictionary encoding can with 9,10,11,12 even more multidigits represent.If for example every
Individual character string is represented with 9, can have 512 different 9 codes.In other words, conversion table has 512 character strings, wherein
256 list items are used for depositing defined character, and remaining 256 are used for depositing (prefix, character string).
The coding principle of lzw algorithm is as follows：Firstly the need of initialization dictionary, first 256 are given 0～255, then often reads
Enter a character pre_char, all must first make a look up in dictionary.If there being this character in dictionary, update current
Character is pre_char, and using current character pre_char as prefix, continues to read in next character current_char work
For trailing character, form a character string " pre_char, current_char ", and search in dictionary again " pre_char,
current_char”.If do not had in dictionary, output character pre_char position code, and by " pre_char, current_
Char " is added in dictionary, then using current_char as prefix.Repeat above step, until all codings complete.
The decoding algorithm of LZW is the inverse process of coding, same show as a kind of adaptive algorithm based on dictionary, one
Side generates dictionary, decoding output.
In sum, the present invention proposes a kind of optimization bit file compression method based on RLE and LZW, including step
As follows：
Step one：FPGA configuration bit file is made up of many partial datas, mainly comprises head control word, data and order
Word, wherein, data and order word format are all with nybble for a unit, scratch and control except the head of FPGA configuration bit file
Word；
Brigade commander's enumerator in initialization RLE is 1；By the run length of RLE according to step one bit file data
Feature is set to 4；
The compression dictionary of initialization LZW arranges for 4096 row * 32, makes dictionary comprise all possible, and described is single
Entry, that is, the first column data of dictionary the 0 to 255th row after initializing is 0-255；
Setting prefix pre_char, makes current prefix pre_char be sky；
Step 2：Setup string pre_string；The FPGA configuration bit literary composition except head control word will be scratched in step one
The first character string of part is assigned to character string pre_string；Described scratch except head control word FPGA configuration bit file be by
Compressed file；
Step 3：
(1) judge whether have character string to need to compress in compressed file, need to compress without character string, then first defeated
Go out character string pre_string, then export the value of brigade commander's enumerator, enter step 4；
(2) need to compress if there are character string, current string current_string is set, makes current string
Current_string is equal to the character late string in character stream；
(3) judge whether pre_string is consistent with current_string：If pre_string and current_
String is consistent, and brigade commander's enumerator adds 1, judges whether brigade commander's Counter Value counts 255；If brigade commander's Counter Value meter is to 255,
First export pre_string, then export brigade commander's Counter Value, make brigade commander's Counter Value be 1,
The unpressed character string reading file to be compressed is to pre_string, return to step three
If brigade commander's Counter Value does not count 255, return to step three；
If pre_string is inconsistent with current_string, export pre_string, export brigade commander's Counter Value,
Now pre_string, return to step three will be assigned to by current_string；
Step 4：(1) set current character current_char,
(2) according to step 3 (1) output order, output is arranged in order into new character stream, reads in this character stream
First character is assigned to current_char；In order to improve compression speed, this step can be same when step 3 has data output
When be actuated for, the first character that read step three exports, to current_char, then carries out subsequent step；
Step 5：Judge that pre_char and current_char is arranged in order the entry of composition whether in dictionary：If
It is in dictionary, pre_char paid in the entry that pre_char and current_char is arranged in order composition；If not in word
In allusion quotation, then export the code word of current prefix pre_char, add the word that pre_char and current_char is arranged in order composition
Bar, in dictionary, the value of current_char is assigned to pre_char；
Step 6：Judge in new character stream whether to also have character to need to compress, need compression if there are character it would be desirable to
The first character of compression pays current_char, return to step five；Need to compress without character, export current prefix
The code word of pre_char, completes to compress.
The compression dictionary of LZW is used for storing the entry producing in compression process.
Brigade commander's enumerator, starts counting up from 1, that is, count 1 when reading first character string, can continuously repeat appearance running into
Character string when Jia 1 every time, when different character strings occur, the value of brigade commander's enumerator can be exported, be reduced to initial value 1；
The value of brigade commander's enumerator is 1-255.
The index value of the compression dictionary of initialization LZW is 1-4096.
In conjunction with above-mentioned optimization bit file compression algorithm and programming thinking, a kind of optimization bit based on RLE and LZW
Compressing file algorithm flow is as shown in Figure 2.
After completing compression, correspond to the data after compression techniques process, decompression is that the inverse process of compression both may be used
To realize restoring initial data, can need to restore, using lzw algorithm, the data optimizing RLE compression, then be optimized RLE
Decompression restores initial initial data.Decompression and compression thinking contrary.
One kind preferably concrete decompression method flow process is given below as shown in figure 3, currently preferred one kind is based on RLE
To include step as follows with the optimization bit file decompression method of LZW：
Step one：Initial decompression dictionary arranges for 4096 row * 32, makes dictionary comprise all possible, described is
Single entry, that is, the first column data of dictionary the 0 to 255th row after initializing is 0-255；
Step 2：Setting variable pw, reads first code stream in data flow to be decompressed, is assigned to pw, with pw for index
Queries dictionary, reads the corresponding entry of this index, and setup string variable pre_char, the corresponding entry of index that pw is represented
It is assigned to pre_char, and export pre_char；As pw be 10, search index be 10 dictionary in string data, be entered as
pre_char；
Step 3：Setting variable cw, continues to read the next code stream in data to be decompressed, is assigned to cw, judges
Whether there is the corresponding entry of the index that cw represents in dictionary, the corresponding entry of index representing if there are cw, output cw represents
Index corresponding string data (the corresponding entry of index that output cw represents, the index corresponding character string number that is, cw represents
According to), variable current_char is set, and the corresponding string data of index that cw is represented is assigned to current_char, if
Put character variable pc, pre_char is assigned to pc, character variable is set, by first of corresponding for current_char entry
Character is assigned to cc, and the character string that pc and cc is arranged in order composition is added in decompression dictionary；The rope representing without cw
Draw corresponding entry, the value of pre_char is assigned to pc, the first character of current_char is assigned to cc, export pc and cc
It is arranged in order the string data of composition, and adds pc and cc and be arranged in order the string data of composition in dictionary；As waited to solve
Compressed data is " 9 18 300 ... ..4095 ", then read first pw value 9, the character string " 9 of output the 9th article of expression of dictionary
256 256 ... .256 " give pre_char, and export " 9 "；Continue read cw value 18, this value in initialized dictionary, output word
The character string " 18 256 256 ... .256 " of the 18th article of expression of allusion quotation gives current_char, and exports " 18 ", adds " 9 18
To in dictionary, this entry indexes as 257 256 ... .256 "；Continue to read cw value 300, this value is not in initialized dictionary, defeated
Go out " 18 18 ", add " 18 18 256 ... .256 " in dictionary, the like, until all data processings complete.
Step 4：Judge in data flow to be decompressed, whether also have data to need to decompress, if also data needs to solve
Compression, return to step three；Need to decompress without data, the output of step 3 is arranged in new character by output order
String data, identifies the head control word in new string data；Setup string head, and head control word is assigned to head
(if new string data is " head control word AA AA AA AA 02 BB BB BB BB 01 CC CC CC CC
09 ... DD DD DD DD 03 ", head control word is assigned to head, to " AA AA AA AA 02 BB BB BB BB 01
CC CC CC CC 09 ... DD DD DD DD 03 " carries out subsequent decompression)；
Step 5：Setup string variable current_string, from the new string data removing head control word
According to brigade commander read character string be assigned to current_string, then read a character after this character string again, that is, swim
The value of long counter, restores original compression data according to the value of the character string reading out and brigade commander's enumerator；As character string " AA
BB CC DD 02 ", reading current_string is " AA BB CC DD ", and brigade commander is 2, and the data after reduction is " AA BB
CC DD AA BB CC DD”；
Step 6：Judge the data whether needing in the new string data remove head control word to be decompressed, if
Have, then replace removing the new string data of head control word, return to step five by the data with compression；If no, will
The data that head is exported with step 5 is arranged in order and obtains initial data, completes to decompress.As carried decompression data to also have " AA
AA AA AA 02 BB BB BB BB 01 CC CC CC CC 09 ... DD DD DD DD 03 ", first reads " AA AA AA
AA ", according to " 02 ", restoring data is " AA AA AA AA AA AA AA AA "；Read " BB BB BB BB " again, according to
" 01 ", restoring data is " BB BB BB BB ", runs through all data successively, until reading " DD DD DD DD ", according to " 03 ",
Restoring data be " DD DD DD DD DD DD DD DD DD DD DD DD " (with head control word splicing complete completely original
Data decompression is condensed to " head control word AA AA AA AA AA AA AA AA BB BB BB BB ... DD DD DD DD DD
DD DD DD DD DD DD DD”).
6th, variable current_char, as call number, decompresses corresponding string data in dictionary.
Correspond to the data after compression techniques process, decompression technique restores optimization firstly the need of using lzw algorithm
RLE compression data, then be optimized RLE decompression restore initial initial data.Algorithm mentality of designing and compression algorithm
Similar, concrete decompression algorithm flow process is as shown in Figure 3.
After the completion of compression and gunzip design, the bit file using five kinds of different resource utilization rates is compressed
Rate test and decompression reduction test, the data after reduction is consistent with initial data, demonstrates the correctness of algorithm realization.Compression
Rate size is as shown in the table：
Table 1 compression test data
Graphic software platform experimental result as shown in figure 4, from experimental result it can be seen that：
With the increase of resource, the compression ratio of all compression methods is all declining；Considerably less data is used for resource
(1%), all more than 95%, the compression ratio of Xilinx only has 85% to three kinds of method compression ratios of our exploitations；
For resource using very many data (99%), the compression method compression ratio of Xilinx is only than the pressure optimizing RLE
Shrinkage is slightly higher, but far below LZW and optimizes RLE_LZW compression method；
For the slightly many data of resource (20%, 56%, 81%), the compression ratio of LZW compression method and other three kinds of methods
It is relatively highest (percent seventy or eighty), the compression ratio of Xilinx is minimum (can only achieve percent twenty or thirty)；
RLE_LZW optimizes compression method, is slightly below LZW for the higher bit compression rate of resource utilization, difference is not
Very big, but improve many compared with LZW method on compression time, this is because data to be compressed first passes through optimization RLE algorithm pressure
Contracting, is then sent through data volume more uncompressed data during LZW compression and has decreased a lot.Optimize RLE algorithm itself not take, and
Lzw algorithm itself needs to be compared one by one with dictionary due to data, and dictionary itself is dynamic renewal, so consuming very much
When.The compression ratio of the more simple lzw algorithm loss of RLE_LZW optimized algorithm is due to optimizing the compression number after RLE algorithm process
Increase compared with initial data according to complexity, lead to compression ratio to receive some impact.But, consider compression time and compression
Rate, a kind of optimization bit file compression ＆ decompression technology based on RLE and LZW is optimum.
Claims (5)
1. a kind of optimization bit file compression method based on RLE and LZW it is characterised in that：As follows including step：
Step one：FPGA configuration bit file is made up of many partial datas, comprises head control word, data and command word, wherein,
Data and order word format are all with nybble for a unit, scratch the head control word except FPGA configuration bit file；
Brigade commander's enumerator in initialization RLE；The bit file data characteristicses setting according to step one by the run length of RLE
For 4；
The compression dictionary of initialization LZW, makes dictionary comprise all possible, and described is single entry；
Setting prefix pre_char, makes current prefix pre_char be sky；
Step 2：Setup string pre_string；The FPGA configuration bit file removing head control word will be scratched in step one
First character string is assigned to character string pre_string；Described scratching except the FPGA configuration bit file of head control word is to be compressed
File；
Step 3：To be compressed by compressed file, step is as follows：
(1) judge whether have character string to need to compress in compressed file, need to compress without character string, then first output word
Symbol string pre_string, then export the value of brigade commander's enumerator, enter step 4；
(2) need to compress if there are character string, current string current_string is set, makes current string current_
String is equal to the character late string in character stream；
(3) judge whether pre_string is consistent with current_string：If pre_string and current_string
Unanimously, brigade commander's enumerator adds 1, judges whether brigade commander's Counter Value counts 255；If brigade commander's Counter Value meter, to 255, first exports
Pre_string, then export brigade commander's Counter Value, make brigade commander's Counter Value be 1,
The unpressed character string reading file to be compressed is to pre_string, return to step three
If brigade commander's Counter Value does not count 255, return to step three；
If pre_string is inconsistent with current_string, export pre_string, export brigade commander's Counter Value, by this
When current_string be assigned to pre_string, return to step three；
Step 4：Set current character current_char, according to step 3 (1) output order, output is arranged in order into new
Character stream, the first character reading in this character stream is assigned to current_char；
Step 5：Judge that pre_char and current_char is arranged in order the entry of composition whether in dictionary：If in word
In allusion quotation, the entry that pre_char and current_char is arranged in order composition is assigned to pre_char；If not in dictionary,
Export the code word of current prefix pre_char, add pre_char and current_char and be arranged in order the entry of composition to dictionary
In, the value of current_char is assigned to pre_char；
Step 6：Judge in new character stream, whether also have character to need to compress, need compression if there are character it would be desirable to compress
First character be assigned to current_char, return to step five；Need to compress without character, export current prefix pre_
The code word of char, completes to compress.
2. a kind of optimization bit file compression method based on RLE and LZW according to claim 1 it is characterised in that：
The compression dictionary of LZW is used for storing the entry producing in compression process.
3. a kind of optimization bit file compression method based on RLE and LZW according to claim 1 it is characterised in that：Trip
Long counter, starts counting up from 1, that is, count 1 when reading first character string, can be when running into the character string continuously repeating appearance
Jia 1 every time, when different character strings occur, the value of brigade commander's enumerator can be exported, reduce initial value 1；Brigade commander's enumerator
It is worth for 1-255, reach and when 255, reduce 1.
4. a kind of optimization bit file compression method based on RLE and LZW according to claim 1 it is characterised in that：Just
The index value of the compression dictionary of beginningization LZW is 1-4096.
5. a kind of optimization bit file decompression method based on RLE and LZW it is characterised in that：As follows including step：
Step one：Initial decompression dictionary, makes dictionary comprise all possible, and described is single entry；
Step 2：Setting variable pw, reads first code stream in data flow to be decompressed, is assigned to pw, with pw as search index
Decompression dictionary, reads the corresponding entry of this index, setup string variable pre_char, the corresponding word of index that pw is represented
Bar is assigned to pre_char, and exports pre_char；
Step 3：Setting variable cw, continues to read the next code stream in data to be decompressed, is assigned to cw, judges in dictionary
In whether have the corresponding entry of the index that cw represents, the corresponding entry of index representing if there are cw, the output index that represents of cw
Corresponding entry, the corresponding string data of index that is, cw represents, variable current_char, the index that cw is represented are set
Corresponding string data is assigned to current_char, arranges character variable pc, and pre_char is assigned to pc, arranges character
Variable cc, the first character of corresponding for current_char entry is assigned to cc, pc and cc is arranged in order the character of composition
String is added in decompression dictionary；The corresponding entry of index representing without cw, the value of pre_char is assigned to pc, will
The first character of current_char is assigned to cc, and output pc and cc is arranged in order the string data of composition, and add pc and
Cc is arranged in order the string data of composition in dictionary；
Step 4：Judge in data flow to be decompressed, whether also have data to need to decompress, if also data needs to decompress,
Return to step three；Need to decompress without data, the output of step 3 is arranged in new character string number by output order
According to the head control word in the new string data of identification；Setup string head, and head control word is assigned to head；
Step 5：Setup string variable current_string, from the new string data removing head control word by
Read character string according to brigade commander and be assigned to current_string, then read a character after this character string again, i.e. brigade commander's meter
The value of number device, restores original compression data according to the value of the character string reading out and brigade commander's enumerator；
Step 6：Judge the data whether needing in the new string data remove head control word to be decompressed, if having,
Data to be decompressed is replaced the new string data removing head control word, return to step five；If no, by head with
The data of step 5 output is arranged in order and obtains initial data, completes to decompress.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610752157.9A CN106407285B (en) | 2016-08-26 | 2016-08-26 | A kind of optimization bit file compression & decompression method based on RLE and LZW |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610752157.9A CN106407285B (en) | 2016-08-26 | 2016-08-26 | A kind of optimization bit file compression & decompression method based on RLE and LZW |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106407285A true CN106407285A (en) | 2017-02-15 |
| CN106407285B CN106407285B (en) | 2019-11-29 |
Family
ID=58003710
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610752157.9A Active CN106407285B (en) | 2016-08-26 | 2016-08-26 | A kind of optimization bit file compression & decompression method based on RLE and LZW |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106407285B (en) |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107294538A (en) * | 2017-06-09 | 2017-10-24 | 深圳市华星光电技术有限公司 | The compensation gauge pressure compression method and decompressing method of OLED display |
| CN107483055A (en) * | 2017-08-04 | 2017-12-15 | 北京世纪铭辰科技有限公司 | A kind of lossless compression method and system |
| CN108184082A (en) * | 2018-01-22 | 2018-06-19 | 豪威科技(上海)有限公司 | Image data transmission system and method |
| CN108259041A (en) * | 2017-12-29 | 2018-07-06 | 中国电子科技集团公司第二十研究所 | A kind of Big Dipper data expansion method based on modified LZW Coding Compression Technologies |
| CN108832935A (en) * | 2018-05-31 | 2018-11-16 | 郑州云海信息技术有限公司 | A kind of RLE algorithm implementation method, system, equipment and computer storage medium |
| WO2018214140A1 (en) * | 2017-05-26 | 2018-11-29 | 深圳配天智能技术研究院有限公司 | Data compression method and data decompression method, and related device |
| CN110518917A (en) * | 2019-07-17 | 2019-11-29 | 华东计算技术研究所（中国电子科技集团公司第三十二研究所） | LZW data compression method and system based on Huffman coding |
| CN110719105A (en) * | 2019-09-16 | 2020-01-21 | 上海御渡半导体科技有限公司 | Lossless compression and decompression method for test vector |
| CN110928809A (en) * | 2019-11-25 | 2020-03-27 | 航天科技控股集团股份有限公司 | Method for programming image data to external Flash through CAN bus |
| WO2020259400A1 (en) * | 2019-06-25 | 2020-12-30 | 比亚迪股份有限公司 | Encoding method and apparatus, storage medium, and computer device |
| CN112579128A (en) * | 2020-12-23 | 2021-03-30 | 恒为科技（上海）股份有限公司 | Data processing method, upper computer, single chip microcomputer and system |
| CN112968706A (en) * | 2021-01-29 | 2021-06-15 | 上海联影医疗科技股份有限公司 | Data compression method, FPGA chip and FPGA online upgrading method |
| CN113709144A (en) * | 2021-08-26 | 2021-11-26 | 哈尔滨工业大学 | High-frequency signal compression method |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105630999A (en) * | 2015-12-28 | 2016-06-01 | 华为技术有限公司 | Data compressing method and device of server |
-
2016
- 2016-08-26 CN CN201610752157.9A patent/CN106407285B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105630999A (en) * | 2015-12-28 | 2016-06-01 | 华为技术有限公司 | Data compressing method and device of server |
Non-Patent Citations (3)
| Title |
|---|
| 刘建: "《多媒体技术基础及应用》", 31 March 2008, 机械工业出版社 * |
| 刘立柱等: "《无失真信源编码纠错译码理论与技术》", 28 February 2008, 国防工业出版社 * |
| 王新年等: "《数字图像压缩技术实用教程》", 31 October 2009, 机械工业出版社 * |
Cited By (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018214140A1 (en) * | 2017-05-26 | 2018-11-29 | 深圳配天智能技术研究院有限公司 | Data compression method and data decompression method, and related device |
| CN110799959B (en) * | 2017-05-26 | 2024-02-27 | 深圳配天智能技术研究院有限公司 | Data compression method, decompression method and related equipment |
| CN110799959A (en) * | 2017-05-26 | 2020-02-14 | 深圳配天智能技术研究院有限公司 | Data compression method, decompression method and related equipment |
| CN107294538A (en) * | 2017-06-09 | 2017-10-24 | 深圳市华星光电技术有限公司 | The compensation gauge pressure compression method and decompressing method of OLED display |
| CN107294538B (en) * | 2017-06-09 | 2020-09-01 | 深圳市华星光电半导体显示技术有限公司 | Compression method and decompression method for compensation gauge of OLED display device |
| CN107483055B (en) * | 2017-08-04 | 2020-06-16 | 北京世纪铭辰科技有限公司 | Lossless compression method and system |
| CN107483055A (en) * | 2017-08-04 | 2017-12-15 | 北京世纪铭辰科技有限公司 | A kind of lossless compression method and system |
| CN108259041A (en) * | 2017-12-29 | 2018-07-06 | 中国电子科技集团公司第二十研究所 | A kind of Big Dipper data expansion method based on modified LZW Coding Compression Technologies |
| CN108184082A (en) * | 2018-01-22 | 2018-06-19 | 豪威科技(上海)有限公司 | Image data transmission system and method |
| CN108832935A (en) * | 2018-05-31 | 2018-11-16 | 郑州云海信息技术有限公司 | A kind of RLE algorithm implementation method, system, equipment and computer storage medium |
| WO2020259400A1 (en) * | 2019-06-25 | 2020-12-30 | 比亚迪股份有限公司 | Encoding method and apparatus, storage medium, and computer device |
| US11750211B2 (en) | 2019-06-25 | 2023-09-05 | Byd Company Limited | Encoding method and apparatus, storage medium, and computer device |
| CN110518917B (en) * | 2019-07-17 | 2023-01-03 | 华东计算技术研究所（中国电子科技集团公司第三十二研究所） | LZW data compression method and system based on Huffman coding |
| CN110518917A (en) * | 2019-07-17 | 2019-11-29 | 华东计算技术研究所（中国电子科技集团公司第三十二研究所） | LZW data compression method and system based on Huffman coding |
| CN110719105A (en) * | 2019-09-16 | 2020-01-21 | 上海御渡半导体科技有限公司 | Lossless compression and decompression method for test vector |
| CN110928809A (en) * | 2019-11-25 | 2020-03-27 | 航天科技控股集团股份有限公司 | Method for programming image data to external Flash through CAN bus |
| CN112579128A (en) * | 2020-12-23 | 2021-03-30 | 恒为科技（上海）股份有限公司 | Data processing method, upper computer, single chip microcomputer and system |
| CN112968706A (en) * | 2021-01-29 | 2021-06-15 | 上海联影医疗科技股份有限公司 | Data compression method, FPGA chip and FPGA online upgrading method |
| CN113709144A (en) * | 2021-08-26 | 2021-11-26 | 哈尔滨工业大学 | High-frequency signal compression method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106407285B (en) | 2019-11-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106407285B (en) | A kind of optimization bit file compression & decompression method based on RLE and LZW | |
| CN1145264C (en) | Data compression and decompression system with immediate dictionary updating interleaved with string search | |
| CN101783788B (en) | File compression method, file compression device, file decompression method, file decompression device, compressed file searching method and compressed file searching device | |
| WO2019153700A1 (en) | Encoding and decoding method, apparatus and encoding and decoding device | |
| CN102122960B (en) | Multi-character combination lossless data compression method for binary data | |
| CN110021369B (en) | Gene sequencing data compression and decompression method, system and computer readable medium | |
| CN107565971B (en) | Data compression method and device | |
| CN116681036B (en) | Industrial data storage method based on digital twinning | |
| CN110518917B (en) | LZW data compression method and system based on Huffman coding | |
| CN103236847A (en) | Multilayer Hash structure and run coding-based lossless compression method for data | |
| Nandi et al. | A compression technique based on optimality of LZW code (OLZW) | |
| CN104156990A (en) | Lossless compressed encoding method and system supporting oversize data window | |
| CN101534124A (en) | Compression algorithm for short natural language | |
| US6748520B1 (en) | System and method for compressing and decompressing a binary code image | |
| CN104682966B (en) | The lossless compression method of table data | |
| Shoba et al. | A Study on Data Compression Using Huffman Coding Algorithms | |
| KR20160100496A (en) | Improved huffman code method and apprartus thereof by using binary clusters | |
| CN114429200A (en) | Standardized Huffman coding and decoding method and neural network computing chip | |
| Mahmood et al. | Efficient compression scheme for large natural text using zipf distribution | |
| JPH0628149A (en) | Method for compressing plural kinds of data | |
| CN206712982U (en) | A kind of Huffman coded systems for VLSI designs | |
| CN109698704B (en) | Comparative gene sequencing data decompression method, system and computer readable medium | |
| Rajput et al. | Comparative Study of Data Compression Techniques | |
| Islam et al. | A Dynamic 5 Bit Data Compression Scheme by Using Set Change Table (5BDC) | |
| Mahmoudi et al. | Comparison of Compression Algorithms in text data for Data Mining |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |