CN107565972A - A kind of compression method, device, equipment and the storage medium of LZ codings - Google Patents
A kind of compression method, device, equipment and the storage medium of LZ codings Download PDFInfo
- Publication number
- CN107565972A CN107565972A CN201710852682.2A CN201710852682A CN107565972A CN 107565972 A CN107565972 A CN 107565972A CN 201710852682 A CN201710852682 A CN 201710852682A CN 107565972 A CN107565972 A CN 107565972A
- Authority
- CN
- China
- Prior art keywords
- length
- data
- compressed
- array
- compression
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Compression, Expansion, Code Conversion, And Decoders (AREA)
Abstract
The invention discloses a kind of compression method of LZ codings, methods described includes：Obtain data to be compressed and the matching length matched with the array in data to be compressed and the set of offset；First that the array in data to be compressed is obtained from set is effectively matched length and first effective offset；If effective offset it is equal with target offset amount and be effectively matched length and object matching length and no more than default maximum matching length, by be effectively matched length be updated to be effectively matched length and object matching length and, and update the set；Set after renewal is stored to compression memory, obtains compressed data.So matching length is superimposed and stored, improves compression ratio, saves the memory space of compression memory.Meanwhile compression set, equipment and the storage medium of a kind of LZ codings disclosed by the invention, similarly with above-mentioned technique effect.
Description
Technical field
The present invention relates to data computing technique field, more specifically to a kind of compression method of LZ codings, device,
Equipment and storage medium.
Background technology
In cloud computing era, mass data transfers need to take the bandwidth of super large, have had a strong impact on cloud computation data center
The performance of middle network transmission.Performance during in order to improve mass data transfers, data compression turn into most important step.
At present, the data compression algorithm being used is numerous, without damage during in order to ensure data compression, and LZ codings are normal
One of compression algorithm.The main implementation process of LZ codings is the input and input before by contrast currently, if
The same section for meeting to require is i.e. to same section using length (identical matching length) and offset (with current matching position
The distance put) substitute current input, exported as former state if there is no matching result, so as to reach the effect to inputting compressed encoding
Fruit.In the prior art, in order to improve the compression efficiency of data, generally use isomerism parallel accelerates platform parallelization processing number
According to still, it is necessary to sacrifice part matching length to expire during isomerism parallel accelerates platform to realize LZ coding compression algorithms
The demand of sufficient parallelization processing, so that the compression ratio of LZ codings decreases.Also, LZ codings are generally compiled with Huffman
Code realizes gzip compression algorithms jointly, and gzip compression algorithms generally apply to isomerism parallel and accelerate platform, so also causing
When realizing LZ coding compression algorithms, compression ratio decreases, so as to cause the compression ratio of whole gzip compression algorithms to reduce.
Therefore, the compression ratio of LZ codings how is improved in the case where isomerism parallel accelerates platform, is those skilled in the art's needs
Solve the problems, such as.
The content of the invention
It is an object of the invention to provide compression method, device, equipment and the storage medium of a kind of LZ codings, to realize
Isomerism parallel accelerates to improve the compression ratio of LZ codings under platform.
To achieve the above object, the embodiments of the invention provide following technical scheme：
A kind of compression method of LZ codings, including：
The matching length and the collection of offset for obtaining data to be compressed and being matched with the array in the data to be compressed
Close；
First of the array in the data to be compressed is obtained from the set being effectively matched length and first has
Imitate offset；
If the effectively offset it is equal with target offset amount and it is described be effectively matched length and object matching length and
No more than default maximum matching length, then the length that is effectively matched is updated to described be effectively matched length and object matching
The sum of length, and update the set；Wherein, the target offset amount and the object matching length are respectively previous operation
Last effective offset of array in data to be compressed is effectively matched length with last；
Set after renewal is stored to compression memory, obtains compressed data.
Wherein, the set for obtaining the matching length matched with the array in the data to be compressed and offset, bag
Include：
Calculate the HASH values of each array in the data to be compressed；
Character string group to be matched is obtained according to the HASH values of each array；
The array in the data to be compressed is matched with the character string group according to default degree of parallelism, obtain with
The matching length of array matching and the set of offset in the data to be compressed.
Wherein, when being unsatisfactory for, described effectively offset is equal with target offset amount and described is effectively matched length and target
With length and during no more than default this condition of maximum matching length, the compression method also includes：
The set is stored to the compression memory, obtains compressed data.
Wherein, the set by after renewal store to compression memory, after obtaining compressed data, in addition to：
The mark of the target offset amount and the storage location of the object matching length is subtracted 2.
A kind of compression set of LZ codings, including：
First acquisition module, for the matching for obtaining data to be compressed and being matched with the array in the data to be compressed
The set of length and offset；
Second acquisition module, for obtaining first effective of the array in the data to be compressed from the set
With length and first effective offset；
Update module, for effectively offset to be equal with target offset amount and described is effectively matched length and target when described
Matching length and no more than default maximum matching length, then the length that is effectively matched is updated to described be effectively matched length
Degree and object matching length and, and update the set；Wherein, the target offset amount and object matching length difference
Length is effectively matched for last effective offset and last of the array in the data to be compressed of previous operation；
First memory module, for the set after renewal to be stored to compression memory, obtain compressed data.
Wherein, first acquisition module includes：
Computing unit, for calculating the HASH values of each array in the data to be compressed；
Acquiring unit, for obtaining character string group to be matched according to the HASH values of each array；
Matching unit, for being entered the array in the data to be compressed and the character string group according to default degree of parallelism
Row matching, obtains the set of the matching length and offset matched with the array in the data to be compressed.
Wherein, in addition to：
Second memory module, for described effectively offset to be equal with target offset amount and described is effectively matched when be unsatisfactory for
Length and object matching length and during no more than default this condition of maximum matching length, the set is stored to described
Memory is compressed, obtains compressed data.
Wherein, in addition to：
Execution module, for subtracting 2 by the mark of the target offset amount and the storage location of the object matching length.
A kind of compression device of LZ codings, including：
Memory, for data storage condensing routine；
Processor, the compression side of the LZ codings described in above-mentioned any one is realized during for performing the data compression program
The step of method.
A kind of computer-readable recording medium, data compression program is stored with the computer-readable recording medium, institute
The step of compression method of the LZ codings described in above-mentioned any one is realized when stating data compression program by the computing device.
By above scheme, a kind of compression method of LZ codings provided in an embodiment of the present invention, methods described includes：
Obtain data to be compressed and the matching length matched with the array in the data to be compressed and the set of offset；From described
First that the array in the data to be compressed is obtained in set is effectively matched length and first effective offset；It is if described
Effective offset it is equal with target offset amount and it is described be effectively matched length and object matching length and no more than it is default most
Big matching length, then by it is described be effectively matched length be updated to it is described be effectively matched length and object matching length and, and more
The new set；Wherein, the target offset amount and the object matching length are respectively in the data to be compressed of previous operation
Array last effective offset and last be effectively matched length；Set after renewal is stored to compression and stored
Device, obtain compressed data.
It can be seen that the above method passes through the matching that obtains data to be compressed and matched with the array in the data to be compressed
The set of length and offset；First that the array in the data to be compressed is obtained from the set is effectively matched length
With first effective offset；By first of the array in data to be compressed effective offset and the number to be compressed of previous operation
Last effective offset of array in, i.e. target offset amount are contrasted, and are meeting that effective offset and target are inclined
Shifting amount is equal and first number being effectively matched in length and the data to be compressed of previous operation of array in data to be compressed
Last of group is effectively matched length, i.e., object matching length and no more than default maximum matching length under conditions of,
By be effectively matched length be updated to be effectively matched length and object matching length and, and update the set；After renewal
Set is stored to compression memory, obtains compressed data.
So first of the array in data to be compressed is effectively matched in length and the data to be compressed of previous operation
Last of array be effectively matched length and be superimposed and store so that the length that is effectively matched after renewal covers object matching
Length, so allow this operation store to compression memory set in be effectively matched length and effective offset is reflected
Data corresponding to object matching length and target offset amount are mapped to, so as to improve the compression efficiency of single operation；Meanwhile update
Set afterwards covers object matching length and target offset amount, reduces the storage of compressed data in memory space, saves
The memory space of memory is compressed, improves compression ratio, and then improves the compression ratio of whole compression process, also saves storage
Cost.
Accordingly, compression set, equipment and the storage medium of a kind of LZ codings provided in an embodiment of the present invention, similarly has
There is above-mentioned technique effect.
Brief description of the drawings
In order to illustrate more clearly about the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing
There is the required accompanying drawing used in technology description to be briefly described, it should be apparent that, drawings in the following description are only this
Some embodiments of invention, for those of ordinary skill in the art, on the premise of not paying creative work, can be with
Other accompanying drawings are obtained according to these accompanying drawings.
Fig. 1 is a kind of compression method flow chart of LZ codings disclosed in the embodiment of the present invention；
Fig. 2 is the compression method flow chart of another LZ codings disclosed in the embodiment of the present invention；
Fig. 3 is a kind of compression set schematic diagram of LZ codings disclosed in the embodiment of the present invention；
Fig. 4 is a kind of compression device schematic diagram of LZ codings disclosed in the embodiment of the present invention.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out clear, complete
Site preparation describes, it is clear that described embodiment is only part of the embodiment of the present invention, rather than whole embodiments.It is based on
Embodiment in the present invention, those of ordinary skill in the art are obtained every other under the premise of creative work is not made
Embodiment, belong to the scope of protection of the invention.
The embodiment of the invention discloses compression method, device, system and the storage medium of a kind of LZ codings, to realize different
Structure accelerates to improve the compression ratio of LZ codings under platform parallel.
Referring to Fig. 1, a kind of compression method of LZ codings provided in an embodiment of the present invention, including：
S101, obtain data to be compressed and the matching length and offset that are matched with the array in the data to be compressed
Set；
In the present embodiment, after getting data to be compressed, by number to be compressed since the minimum subscript of data to be compressed
According to being divided into some arrays, and the HASH values of each array are calculated respectively, obtained according to the HASH values of each array from dictionary pair
The character string answered, and each array is matched with each character string, obtain the matching length that is matched with each array and partially
Shifting amount.Because each array is corresponding with character string, what is got from dictionary is character string group, the matching length that the match is successful
Degree and offset also form set.
S102, obtain from the set first of array in the data to be compressed and be effectively matched length and first
Individual effective offset；
Specifically, the matching knot of all arrays of data to be compressed is contained in the matching result obtained according to above-mentioned steps
Fruit, due to being each array of parallel processing, it is possible that after matching length and offset that previous array matches cover
The matching length and offset of one array, therefore the matching result of the latter array is as invalid.Need from the set
Middle obtain the array in the data to be compressed first is effectively matched length and first effective offset.
S103, if described effectively offset is equal with target offset amount and described is effectively matched length and object matching length
And no more than default maximum matching length, then the length that is effectively matched is updated to described be effectively matched length and target
The sum of matching length, and update the set；
Wherein, the target offset amount and the object matching length are respectively the number in the data to be compressed of previous operation
Last effective offset of group is effectively matched length with last；
Specifically, after getting and being effectively matched length and effective offset, if effectively offset and target offset amount are equal
And be effectively matched length and object matching length and no more than default maximum matching length, then be effectively matched length by described
Be updated to it is described be effectively matched length and object matching length and, increase the size of matching length, and then raising single operation
Compression efficiency.
It should be noted that the size that maximum matching length can be adjusted flexibly according to being actually needed in technical staff, typically
System default maximum matching length is 258.
S104, by the set after renewal store to compression memory, obtain compressed data.
Specifically, by after renewal be effectively matched length and effective offset is stored to the set, by it is described set deposit
Storage obtains compressed data to compression memory.Wherein, because target offset amount is equal with effective offset, and having after renewal
Effect matching length is the splicing and superposition of two matching lengths, so, the compressed data of this operation gained, i.e., in described set
Both included being effectively matched length before renewal, also including object matching length.Thus, object matching length and target offset amount quilt
Covering storage, reduces the data storage of compression memory, has saved storage space.
It can be seen that present embodiments provide a kind of compression method of LZ coding, by obtain data to be compressed and with it is described
The matching length of array matching and the set of offset in data to be compressed；The data to be compressed are obtained from the set
In first of array be effectively matched length and first effective offset；First of array in data to be compressed is had
Imitate last effective offset of the array in the data to be compressed of offset and previous operation, i.e. target offset amount is carried out pair
Than, be effectively matched at first of the array for meeting that effective offset is equal with target offset amount and in data to be compressed length with
Last of array in the data to be compressed of previous operation is effectively matched length, i.e., object matching length and no more than pre-
If maximum matching length under conditions of, by be effectively matched length be updated to be effectively matched length and object matching length and,
And update the set；Set after renewal is stored to compression memory, obtains compressed data.
So first of the array in data to be compressed is effectively matched in length and the data to be compressed of previous operation
Last of array be effectively matched length and be superimposed and store so that the length that is effectively matched after renewal covers object matching
Length, so allow this operation store to compression memory set in be effectively matched length and effective offset is reflected
Data corresponding to object matching length and target offset amount are mapped to, so as to improve the compression efficiency of single operation；Meanwhile update
Set afterwards covers object matching length and target offset amount, reduces the storage of compressed data in memory space, saves
The memory space of memory is compressed, improves compression ratio, and then improves the compression ratio of whole compression process, also saves storage
Cost.
Referring to Fig. 2, the compression method of another LZ codings provided in an embodiment of the present invention, including：
S201, obtain data to be compressed；
S202, the HASH values for calculating each array in the data to be compressed；
S203, according to the HASH values acquisition of each array character string group to be matched；
S204, according to default degree of parallelism the array in the data to be compressed is matched with the character string group,
Obtain the set of the matching length and offset matched with the array in the data to be compressed；
In the present embodiment, the packet to be compressed got is handled, such as：The data to be compressed of 8 bytes are obtained,
Default degree of parallelism is 4, then is compressed the data to be compressed one by one to each array according to different arrays.
Specifically, being directed to first array, its HASH value is calculated, and obtained from dictionary and first according to its HASH value
The character string of array matching.Processing to other arrays is similar to the above, it is necessary to illustrate, for the HASH values of each array
It is parallel processing to calculate, obtain the operation such as character string and matching length and offset, to improve compression efficiency.I.e. according to pre-
If degree of parallelism the array in the data to be compressed is matched with the character string group, obtain and the data to be compressed
In array matching matching length and offset set.Because each array has its corresponding matching length and skew
Amount, thus matched to obtain as a result, including the set of matching length and offset corresponding to all arrays.
It should be noted that the length of the HASH values of each data is fixed length digit；Also, specific HASH algorithms
Technical staff can be self-defined according to the actual requirements, and the present embodiment does not limit specifically.
S205, obtain from the set first of array in the data to be compressed and be effectively matched length and first
Individual effective offset；
S206, if described effectively offset is equal with target offset amount and described is effectively matched length and object matching length
And no more than default maximum matching length, then the length that is effectively matched is updated to described be effectively matched length and target
The sum of matching length, and update the set；
Wherein, the target offset amount and the object matching length are respectively the number in the data to be compressed of previous operation
Last effective offset of group is effectively matched length with last；
S207, by the set after renewal store to compression memory, obtain compressed data.
Specifically, the parallel processing of the array by treating compressed data, improves compression efficiency；Obtained from the set
Take first of the array in the data to be compressed to be effectively matched length and first effective offset, and ought effectively offset
Amount is equal with target offset amount and described being effectively matched length and object matching length and matches length no more than default maximum
When spending, by it is described be effectively matched length be updated to it is described be effectively matched length and object matching length and, and update the collection
Close；Set after renewal is stored to compression memory, obtains compressed data.
It can be seen that a kind of compression method for LZ codings that the present embodiment provides, by obtaining data to be compressed；Treated described in calculating
The HASH values of each array in compressed data；Character string group to be matched is obtained according to the HASH values of each array；According to
Default degree of parallelism is matched the array in the data to be compressed with the character string group, is obtained and the number to be compressed
The matching length of array matching and the set of offset in；The array in the data to be compressed is obtained from the set
First be effectively matched length and first effective offset；By first effective offset of the array in data to be compressed
With last effective offset of the array in the data to be compressed of previous operation, i.e. target offset amount is contrasted, full
Foot effectively offset is equal with target offset amount and first of array in data to be compressed is effectively matched length and previous behaviour
Last of array in the data to be compressed made is effectively matched length, i.e., object matching length and no more than it is default most
Under conditions of big matching length, by be effectively matched length be updated to be effectively matched length and object matching length and, and update
The set；Set after renewal is stored to compression memory, obtains compressed data.
So first of the array in data to be compressed is effectively matched in length and the data to be compressed of previous operation
Last of array be effectively matched length and be superimposed and store so that the length that is effectively matched after renewal covers object matching
Length, so allow this operation store to compression memory set in be effectively matched length and effective offset is reflected
Data corresponding to object matching length and target offset amount are mapped to, so as to improve the compression efficiency of single operation；Meanwhile update
Set afterwards covers object matching length and target offset amount, reduces the storage of compressed data in memory space, saves
The memory space of memory is compressed, improves compression ratio, and then improves the compression ratio of whole compression process, also saves storage
Cost.
Based on above-mentioned any embodiment, it is necessary to explanation, when being unsatisfactory for effectively offset and the target offset amount phase
Deng and it is described be effectively matched length and object matching length and during no more than default this condition of maximum matching length, it is described
Compression method also includes：The set is stored to the compression memory, obtains compressed data.
Specifically, if effectively offset and target offset amount are unequal, or it is effectively matched length and object matching length
And be all unsatisfactory for more than default maximum matching length, or the two above-mentioned conditions, then by it is described set store to described
Memory is compressed, the set is the compressed data that this operation obtains.
Based on above-mentioned any embodiment, it is necessary to which explanation, the set by after renewal are stored to compression memory, obtained
To after compressed data, in addition to：The mark of the target offset amount and the storage location of the object matching length is subtracted 2.
Specifically, because the position for compressing memory has mark, so the storage location of compressed data can be clearly
Difference；Meanwhile the mark can need to be changed accordingly according to storage.It should be noted that matching length and skew
It is 2 to measure occupy-place length, so when storing the set, object matching length and the capped storage of target offset amount, that is, is needed
The mark of the target offset amount and the storage location of the object matching length is subtracted 2.
Based on above-mentioned any embodiment, it is necessary to explanation, when equal and effective with target offset amount of effective offset
It is with length and object matching length and during no more than default maximum matching length, the length that is effectively matched is updated to institute
State be effectively matched length and object matching length and specific implementation process it is as follows：
Such as：The object matching length of previous operation is 4, and target offset amount is 13；This operation is effectively matched length
For 3, effective offset is 13.Wherein, effective offset and target offset amount is 13, and is effectively matched length and target
It is with length and no more than default maximum matching length 258, so, it can will be effectively matched length and object matching length phase
Add to obtain 7, now, length is effectively matched using 7 as after renewal.
Based on above-mentioned any embodiment, it is necessary to which explanation, the operation platform of the compression method of the LZ codings can be
Any platform, such as：CPU+FPGA isomerism parallels platform, GPU+FPGA isomerism parallels platform, CPU+GPU isomerism parallel platforms
Deng.
Specifically, encoded below with CPU+FPGA isomerism parallel platforms, running LZ provided in an embodiment of the present invention
Exemplified by the compression algorithm that compression method is realized, it is specifically described.
Calculating logic is divided into respectively in host side according to the amount of calculation of LZ coding compression algorithms and algorithmic characteristic first
The kernel parts for performing and being performed on FPGA ends on CPU, can generally the language such as OpenCL or JAVA be used to complete pressure
The description of compression algorithm, it is described to generate the host side program run on host side CPU, and the Kernel towards FPGA ends respectively
Program.Then, host side program is compiled using GCC compilers, generates the executable journey that can be performed on host side CPU
Preface part；Kernel program files are compiled using Altera SDK for OpenCL (AOC) High Level Synthesis instruments comprehensive
Close, generate the AOCX files that can be run on FPGA ends.Finally, host side program is run on host side CPU, calls FPGA ends
On compressed core hardware algorithm circuit carry out it is hardware-accelerated.Wherein, PCI-E interface is used between host side CPU and FPGA ends
Connection, enters row data communication, using the DDR internal memories on the development board of FPGA ends as data buffer storage.
Specifically, the host side CPU, enters the caching of row data communication, the caching for creating host side with FPGA ends
It is stored on the DDR internal memories of FPGA ends development board；By data transfer to be compressed to the DDR internal memories；When the operation of FPGA ends is set
Required parameter information；Control starts the compression algorithm at FPGA ends；After the compression algorithm end of run at FPGA ends, from FPGA
Hold in the caching on DDR internal memories, by the digital independent after compression to host side.
Specifically, the FPGA ends perform the specific steps of the compression method of LZ codings provided in an embodiment of the present invention.Example
Such as：FPGA ends obtain data to be compressed and the matching length and the collection of offset that are matched with the array in the data to be compressed
Close；First that the array in the data to be compressed is obtained from the set is effectively matched length and first effectively skew
Amount；If the effectively offset is equal with target offset amount and described being effectively matched length and object matching length and is not more than
Default maximum matching length, then the length that is effectively matched is updated to described be effectively matched length and object matching length
With, and update the set；Wherein, the target offset amount and the object matching length are respectively the to be compressed of previous operation
Last effective offset of array in data is effectively matched length with last；Set after renewal is stored to pressure
Contracting memory, obtains compressed data.Wherein, the compression memory is the DDR internal memories at FPGA ends, when completion data compression
Afterwards, host side CPU reads compressed data from the DDR internal memories at FPGA ends.
A kind of compression set of LZ codings provided in an embodiment of the present invention is introduced below, a kind of LZ described below
The compression set of coding can be with cross-referenced with a kind of above-described compression method of LZ codings.
Referring to Fig. 3, a kind of compression set of LZ codings provided in an embodiment of the present invention, including：
First acquisition module 301, for obtaining data to be compressed and matching with the array in the data to be compressed
The set of matching length and offset；
Second acquisition module 302, first for obtaining the array in the data to be compressed from the set has
Imitate matching length and first effective offset；
Update module 303, for when it is described effectively offset it is equal with target offset amount and it is described be effectively matched length and
Object matching length and no more than default maximum matching length, then the length that is effectively matched is updated to described effective
It is with length and object matching length and, and update the set；Wherein, the target offset amount and the object matching length
Last effective offset of array respectively in the data to be compressed of previous operation is effectively matched length with last；
First memory module 304, for the set after renewal to be stored to compression memory, obtain compressed data.
Wherein, first acquisition module includes：
Computing unit, for calculating the HASH values of each array in the data to be compressed；
Acquiring unit, for obtaining character string group to be matched according to the HASH values of each array；
Matching unit, for being entered the array in the data to be compressed and the character string group according to default degree of parallelism
Row matching, obtains the set of the matching length and offset matched with the array in the data to be compressed.
Wherein, in addition to：
Second memory module, for described effectively offset to be equal with target offset amount and described is effectively matched when be unsatisfactory for
Length and object matching length and during no more than default this condition of maximum matching length, the set is stored to described
Memory is compressed, obtains compressed data.
Wherein, in addition to：
Execution module, for subtracting 2 by the mark of the target offset amount and the storage location of the object matching length.
A kind of compression device of LZ codings provided in an embodiment of the present invention is introduced below, a kind of LZ described below
The compression device of coding can be with cross-referenced with above-described a kind of compression method, the device of LZ codings.
Referring to Fig. 4, a kind of compression device of LZ codings provided in an embodiment of the present invention, including：
Memory 401, for data storage condensing routine；
Processor 402, the pressure of the LZ codings described in above-mentioned any one is realized during for performing the data compression program
The step of compression method.
Specifically, when compression device is FPGA, the memory is the DDR internal memories at FPGA ends, for data storage
Condensing routine；The processor is the process chip at FPGA ends, realizes above-mentioned during for performing the data compression program
The step of compression method of the LZ codings of meaning one.
A kind of computer-readable recording medium, data compression program is stored with the computer-readable recording medium, institute
The step of compression method of the LZ codings described in above-mentioned any one is realized when stating data compression program by the computing device.
Each embodiment is described by the way of progressive in this specification, what each embodiment stressed be and other
The difference of embodiment, between each embodiment identical similar portion mutually referring to.
The foregoing description of the disclosed embodiments, professional and technical personnel in the field are enable to realize or using the present invention.
A variety of modifications to these embodiments will be apparent for those skilled in the art, as defined herein
General Principle can be realized in other embodiments without departing from the spirit or scope of the present invention.Therefore, it is of the invention
The embodiments shown herein is not intended to be limited to, and is to fit to and principles disclosed herein and features of novelty phase one
The most wide scope caused.
Claims (10)
- A kind of 1. compression method of LZ codings, it is characterised in that including：Obtain data to be compressed and the matching length matched with the array in the data to be compressed and the set of offset；First of the array in the data to be compressed is obtained from the set and is effectively matched length and first effectively partially Shifting amount；If it is described effectively offset it is equal with target offset amount and it is described be effectively matched length with it is object matching length and little In default maximum matching length, then the length that is effectively matched is updated to described be effectively matched length and object matching length Sum, and update the set；Wherein, the target offset amount and the object matching length are respectively that previous operation is waited to press Last effective offset of array in contracting data is effectively matched length with last；Set after renewal is stored to compression memory, obtains compressed data.
- 2. the compression method of LZ codings according to claim 1, it is characterised in that the acquisition and the data to be compressed In array matching matching length and offset set, including：Calculate the HASH values of each array in the data to be compressed；Character string group to be matched is obtained according to the HASH values of each array；The array in the data to be compressed is matched with the character string group according to default degree of parallelism, obtain with it is described The matching length of array matching and the set of offset in data to be compressed.
- 3. the compression method of LZ according to claim 1 coding, it is characterised in that when be unsatisfactory for the effectively offset with Target offset amount it is equal and it is described be effectively matched length and object matching length and no more than default maximum matching length this During one condition, the compression method also includes：The set is stored to the compression memory, obtains compressed data.
- 4. the compression method of the LZ codings according to claim 1-3 any one, it is characterised in that described by after renewal Set store to compression memory, after obtaining compressed data, in addition to：The mark of the target offset amount and the storage location of the object matching length is subtracted 2.
- A kind of 5. compression set of LZ codings, it is characterised in that including：First acquisition module, for the matching length for obtaining data to be compressed and being matched with the array in the data to be compressed With the set of offset；Second acquisition module, first for obtaining the array in the data to be compressed from the set is effectively matched length Degree and first effective offset；Update module, for effectively offset to be equal with target offset amount and described is effectively matched length and object matching when described Length and no more than default maximum matching length, then by it is described be effectively matched length be updated to it is described be effectively matched length with The sum of object matching length, and update the set；Wherein, before the target offset amount and the object matching length are respectively Last effective offset of array in the data to be compressed of secondary operation is effectively matched length with last；First memory module, for the set after renewal to be stored to compression memory, obtain compressed data.
- 6. the compression set of LZ codings according to claim 5, it is characterised in that first acquisition module includes：Computing unit, for calculating the HASH values of each array in the data to be compressed；Acquiring unit, for obtaining character string group to be matched according to the HASH values of each array；Matching unit, for being carried out the array in the data to be compressed and the character string group according to default degree of parallelism Match somebody with somebody, obtain the set of the matching length and offset matched with the array in the data to be compressed.
- 7. the compression set of LZ codings according to claim 5, it is characterised in that also include：Second memory module, for described effectively offset to be equal with target offset amount and described is effectively matched length when be unsatisfactory for With object matching length and during no more than default this condition of maximum matching length, the set is stored to the compression Memory, obtain compressed data.
- 8. the compression set of the LZ codings according to claim 5-7 any one, it is characterised in that also include：Execution module, for subtracting 2 by the mark of the target offset amount and the storage location of the object matching length.
- A kind of 9. compression device of LZ codings, it is characterised in that including：Memory, for data storage condensing routine；Processor, realizing that the LZ as described in claim 1-4 any one is encoded during for performing the data compression program The step of compression method.
- 10. a kind of computer-readable recording medium, it is characterised in that data pressure is stored with the computer-readable recording medium Contracting program, the data compression program are realized that the LZ as described in claim 1-4 any one is compiled during the computing device The step of compression method of code.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710852682.2A CN107565972A (en) | 2017-09-19 | 2017-09-19 | A kind of compression method, device, equipment and the storage medium of LZ codings |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710852682.2A CN107565972A (en) | 2017-09-19 | 2017-09-19 | A kind of compression method, device, equipment and the storage medium of LZ codings |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107565972A true CN107565972A (en) | 2018-01-09 |
Family
ID=60982167
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710852682.2A Pending CN107565972A (en) | 2017-09-19 | 2017-09-19 | A kind of compression method, device, equipment and the storage medium of LZ codings |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107565972A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108832935A (en) * | 2018-05-31 | 2018-11-16 | 郑州云海信息技术有限公司 | A kind of RLE algorithm implementation method, system, equipment and computer storage medium |
| CN111817724A (en) * | 2020-07-22 | 2020-10-23 | 山东云海国创云计算装备产业创新中心有限公司 | Data compression circuit |
| CN112199343A (en) * | 2020-12-03 | 2021-01-08 | 南京烽火星空通信发展有限公司 | Data compression/decompression acceleration method for small data block scene |
| CN116634029A (en) * | 2023-07-21 | 2023-08-22 | 众科云(北京)科技有限公司 | Work platform data rapid transmission method based on block chain |
| CN117119120A (en) * | 2023-10-25 | 2023-11-24 | 上海伯镭智能科技有限公司 | Cooperative control method based on multiple unmanned mine cars |
| WO2024153011A1 (en) * | 2023-01-16 | 2024-07-25 | 杭州阿里云飞天信息技术有限公司 | Lossless compression method, electronic device, and storage medium |
| CN118399976A (en) * | 2024-06-18 | 2024-07-26 | 陕西联众电力科技有限公司 | Remote monitoring method and system for equipment operation data |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110154169A1 (en) * | 2009-12-22 | 2011-06-23 | Vinodh Gopal | System, method, and apparatus for a scalable processor architecture for a variety of string processing applications |
| US20120262314A1 (en) * | 2011-04-15 | 2012-10-18 | Cavium Networks, Inc. | System and Method of Compression and Decompression |
| CN103326730A (en) * | 2013-06-06 | 2013-09-25 | 清华大学 | Data parallelism compression method |
| CN106656198A (en) * | 2016-11-23 | 2017-05-10 | 郑州云海信息技术有限公司 | LZ77-based coding method |
| CN106788447A (en) * | 2016-11-29 | 2017-05-31 | 郑州云海信息技术有限公司 | The matching length output intent and device of a kind of LZ77 compression algorithms |
-
2017
- 2017-09-19 CN CN201710852682.2A patent/CN107565972A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110154169A1 (en) * | 2009-12-22 | 2011-06-23 | Vinodh Gopal | System, method, and apparatus for a scalable processor architecture for a variety of string processing applications |
| US20120262314A1 (en) * | 2011-04-15 | 2012-10-18 | Cavium Networks, Inc. | System and Method of Compression and Decompression |
| CN103326730A (en) * | 2013-06-06 | 2013-09-25 | 清华大学 | Data parallelism compression method |
| CN106656198A (en) * | 2016-11-23 | 2017-05-10 | 郑州云海信息技术有限公司 | LZ77-based coding method |
| CN106788447A (en) * | 2016-11-29 | 2017-05-31 | 郑州云海信息技术有限公司 | The matching length output intent and device of a kind of LZ77 compression algorithms |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108832935A (en) * | 2018-05-31 | 2018-11-16 | 郑州云海信息技术有限公司 | A kind of RLE algorithm implementation method, system, equipment and computer storage medium |
| CN108832935B (en) * | 2018-05-31 | 2022-05-10 | 郑州云海信息技术有限公司 | RLE algorithm implementation method, system, equipment and computer storage medium |
| CN111817724A (en) * | 2020-07-22 | 2020-10-23 | 山东云海国创云计算装备产业创新中心有限公司 | Data compression circuit |
| CN111817724B (en) * | 2020-07-22 | 2022-03-22 | 山东云海国创云计算装备产业创新中心有限公司 | Data compression circuit |
| CN112199343A (en) * | 2020-12-03 | 2021-01-08 | 南京烽火星空通信发展有限公司 | Data compression/decompression acceleration method for small data block scene |
| WO2024153011A1 (en) * | 2023-01-16 | 2024-07-25 | 杭州阿里云飞天信息技术有限公司 | Lossless compression method, electronic device, and storage medium |
| CN116634029A (en) * | 2023-07-21 | 2023-08-22 | 众科云(北京)科技有限公司 | Work platform data rapid transmission method based on block chain |
| CN116634029B (en) * | 2023-07-21 | 2023-09-29 | 众科云(北京)科技有限公司 | Work platform data rapid transmission method based on block chain |
| CN117119120A (en) * | 2023-10-25 | 2023-11-24 | 上海伯镭智能科技有限公司 | Cooperative control method based on multiple unmanned mine cars |
| CN117119120B (en) * | 2023-10-25 | 2023-12-22 | 上海伯镭智能科技有限公司 | Cooperative control method based on multiple unmanned mine cars |
| CN118399976A (en) * | 2024-06-18 | 2024-07-26 | 陕西联众电力科技有限公司 | Remote monitoring method and system for equipment operation data |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107565972A (en) | A kind of compression method, device, equipment and the storage medium of LZ codings | |
| CN107027036A (en) | A kind of FPGA isomeries accelerate decompression method, the apparatus and system of platform | |
| TWI730654B (en) | Method and device for deploying and executing smart contract | |
| US20200244282A1 (en) | Processing core data compression and storage system | |
| RU2019136750A (en) | NEURAL NETWORK PROCESSOR USING COMPRESSION AND UNPACKING OF ACTIVATION DATA TO REDUCE MEMORY BANDWIDTH USE | |
| CN105264779B (en) | Data compression and decompression using SIMD instruction | |
| CN104199951B (en) | Web page processing method and device | |
| US10158376B2 (en) | Techniques to accelerate lossless compression | |
| EP3963853B1 (en) | Optimizing storage and retrieval of compressed data | |
| CN103248369A (en) | Compression system and method based on FPFA (Field Programmable Gate Array) | |
| CN108886367A (en) | Method, apparatus and system for compression and decompression data | |
| CN110943744A (en) | Data compression, decompression and processing method and device based on data compression and decompression | |
| Funasaka et al. | Adaptive loss‐less data compression method optimized for GPU decompression | |
| CN107623855A (en) | A kind of embedded rate steganography device of height based on compressed encoding and steganography method | |
| CN107704268A (en) | MD5 hash functions computational methods, system and computer-readable recording medium | |
| CN113497627A (en) | Data compression and decompression method, device and system | |
| CN111695885A (en) | Digital certificate block chain compression method based on transaction input reduction | |
| CN107612681A (en) | A kind of data processing method based on SM3 algorithms, apparatus and system | |
| JP5362592B2 (en) | Optimal selection of compression entries for compression of program instructions | |
| CN104967856A (en) | Coding method and corresponding device | |
| JP2011090526A (en) | Compression program, method, and device, and decompression program, method, and device | |
| CN107612682A (en) | A kind of data processing method based on SHA512 algorithms, apparatus and system | |
| CN115567059A (en) | Data compression and decompression method and device | |
| CN112100453B (en) | Character string distribution statistical method, system, equipment and computer storage medium | |
| De Agostino | LZW Data Compression on Large Scale and Extreme Distributed Systems. |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180109 |