CN107688439A - The generation method and device of onrelevant compression blocks based on Deflate - Google Patents

The generation method and device of onrelevant compression blocks based on Deflate Download PDF

Info

Publication number
CN107688439A
CN107688439A CN201710698605.6A CN201710698605A CN107688439A CN 107688439 A CN107688439 A CN 107688439A CN 201710698605 A CN201710698605 A CN 201710698605A CN 107688439 A CN107688439 A CN 107688439A
Authority
CN
China
Prior art keywords
length
distance
demarcation
onrelevant
line
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201710698605.6A
Other languages
Chinese (zh)
Other versions
CN107688439B (en
Inventor
张宇弘
王界兵
张伟
董迪马
耿涛
黄嘉乐
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Frontsurf Information Technology Co Ltd
Original Assignee
Shenzhen Frontsurf Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Frontsurf Information Technology Co Ltd filed Critical Shenzhen Frontsurf Information Technology Co Ltd
Priority to CN201710698605.6A priority Critical patent/CN107688439B/en
Publication of CN107688439A publication Critical patent/CN107688439A/en
Application granted granted Critical
Publication of CN107688439B publication Critical patent/CN107688439B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G06F3/064—Management of blocks
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
    • H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
    • H03M7/40—Conversion to or from variable length codes, e.g. Shannon-Fano code, Huffman code, Morse code

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)

Abstract

The present invention discloses a kind of generation method and device of the onrelevant compression blocks based on Deflate, wherein, this method includes：When line of demarcation between two compression blocks of onrelevant determines, [distance, the length] matching symbols found in the past through LZ77 conversion that define boundaries are carried out with the LZ77 inverse transformations of selectivity；Check [distance, length] whether matching symbols point to file content before line of demarcation, in the file content before finding that [distance, length] matching symbols point to line of demarcation, the output of the symbol data alternatively LZ77 inverse transformations of property is replaced using former input text；Output of the LZ77 reinfusion buffering area to the LZ77 inverse transformations of selectivity is distributed to keep in；And produce compression data block according to the data of LZ77 reinfusion buffering areas.The present invention can realize the data onrelevant between adjacent container, will not the interrupt operation when container switches, it is possible to increase the stationary performance of system.

Description

The generation method and device of onrelevant compression blocks based on Deflate
Technical field
The present invention relates to technical field of data processing, more particularly to a kind of life of the onrelevant compression blocks based on Deflate Into method and device.
Background technology
At present, many storage devices typically carry out data storage using the memory space of regular length.We are unified fixed The memory space of length is referred to as container.The length of container can be in different storage devices it is different, than if any storage set Alternative 4K bytes, some selection 1M bytes.When container length than allowing after compressing original data is bigger, Neng Gou Output is blocked on current container border, latter half initial data is stored in next container.In many storage systems, No state relation is wished between container and container.In other words, when user accesses a container, he is not intended to access Another container just can know that some information of current container.Original Deflate algorithms are stateful pass between data block Connection.The content of previous data block can be in the sliding window of the latter data block, [distance, the length] of the latter data block Symbol may point to the content of previous data block.Inside same container, continuous compression data block is desirable to stateful Association, that is to say, that the latter compression data block, so can be with by the use of the original text of previous compression data block as sliding window Increase the possibility of LZ77 Transformation Matchings.But no state relation is desirable between container and container.
In order to ensure do not have state relation between container, prior art is using it is determined that the border of previous container Afterwards, history and the output before that LZ77 conversion preserves are emptied, since the border of previous container, again to being originally inputted number According to the compression work for starting new container.Namely a part of initial data needs to be accessed twice, and the knot of part matching Fruit is dropped.For hardware design, this method has interrupted streamline, and redundancy week is inserted between container and container Phase, so that sometime compression performance drastically declines, cause the performance at system of users interface unstable.
In view of this, it is necessary to it is further to propose that the technology that onrelevant compression blocks are produced to being currently based on Deflate is carried out Improvement.
The content of the invention
To solve an above-mentioned at least technical problem, the main object of the present invention is to provide a kind of based on the unrelated of Deflate Join the generation method of compression blocks.
To achieve the above object, one aspect of the present invention is：A kind of onrelevant based on Deflate is provided The generation method of compression blocks, including：
When line of demarcation between two compression blocks of onrelevant determines, to being found before defining boundaries through LZ77 conversion [distance, length] matching symbols carry out selectivity LZ77 inverse transformations；
Check whether [distance, length] matching symbols point to the file content before line of demarcation, in any [distance, length] Matching symbols point to line of demarcation before file content when, using original input text replace should [distance, length] matching symbols make For output；
Output of the LZ77 reinfusion buffering area to the LZ77 inverse transformations of selectivity is distributed to keep in；And
Compression data block is produced according to the data of LZ77 reinfusion buffering areas.
In a specific embodiment, it is determined that also being wrapped before the step of line of demarcation between two compression blocks of onrelevant Include：
The line of demarcation between two compression blocks of onrelevant is determined according to the size of container.
In a specific embodiment, described pair of [distance, the length] found in the past through LZ77 conversion that defines boundaries The LZ77 inverse transformations that selectivity is carried out with symbol specifically include：
An area undetermined is marked off from the output buffer of LZ77 conversion；
[distance, length] matching symbols that LZ77 conversion is found before defining boundaries are marked and are included into undetermined Area；And
The LZ77 inverse transformations for carrying out selectivity to owning [distance, length] matching symbols in area undetermined.
In a specific embodiment, the selective LZ77 inverse transformations specifically include：
Parse all [distance, length] matching symbols in area undetermined；
The absolute drift corresponding with input data of all [distance, length] matching symbols is calculated, and will all [distance, length Degree] absolute drifts of matching symbols draws comparative result compared with the absolute drift in line of demarcation；
If [distance, length] matching symbols point to the file content before line of demarcation, then [distance, length] matching symbol is used Number signified file content recovers original text；
If [distance, length] matching symbols point to the later file content in line of demarcation, then retain [distance, length] matching Symbol.
In a specific embodiment, described pair of [distance, the length] found in the past through LZ77 conversion that defines boundaries Also include before the step of carrying out the LZ77 inverse transformations of selectivity with symbol：
Sliding window before converting the line of demarcation preserved through LZ77 is all cancelled, wherein, the sliding window is to appoint The byte sequence of meaning.
To achieve the above object, another technical solution used in the present invention is：There is provided a kind of based on the unrelated of Deflate Join the generating means of compression blocks, including：
LZ77 inverse transform blocks, when the line of demarcation between two compression blocks of onrelevant determines, boundary is determined for pair [distance, the length] matching symbols found before line through LZ77 conversion carry out the LZ77 inverse transformations of selectivity；
Check module, to check [distance, length] matching symbols whether point to line of demarcation before file content, it is in office When one [distance, length] matching symbols point to the file content before line of demarcation, using original input text replace should [distance, it is long Degree] matching symbols conduct output；
Distribute module, carried out temporarily for distributing output of the LZ77 reinfusion buffering area to the LZ77 inverse transformations of selectivity Deposit；And
Generation module, for producing compression data block according to the data of LZ77 reinfusion buffering areas.
In a specific embodiment, the generating means of the onrelevant compression blocks based on Deflate also include determining Module is used for：
The line of demarcation between two compression blocks of onrelevant is determined according to the size of container.
In a specific embodiment, the LZ77 inverse transform blocks are used for：
An area undetermined is marked off from the output buffer of LZ77 conversion；
[distance, length] matching symbols that LZ77 conversion is found before defining boundaries are marked and are included into undetermined Area；And
The LZ77 inverse transformations for carrying out selectivity to owning [distance, length] matching symbols in area undetermined.
In a specific embodiment, the LZ77 inverse transform blocks are additionally operable to：
Parse all [distance, length] matching symbols in area undetermined；
The absolute drift corresponding with input data of all [distance, length] matching symbols is calculated, and will all [distance, length Degree] absolute drifts of matching symbols draws comparative result compared with the absolute drift in line of demarcation；
If [distance, length] matching symbols point to the file content before line of demarcation, then [distance, length] matching symbol is used Number signified file content recovers original text.
If [distance, length] matching symbols point to the later file content in line of demarcation, then retain [distance, length] matching Symbol.
In a specific embodiment, the generating means of the onrelevant compression blocks based on Deflate also include：
Module is abandoned, for the sliding window before converting the line of demarcation preserved through LZ77 all to be cancelled, wherein, it is described Sliding window is arbitrary byte sequence.
It is right first when technical scheme is mainly determined using the line of demarcation between two compression blocks of onrelevant [distance, the length] matching symbols found in the past through LZ77 conversion that define boundaries carry out the LZ77 inverse transformations of selectivity；Then Check whether [distance, length] matching symbols point to the file content before line of demarcation, in any [distance, length] matching symbols Point to line of demarcation before file content when, using original input text replace should [distance, length] matching symbols as output；And Output of the LZ77 reinfusion buffering area to the LZ77 inverse transformations of selectivity is distributed afterwards to keep in；Finally according to LZ77 reinfusion The data of buffering area produce compression data block, because the compression data block in this programme uses former input text replacement and line of demarcation [distance, length] matching symbols associated in the past, that is, data after replacing no longer with before line of demarcation [distance, it is long Degree] matching symbols are associated, so, it is possible to achieve the data onrelevant between adjacent container, can be kept by above-mentioned scheme Data path streamline it is unobstructed, will not the interrupt operation when container switches, also will not repeated accesses input data, it is possible to increase The stationary performance of system.
Brief description of the drawings
In order to illustrate more clearly about the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing There is the required accompanying drawing used in technology description to be briefly described, it should be apparent that, drawings in the following description are only this Some embodiments of invention, for those of ordinary skill in the art, on the premise of not paying creative work, can be with Structure according to these accompanying drawings obtains other accompanying drawings.
Fig. 1 is the method flow diagram of the generation method of onrelevant compression blocks of the one embodiment of the invention based on Deflate；
Fig. 2 is the method flow diagram of the generation method of onrelevant compression blocks of the another embodiment of the present invention based on Deflate；
Fig. 3 is the method flow diagram of LZ77 inverse transformations selective in step S20 of the invention；
Fig. 4 is the module square frame of the generating means of onrelevant compression blocks of the one embodiment of the invention based on Deflate；
Fig. 5 is the module square frame of the generating means of onrelevant compression blocks of the another embodiment of the present invention based on Deflate；
Fig. 6 is compression blocks process chart in one embodiment of the invention.
The realization, functional characteristics and advantage of the object of the invention will be described further referring to the drawings in conjunction with the embodiments.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out clear, complete Site preparation describes, it is clear that described embodiment is only the part of the embodiment of the present invention, rather than whole embodiments.Base Embodiment in the present invention, those of ordinary skill in the art obtained under the premise of creative work is not made it is all its His embodiment, belongs to the scope of protection of the invention.
It is to be appreciated that the description for being related to " first ", " second " etc. in the present invention be only used for describe purpose, and it is not intended that Indicate or imply its relative importance or the implicit quantity for indicating indicated technical characteristic.Thus, define " first ", At least one this feature can be expressed or be implicitly included to the feature of " second ".In addition, the technical side between each embodiment Case can be combined with each other, but must can be implemented as basis with those of ordinary skill in the art, when the combination of technical scheme Occur conflicting or will be understood that the combination of this technical scheme is not present when can not realize, also not in the guarantor of application claims Within the scope of shield.
For understanding technical scheme preferably, Deflate compression algorithms are described in detail below. When being compressed using Deflate, the mutation that (LZ77 conversion can be realized by LZ algorithms) is first converted using LZ77 is compressed, such as If having two pieces of contents identical in fruit file, then only it is to be understood that previous piece of position and size, may thereby determine that latter piece Content, in this way, [distance, length] (distance between the two, the length of identical content) such a matching symbols can be used, To replace latter piece of content.Because the bit (bit) of (distance between the two, the length of identical content) this pair of information counts, Less than the bit number for being replaced content, therefore data are compressed.
The method that LZ77 conversion uses " sliding window ", to find the same section in file, that is, matched character string, Abbreviation matching string.Above-mentioned matching string refers to the sequence of any byte, rather than just can be shown in text The sequence of those bytes out.Here matching string is it is emphasised that its position hereof, its length is with matching Situation and change.
Since the beginning of file, byte-by-byte is handled LZ77 backward.The sliding window of one fixed size (before currently processed byte, and being close to currently processed byte), constantly slides backward with the byte of processing.For Each byte in file, and each byte in sliding window are matched, and find most long matching string.It is if currently processed The character string that byte starts has matching to go here and there in sliding window, just with (distance between the two, matching length) such a pair of letters Breath, to replace current string, then from next byte after the character string handled just now, is continued with.If work as The character string that pre-treatment byte starts without matching string, does not just do the currently processed byte of the output changed in the window.Processing text In part when first character section, window is not slided on file before currently processed byte, that is, also, at this moment in window There is no any content, processed byte will not do the output changed.With processing constantly backward, window is increasing File is slipped into, last whole window slips into file, and then whole window slides backward on file, until the whole end of file.
Result through LZ77 conversion compressions is referred to as symbol data, and symbol data is temporarily stored in output buffer l_ In buf [], d_buf [], flag_buf [].For the compression result of LZ77 algorithms, one piece of output may be used or be divided into Polylith output (after LZ77 compresses certain part, exports with regard to carrying out once block, exports one piece).In this way, Deflate algorithms One or more IOB is included in output file, and the size of each IOB is not fixed.And when l_buf [] is full, Or d_buf [] it is full when, by result compress to existing LZ77 progress Huffman codings and be output to a compressed data In block.The method of Huffman codings can be used the obtained result of LZ77 conversion compressions (actually Deflate according to circumstances, Selection is encoded using static Huffman or dynamic Huffman code) it is compressed.Huffman codings utilize statistics probability Result is encoded, can further compressed data.Inside same container, continuous compression data block has been desirable to shape State association, that is to say, that the latter compression data block so may be used by the use of the original text of previous compression data block as sliding window To increase the possibility of LZ77 matchings.But no state relation is desirable between container and container.Therefore, the invention provides A kind of generation method of the onrelevant compression blocks based on Deflate, concrete scheme refer to following embodiments.
Fig. 1 is refer to, in embodiments of the present invention, is somebody's turn to do the generation method of the onrelevant compression blocks based on Deflate, bag Include：
Step S10, when the line of demarcation between two compression blocks of onrelevant determines, to defining boundaries in the past through LZ77 Convert the LZ77 inverse transformations that [distance, the length] matching symbols found carry out selectivity；
Step S20, check whether [distance, length] matching symbols point to file content before line of demarcation, it is any [away from From length] for matching symbols when pointing to the file content before line of demarcation, being replaced using original input text should [distance, length] With symbol as output；
Step S30, output of the LZ77 reinfusion buffering area to the LZ77 inverse transformations of selectivity is distributed to keep in；And
Step S40, compression data block is produced according to the data of LZ77 reinfusion buffering areas.
Before line of demarcation determines, the LZ77 conversion input data after line of demarcation is handled, and may be It has matched the content before line of demarcation.In order to ensure there is no state relation between container, before line of demarcation determines, LZ77 conversion Produce [distance, length] symbol be in fact it is invalid, it is necessary to these symbols carry out selectivity LZ77 inverse transformations.This reality Apply in example, add the LZ77 inverse transformations of one-level selectivity.The LZ77 inverse transformations of selectivity check output buffer l_buf [], [distance, length] matching symbols in d_buf [], flag_buf [], divide if any [distance, length] matching symbols point to File content before boundary line, then being replaced using former input text should [distance, length] matching symbols；The LZ77 contravariant of selectivity The output changed is kept in into the LZ77 newly increased a reinfusion buffering area, finally by Deflate algorithms and anti-according to LZ77 The content of output buffer produces compression data block.
It is right first when technical scheme is mainly determined using the line of demarcation between two compression blocks of onrelevant [distance, the length] matching symbols found in the past through LZ77 conversion that define boundaries carry out the LZ77 inverse transformations of selectivity；Then Check whether [distance, length] matching symbols point to the file content before line of demarcation, in any [distance, length] matching symbols Point to line of demarcation before file content when, using original input text replace should [distance, length] matching symbols as output；And Output of the LZ77 reinfusion buffering area to the LZ77 inverse transformations of selectivity is distributed afterwards to keep in；Finally according to LZ77 reinfusion The data of buffering area produce compression data block, because the compression data block in this programme uses former input text replacement and line of demarcation [distance, length] matching symbols associated in the past, that is, data after replacing no longer with before line of demarcation [distance, it is long Degree] matching symbols are associated, so, it is possible to achieve the data onrelevant between adjacent container, can be kept by above-mentioned scheme Data path streamline it is unobstructed, will not the interrupt operation when container switches, also will not repeated accesses input data, it is possible to increase The stationary performance of system.
Fig. 2 is refer to, in a specific embodiment, it is determined that the step in the line of demarcation between two compression blocks of onrelevant Also include before rapid：
Step S01, the line of demarcation between two compression blocks of onrelevant is determined according to the size of container.
, it is necessary to first calculate the size of container in the present embodiment, two pressures of onrelevant are then determined according to container size Line of demarcation between contracting block.Specifically, when last compression data block of container storage, last compression data block is The line of demarcation of current container and subsequent vessel, the absolute drift in input data corresponding to it is recorded, as follow-up Processing.
In a specific embodiment, described pair of [distance, the length] found in the past through LZ77 conversion that defines boundaries The LZ77 inverse transformations that selectivity is carried out with symbol specifically include：
An area undetermined is marked off from the output buffer of LZ77 conversion；
[distance, length] matching symbols that LZ77 conversion is found before defining boundaries are marked and are included into undetermined Area；And
The LZ77 inverse transformations for carrying out selectivity to owning [distance, length] matching symbols in area undetermined.
In the present embodiment, [distance, length] matching symbols for converting to obtain through LZ77 are operated for convenience, can be with An area undetermined is marked off in the output buffer of LZ77 conversion, then in output buffer l_buf [], d_buf [], flag_ In buf [] and LZ77 streamlines, LZ77 [distance, length] matching symbols for converting to obtain are marked and are included into undetermined Area, selective LZ77 inverse transformations are finally carried out, in this way, can prevent from omitting [distance, length] matching symbols, and can protected Hold the unobstructed of data path streamline.
Fig. 3 is refer to, in a specific embodiment, the selective LZ77 inverse transformations specifically include：
Step S21, all [distance, length] matching symbols in area undetermined are parsed；
Step S22, the absolute drift corresponding with input data of all [distance, length] matching symbols is calculated, and will be all The absolute drift of [distance, length] matching symbols draws comparative result compared with the absolute drift in line of demarcation；
If step S23, [distance, length] matching symbols point to the file content before line of demarcation, then with [distance, length Degree] matching symbols signified file contents recovers original text；[if distance, length] matching symbols point to the later text in line of demarcation Part content, then retain [distance, length] matching symbols.
In the present embodiment, the selective LZ77 inverse transformations can be parsed to [distance, length] matching symbols, That is, equivalent to decompression [distance, length] matching symbols, then by the way that [distance, length] matching symbols of decompression are calculated Go out itself and the absolute displacement of output data and the absolute displacement in line of demarcation, then compare both absolute displacements and draw and compare knot Fruit.If it find that the content pointed by this [distance, length] matching symbols then removes original text output buffer handle earlier than line of demarcation The content of sensing, which is read back, to be come, and is stored in LZ77 reinfusion buffering areas.
Refer to Fig. 2, in a specific embodiment, described pair define boundaries in the past through LZ77 conversion find [away from From length] matching symbols carry out selectivity LZ77 inverse transformations the step of before also include：
Step S02, the sliding window before converting the line of demarcation preserved through LZ77 is all cancelled, wherein, the slip Window is arbitrary byte sequence.
In the present embodiment, the history before by above-mentioned scheme follow-up input being made to go to matching line of demarcation. In addition, original text output buffer needs to preserve the record of the previous LZ77 conversion sliding windows in line of demarcation, the record only exists The LZ77 inverse transformations of selectivity can just be cancelled after converting.Because [distance, the length] matching symbols in area undetermined are possible to The partial content in this segment record is pointed to, before the LZ77 inverse transformations of the selectivity in area undetermined are completed, original text output buffer The record of the previous LZ77 algorithms sliding window in preservation line of demarcation is needed, after the LZ77 inverse transformations of selectivity convert, Area undetermined does not just use, therefore can abolish the record.
Refer to Fig. 4 and Fig. 6, in embodiments of the invention, should onrelevant compression blocks based on Deflate generation dress Put, including：
LZ77 inverse transform blocks 10, when the line of demarcation between two compression blocks of onrelevant determines, determine to divide for pair [distance, the length] matching symbols found before boundary line through LZ77 conversion carry out the LZ77 inverse transformations of selectivity；
Check module 20, to check [distance, length] matching symbols whether point to line of demarcation before file content, When any [distance, length] matching symbols point to the file content before line of demarcation, using original input text replace should [distance, Length] matching symbols conduct output；
Distribute module 30, carried out temporarily for distributing output of the LZ77 reinfusion buffering area to the LZ77 inverse transformations of selectivity Deposit；And
Generation module 40, for producing compression data block according to the data of LZ77 reinfusion buffering areas.
In the present embodiment, one-level reverse L Z77 conversion is added by LZ77 inverse transform blocks 10；By checking that module 20 can To check output buffer l_buf [], [distance, length] matching symbols in d_buf [], flag_buf [], if any File content before [distance, length] matching symbols sensing line of demarcation, then replaced using former input text and be somebody's turn to do [distance, length] Matching symbols；The output of the LZ77 inverse transformations of selectivity is kept in the LZ77 newly increased a reinfusion by distribute module 30 In buffering area, finally by generation module 40, produced using Deflate algorithms and according to the contents of LZ77 reinfusion buffering areas Compression data block.
Fig. 5 and Fig. 6 are refer to, in a specific embodiment, the generation of the onrelevant compression blocks based on Deflate Device is also used for including determining module 50：
The line of demarcation between two compression blocks of onrelevant is determined according to the size of container.
In the present embodiment, the line of demarcation between two compression blocks of onrelevant is determined according to container size.Specifically, when appearance When device stores last compression data block, last compression data block is the line of demarcation of current container and subsequent vessel, The absolute drift in input data corresponding to it is recorded, as subsequent treatment.
Fig. 4 and Fig. 6 are refer to, in a specific embodiment, the selective LZ77 inverse transform blocks 10 are used for：
An area undetermined is marked off from the output buffer of LZ77 conversion；
[distance, length] matching symbols that LZ77 conversion is found before defining boundaries are marked and are included into undetermined Area；And
The LZ77 inverse transformations for carrying out selectivity to owning [distance, length] matching symbols in area undetermined.
In the present embodiment, [distance, the length] matching symbols for converting to obtain through LZ77 are operated for convenience, the choosing The LZ77 inverse transform blocks 10 of selecting property can mark off an area undetermined in the output buffer that LZ77 is converted, then slow in output Rush area l_buf [], in d_buf [], flag_buf [] and LZ77 streamlines, [distance, length] that LZ77 is converted to obtain Matching symbols are marked and are included into area undetermined, finally carry out selective LZ77 inverse transformations, in this way, can prevent omit [away from From length] matching symbols, and the unobstructed of data path streamline can be kept.
In a specific embodiment, the LZ77 inverse transform blocks 10 are additionally operable to：
Parse all [distance, length] matching symbols in area undetermined；
The absolute drift corresponding with input data of all [distance, length] matching symbols is calculated, and will all [distance, length Degree] absolute drifts of matching symbols draws comparative result compared with the absolute drift in line of demarcation；And
If [distance, length] matching symbols point to the file content before line of demarcation, then [distance, length] matching symbol is used Number signified file content recovers original text.
If [distance, length] matching symbols point to the later file content in line of demarcation, then retain [distance, length] matching Symbol.
In the present embodiment, the LZ77 inverse transform blocks 10 can be parsed to [distance, length] matching symbols, that is, Equivalent to decompression [distance, length] matching symbols, then by the way that [distance, length] matching symbols of decompression are calculated into it The absolute displacement of absolute displacement and line of demarcation with output data, the absolute displacement for then comparing both draw comparative result.Such as Fruit finds content pointed by this [distance, length] matching symbols earlier than line of demarcation, then removes original text output buffer pointing to Content read back come, be stored in reverse L Z77 output buffers.
Fig. 5 and Fig. 6 are refer to, in a specific embodiment, the generation of the onrelevant compression blocks based on Deflate Device also includes：
Module 60 is abandoned, for the sliding window before converting the line of demarcation preserved through LZ77 all to be cancelled, wherein, institute It is arbitrary byte sequence to state sliding window.
In the present embodiment, by abandon module 60 can make follow-up input would not go match line of demarcation before history.
The preferred embodiments of the present invention are the foregoing is only, are not intended to limit the scope of the invention, it is every at this Under the inventive concept of invention, the equivalent structure transformation made using description of the invention and accompanying drawing content, or directly/use indirectly It is included in other related technical areas in the scope of patent protection of the present invention.

Claims (10)

  1. A kind of 1. generation method of the onrelevant compression blocks based on Deflate, it is characterised in that the nothing based on Deflate The generation method of association compression blocks includes：
    When line of demarcation between two compression blocks of onrelevant determines, to defining boundaries what is found in the past through LZ77 conversion [distance, length] matching symbols carry out the LZ77 inverse transformations of selectivity；
    Check whether symbol data points to the file content before line of demarcation, the text before any symbol data points to line of demarcation During part content, the symbol data is replaced as output using original input text；
    Output of the LZ77 reinfusion buffering area to the LZ77 inverse transformations of selectivity is distributed to keep in；And
    Compression data block is produced according to the data of LZ77 reinfusion buffering areas.
  2. 2. the generation method of the onrelevant compression blocks based on Deflate as claimed in claim 1, it is characterised in that it is determined that Also include before the step of line of demarcation between two compression blocks of onrelevant：
    The line of demarcation between two compression blocks of onrelevant is determined according to the size of container.
  3. 3. the generation method of the onrelevant compression blocks based on Deflate as claimed in claim 1, it is characterised in that described right The symbol data found in the past through LZ77 conversion that defines boundaries carries out LZ77 inverse transformations and specifically included：
    An area undetermined is marked off from the output buffer of LZ77 conversion；
    The symbol data that LZ77 conversion is found before defining boundaries is marked and is included into area undetermined；And
    The LZ77 inverse transformations of selectivity are carried out to all symbol datas in area undetermined.
  4. 4. the generation method of the onrelevant compression blocks based on Deflate as claimed in claim 3, it is characterised in that the choosing The LZ77 inverse transformations of selecting property specifically include：
    Parse all symbol datas in area undetermined；
    The absolute drift corresponding with input data of all [distance, length] matching symbols is calculated, and will own [distance, length] Absolute drift with symbol draws comparative result compared with the absolute drift in line of demarcation；
    If [distance, length] matching symbols point to the file content before line of demarcation, then [distance, length] matching symbols institute is used The file content of finger recovers original text；
    If [distance, length] matching symbols point to the later file content in line of demarcation, then retain [distance, length] matching symbol Number.
  5. 5. the generation method of the onrelevant compression blocks based on Deflate as claimed in claim 1, it is characterised in that described right The step of [distance, the length] matching symbols found in the past through LZ77 conversion that define boundaries carry out the LZ77 inverse transformations of selectivity Also include before：
    Sliding window before converting the line of demarcation preserved through LZ77 is all cancelled, wherein, the sliding window is arbitrary Byte sequence.
  6. A kind of 6. generating means of the onrelevant compression blocks based on Deflate, it is characterised in that the nothing based on Deflate The generating means of association compression blocks include：
    LZ77 inverse transform blocks, when the line of demarcation between two compression blocks of onrelevant determines, for define boundaries with Preceding [distance, length] matching symbols found through LZ77 conversion carry out the LZ77 inverse transformations of selectivity；
    Check module, to check [distance, length] matching symbols whether point to line of demarcation before file content, any When [distance, length] matching symbols point to the file content before line of demarcation, being replaced using original input text should [distance, length] Matching symbols are as output；
    Distribute module, kept in for distributing output of the LZ77 reinfusion buffering area to the LZ77 inverse transformations of selectivity；With And
    Generation module, for producing compression data block according to the data of LZ77 reinfusion buffering areas.
  7. 7. the generating means of the onrelevant compression blocks based on Deflate as claimed in claim 6, it is characterised in that the base Also it is used in the generating means of Deflate onrelevant compression blocks including determining module：
    The line of demarcation between two compression blocks of onrelevant is determined according to the size of container.
  8. 8. the generating means of the onrelevant compression blocks based on Deflate as claimed in claim 6, it is characterised in that described LZ77 inverse transform blocks are used for：
    An area undetermined is marked off from the output buffer of LZ77 conversion；
    [distance, length] matching symbols that LZ77 conversion is found before defining boundaries are marked and are included into area undetermined；With And
    Markd [distance, the length] matching symbols of institute in area undetermined are carried out with the LZ77 inverse transformations of selectivity.
  9. 9. the generating means of the onrelevant compression blocks based on Deflate as claimed in claim 8, it is characterised in that described LZ77 inverse transform blocks are additionally operable to：
    Parse markd [distance, the length] matching symbols of institute in area undetermined；
    The absolute drift corresponding with input data of all [distance, length] matching symbols is calculated, and will own [distance, length] Absolute drift with symbol draws comparative result compared with the absolute drift in line of demarcation；
    If [distance, length] matching symbols point to the file content before line of demarcation, then [distance, length] matching symbols institute is used The file content of finger recovers original text；
    If [distance, length] matching symbols point to the later file content in line of demarcation, then retain [distance, length] matching symbol Number.
  10. 10. the generating means of the onrelevant compression blocks based on Deflate as claimed in claim 9, it is characterised in that the base Also include in the generating means of Deflate onrelevant compression blocks：
    Module is abandoned, for the sliding window before converting the line of demarcation preserved through LZ77 all to be cancelled, wherein, the slip Window is arbitrary byte sequence.
CN201710698605.6A 2017-08-15 2017-08-15 The generation method and device of onrelevant compression blocks based on Deflate Active CN107688439B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710698605.6A CN107688439B (en) 2017-08-15 2017-08-15 The generation method and device of onrelevant compression blocks based on Deflate

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710698605.6A CN107688439B (en) 2017-08-15 2017-08-15 The generation method and device of onrelevant compression blocks based on Deflate

Publications (2)

Publication Number Publication Date
CN107688439A true CN107688439A (en) 2018-02-13
CN107688439B CN107688439B (en) 2019-03-22

Family

ID=61153402

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710698605.6A Active CN107688439B (en) 2017-08-15 2017-08-15 The generation method and device of onrelevant compression blocks based on Deflate

Country Status (1)

Country Link
CN (1) CN107688439B (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108650256A (en) * 2018-05-09 2018-10-12 南京思达捷信息科技有限公司 A kind of acquirement system of big data
CN114172521A (en) * 2022-02-08 2022-03-11 苏州浪潮智能科技有限公司 Decompression chip verification method, device and equipment and readable storage medium
CN114900555A (en) * 2021-12-14 2022-08-12 合肥哈工轩辕智能科技有限公司 Data distribution method and device based on lossless compression algorithm

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103248369A (en) * 2013-05-21 2013-08-14 南京畅洋科技有限公司 Compression system and method based on FPFA (Field Programmable Gate Array)
CN104811209A (en) * 2015-04-22 2015-07-29 北京理工大学 Compressed file data embedding method and device capable of resisting longest matching detection
CN105610447A (en) * 2015-10-29 2016-05-25 吴均 LZ77 algorithm based zonal coding and compression method
CN106503165A (en) * 2016-10-31 2017-03-15 杭州华为数字技术有限公司 Compression, decompressing method, device and equipment
CN106850141A (en) * 2017-01-20 2017-06-13 济南浪潮高新科技投资发展有限公司 A kind of physical message system data Lossless Compression transmission method of use DEFLATE algorithms

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103248369A (en) * 2013-05-21 2013-08-14 南京畅洋科技有限公司 Compression system and method based on FPFA (Field Programmable Gate Array)
CN104811209A (en) * 2015-04-22 2015-07-29 北京理工大学 Compressed file data embedding method and device capable of resisting longest matching detection
CN105610447A (en) * 2015-10-29 2016-05-25 吴均 LZ77 algorithm based zonal coding and compression method
CN106503165A (en) * 2016-10-31 2017-03-15 杭州华为数字技术有限公司 Compression, decompressing method, device and equipment
CN106850141A (en) * 2017-01-20 2017-06-13 济南浪潮高新科技投资发展有限公司 A kind of physical message system data Lossless Compression transmission method of use DEFLATE algorithms

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108650256A (en) * 2018-05-09 2018-10-12 南京思达捷信息科技有限公司 A kind of acquirement system of big data
CN114900555A (en) * 2021-12-14 2022-08-12 合肥哈工轩辕智能科技有限公司 Data distribution method and device based on lossless compression algorithm
CN114172521A (en) * 2022-02-08 2022-03-11 苏州浪潮智能科技有限公司 Decompression chip verification method, device and equipment and readable storage medium

Also Published As

Publication number Publication date
CN107688439B (en) 2019-03-22

Similar Documents

Publication Publication Date Title
US9390099B1 (en) Method and apparatus for improving a compression ratio of multiple documents by using templates
KR100271861B1 (en) Data compression, expansion method and apparatus and data processing unit and network
JP6319740B2 (en) Method for speeding up data compression, computer for speeding up data compression, and computer program therefor
US9041567B2 (en) Using variable encodings to compress an input data stream to a compressed output data stream
CN107688439A (en) The generation method and device of onrelevant compression blocks based on Deflate
US8669889B2 (en) Using variable length code tables to compress an input data stream to a compressed output data stream
CN108322220A (en) Decoding method, device and coding/decoding apparatus
US10903852B2 (en) Computer system supporting multiple encodings with static data support
CN104199951B (en) Web page processing method and device
US10897270B2 (en) Dynamic dictionary-based data symbol encoding
CN105052040A (en) System and method for multi-stream compression and decompression
US9100042B2 (en) High throughput decoding of variable length data symbols
KR102393743B1 (en) Encoder, decoder and method using mode symbols
CN105791832B (en) Data-encoding scheme and data decoding method and its system
US20220171724A1 (en) Memory system and information processing system
US9787323B1 (en) Huffman tree decompression
CN114282141A (en) Processing method and device for compression format data, electronic equipment and readable storage medium
KR20200121760A (en) Conditional transcoding for encoded data
EP2843842A1 (en) Method and system for LZW based decompression
KR101705461B1 (en) Method and apparatus for encoding and decoding strings
Dhanawe et al. Hiding file on Android Mobile and Sending APK file through whats app using Steganography and Compression techniques
JP2003318739A (en) System and method for compressing data sequence, and computer readable medium
JP2003513313A (en) Variable length decoding
Senthil et al. RIDBE: A lossless, reversible text transformation scheme for better compression
Senthil et al. Reinforced intelligent dictionary-based encoding scheme

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant