CN101237301B - Dynamic data compression technology - Google Patents
Dynamic data compression technology Download PDFInfo
- Publication number
- CN101237301B CN101237301B CN2008100653977A CN200810065397A CN101237301B CN 101237301 B CN101237301 B CN 101237301B CN 2008100653977 A CN2008100653977 A CN 2008100653977A CN 200810065397 A CN200810065397 A CN 200810065397A CN 101237301 B CN101237301 B CN 101237301B
- Authority
- CN
- China
- Prior art keywords
- compression
- round
- data
- test
- algorithm
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Compression, Expansion, Code Conversion, And Decoders (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention relates to a dynamic data compression technology which is a method as follows: by compressively testing data flow, an appropriate compression strategy is selected to balance the load of a system to achieve the promotion of data forwarding speed between wide area networks under the given system ability. The main steps included by the dynamic data compression technology are as follows: (1) dynamic selection of a compression algorithm: a buffer which is sent back to a client end for each round is subject to a new-round dynamic compression test according to the compression state of the buffer last time, the compression algorithm is dynamically selected and the next-round compression test round is determined according to the test result; (2) the calculation of the compression test round: when the compression test has worse effect, the test is considered to be unsuccessful, the next test round can be after the step round, while if the test fails for continuous N times, the next test round is set after the N multiplied by step round. The dynamic data compression technology can be widely used in agent servers, various VPN products or other systems based on C/S.
Description
Technical field
The present invention relates to a kind of data compression technique, particularly a kind of by data flow is done compression verification with select appropriate Compression Strategies come the balance sysmte load, to reach at the dynamic data compression technology that promotes data retransmission speed between wide area network under the set system capability.
Background technology
Along with popularizing rapidly of the Internet and constantly rising violently of netizen's quantity, the speed speed of obtaining information has become the key factor of restriction internet development.How under existing Internet resources, to improve the speed of user's obtaining information, become a great problem of puzzlement numerous data centers, ISP and enterprise.Therefore the network acceleration product arises at the historic moment, and smoothes out with the fingers and become a part indispensable on the Future Internet.80% above data of transmission over networks all are based on the hypertext data of HTTP/HTTPS, thereby the network acceleration product also accelerates to process mainly for this part.
Main network acceleration technology has data compression transmission, buffer memory and use SSL accelerator etc. at present, these technology can be accelerated access to netwoks speed to a certain extent aspect the saving network bandwidth, shortening network latency and the minimizing server operation burden significant effect being arranged.In addition, when the net environment of high packet loss high latency used the TCP upper layer application, many network acceleration products also adopted the method that concrete application layer protocol even Transmission Control Protocol itself are optimized, to reach the effect of acceleration.
Summary of the invention
The object of the invention is to overcome above-mentioned weak point of the prior art, provide a kind of by the quick recognition network input of feature extracting method data, and become efficient coding according to the data replacement that repetition is smoothed out with the fingers in the data flow warehouse, can smooth out with the fingers coding at the stream cache of far-end equipment and translate back original data, thereby reduce the data volume of transmitting at wide area network.Because data flow warehouse very large (can preserve by hard disk, hundreds of million to hundreds of thousand GB does not wait) can preserve the more repeat pattern of this common compression method.Therefore, can in the data flow storehouse, find most redundant informations, significantly reduce the network data transmission amount, improve a kind of of bandwidth availability ratio and realize the dynamic data compression technology that transfer of data is encrypted between net by the stream buffer memory.The object of the invention be to provide a kind of by data flow is made compression verification with select appropriate Compression Strategies come the balance sysmte load, to reach at the dynamic data compression technology that promotes under the set system capability between wide area network.It comprises the Dynamic Selection of compression algorithm and the dynamic calculation of compression verification round: the Dynamic Selection of compression algorithm: for every buffering that will postback to client of taking turns, according to its last compression state, do the dynamic compression test of a new round, according to test result Dynamic Selection compression algorithm, and determine next round compression verification round.The calculating of compression verification round: when the compression verification effect is relatively poor, think test crash, next test round will be after step time, if continuous failed N time, then the next one is tested round and is fixed on larger striding, for example after N*step time.
Purpose of the present invention can reach by following measures:
This dynamic data compression technology (dynamic calculation of the Dynamic Selection of compression algorithm and compression verification round), its special character be, it may further comprise the steps:
(1) initialization (step 101);
(2) extract the compressive state S that connects, check round T (step 102);
Whether (3) detect S is a kind of (step 103) in the algorithm to be selected;
(4) whether if S is not a kind of in the algorithm to be selected, then detecting current is to check round (step 104);
(5) if the testing result of step 104 is no, then send not packed data (step 110);
(6) if the testing result of step 104 is yes, then use successively all compression algorithm data to be selected, check its compression ratio (step 105);
(7) detect its compression ratio and whether reach minimum require (step 106);
(8) if the testing result of step 106 is yes, then upgrade S, and after sending data (step 107) after this compression algorithm, finish (step 115);
(9) if the testing result of step 106 is no, then detect whether also have algorithm to be selected (step 108);
(10) if the testing result of step 108 is yes, then return step 105;
(11) if the testing result of step 108 is no, then adjust T to larger stride (step 109);
(12) sending not, packed data (step 110) extremely finishes (step 115);
(13) if S is a kind of in the algorithm to be selected, then adopt corresponding compression algorithm data (step 111);
(14) detect compression ratio and whether reach minimum require (step 112);
(15) if the testing result of step 112 is yes, then send the data (step 113) after compressing;
(16) if the testing result of step 112 is no, then S and T are reinitialized (step 114), and send not packed data (step 114);
(17) finish (step 115)
The present invention has following advantage:
1. with respect to existing speed technology, the characteristics of maximum of the present invention are to realize simply, implementation cost is low.It needs expensive equipment investment unlike caching technology or hardware accelerator etc. when implementing, also need to go deep into the protocol theory that bottom is understood various complexity unlike protocol optimization etc.The present invention takes full advantage of the own characteristic based on the transfer of data of same connection in realization, has only needed the characteristics of decompression algorithm to get final product.
2. the present invention takes full advantage of the characteristics based on the transfer of data of same connection, and appropriate Compression Strategies has been adopted in different connections, thereby has improved data retransmission speed under set system capability, has greatly strengthened user's experience.
3. the dynamic compression technology can also be combined with existing speed technology easily, further to promote network speed.Only need at client deployment one client-side program during enforcement, the AppProxy of administration server gets final product in the application service end, can be widely used in acting server, various vpn products or other systems based on C/S.
4. the present invention has developed a kind of application proxy AppProxy of system, and in the research and development process of the compress technique of network data, various compression algorithms has been made test experiments.Experimental result shows, under same running environment, to dissimilar data, different algorithms is at compression ratio and take and have larger gap aspect the system load.Most representative is to hypertext data, and algorithm 2 relative algorithms 1 have larger compression ratio gap and larger system load to take gap, and to some other test data, the compression ratio gap will be little many.
5. in the research and development process of transmission technology, a large amount of Internet Transmissions based on connecting has been made test statistics, statistics shows, the data flow of transmitting in the same connection, if they are divided into polylith by fixed length X, be consistent basically in the characteristic aspect the polylith data compression of close forwarding then, comprise that whether having compression is worth, and the compression algorithm that the adopts compression ratio that can reach etc.
Description of drawings
Fig. 1 is the flow chart of dynamic data compression technology of the present invention.
Fig. 2 is the flow chart of a preferred embodiment of the present invention.
Embodiment
The present invention is further detailed in conjunction with the accompanying drawings below:
See also shown in the accompanying drawing, described dynamic data compression technology may further comprise the steps:
(1) AppProxy begins to turn when connecting, and is related with connection following data:
Last compression state S=postbacks data round A=0 without compression;
Check round B=1 next time; Continuous frequency of failure F=1 (step 201);
(2) when having in the connection of step 201 before new round service end data postback client, establishing data length is X; A then increases progressively one, that is: A++ (step 202);
(3) judge compressive state S (step 203);
(4) if the determination result is NO for step 203, the value of then further judging data round A whether with the value identical (step 204) that checks round B;
(5) if the determination result is NO for step 204, then transmit uncompressed data (step 205);
(6) if the judged result of step 204 is yes, then do algorithm 1 compression, get length Y (step 206);
(7) judge data length X whether greater than the minimum compression ratio m of system's appointment with make algorithm 1 compression, the product (step 207) of Len got Y;
(8) if the determination result is NO for step 207, then think test crash, next test round will be after step time, if continuous failed F time, then the next one is tested round and is fixed on after F*step time, i.e. B+=*F, F++ (step 208);
(9) transmit uncompressed data (step 205);
(10) if the judged result of step 207 is yes, then do algorithm 2 compressions, get length Z (step 209);
(11) judge that whether data length X is greater than the minimum compression ratio m of system's appointment and the product (step 210) of m, step 209 length Z;
(12) if the determination result is NO for step 210, forwarding algorithm 1 packed data S=algorithm 1 (step 213) then;
(13) if the judged result of step 210 is yes, forwarding algorithm 2 packed data S=algorithms 2 (steps 217) then;
(14) if the judged result of step 203 is yes, then use first algorithm 1 test, do algorithm 1 compression, get length Y (step 211);
(15) judge whether compression ratio reaches minimum requirement, namely whether data length X is greater than the minimum compression ratio m of system's appointment and the product (step 212) of step 211 length Y;
(16) if the judged result of step 212 is yes, forwarding algorithm 1 packed data then, S=algorithm 1 (step 213);
(17) if the determination result is NO for step 212, then transmit uncompressed data, S=checks round B=data round A+step, continuously frequency of failure F=1 (step 214) without compression;
(18) if the determination result is NO for step 212, behind algorithm 1 test crash, use algorithm 2 compressions, get length Z (step 215);
(19) judge whether compression ratio reaches minimum requirement, namely whether data length X specifies the product (step 216) of minimum compression ratio m and m, step 215 length Z greater than system;
(20) if the determination result is NO for step 216, then transmit uncompressed data, S=checks round B=data round A+step, continuously frequency of failure F=1 (step 214) without compression;
(21) if the judged result of step 216 is yes, forwarding algorithm 2 packed data S=algorithms 2 (steps 217) then.
The above only is preferred embodiment of the present invention, and all equalizations of doing according to claim scope of the present invention change and modify, and all should belong to the covering scope of claim of the present invention.
Claims (1)
1. a dynamic data compression method is characterized in that, may further comprise the steps:
(1) initialization;
(2) extract the compressive state S that connects, check round T;
Whether (3) detect S is a kind of in the algorithm to be selected, if testing result is no, and execution in step (4) then, if testing result is yes, execution in step (10) then;
Whether be check round, if testing result is no, then go to step (9) if (4) detecting current, if testing result is yes, and execution in step (5) then;
(5) use successively all compression algorithm data to be selected, check its compression ratio;
(6) detect its compression ratio and whether reach minimum requirement, if testing result is yes, then upgrade S, and after sending data after this compression algorithm, go to step (14), if testing result is no, execution in step (7) then;
(7) detect whether to also have algorithm to be selected, if testing result is yes, then return step (5), if testing result is no, execution in step (8) then;
(8) adjust T to larger striding;
(9) send not packed data and go to step (14);
(10) adopt corresponding compression algorithm data;
(11) detect compression ratio and whether reach minimum requirement, if testing result is yes, execution in step (12) then, if testing result is no, execution in step (13) then;
(12) data after the transmission compression;
(13) S and T are reinitialized, and go to step (9);
(14) finish.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2008100653977A CN101237301B (en) | 2008-02-22 | 2008-02-22 | Dynamic data compression technology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2008100653977A CN101237301B (en) | 2008-02-22 | 2008-02-22 | Dynamic data compression technology |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101237301A CN101237301A (en) | 2008-08-06 |
| CN101237301B true CN101237301B (en) | 2013-01-23 |
Family
ID=39920678
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2008100653977A Active CN101237301B (en) | 2008-02-22 | 2008-02-22 | Dynamic data compression technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101237301B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102571820B (en) * | 2012-02-21 | 2015-11-25 | 华为技术有限公司 | For transmitting the method for data, compression service device and terminal |
| US9355613B2 (en) | 2012-10-09 | 2016-05-31 | Mediatek Inc. | Data processing apparatus for transmitting/receiving compression-related indication information via display interface and related data processing method |
| CN103516369B (en) * | 2013-06-20 | 2016-12-28 | 易乐天 | A kind of self-adapting data compression and decompress method and system and storage device |
| CN108322220A (en) * | 2018-02-08 | 2018-07-24 | 华为技术有限公司 | Decoding method, device and coding/decoding apparatus |
| CN114665886A (en) * | 2022-04-13 | 2022-06-24 | 杭州脑芯科技有限公司 | Data compression method, chip, equipment and storage medium for electroencephalogram data |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1765103A (en) * | 2003-03-26 | 2006-04-26 | 英国电讯有限公司 | Transmitting over a network |
| US20060176953A1 (en) * | 2005-02-04 | 2006-08-10 | Nader Mohsenian | Method and system for video encoding with rate control |
| CN101035086A (en) * | 2007-04-20 | 2007-09-12 | 杭州华三通信技术有限公司 | Data transfer method and device |
-
2008
- 2008-02-22 CN CN2008100653977A patent/CN101237301B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1765103A (en) * | 2003-03-26 | 2006-04-26 | 英国电讯有限公司 | Transmitting over a network |
| US20060176953A1 (en) * | 2005-02-04 | 2006-08-10 | Nader Mohsenian | Method and system for video encoding with rate control |
| CN101035086A (en) * | 2007-04-20 | 2007-09-12 | 杭州华三通信技术有限公司 | Data transfer method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101237301A (en) | 2008-08-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10411956B2 (en) | Enabling planned upgrade/downgrade of network devices without impacting network sessions | |
| US9253065B2 (en) | Latency measurement in resource requests | |
| US7826487B1 (en) | Coalescing acknowledgement responses to improve network communications | |
| CN101237301B (en) | Dynamic data compression technology | |
| CN106304203B (en) | Data transmission method and device | |
| EP2493149B1 (en) | Cloud server/thin client/gateway selective browser instantiation | |
| CN101119385A (en) | Method for enhancing HTTP network velocity using WebPush | |
| US9491113B2 (en) | Managing network connections for processing network resources | |
| CN101796491A (en) | Improve the method and apparatus of HTTP performance on the long delay link | |
| CN106612284B (en) | Streaming data transmission method and device | |
| US9787521B1 (en) | Concurrent loading of session-based information | |
| CN101997822A (en) | Streaming media content delivery method, system and equipment | |
| US8516156B1 (en) | Adaptive compression | |
| WO2015142752A1 (en) | Transport accelerator implementing a multiple interface architecture | |
| US9900368B2 (en) | Method for optimising downloading of data | |
| CN103873443B (en) | Information processing method, local proxy server and network proxy server | |
| WO2014019287A1 (en) | System and method for optimizing inter-node communication in content distribution network | |
| US20140074912A1 (en) | Communication apparatus, relay apparatus and communication method | |
| MacLarty et al. | Policy-based content delivery: an active network approach | |
| US11082474B2 (en) | Data buffering method and apparatus in adaptive streaming service | |
| CN104283957A (en) | CDN cache method based on continuous connectionism | |
| CN104580392A (en) | Method, device and equipment for retaining long connection | |
| CN104883398B (en) | A kind of document down loading method, apparatus and system | |
| CN112769730B (en) | Page compression method, device, client and server | |
| Armstrong et al. | Just-in-time push prefetching: Accelerating the mobile web |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CP03 | Change of name, title or address |
Address after: Nanshan District Xueyuan Road in Shenzhen city of Guangdong province 518000 No. 1001 Nanshan Chi Park building A1 layer Patentee after: SINFOR Polytron Technologies Inc Address before: 518052 room 410, science and technology innovation center, 1 Qilin Road, Shenzhen, Guangdong, Nanshan District Patentee before: Shenxinfu Electronics Science and Technology Co., Ltd., Shenzhen |
|
| CP03 | Change of name, title or address |