CN101237301A - Dynamic data compression technology - Google Patents
Dynamic data compression technology Download PDFInfo
- Publication number
- CN101237301A CN101237301A CNA2008100653977A CN200810065397A CN101237301A CN 101237301 A CN101237301 A CN 101237301A CN A2008100653977 A CNA2008100653977 A CN A2008100653977A CN 200810065397 A CN200810065397 A CN 200810065397A CN 101237301 A CN101237301 A CN 101237301A
- Authority
- CN
- China
- Prior art keywords
- compression
- round
- data
- test
- algorithm
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
The invention relates to a dynamic data compression technology which is a method as follows: by compressively testing data flow, an appropriate compression strategy is selected to balance the load of a system to achieve the promotion of data forwarding speed between wide area networks under the given system ability. The main steps included by the dynamic data compression technology are as follows: (1) dynamic selection of a compression algorithm: a buffer which is sent back to a client end for each round is subject to a new-round dynamic compression test according to the compression state of the buffer last time, the compression algorithm is dynamically selected and the next-round compression test round is determined according to the test result; (2) the calculation of the compression test round: when the compression test has worse effect, the test is considered to be unsuccessful, the next test round can be after the step round, while if the test fails for continuous N times, the next test round is set after the N multiplied by step round. The dynamic data compression technology can be widely used in agent servers, various VPN products or other systems based on C/S.
Description
Technical field
The present invention relates to a kind of data compression technique, particularly a kind of by data flow is done compression verification with select appropriate Compression Strategies come the balance sysmte load, to reach at the dynamic data compression technology that promotes data forwarding speed between wide area network under the set system capability.
Background technology
Along with popularizing rapidly of the Internet and constantly rising violently of netizen's quantity, the speed speed that obtains information has become the key factor of restriction internet development.How under existing Internet resources, to improve the speed that the user obtains information, become a great problem of puzzlement numerous data centers, ISP and enterprise.Therefore the network acceleration product arises at the historic moment, and will become a part indispensable on the following the Internet.80% above data of transmission over networks all are based on the hypertext data of HTTP/HTTPS, thereby the network acceleration product also is primarily aimed at this part and quickens to handle.
Main network acceleration technology has data compression transmission, buffer memory and use SSL accelerator etc. at present, these technology can be accelerated access to netwoks speed to a certain extent aspect the saving network bandwidth, shortening network latency and the minimizing server operation burden significant effect being arranged.In addition, when using the TCP upper layer application under the network environment of high packet loss high latency, many network acceleration products also adopt the method that concrete application layer protocol even Transmission Control Protocol itself are optimized, to reach the effect of acceleration.
Summary of the invention
The objective of the invention is to overcome above-mentioned weak point of the prior art, provide a kind of by feature extracting method quick identification network input data, and the data that repeat are replaced to efficient coding according to the data flow warehouse, coding can be translated back original data at the stream cache of far-end equipment, thereby reduce data quantity transmitted on wide area network.Because data flow warehouse very big (can preserve by hard disk, hundreds of million to hundreds of thousand GB does not wait) can preserve than the more repeat pattern of common compression method.Therefore, can in the data flow storehouse, find most redundant informations, significantly reduce the network data transmission amount, improve a kind of of bandwidth availability ratio and realize the dynamic data compression technology that transfer of data is quickened between net by the stream buffer memory.The object of the present invention is to provide a kind of by data flow is done compression verification with select appropriate Compression Strategies come the balance sysmte load, to reach at the dynamic data compression technology that promotes under the set system capability between wide area network.It comprises the Dynamic Selection of compression algorithm and the dynamic calculation of compression verification round: the Dynamic Selection of compression algorithm: for every buffering that will postback to client of taking turns, according to its last compression state, do the dynamic compression test of a new round, according to test result Dynamic Selection compression algorithm and decision next round compression verification round.The calculating of compression verification round: when the compression verification effect is relatively poor, think test crash, next test round will be after step time, if failure N time continuously, then the next one is tested round and fixed on bigger striding, for example after N*step time.
Purpose of the present invention can reach by following measure:
This dynamic data compression technology (dynamic calculation of the Dynamic Selection of compression algorithm and compression verification round), its special character is that it may further comprise the steps:
(1) initialization (step 101);
(2) extract the compressive state S that connects, check round T (step 102);
Whether (3) detect S is a kind of (step 103) in the algorithm to be selected;
(4) if whether S is not to check round (step 104) for a kind of in the algorithm to be selected then detects current;
(5) if the testing result of step 104 for not, then sends not packed data (step 110);
(6) if the testing result of step 104 for being, is then used all compression algorithm data to be selected successively, check its compression ratio (step 105);
(7) detect its compression ratio and whether reach minimum require (step 106);
(8) if the testing result of step 106 for being, is then upgraded S, and after sending data (step 107) after this compression algorithm, end (step 115);
(9) if the testing result of step 106 for not, then detects whether also have algorithm to be selected (step 108);
(10) if the testing result of step 108 is for being then to return step 105;
(11) if the testing result of step 108 for not, is then adjusted T to bigger stride (step 109);
(12) sending not, packed data (step 110) extremely finishes (step 115);
(13) if S is a kind of in the algorithm to be selected, then adopt corresponding compression algorithm data (step 111);
(14) detect compression ratio and whether reach minimum require (step 112);
(15) if the testing result of step 112 is for being the data (step 113) after then transmission is compressed;
(16) if the testing result of step 112 for not, then reinitializes (step 114) to S and T, and sends not packed data (step 114);
(17) finish (step 115)
The present invention has following advantage:
1. with respect to existing speed technology, the characteristics of maximum of the present invention are to realize simply, implementation cost is low.It needs expensive equipment input unlike caching technology or hardware accelerator etc. when implementing, also need go deep into the protocol theory that bottom is understood various complexity unlike protocol optimization etc.The present invention makes full use of the own characteristic based on the transfer of data of same connection in realization, has only needed the characteristics of decompression algorithm to get final product.
2. the present invention has made full use of the characteristics based on the transfer of data of same connection, and appropriate Compression Strategies has been adopted in different connections, thereby has improved data forwarding speed under set system capability, has strengthened user experience greatly.
3. the dynamic compression technology can also be used in combination with existing speed technology easily, further to promote network speed.Only need during enforcement at client deployment one client-side program, the AppProxy of administration server gets final product in the application service end, can be widely used in acting server, various vpn products or other systems based on C/S.
4. the present invention has developed a kind of application proxy AppProxy of system, and in the research and development process of the compress technique of network data, various compression algorithms has been made test experiments.Experimental result shows, under same running environment, to data of different types, different algorithms is at compression ratio and take and have bigger gap aspect the system load.Most representative is to hypertext data, and algorithm 2 relative algorithms 1 have bigger compression ratio gap and bigger system load to take gap, and to some other test data, the compression ratio gap will be little many.
5. in the research and development process of transmission technology, a large amount of Network Transmission based on connecting has been made test statistics, statistics shows, institute's data flow transmitted in the same connection, if they are divided into polylith by fixed length X, be consistent basically then, comprise whether having compression ratio that compression algorithm that compression is worth and adopts can reach etc. in the characteristic aspect the compression of the polylith data of close forwarding.
Description of drawings
Fig. 1 is the flow chart of dynamic data compression technology of the present invention.
Fig. 2 is the flow chart of a preferred embodiment of the present invention.
Embodiment
The present invention is further detailed in conjunction with the accompanying drawings below:
See also shown in the accompanying drawing, described dynamic data compression technology may further comprise the steps:
(1) AppProxy begins to change when connecting, and is related with connection following data:
Last compression state S=does not have compression, postbacks data round A=0;
Check round B=1 next time; Continuous frequency of failure F=1 (step 201);
(2) when having in the connection of step 201 before new round service end data postback client, establishing data length is X; A then increases progressively one, that is: A++ (step 202);
(3) judge compressive state S (step 203);
(4) if the judged result of step 203 for not, then further the value of judgment data round A whether with the value identical (step 204) of checking round B;
(5) if the judged result of step 204 for not, is then transmitted uncompressed data (step 205);
(6) if the judged result of step 204 for being, is then done algorithm 1 compression, get length Y (step 206);
(7) the judgment data length X whether greater than the minimum compression ratio m of system's appointment with make algorithm 1 compression, the product (step 207) of Len got Y;
(8) if the judged result of step 207, is then thought test crash for not, next test round will be step time after if continuously failure F time then the next one test round and fix on after F*step time, be i.e. B+=
*F, F++ (step 208);
(9) transmit uncompressed data (step 205);
(10) if the judged result of step 207 for being, is then done algorithm 2 compressions, get length Z (step 209);
(11) whether the judgment data length X is greater than the minimum compression ratio m of system's appointment and the product (step 210) of m, step 209 length Z;
(12) judged result as if step 210 is otherwise forwarding algorithm 1 packed data S=algorithm 1 (step 213);
(13) if the judged result of step 210 is for being forwarding algorithm 2 packed data S=algorithms 2 (steps 217) then;
(14) if the judged result of step 203 for being, is then used algorithm 1 test earlier, do algorithm 1 compression, get length Y (step 211);
(15) judge whether compression ratio reaches minimum requirement, promptly whether data length X is greater than the minimum compression ratio m of system's appointment and the product (step 212) of step 211 length Y;
(16) if the judged result of step 212 is for being, forwarding algorithm 1 packed data then, S=algorithm 1 (step 213);
(17) if the judged result of step 212 for not, is then transmitted uncompressed data, S=does not have compression, checks round B=data round A+step, continuously frequency of failure F=1 (step 214);
(18) judged result as if step 212 is not, behind algorithm 1 test crash, uses algorithm 2 compressions, gets length Z (step 215);
(19) judge whether compression ratio reaches minimum requirement, promptly whether data length X specifies the product (step 216) of minimum compression ratio m and m, step 215 length Z greater than system;
(20) if the judged result of step 216 for not, is then transmitted uncompressed data, S=does not have compression, checks round B=data round A+step, continuously frequency of failure F=1 (step 214);
(21) if the judged result of step 216 is for being, forwarding algorithm 2 packed data S=algorithms 2 (steps 217) then.
The above only is preferred embodiment of the present invention, and all equalizations of being done according to claim scope of the present invention change and modify, and all should belong to the covering scope of claim of the present invention.
Claims (1)
1, a kind of dynamic data compression technology is characterized in that, it may further comprise the steps:
(1) initialization (step 101);
(2) extract the compressive state S that connects, check round T (step 102);
Whether (3) detect S is a kind of (step 103) in the algorithm to be selected;
(4) whether if S be a kind of in the algorithm to be selected, then detecting current is inspection round (step 104);
(5) if the testing result of step 104 for not, then sends not packed data (step 110);
(6) if the testing result of step 104 for being, is then used all compression algorithm data to be selected successively, check its compression ratio (step 105);
(7) detect its compression ratio and whether reach minimum require (step 106);
(8) if the testing result of step 106 for being, is then upgraded S, and after sending data (step 107) after this compression algorithm, end (step 115);
(9) if the testing result of step 106 for not, then detects whether also have algorithm to be selected (step 108);
(10) if the testing result of step 108 is for being then to return step 105;
(11) if the testing result of step 108 for not, is then adjusted T to bigger stride (step 109);
(12) sending not, packed data (step 110) extremely finishes (step 115);
(13) if S is a kind of in the algorithm to be selected, then adopt corresponding compression algorithm data (step 111);
(14) detect compression ratio and whether reach minimum require (step 112);
(15) if the testing result of step 112 is for being the data (step 113) after then transmission is compressed;
(16) if the testing result of step 112 for not, then reinitializes (step 114) to S and T, and sends not packed data (step 114);
(17) finish (step 115).
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2008100653977A CN101237301B (en) | 2008-02-22 | 2008-02-22 | Dynamic data compression technology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2008100653977A CN101237301B (en) | 2008-02-22 | 2008-02-22 | Dynamic data compression technology |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101237301A true CN101237301A (en) | 2008-08-06 |
| CN101237301B CN101237301B (en) | 2013-01-23 |
Family
ID=39920678
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2008100653977A Active CN101237301B (en) | 2008-02-22 | 2008-02-22 | Dynamic data compression technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101237301B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102571820A (en) * | 2012-02-21 | 2012-07-11 | 华为技术有限公司 | Method used for transmitting data, compression server and terminal |
| CN103516369A (en) * | 2013-06-20 | 2014-01-15 | 易乐天 | Method and system for self-adaptation data compression and decompression and storage device |
| WO2014056407A1 (en) * | 2012-10-09 | 2014-04-17 | Mediatek Inc. | Data processing apparatus with adaptive compression algorithm selection for data communication based on sensor input/display configuration over display interface and related data processing method |
| CN108322220A (en) * | 2018-02-08 | 2018-07-24 | 华为技术有限公司 | Decoding method, device and coding/decoding apparatus |
| WO2023198150A1 (en) * | 2022-04-13 | 2023-10-19 | 杭州脑芯科技有限公司 | Data compression method for electroencephalogram data, chip, device, and storage medium |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB0306973D0 (en) * | 2003-03-26 | 2003-04-30 | British Telecomm | Transmitting video |
| US20060176953A1 (en) * | 2005-02-04 | 2006-08-10 | Nader Mohsenian | Method and system for video encoding with rate control |
| CN100596108C (en) * | 2007-04-20 | 2010-03-24 | 杭州华三通信技术有限公司 | Data transfer method and device |
-
2008
- 2008-02-22 CN CN2008100653977A patent/CN101237301B/en active Active
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102571820B (en) * | 2012-02-21 | 2015-11-25 | 华为技术有限公司 | For transmitting the method for data, compression service device and terminal |
| CN102571820A (en) * | 2012-02-21 | 2012-07-11 | 华为技术有限公司 | Method used for transmitting data, compression server and terminal |
| US9514705B2 (en) | 2012-10-09 | 2016-12-06 | Mediatek Inc. | Data processing apparatus with adaptive compression algorithm selection based on visibility of compression artifacts for data communication over display interface and related data processing method |
| WO2014056407A1 (en) * | 2012-10-09 | 2014-04-17 | Mediatek Inc. | Data processing apparatus with adaptive compression algorithm selection for data communication based on sensor input/display configuration over display interface and related data processing method |
| US9355613B2 (en) | 2012-10-09 | 2016-05-31 | Mediatek Inc. | Data processing apparatus for transmitting/receiving compression-related indication information via display interface and related data processing method |
| US9466258B2 (en) | 2012-10-09 | 2016-10-11 | Mediatek Inc. | Data processing apparatus with adaptive compression algorithm selection for data communication based on sensor input/display configuration over display interface and related data processing method |
| US9633624B2 (en) | 2012-10-09 | 2017-04-25 | Mediatek Inc. | Data processing apparatus for transmitting/receiving compression-related indication information via display interface and related data processing method |
| US9711109B2 (en) | 2012-10-09 | 2017-07-18 | Mediatek Inc. | Data processing apparatus for transmitting/receiving compression-related indication information via display interface and related data processing method |
| US9773469B2 (en) | 2012-10-09 | 2017-09-26 | Mediatek Inc. | Data processing apparatus with adaptive compression/de-compression algorithm selection for data communication over display interface and related data processing method |
| CN103516369A (en) * | 2013-06-20 | 2014-01-15 | 易乐天 | Method and system for self-adaptation data compression and decompression and storage device |
| CN103516369B (en) * | 2013-06-20 | 2016-12-28 | 易乐天 | A kind of self-adapting data compression and decompress method and system and storage device |
| CN108322220A (en) * | 2018-02-08 | 2018-07-24 | 华为技术有限公司 | Decoding method, device and coding/decoding apparatus |
| WO2019153700A1 (en) * | 2018-02-08 | 2019-08-15 | 华为技术有限公司 | Encoding and decoding method, apparatus and encoding and decoding device |
| WO2023198150A1 (en) * | 2022-04-13 | 2023-10-19 | 杭州脑芯科技有限公司 | Data compression method for electroencephalogram data, chip, device, and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101237301B (en) | 2013-01-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10411956B2 (en) | Enabling planned upgrade/downgrade of network devices without impacting network sessions | |
| US9253065B2 (en) | Latency measurement in resource requests | |
| US9185012B2 (en) | Latency measurement in resource requests | |
| US20190020562A1 (en) | Latency measurement in resource requests | |
| KR101383905B1 (en) | method and apparatus for processing server load balancing with the result of hash function | |
| CN101237301B (en) | Dynamic data compression technology | |
| CN102594877B (en) | In conjunction with the method, the system that are redirected download request and the service of agency service accelerating network | |
| US20060059246A1 (en) | System and method for connection optimization | |
| CN110225417A (en) | Data processing method and server, the method and server that detect Caton | |
| US9491113B2 (en) | Managing network connections for processing network resources | |
| US9787521B1 (en) | Concurrent loading of session-based information | |
| CN101997822A (en) | Streaming media content delivery method, system and equipment | |
| CN103580988A (en) | Method for message receiving, pushing and transmitting, device, server group and system | |
| US8516156B1 (en) | Adaptive compression | |
| CN104768079A (en) | Method, device and system for distributing multimedia resources | |
| US9900368B2 (en) | Method for optimising downloading of data | |
| CN103873443B (en) | Information processing method, local proxy server and network proxy server | |
| CN104519088A (en) | Buffer memory system realization method and buffer memory system | |
| WO2014019287A1 (en) | System and method for optimizing inter-node communication in content distribution network | |
| CN105893429A (en) | Method and system for optimizing WEB cache | |
| US20140074912A1 (en) | Communication apparatus, relay apparatus and communication method | |
| US11082474B2 (en) | Data buffering method and apparatus in adaptive streaming service | |
| El Khoury et al. | Analysis of scalable TCP congestion control algorithm | |
| CN104283957A (en) | CDN cache method based on continuous connectionism | |
| CN104580392A (en) | Method, device and equipment for retaining long connection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CP03 | Change of name, title or address | ||
| CP03 | Change of name, title or address |
Address after: Nanshan District Xueyuan Road in Shenzhen city of Guangdong province 518000 No. 1001 Nanshan Chi Park building A1 layer Patentee after: SINFOR Polytron Technologies Inc Address before: 518052 room 410, science and technology innovation center, 1 Qilin Road, Shenzhen, Guangdong, Nanshan District Patentee before: Shenxinfu Electronics Science and Technology Co., Ltd., Shenzhen |