CN105322969B - The method and device of data compression and decompression - Google Patents

The method and device of data compression and decompression Download PDF

Info

Publication number
CN105322969B
CN105322969B CN201510907676.3A CN201510907676A CN105322969B CN 105322969 B CN105322969 B CN 105322969B CN 201510907676 A CN201510907676 A CN 201510907676A CN 105322969 B CN105322969 B CN 105322969B
Authority
CN
China
Prior art keywords
data
key
compression
original transmitted
value pair
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201510907676.3A
Other languages
Chinese (zh)
Other versions
CN105322969A (en
Inventor
梁敬彪
任建峰
李勇智
刘文娇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Qihoo Technology Co Ltd
Original Assignee
Beijing Qihoo Technology Co Ltd
Qizhi Software Beijing Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Qihoo Technology Co Ltd, Qizhi Software Beijing Co Ltd filed Critical Beijing Qihoo Technology Co Ltd
Priority to CN201510907676.3A priority Critical patent/CN105322969B/en
Publication of CN105322969A publication Critical patent/CN105322969A/en
Priority to PCT/CN2016/104567 priority patent/WO2017097071A1/en
Application granted granted Critical
Publication of CN105322969B publication Critical patent/CN105322969B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
    • H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)

Abstract

The present invention provides a kind of data compression and the schemes of decompression, original transmitted data is analyzed, determine to include the data structure of original transmitted data and the data characteristic information of size of data, then, according to data characteristic information, judge whether to carry out compression processing to original transmitted data, when judging result indicates to execute compression processing, based on predetermined key compress mode in key compression fit list, conversion is carried out to the key of key-value pair original in original transmitted data and generates corresponding compression bonding, the compressed transmission data including corresponding compression key-value pair is generated based on compression bonding.Network bandwidth can be saved in data transmission procedure by this programme, avoid data volume it is excessive when not expected loss of data the case where, realize the high efficiency of transmission of data；Meanwhile the response time of computer digital animation is improved, to provide guarantee for the transmission data that later period efficient process receives, improve the experience of user.

Description

The method and device of data compression and decompression
Technical field
The present invention relates to field of computer technology, specifically, the present invention relates to a kind of data compression and the methods of decompression And device.
Background technique
With the continuous development of computer technology, various SDK (the Software Development in terms of software development Kit, Software Development Kit) and development approach also constantly updating.During software application, it usually needs by connecing Various types of data are passed in oral instructions, when the data volume of transmission is larger, as contained a large amount of longer character in transmission data Data name and its corresponding specific value, on the one hand, a large amount of CPU (Central of system will be expended after data transmitting Processing Unit, central processing unit) resource goes to parse each data name and its corresponding numerical value, to influence the place of terminal Speed is managed, and the data name of a large amount of longer characters will waste a large amount of terminal data memory space in storage；On the other hand, if Data are transmitted by network mode, will expend a large amount of network bandwidth, or even there is a situation where data to be truncated, and cause to count According to loss.
Therefore, it is necessary to a kind of scheme for being compressed of parameter name to longer character in transmission data, make comprising largely compared with The transmission data name of long character and its corresponding specific value realize efficient transmitting during interface transmits, and reach saving The purpose of network bandwidth resources and system CPU resource, and realize and transmission data are efficiently read and write in the database, thus The further experience for improving user.
Summary of the invention
To overcome above-mentioned technical problem or at least being partially solved above-mentioned technical problem, spy proposes following technical scheme:
The embodiment of the present invention proposes a kind of method of data compression, comprising:
Original transmitted data is analyzed, determines to include the data structure of original transmitted data and the data of size of data Characteristic information；
According to data characteristic information, judge whether to carry out compression processing to original transmitted data；
When judging result indicates to execute compression processing, based on predetermined key compress mode in the list of key compression fit, to original The key of original key-value pair carries out conversion and generates corresponding compression bonding in the transmission data that begin；
Being generated based on compression bonding includes the corresponding compressed transmission data for compressing key-value pair.
Preferably, according to data characteristic information, judge whether to carry out compression processing to original transmitted data, comprising:
Judge the size of data of original transmitted data and the relationship of the first tentation data size threshold value；
If the size of data that judging result is original transmitted data is greater than the first tentation data size threshold value, determine to original The transmission data that begin carry out compression processing.
Preferably, according to data characteristic information, judge whether to carry out compression processing to original transmitted data, comprising:
Judge that the size of data of original transmitted data is big with the first tentation data size threshold value and the second tentation data respectively The relationship of small threshold value, the first tentation data size threshold value are greater than the second tentation data size threshold value；
If the size of data that judging result is original transmitted data is greater than the second tentation data size threshold value and less than first When tentation data size threshold value, judged whether to carry out compression processing to original transmitted data according to data structure.
Preferably, judged whether to carry out compression processing to original transmitted data according to data structure, be specifically included:
Include in the quantity and original transmitted data of the key-value pair of the first key having the same in statistics original transmitted data Key-value pair total quantity the first ratio；
When the first ratio be greater than the first predetermined ratio threshold value, and the character quantity of the first key be greater than the first predetermined character quantity When threshold value, judgement needs to carry out compression processing to original transmitted data.
Preferably, judged whether to carry out compression processing to original transmitted data according to data structure, be specifically included:
It is predetermined to judge whether the character quantity of the second key in original transmitted data with longest character length is greater than second Character quantity threshold value；
If judge that the character quantity of the second key is greater than the second predetermined character amount threshold, count complete in original transmitted data The character quantity of the second key of portion accounts for the second ratio of total character quantity of original transmitted data；
When the second ratio is greater than the second predetermined ratio threshold value, judgement needs to carry out compression processing to original transmitted data.
Preferably, this method further include:
Determine the history frequency of occurrences of the key of each key-value pair in original transmitted data within a predetermined period of time；
Wherein, according to data characteristic information, judge whether to carry out compression processing to original transmitted data, comprising:
According to data characteristic information, and the history frequency of occurrences is combined, judges whether to carry out at compression original transmitted data Reason.
It is preferably based on compression bonding and generates and include the steps that the compressed transmission data for compressing key-value pair accordingly further wraps It includes:
Corresponding compression key-value pair is generated based on compression bonding, and associated compressive state mark is set；
By compression key-value pair and associated compressive state identifier combination to generate compressed transmission data.
Preferably, this method further include:
Corresponding key, which is generated, based on key compression fit list compresses dictionary file；
According to the application acquisition request received, requesting party is sent to using key compression dictionary file as configuration file.
Another embodiment of the present invention proposes a kind of method of data decompression, comprising:
Judge whether the transmission data received are compressed transmission data；
When judging to determine transmission data as compressed transmission data, parses and extract the compression key assignments in compressed transmission data Pair compression bonding；
Predetermined key in key compression fit list based on pre-configuration decompresses mode, carries out decompression processing to compression bonding, with Acquire corresponding original key-value pair.
Preferably, judge that the step of whether the transmission data received are compressed transmission data further comprises:
Judge to transmit whether the key-value pair in data includes associated compressive state mark；
Wherein, it when judging to determine transmission data as compressed transmission data, parses and extracts the pressure in compressed transmission data The step of compression bonding of contracting key-value pair, further comprises:
When the key-value pair in transmission data includes associated compressive state mark, it is determined that transmission data are compression transmission Data；And extract the compression bonding for identifying relevant compression key-value pair in compressed transmission data to compressive state.
Preferably, this method further include:
Configuration file is received, includes that key compresses dictionary file in configuration file；
Dictionary file configuration, which is compressed, according to key generates local key compression fit list.
The embodiment of the present invention proposes a kind of device of data compression, comprising:
First determining module determines the data knot including original transmitted data for analyzing original transmitted data The data characteristic information of structure and size of data；
First judgment module carries out compression processing to original transmitted data for judging whether according to data characteristic information；
Conversion module, for being based on predetermined key in the list of key compression fit when judging result indicates to execute compression processing Compress mode carries out conversion to the key of key-value pair original in original transmitted data and generates corresponding compression bonding；
First generation module, being generated based on compression bonding includes the corresponding compressed transmission data for compressing key-value pair.
Preferably, first judgment module specifically includes:
Second judgment submodule, for judging the size of data and the first tentation data size threshold value of original transmitted data Relationship；
Second determines submodule, if the size of data for judging result to be original transmitted data is greater than the first tentation data When size threshold value, determines and compression processing is carried out to original transmitted data.
Preferably, first judgment module includes:
Third judging submodule, for judge the size of data of original transmitted data respectively with the first tentation data size threshold The relationship of value and the second tentation data size threshold value, the first tentation data size threshold value are greater than the second tentation data size threshold value；
4th judging submodule, if the size of data for judging result to be original transmitted data is greater than the second tentation data Size threshold value and when less than the first tentation data size threshold value, judges whether to press original transmitted data according to data structure Contracting processing.
Preferably, the 4th judging submodule specifically includes:
First statistic unit, for counting in original transmitted data the quantity of the key-value pair of the first key having the same and former First ratio of the key-value pair total quantity for including in the transmission data that begin；
First judging unit, for being greater than the first predetermined ratio threshold value when the first ratio, and the character quantity of the first key is big When the first predetermined character amount threshold, judgement needs to carry out compression processing to original transmitted data.
Preferably, the 4th judging submodule specifically includes:
Second judgment unit, for judging the character quantity of the second key in original transmitted data with longest character length Whether the second predetermined character amount threshold is greater than；
Second statistic unit, if when for judging that the character quantity of the second key is greater than the second predetermined character amount threshold, system All the character quantity of the second key accounts for the second ratio of total character quantity of original transmitted data in meter original transmitted data；
Third judging unit, for when the second ratio is greater than the second predetermined ratio threshold value, judgement to be needed to original transmitted Data carry out compression processing.
Preferably, the device further include:
Third determining module, for determining the history of the key of each key-value pair in original transmitted data within a predetermined period of time The frequency of occurrences；
Wherein, first judgment module is specifically used for according to data characteristic information, and combines the history frequency of occurrences, judges whether Compression processing is carried out to original transmitted data.
Preferably, the first generation module includes:
Setting unit is identified, for generating corresponding compression key-value pair based on compression bonding, and associated compression shape is set State mark；
Combination producing unit, for key-value pair and associated compressive state identifier combination will to be compressed to generate compression transmission Data.
Preferably, the device further include:
Second generation module compresses dictionary file for generating corresponding key based on the list of key compression fit；
Sending module, for being sent out using key compression dictionary file as configuration file according to the application acquisition request received It send to requesting party.
Another embodiment of the present invention proposes a kind of device of data decompression, comprising:
6th judgment module, whether the transmission data for judging to receive are compressed transmission data；
Extraction module is parsed, is passed for when judging to determine transmission data as compressed transmission data, parsing and extracting compression The compression bonding of compression key-value pair in transmission of data；
Decompression processing module, the predetermined key in the key compression fit list based on pre-configuration decompress mode, to compression bonding into Row decompression processing, to acquire corresponding original key-value pair.
Preferably, the 6th judgment module is further used for judging whether the key-value pair transmitted in data includes associated pressure Contracting status indicator；
Wherein, parsing extraction module is further used for when the key-value pair in transmission data including associated compressive state mark Know, it is determined that transmission data are compressed transmission data；And extract pressure relevant to compressive state mark in compressed transmission data The compression bonding of contracting key-value pair.
Preferably, the device further include:
Receiving module includes that key compresses dictionary file for receiving configuration file, in configuration file；
Third generation module generates local key compression fit list for compressing dictionary file configuration according to key.
In the embodiment of the present invention, the scheme of a kind of data compression and decompression is proposed, it is former by analysis in server end Begin transmission data, judges whether to carry out compression processing to original transmitted data, when judging result indicates to execute compression processing, base The predetermined key compress mode in key compression fit list carries out conversion to the key of key-value pair original in original transmitted data and generates phase The compression bonding answered, then being generated based on compression bonding includes the corresponding compressed transmission data for compressing key-value pair；If it is logical to transmit data When crossing network mode and being transmitted, conversion is carried out by the key to key-value pair original in original transmitted data and generates corresponding compression Key can save network bandwidth in data transmission procedure, avoid data volume it is excessive when not expected loss of data the case where, Realize the high efficiency of transmission of data；Meanwhile improving the response time of computer digital animation；In the client of terminal device, When the transmission data for judging to receive is compressed transmission datas, parses and extract the compression key-value pair in compressed transmission data Compression bonding, then the predetermined key in the key compression fit list based on pre-configuration decompresses mode, carries out decompression processing to compression bonding, To acquire corresponding original key-value pair, it can quickly and accurately decompress to obtain the original value of compression bonding, realize and efficiently obtain Transmission data are taken, to provide guarantee for the transmission data that later period efficient process receives, improve the experience of user.
The additional aspect of the present invention and advantage will be set forth in part in the description, these will become from the following description Obviously, or practice through the invention is recognized.
Detailed description of the invention
Above-mentioned and/or additional aspect and advantage of the invention will become from the following description of the accompanying drawings of embodiments Obviously and it is readily appreciated that, in which:
Fig. 1 is the flow diagram of the method for the data compression of one embodiment in the present invention；
Fig. 2 is the flow diagram of the method for the data compression of a preferred embodiment in the present invention；
Fig. 3 is the flow diagram of the method for the data compression of a preferred embodiment in the present invention；
Fig. 4 is the flow diagram of the method for the data compression of a preferred embodiment in the present invention；
Fig. 5 is the flow diagram of the method for the data decompression of another embodiment in the present invention；
Fig. 6 is the structural schematic diagram of the device of the data compression of another embodiment in the present invention；
Fig. 7 is the structural schematic diagram of the device of the data compression of another preferred embodiment in the present invention；
Fig. 8 is the structural schematic diagram of the device of the data compression of another preferred embodiment in the present invention；
Fig. 9 is the structural schematic diagram of the device of the data compression of another preferred embodiment in the present invention；
Figure 10 is the structural schematic diagram of the device of the data decompression of another embodiment in the present invention.
Specific embodiment
The embodiment of the present invention is described below in detail, the example of embodiment is shown in the accompanying drawings, wherein identical from beginning to end Or similar label indicates same or similar element or element with the same or similar functions.It is retouched below with reference to attached drawing The embodiment stated is exemplary, and for explaining only the invention, and is not construed as limiting the claims.
Those skilled in the art of the present technique are appreciated that unless expressly stated, singular " one " used herein, " one It is a ", " " and "the" may also comprise plural form.It is to be further understood that " the packet of wording used in specification of the invention Include " refer to existing characteristics, integer, step, operation, element and/or component, but it is not excluded that in the presence of or addition it is one or more Other features, integer, step, operation, element, component and/or their group.It should be understood that when we claim element to be " connected " Or when " coupled " to another element, it can be directly connected or coupled to other elements, or there may also be intermediary elements.This Outside, " connection " or " coupling " used herein may include being wirelessly connected or wirelessly coupling.Wording "and/or" packet used herein Include one or more associated wholes for listing item or any cell and all combination.
Those skilled in the art of the present technique are appreciated that unless otherwise defined, all terms used herein (including technology art Language and scientific term), there is meaning identical with the general understanding of those of ordinary skill in fields of the present invention.Should also Understand, those terms such as defined in the general dictionary, it should be understood that have in the context of the prior art The consistent meaning of meaning, and unless idealization or meaning too formal otherwise will not be used by specific definitions as here To explain.
Fig. 1 is the flow diagram of the method for the data compression of one embodiment in the present invention.
Step S110: analyzing original transmitted data, determines data structure and data including original transmitted data The data characteristic information of size；Step S120: according to data characteristic information, judge whether to carry out at compression original transmitted data Reason；Step S130: when judging result indicates to execute compression processing, being based on predetermined key compress mode in the list of key compression fit, Conversion is carried out to the key of key-value pair original in original transmitted data and generates corresponding compression bonding；Step S140: raw based on compression bonding At the compressed transmission data for including corresponding compression key-value pair.
In the embodiment of the present invention, the scheme of a kind of data compression and decompression is proposed, it is former by analysis in server end Begin transmission data, judges whether to carry out compression processing to original transmitted data, when judging result indicates to execute compression processing, base The predetermined key compress mode in key compression fit list carries out conversion to the key of key-value pair original in original transmitted data and generates phase The compression bonding answered, then being generated based on compression bonding includes the corresponding compressed transmission data for compressing key-value pair；If it is logical to transmit data When crossing network mode and being transmitted, conversion is carried out by the key to key-value pair original in original transmitted data and generates corresponding compression Key can save network bandwidth in data transmission procedure, avoid data volume it is excessive when not expected loss of data the case where, Realize the high efficiency of transmission of data；Meanwhile improving the response time of computer digital animation；In the client of terminal device, When the transmission data for judging to receive is compressed transmission datas, parses and extract the compression key-value pair in compressed transmission data Compression bonding, then the predetermined key in the key compression fit list based on pre-configuration decompresses mode, carries out decompression processing to compression bonding, To acquire corresponding original key-value pair, it can quickly and accurately decompress to obtain the original value of compression bonding, realize and efficiently obtain Transmission data are taken, to provide guarantee for the transmission data that later period efficient process receives, improve the experience of user.
In the embodiment of the present invention, the data structure of data in data is transmitted by taking key assignments (Key-Value) structure as an example.Its In, the content that key compresses key compression in dictionary file such as can be with 26 upper case and lower case English characters and 0 to 9 10 numbers Word, which combines, to be indicated, and can form the compression bonding of 62 different monocases, 3844 different two characters by this method Compression bonding, the compression bonding of 238328 different three characters, be fully able to meet in existing application compress key value structure in The use of key.
Step S110: analyzing original transmitted data, determines data structure and data including original transmitted data The data characteristic information of size.
For example, in the original transmitted data " userinfo " comprising three groups of key-value pair structures, three groups of key-value pairs are respectively " user_id:22 ", " user_id:23 " and " user_id:24 ", for one group of key-value pair " user_id:22 ", the part Key For " user_id ", the part Value is " 22 ", determines that the data structure of original transmitted data " userinfo " is key-value pair structure, And the specific size for calculating multiple groups key-value pair in determining original transmitted data " userinfo " is 30 bytes.
Step S120: according to data characteristic information, judge whether to carry out compression processing to original transmitted data.
For example, being key-value pair structure according to the data structure of original transmitted data " userinfo ", and size of data is 30 A byte judges whether to carry out compression processing, judgement to the part Key " user_id " of the original transmitted data of " userinfo " Mode is specifically described in following embodiments as in Figure 2-4.
Step S130: when judging result indicates to execute compression processing, based on predetermined key compression in the list of key compression fit Mode carries out conversion to the key of key-value pair original in original transmitted data and generates corresponding compression bonding.
For example, when judging result instruction is executed to the part Key of 3 groups of key-value pairs of original transmitted data " userinfo " When data " user_id " carry out compression processing, based on predetermined key compress mode in the list of key compression fit, such as " user_id " is pressed Corresponding value is " uid " after contracting, and the key assignments " user_id " of 3 groups of key-value pairs of original transmitted data " userinfo " is carried out Conversion generates corresponding compression bonding " uid ".
Step S140: being generated based on compression bonding includes the corresponding compressed transmission data for compressing key-value pair.
For example, based on compression bonding " uid " generate include corresponding 3 groups of compression key-value pairs compressed transmission data " uid: 22 ", " uid:23 " and " uid:24 ".
In a preferred embodiment, as shown in Fig. 2, according to data characteristic information, judge whether to original transmitted data into The step of row compression processing, specifically includes step S221 and step S222.Step S221: judge that the data of original transmitted data are big The small relationship with the first tentation data size threshold value；Step S222: if judging result is big for the size of data of original transmitted data When the first tentation data size threshold value, determines and compression processing is carried out to original transmitted data.
For example, the first tentation data size threshold value is 20 bytes, the data according to original transmitted data " userinfo " are big Small is 30 bytes, and the size of data that judgement obtains original transmitted data " userinfo " is greater than the first tentation data size threshold value, Then determine that the part Key " user_id " to three groups of key-value pairs in original transmitted data " userinfo " carries out compression processing, obtains The compressed part Key is " username " corresponding data " uid " in key compression fit list.
In another preferred embodiment, as shown in figure 3, according to data characteristic information, judge whether to original transmitted data The step of carrying out compression processing, specifically includes step S323 and step S324.Step S323: judge the data of original transmitted data The size relationship with the first tentation data size threshold value and the second tentation data size threshold value respectively, the first tentation data size threshold Value is greater than the second tentation data size threshold value；Step S324: if the size of data that judging result is original transmitted data is greater than the Two tentation data size threshold values and when less than the first tentation data size threshold value, judge whether according to data structure to original transmitted Data carry out compression processing.
For example, the first tentation data size threshold value be 50 bytes, the second tentation data size threshold value be 20 bytes, comprising Include following format key-value pair data in the original transmitted data for transmitting customer information of multiple groups key-value pair structure: "username:tracy；age:18；username:tom；age:32；", wherein the data of the part Key are respectively " username ", " age ", " username " and " age ", the data of the part Value be respectively " tracy ", " 18 ", " tom " and " 32 " in the original transmitted data for transmitting customer information, determine that the data structure of original transmitted data is key-value pair structure, And the size of data of original transmitted data is 42 bytes, 42 byte of size of data that can determine whether to obtain original transmitted data is greater than the Two tentation data size threshold values, 20 byte and less than 50 byte of the first tentation data size threshold value, then continue to be sentenced according to data structure It is disconnected whether compression processing to be carried out to original transmitted data.
Preferably (referring to Fig. 3), step S324 includes that step S3241 (not shown) and step S3242 (do not show in figure Out).Step S3241: the quantity and original transmitted data of the key-value pair of the first key having the same in statistics original transmitted data In include key-value pair total quantity the first ratio；Step S3242: when the first ratio is greater than the first predetermined ratio threshold value, and the When the character quantity of one key is greater than the first predetermined character amount threshold, judgement needs to carry out compression processing to original transmitted data.
For example, the first tentation data size threshold value is 50 bytes, the second tentation data size threshold value is 20 bytes, and first is pre- Certainty ratio threshold value is 40%, and the first predetermined character amount threshold is 15, and the highest key of the frequency of occurrences is the in original transmitted data Two keys；In the original transmitted data " username:tracy comprising multiple groups key-value pair structure；age:18；username:tom； age:32；" in, the size of data of original transmitted data is 42 bytes, can determine whether the size of data 42 for determining original transmitted data Byte is greater than 20 byte of the second tentation data size threshold value and less than 50 byte of the first tentation data size threshold value, and then statistics is former The quantity of the key-value pair of the first key " username " having the same is 2 in the transmission data that begin, the first key having the same The first ratio is calculated divided by the key-value pair total quantity 4 for including in original transmitted data in the quantity 2 of the key-value pair of " username " Example is 50%, and the first ratio 50% is greater than the first predetermined ratio threshold value 40%, and the first key in original transmitted data The character quantity of " username " is 16 greater than the first predetermined character amount threshold 15, and judgement is needed to the first key " username " Carry out compression processing；Then based on predetermined key compress mode in the list of key compression fit, the part Key to original transmitted data is " username " in the key-value pair of " username " carries out compression processing, and obtaining the compressed part Key is " username " Key-value pair in " username " corresponding compression bonding be " uid "；Count key having the same in original transmitted data The quantity of the key-value pair of " age " is 2, and the quantity 2 of the key-value pair of key " age " having the same in original transmitted data divided by wrapping The first ratio 50% that the key-value pair total quantity 4 included obtains, then the first ratio 50% is greater than the first predetermined ratio threshold value 40%, but The character quantity of key " age " is 4 less than the first predetermined character amount threshold 15, therefore not to the part Key of original transmitted data Compression processing is carried out for " age " in the key-value pair of " age ".
Preferably (referring to Fig. 3), step S324 includes that step S3243 (not shown), step S3244 (do not show in figure Out) and step S3245 (not shown).Step S3243: judge second with longest character length in original transmitted data Whether the character quantity of key is greater than the second predetermined character amount threshold；Step S3244: if judging, the character quantity of the second key is greater than When the second predetermined character amount threshold, the character quantity for counting whole second keys in original transmitted data accounts for original transmitted data Second ratio of total character quantity；Step S3245: when the second ratio is greater than the second predetermined ratio threshold value, judge to need to original The transmission data that begin carry out compression processing.
For example, the first tentation data size threshold value is 50 bytes, the second tentation data size threshold value is 20 bytes, and second is pre- Determining character quantity threshold value is 2, and the second predetermined ratio threshold value is 20%, and the key with longest character length is in original transmitted data Second key；In the original transmitted data comprising multiple groups key-value pair structure, the size of data of original transmitted data is 42 bytes, can Judge to determine that 42 byte of size of data of original transmitted data is greater than 20 byte of the second tentation data size threshold value and less than first 50 byte of tentation data size threshold value subsequently determines whether the second key in original transmitted data with longest character length The character quantity of " username " is 8 greater than the second predetermined character amount threshold 2, by two the second keys in initial data The character quantity 16 of " username " obtains the second ratio 38% divided by total character quantity 42 of original transmitted data, judges to determine Second ratio 38% is greater than the second predetermined ratio threshold value 20%, then judgement need be to the part the Key of original transmitted data " username " in the key-value pair of " username " carries out compression processing, based on predetermined key compression side in the list of key compression fit Formula, it is " uid " that the corresponding compression bonding that the part Key is " username " in the key-value pair of " username " is obtained after compression.
In a preferred embodiment, as shown in figure 4, the method comprising the steps of S410, step S420, step S430, step S440 and step S450.Step S410: analyzing original transmitted data, determines the data structure including original transmitted data And the data characteristic information of size of data；Step S420: each key-value pair in original transmitted data within a predetermined period of time is determined In key the history frequency of occurrences；Step S430: according to data characteristic information, and combining the history frequency of occurrences, judges whether pair Original transmitted data carries out compression processing；Step S440: when judging result indicates to execute compression processing, it is based on key compression fit Predetermined key compress mode in list carries out conversion to the key of key-value pair original in original transmitted data and generates corresponding compression bonding； Step S450: being generated based on compression bonding includes the corresponding compressed transmission data for compressing key-value pair.
Wherein, the content executed in step S410, step S440 and step S450 in this preferred embodiment with it is shown in FIG. 1 Step S110, the content executed in step S130 and step S140 is same or similar, and details are not described herein.
For example, including following format key-value pair data in the original transmitted data comprising multiple groups key-value pair structure: "username:tracy；age:18；username:tom；age:32；", remembered according to nearly one month history in database Record, obtains within nearly one month period that the history frequency of occurrences of Key key is as follows in each key-value pair in original transmitted data: " username " frequency of occurrences is one month 800 times, and " age " frequency of occurrences is one month 900 times；According to the judgment method of Fig. 3 Judge to be " username " progress compression processing in the key-value pair of " username " to the part Key of original transmitted data, no The part Key to original transmitted data is that " age " in the key-value pair of " age " carries out compression processing, due to " age " frequency of occurrences Greater than the frequency that " username " occurs, then combine the judgement of the history frequency of occurrences need be to the part the Key of original transmitted data " age " in the key-value pair of " age " carries out compression processing.
In a preferred embodiment, step S140 further include step S141 (not shown) and step S142 (in figure not It shows).Step S141: corresponding compression key-value pair is generated based on compression bonding, and associated compressive state mark is set；Step S142: by compression key-value pair and associated compressive state identifier combination to generate compressed transmission data.
For example, generating corresponding compression key-value pair based on original transmitted data, associated compressive state mark is set, it can Compressing mark state is configured by compressing mark such as " IsCompressed ", may be configured as when compressive state is compression " IsCompressed=true；", it may be configured as " IsCompressed=false when compressive state is not compress；", according to The part Key to original transmitted data is that " username " and " age " in the key-value pair of " username " and " age " is pressed Contracting handles to obtain corresponding compression bonding to be " un " and " ag ", by compression key-value pair and associated compressive state identifier combination to generate Compressed transmission data " IsCompressed=true；un:tracy；ag:18；un:tom；ag:32；".
In a preferred embodiment, this method further includes that step S150 (not shown) and step S160 (do not show in figure Out).Step S150: corresponding key is generated based on the list of key compression fit and compresses dictionary file；Step S160: according to what is received Using acquisition request, requesting party is sent to using key compression dictionary file as configuration file.
For example, generating corresponding key based on scheduled key compression fit list compresses dictionary file, file is entitled CompFile, file content are " username:un；age:ag；", according to the client application acquisition request received, by key Dictionary file CompFile, which is compressed, as configuration file is sent to corresponding requesting party.
Fig. 5 is the flow diagram of the method for the data decompression of one embodiment in the present invention.
Step S510: judge whether the transmission data received are compressed transmission data；Step S520: it is passed when judgement determines When transmission of data is compressed transmission data, the compression bonding of the compression key-value pair in compressed transmission data is parsed and extracted；Step S530: Predetermined key in key compression fit list based on pre-configuration decompresses mode, decompression processing is carried out to compression bonding, to acquire Corresponding original key-value pair.
For example, after sending the relevant request of data of acquisition APP using the client of APP in terminal device, Terminal device receives the transmission data of server end return, when the transmission data of client received using APP judgement are When compressed transmission data, data parsing is carried out to compressed transmission data and extracts the pressure of the compression key-value pair in compressed transmission data Contracting key decompresses mode based on the predetermined key in the key compression fit list that client is pre-configured, returns and transmit to server end Compression bonding in data carries out decompression processing, compresses key-value pair original key-value pair accordingly to acquire in APP transmission data.
In a preferred embodiment, step S510 further includes step S511 (not shown).Step S511: judgement transmission Whether the key-value pair in data includes associated compressive state mark.
Wherein, when judge determine transmission data for compressed transmission data when, step S520 further include step S521 (in figure not It shows).Step S521: when the key-value pair in transmission data includes associated compressive state mark, it is determined that transmitting data is Compressed transmission data；And extract the compression bonding for identifying relevant compression key-value pair in compressed transmission data to compressive state.
For example, after sending the relevant request of data of acquisition APP using the client of APP in terminal device, Terminal device receives the transmission data " IsCompressed=true of server-side return；un:tracy；ag:18；un:tom； ag:32；", transmission data are parsed using the client of APP, can obtain in the key-value pair in transmission data including correlation The compressive state of connection identifies " IsCompressed ", can identify " IsCompressed=according to the compressive state in transmission data true；" judge to transmit data for compressed transmission data, then extract pressure relevant to compressive state mark in compressed transmission data The compression bonding of contracting key-value pair obtains " un；ag；un；ag；".
In a preferred embodiment, this method further includes that step S531 (not shown) and step S532 (do not show in figure Out).Step S531: receiving configuration file, includes that key compresses dictionary file in configuration file；Step S532: word is compressed according to key Allusion quotation file configuration generates local key compression fit list.
For example, receiving in the configuration file of server end return using the client of APP includes that key compresses dictionary file CompFile, content are " username:un；age:ag；", it is raw that the data configuration in dictionary file CompFile is compressed according to key At local key compression fit list, content is " un:username；ag:age；"；Subsequently, based on the key compression fit of generation Predetermined key in list decompresses mode, to the compression bonding " un extracted；ag；un；ag；" decompression processing is carried out, obtain compression bonding "un；ag；un；ag；" original key be " username；age；username；age；", therefore corresponding original key assignments can be obtained To for " username:tracy；age:18；username:tom；age:32；".
Fig. 6 is the structural schematic diagram of the device of the data compression of another embodiment in the present invention.
First determining module 610 analyzes original transmitted data, determines the data structure including original transmitted data And the data characteristic information of size of data；According to data characteristic information, first judgment module 620 judges whether to original transmitted number According to progress compression processing；When judging result indicates to execute compression processing, based on predetermined key compression side in the list of key compression fit Formula, conversion module 630 carry out conversion to the key of key-value pair original in original transmitted data and generate corresponding compression bonding；First generates It includes the corresponding compressed transmission data for compressing key-value pair that module 640 is generated based on compression bonding.
In the embodiment of the present invention, the scheme of a kind of data compression and decompression is proposed, it is former by analysis in server end Begin transmission data, judges whether to carry out compression processing to original transmitted data, when judging result indicates to execute compression processing, base The predetermined key compress mode in key compression fit list carries out conversion to the key of key-value pair original in original transmitted data and generates phase The compression bonding answered, then being generated based on compression bonding includes the corresponding compressed transmission data for compressing key-value pair；If it is logical to transmit data When crossing network mode and being transmitted, conversion is carried out by the key to key-value pair original in original transmitted data and generates corresponding compression Key can save network bandwidth in data transmission procedure, avoid data volume it is excessive when not expected loss of data the case where, Realize the high efficiency of transmission of data；Meanwhile improving the response time of computer digital animation；In the client of terminal device, When the transmission data for judging to receive is compressed transmission datas, parses and extract the compression key-value pair in compressed transmission data Compression bonding, then the predetermined key in the key compression fit list based on pre-configuration decompresses mode, carries out decompression processing to compression bonding, To acquire corresponding original key-value pair, it can quickly and accurately decompress to obtain the original value of compression bonding, realize and efficiently obtain Transmission data are taken, to provide guarantee for the transmission data that later period efficient process receives, improve the experience of user.
In the embodiment of the present invention, the data structure of data in data is transmitted by taking key assignments (Key-Value) structure as an example.Its In, the content that key compresses key compression in dictionary file such as can be with 26 upper case and lower case English characters and 0 to 9 10 numbers Word, which combines, to be indicated, and can form the compression bonding of 62 different monocases, 3844 different two characters by this method Compression bonding, the compression bonding of 238328 different three characters, be fully able to meet in existing application compress key value structure in The use of key.
First determining module 610 analyzes original transmitted data, determines the data structure including original transmitted data And the data characteristic information of size of data.
For example, in the original transmitted data " userinfo " comprising three groups of key-value pair structures, three groups of key-value pairs are respectively " user_id:22 ", " user_id:23 " and " user_id:24 ", for one group of key-value pair " user_id:22 ", the part Key For " user_id ", the part Value is " 22 ", determines that the data structure of original transmitted data " userinfo " is key-value pair structure, And the specific size for calculating multiple groups key-value pair in determining original transmitted data " userinfo " is 30 bytes.
According to data characteristic information, first judgment module 620 judges whether to carry out compression processing to original transmitted data.
For example, being key-value pair structure according to the data structure of original transmitted data " userinfo ", and size of data is 30 A byte judges whether to carry out compression processing, judgement to the part Key " user_id " of the original transmitted data of " userinfo " Mode is specifically described in following embodiments as in Figure 2-4.
When judging result indicates to execute compression processing, based on predetermined key compress mode in the list of key compression fit, conversion Module 630 carries out conversion to the key of key-value pair original in original transmitted data and generates corresponding compression bonding.
For example, when judging result instruction is executed to the part Key of 3 groups of key-value pairs of original transmitted data " userinfo " When data " user_id " carry out compression processing, based on predetermined key compress mode in the list of key compression fit, such as " user_id " is pressed Corresponding value is " uid " after contracting, and the key assignments " user_id " of 3 groups of key-value pairs of original transmitted data " userinfo " is carried out Conversion generates corresponding compression bonding " uid ".
It includes the corresponding compressed transmission data for compressing key-value pair that first generation module 640 is generated based on compression bonding.
For example, based on compression bonding " uid " generate include corresponding 3 groups of compression key-value pairs compressed transmission data " uid: 22 ", " uid:23 " and " uid:24 ".
In a preferred embodiment, as shown in fig. 7, first judgment module specifically includes second judgment submodule 721 and Two determine submodule 722.Second judgment submodule 721 judges the size of data and the first tentation data size of original transmitted data The relationship of threshold value；If the size of data that judging result is original transmitted data is greater than the first tentation data size threshold value, second It determines that submodule 722 determines and compression processing is carried out to original transmitted data.
For example, the first tentation data size threshold value is 20 bytes, the data according to original transmitted data " userinfo " are big Small is 30 bytes, and the size of data that judgement obtains original transmitted data " userinfo " is greater than the first tentation data size threshold value, Then determine that the part Key " user_id " to three groups of key-value pairs in original transmitted data " userinfo " carries out compression processing, obtains The compressed part Key is " username " corresponding data " uid " in key compression fit list.
In another preferred embodiment, as shown in figure 8, first judgment module specifically includes 823 He of third judging submodule 4th judging submodule 824.Third judging submodule 823 judge the size of data of original transmitted data respectively with the first predetermined number According to size threshold value and the relationship of the second tentation data size threshold value, it is big that the first tentation data size threshold value is greater than the second tentation data Small threshold value；If judging result is that the size of data of original transmitted data is greater than the second tentation data size threshold value and pre- less than first When determining data size thresholds, the 4th judging submodule 824 judges whether to compress original transmitted data according to data structure Processing.
For example, the first tentation data size threshold value be 50 bytes, the second tentation data size threshold value be 20 bytes, comprising Include following format key-value pair data in the original transmitted data for transmitting customer information of multiple groups key-value pair structure: "username:tracy；age:18；username:tom；age:32；", wherein the data of the part Key are respectively " username ", " age ", " username " and " age ", the data of the part Value be respectively " tracy ", " 18 ", " tom " and " 32 " in the original transmitted data for transmitting customer information, determine that the data structure of original transmitted data is key-value pair structure, And the size of data of original transmitted data is 42 bytes, 42 byte of size of data that can determine whether to obtain original transmitted data is greater than the Two tentation data size threshold values, 20 byte and less than 50 byte of the first tentation data size threshold value, then continue to be sentenced according to data structure It is disconnected whether compression processing to be carried out to original transmitted data.
Preferably (referring to Fig. 8), the 4th judging submodule includes the first statistic unit (not shown) and the first judgement Unit (not shown).First statistic unit counts the quantity of the key-value pair of the first key having the same in original transmitted data With the first ratio of the key-value pair total quantity for including in original transmitted data；When the first ratio be greater than the first predetermined ratio threshold value, And first key character quantity be greater than the first predetermined character amount threshold when, the first judging unit judgement need to original transmitted number According to progress compression processing.
For example, the first tentation data size threshold value is 50 bytes, the second tentation data size threshold value is 20 bytes, and first is pre- Certainty ratio threshold value is 40%, and the first predetermined character amount threshold is 15, and the highest key of the frequency of occurrences is the in original transmitted data Two keys；In the original transmitted data " username:tracy comprising multiple groups key-value pair structure；age:18；username:tom； age:32；" in, the size of data of original transmitted data is 42 bytes, can determine whether the size of data 42 for determining original transmitted data Byte is greater than 20 byte of the second tentation data size threshold value and less than 50 byte of the first tentation data size threshold value, and then statistics is former The quantity of the key-value pair of the first key " username " having the same is 2 in the transmission data that begin, the first key having the same The first ratio is calculated divided by the key-value pair total quantity 4 for including in original transmitted data in the quantity 2 of the key-value pair of " username " Example is 50%, and the first ratio 50% is greater than the first predetermined ratio threshold value 40%, and the first key in original transmitted data The character quantity of " username " is 16 greater than the first predetermined character amount threshold 15, and judgement is needed to the first key " username " Carry out compression processing；Then based on predetermined key compress mode in the list of key compression fit, the part Key to original transmitted data is " username " in the key-value pair of " username " carries out compression processing, and obtaining the compressed part Key is " username " Key-value pair in " username " corresponding compression bonding be " uid "；Count key having the same in original transmitted data The quantity of the key-value pair of " age " is 2, and the quantity 2 of the key-value pair of key " age " having the same in original transmitted data divided by wrapping The first ratio 50% that the key-value pair total quantity 4 included obtains, then the first ratio 50% is greater than the first predetermined ratio threshold value 40%, but The character quantity of key " age " is 4 less than the first predetermined character amount threshold 15, therefore not to the part Key of original transmitted data Compression processing is carried out for " age " in the key-value pair of " age ".
Preferably (referring to Fig. 8), the 4th judging submodule includes second judgment unit (not shown), the second statistics list First (not shown) and third judging unit (not shown).Second judgment unit judges have most in original transmitted data Whether the character quantity of the second key of long character length is greater than the second predetermined character amount threshold；If judging the number of characters of the second key When amount is greater than the second predetermined character amount threshold, the second statistic unit counts the number of characters of whole second keys in original transmitted data Amount accounts for the second ratio of total character quantity of original transmitted data；When the second ratio is greater than the second predetermined ratio threshold value, third Judging unit judgement needs to carry out compression processing to original transmitted data.
For example, the first tentation data size threshold value is 50 bytes, the second tentation data size threshold value is 20 bytes, and second is pre- Determining character quantity threshold value is 2, and the second predetermined ratio threshold value is 20%, and the key with longest character length is in original transmitted data Second key；In the original transmitted data comprising multiple groups key-value pair structure, the size of data of original transmitted data is 42 bytes, can Judge to determine that 42 byte of size of data of original transmitted data is greater than 20 byte of the second tentation data size threshold value and less than first 50 byte of tentation data size threshold value subsequently determines whether the second key in original transmitted data with longest character length The character quantity of " username " is 8 greater than the second predetermined character amount threshold 2, by two the second keys in initial data The character quantity 16 of " username " obtains the second ratio 38% divided by total character quantity 42 of original transmitted data, judges to determine Second ratio 38% is greater than the second predetermined ratio threshold value 20%, then judgement need be to the part the Key of original transmitted data " username " in the key-value pair of " username " carries out compression processing, based on predetermined key compression side in the list of key compression fit Formula, it is " uid " that the corresponding compression bonding that the part Key is " username " in the key-value pair of " username " is obtained after compression.
In a preferred embodiment, as shown in figure 9, the device include the first determining module 910, third determining module 920, First judgment module 930, conversion module 940 and the first generation module 950.First determining module 910 to original transmitted data into Row analysis determines to include the data structure of original transmitted data and the data characteristic information of size of data；Third determining module 920 Determine the history frequency of occurrences of the key in original transmitted data within a predetermined period of time in each key-value pair；Believed according to data characteristics Breath, and the history frequency of occurrences is combined, first judgment module 930 judges whether to carry out compression processing to original transmitted data；When sentencing When disconnected result instruction executes compression processing, based on predetermined key compress mode in the list of key compression fit, conversion module 940 is to original The key of original key-value pair carries out conversion and generates corresponding compression bonding in transmission data；First generation module 950 is raw based on compression bonding At the compressed transmission data for including corresponding compression key-value pair.
Wherein, the first determining module 910 in this preferred embodiment, conversion module 940 and the first generation module 950 execute Content is identical as the content that the first determining module 610 shown in fig. 6, conversion module 630 and the first generation module 640 execute or phase Seemingly, details are not described herein.
For example, including following format key-value pair data in the original transmitted data comprising multiple groups key-value pair structure: "username:tracy；age:18；username:tom；age:32；", remembered according to nearly one month history in database Record, obtains within nearly one month period that the history frequency of occurrences of Key key is as follows in each key-value pair in original transmitted data: " username " frequency of occurrences is one month 800 times, and " age " frequency of occurrences is one month 900 times；According to the judgment means of Fig. 8 Judge to be " username " progress compression processing in the key-value pair of " username " to the part Key of original transmitted data, no The part Key to original transmitted data is that " age " in the key-value pair of " age " carries out compression processing, due to " age " frequency of occurrences Greater than the frequency that " username " occurs, then combine the judgement of the history frequency of occurrences need be to the part the Key of original transmitted data " age " in the key-value pair of " age " carries out compression processing.
In a preferred embodiment, the first generation module further includes mark setting unit (not shown) and combination producing Unit (not shown).Corresponding compression key-value pair is generated based on compression bonding, setting unit is identified and associated pressure is set Contracting status indicator；Combination producing unit will compress key-value pair and associated compressive state identifier combination to generate compression transmission number According to.
For example, generating corresponding compression key-value pair based on original transmitted data, associated compressive state mark is set, it can Compressing mark state is configured by compressing mark such as " IsCompressed ", may be configured as when compressive state is compression " IsCompressed=true；", it may be configured as " IsCompressed=false when compressive state is not compress；", according to The part Key to original transmitted data is that " username " and " age " in the key-value pair of " username " and " age " is pressed Contracting handles to obtain corresponding compression bonding to be " un " and " ag ", by compression key-value pair and associated compressive state identifier combination to generate Compressed transmission data " IsCompressed=true；un:tracy；ag:18；un:tom；ag:32；".
In a preferred embodiment, which further includes the second generation module (not shown) and sending module (in figure It is not shown).Second generation module is based on the list of key compression fit and generates corresponding key compression dictionary file；It is answered according to what is received With acquisition request, sending module is sent to requesting party using key compression dictionary file as configuration file.
For example, generating corresponding key based on scheduled key compression fit list compresses dictionary file, file is entitled CompFile, file content are " username:un；age:ag；", according to the client application acquisition request received, by key Dictionary file CompFile, which is compressed, as configuration file is sent to corresponding requesting party.
Figure 10 is the structural schematic diagram of the device of the data decompression of another preferred embodiment in the present invention.
Whether the transmission data that the judgement of the 6th judgment module 1010 receives are compressed transmission data；When judgement determines transmission When data are compressed transmission data, parsing extraction module 1020 parses and extracts the pressure of the compression key-value pair in compressed transmission data Contracting key；Predetermined key in key compression fit list based on pre-configuration decompresses mode, decompression processing module 1030 to compression bonding into Row decompression processing, to acquire corresponding original key-value pair.
For example, after sending the relevant request of data of acquisition APP using the client of APP in terminal device, Terminal device receives the transmission data of server end return, when the transmission data of client received using APP judgement are When compressed transmission data, data parsing is carried out to compressed transmission data and extracts the pressure of the compression key-value pair in compressed transmission data Contracting key decompresses mode based on the predetermined key in the key compression fit list that client is pre-configured, returns and transmit to server end Compression bonding in data carries out decompression processing, compresses key-value pair original key-value pair accordingly to acquire in APP transmission data.
In a preferred embodiment, whether the 6th judgment module 1010 is further used for the key-value pair judged transmit in data It is identified including associated compressive state.
Wherein, when judging to determine transmission data as compressed transmission data, parsing extraction module 1020 is when in transmission data Key-value pair include associated compressive state mark, it is determined that transmission data are compressed transmission data；And it extracts compression and passes The compression bonding of relevant compression key-value pair is identified in transmission of data to compressive state.
For example, after sending the relevant request of data of acquisition APP using the client of APP in terminal device, Terminal device receives the transmission data " IsCompressed=true of server-side return；un:tracy；ag:18；un:tom； ag:32；", transmission data are parsed using the client of APP, can obtain in the key-value pair in transmission data including correlation The compressive state of connection identifies " IsCompressed ", can identify " IsCompressed=according to the compressive state in transmission data true；" judge to transmit data for compressed transmission data, then extract pressure relevant to compressive state mark in compressed transmission data The compression bonding of contracting key-value pair obtains " un；ag；un；ag；".
In a preferred embodiment, which further includes 1031 (not shown) of receiving module and third generation module 1032 (not shown)s.Receiving module 1031 receives configuration file, includes that key compresses dictionary file in configuration file；Third is raw Dictionary file configuration is compressed according to key at module 1032 and generates local key compression fit list.
For example, receiving in the configuration file of server end return using the client of APP includes that key compresses dictionary file CompFile, content are " username:un；age:ag；", it is raw that the data configuration in dictionary file CompFile is compressed according to key At local key compression fit list, content is " un:username；ag:age；"；Subsequently, based on the key compression fit of generation Predetermined key in list decompresses mode, to the compression bonding " un extracted；ag；un；ag；" decompression processing is carried out, obtain compression bonding "un；ag；un；ag；" original key be " username；age；username；age；", therefore corresponding original key assignments can be obtained To for " username:tracy；age:18；username:tom；age:32；".
Those skilled in the art of the present technique are appreciated that the present invention includes being related to for executing one in the application in operation Or multinomial equipment.These equipment can specially design and manufacture for required purpose, or also may include general-purpose computations Known device in machine.These equipment have the computer program being stored in it, these computer programs selectively activate Or reconstruct.Such computer program, which can be stored in equipment (for example, computer) readable medium or be stored in, to be suitable for Storage e-command is simultaneously coupled in any kind of medium of bus respectively, and computer-readable medium is including but not limited to any The disk (including floppy disk, hard disk, CD, CD-ROM and magneto-optic disk) of type, ROM (Read-Only Memory, read-only storage Device), RAM (Random Access Memory, immediately memory), EPROM (Erasable Programmable Read- Only Memory, Erarable Programmable Read only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory, Electrically Erasable Programmable Read-Only Memory), flash memory, magnetic card or light card.It is, readable Medium includes by equipment (for example, computer) with any medium for the form storage or transmission information that can be read.
Those skilled in the art of the present technique be appreciated that can be realized with computer program instructions these structure charts and/or The combination of each frame and these structure charts and/or the frame in block diagram and/or flow graph in block diagram and/or flow graph.This technology neck Field technique personnel be appreciated that these computer program instructions can be supplied to general purpose computer, special purpose computer or other The processor of programmable data processing method is realized, to pass through the processing of computer or other programmable data processing methods The scheme specified in frame or multiple frames of the device to execute structure chart and/or block diagram and/or flow graph disclosed by the invention.
Those skilled in the art of the present technique have been appreciated that in the present invention the various operations crossed by discussion, method, in process Steps, measures, and schemes can be replaced, changed, combined or be deleted.Further, each with having been crossed by discussion in the present invention Kind of operation, method, other steps, measures, and schemes in process may also be alternated, changed, rearranged, decomposed, combined or deleted. Further, in the prior art to have and the step in various operations, method disclosed in the present invention, process, measure, scheme It may also be alternated, changed, rearranged, decomposed, combined or deleted.
The above is only some embodiments of the invention, it is noted that those skilled in the art are come It says, various improvements and modifications may be made without departing from the principle of the present invention, these improvements and modifications also should be regarded as Protection scope of the present invention.

Claims (22)

1. a kind of method of data compression, comprising:
Original transmitted data is analyzed, determines to include the data structure of the original transmitted data and the data of size of data Characteristic information；
According to the data characteristic information, judge whether to carry out compression processing to the original transmitted data；
When judging result indicates to execute compression processing, the determining and original transmitted data Central Plains from key compression fit list The matched compression bonding of the key of beginning key-value pair；
Being generated based on the compression bonding includes the corresponding compressed transmission data for compressing key-value pair.
2. the method for data compression according to claim 1, wherein according to the data characteristic information, judge whether pair The original transmitted data carries out compression processing, comprising:
Judge the size of data of the original transmitted data and the relationship of the first tentation data size threshold value；
If the size of data that judging result is the original transmitted data is greater than the first tentation data size threshold value, determine to institute It states original transmitted data and carries out compression processing.
3. the method for data compression according to claim 1, wherein according to the data characteristic information, judge whether pair The original transmitted data carries out compression processing, comprising:
Judge that the size of data of the original transmitted data is big with the first tentation data size threshold value and the second tentation data respectively The relationship of small threshold value, the first tentation data size threshold value are greater than the second tentation data size threshold value；
If the size of data that judging result is the original transmitted data is greater than the second tentation data size threshold value and is less than When the first tentation data size threshold value, judge whether to compress the original transmitted data according to the data structure Processing.
4. the method for data compression according to claim 3, wherein judged whether according to the data structure to the original The transmission data that begin carry out compression processing, specifically include:
It counts in the original transmitted data in the quantity and the original transmitted data of the key-value pair of the first key having the same Including key-value pair total quantity the first ratio；
When first ratio be greater than the first predetermined ratio threshold value, and the character quantity of first key be greater than the first predetermined character When amount threshold, judge to need to carry out compression processing to the original transmitted data.
5. the method for data compression according to claim 3, wherein judged whether according to the data structure to the original The transmission data that begin carry out compression processing, specifically include:
It is predetermined to judge whether the character quantity of the second key in the original transmitted data with longest character length is greater than second Character quantity threshold value；
If judge that the character quantity of second key is greater than the second predetermined character amount threshold, the original transmitted is counted All the character quantity of the second key accounts for the second ratio of total character quantity of the original transmitted data in data；
When second ratio is greater than the second predetermined ratio threshold value, judge to need to carry out at compression the original transmitted data Reason.
6. the method for data compression according to claim 1, further includes:
Determine the history frequency of occurrences of the key of each key-value pair in the original transmitted data described within a predetermined period of time；
Wherein, according to the data characteristic information, judge whether to carry out compression processing to the original transmitted data, comprising:
According to the data characteristic information, and in conjunction with the history frequency of occurrences, judge whether to the original transmitted data into Row compression processing.
7. the method for data compression according to claim 1, wherein being generated based on the compression bonding includes corresponding compression The step of compressed transmission data of key-value pair, further comprises:
Corresponding compression key-value pair is generated based on the compression bonding, and associated compressive state mark is set；
By the compression key-value pair and the associated compressive state identifier combination to generate compressed transmission data.
8. the method for data compression according to claim 1, further includes:
Corresponding key, which is generated, based on the key compression fit list compresses dictionary file；
According to the application acquisition request received, requesting party is sent to using key compression dictionary file as configuration file.
9. a kind of method of data decompression, comprising:
Judge whether the transmission data received are compressed transmission data；
When judging to determine the transmission data as compressed transmission data, parses and extract the compression in the compressed transmission data The compression bonding of key-value pair；
The original key is replaced institute by the determining and matched original key of the compression bonding from the key compression fit list of pre-configuration The compression bonding of compression key-value pair is stated, corresponding original key-value pair is obtained.
10. the method for data decompression according to claim 9, wherein judge whether the transmission data received are compression The step of transmitting data further comprises:
Judge to transmit whether the key-value pair in data includes associated compressive state mark；
Wherein, it when judging to determine the transmission data as compressed transmission data, parses and extracts in the compressed transmission data Compression key-value pair compression bonding the step of further comprise:
When the key-value pair in the transmission data includes associated compressive state mark, it is determined that the transmission data are compression Transmit data；And extract the compression for identifying relevant compression key-value pair in the compressed transmission data to the compressive state Key.
11. the method for data decompression according to claim 9 or 10, further includes:
Configuration file is received, includes that key compresses dictionary file in the configuration file；
Dictionary file configuration, which is compressed, according to the key generates the local key compression fit list.
12. a kind of device of data compression, comprising:
First determining module determines the data knot including the original transmitted data for analyzing original transmitted data The data characteristic information of structure and size of data；
First judgment module, for judging whether to compress the original transmitted data according to the data characteristic information Processing；
Conversion module is used for when judging result indicates to execute compression processing, the determining and original from key compression fit list The matched compression bonding of key of original key-value pair in the transmission data that begin；
First generation module, being generated based on the compression bonding includes the corresponding compressed transmission data for compressing key-value pair.
13. the device of data compression according to claim 12, wherein the first judgment module specifically includes:
Second judgment submodule, for judging the size of data and the first tentation data size threshold value of the original transmitted data Relationship；
Second determines submodule, if the size of data for judging result to be the original transmitted data is greater than the first tentation data When size threshold value, determines and compression processing is carried out to the original transmitted data.
14. the device of data compression according to claim 12, wherein the first judgment module includes:
Third judging submodule, for judge the size of data of the original transmitted data respectively with the first tentation data size threshold The relationship of value and the second tentation data size threshold value, the first tentation data size threshold value are greater than the second tentation data size threshold Value；
4th judging submodule, if it is predetermined to be greater than described second for the size of data that judging result is the original transmitted data Data size thresholds and be less than the first tentation data size threshold value when, judged whether according to the data structure to the original The transmission data that begin carry out compression processing.
15. the device of data compression according to claim 14, wherein the 4th judging submodule specifically includes:
First statistic unit, for counting the quantity of the key-value pair of the first key having the same and institute in the original transmitted data State the first ratio of the key-value pair total quantity for including in original transmitted data；
First judging unit, for being greater than the first predetermined ratio threshold value, and the number of characters of first key when first ratio When amount is greater than the first predetermined character amount threshold, judge to need to carry out compression processing to the original transmitted data.
16. the device of data compression according to claim 14, wherein the 4th judging submodule specifically includes:
Second judgment unit, for judging the character quantity of the second key in the original transmitted data with longest character length Whether the second predetermined character amount threshold is greater than；
Second statistic unit, if for judging that the character quantity of second key is greater than the second predetermined character amount threshold When, the character quantity for counting in the original transmitted data whole the second keys accounts for total character quantity of the original transmitted data Second ratio；
Third judging unit, for judging to need to described original when second ratio is greater than the second predetermined ratio threshold value It transmits data and carries out compression processing.
17. the device of data compression according to claim 12, further includes:
Third determining module, the history for the key of each key-value pair in the determination within a predetermined period of time original transmitted data The frequency of occurrences；
Wherein, first judgment module is specifically used for according to the data characteristic information, and in conjunction with the history frequency of occurrences, judgement Whether compression processing is carried out to the original transmitted data.
18. the device of data compression according to claim 12, wherein the first generation module includes: mark setting unit, For generating corresponding compression key-value pair based on the compression bonding, and associated compressive state mark is set；
Combination producing unit, for compressing the compression key-value pair and the associated compressive state identifier combination to generate Transmit data.
19. the device of data compression according to claim 12, further includes:
Second generation module compresses dictionary file for generating corresponding key based on the key compression fit list；
Sending module, for being sent out using key compression dictionary file as configuration file according to the application acquisition request received It send to requesting party.
20. a kind of device of data decompression, comprising:
6th judgment module, whether the transmission data for judging to receive are compressed transmission data；
Extraction module is parsed, for parsing and extracting the pressure when judging to determine the transmission data as compressed transmission data The compression bonding of compression key-value pair in contracting transmission data；
Decompression processing module, for determining with the matched original key of the compression bonding from the key compression fit list of pre-configuration, The compression bonding that the original key is replaced to the compression key-value pair, obtains corresponding original key-value pair.
21. the device of data decompression according to claim 20, wherein the 6th judgment module is further used for judging Whether the key-value pair in transmission data includes associated compressive state mark；
Wherein, the parsing extraction module is further used for when the key-value pair in the transmission data including associated compression shape State mark, it is determined that the transmission data are compressed transmission data；And extract in the compressed transmission data with the compression The compression bonding of the relevant compression key-value pair of status indicator.
22. the device of the data decompression according to claim 20 or 21, further includes:
Receiving module includes that key compresses dictionary file for receiving configuration file, in the configuration file；
Third generation module generates the local key compression fit list for compressing dictionary file configuration according to the key.
CN201510907676.3A 2015-12-09 2015-12-09 The method and device of data compression and decompression Active CN105322969B (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201510907676.3A CN105322969B (en) 2015-12-09 2015-12-09 The method and device of data compression and decompression
PCT/CN2016/104567 WO2017097071A1 (en) 2015-12-09 2016-11-04 Method and apparatus for compressing and decompressing data

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201510907676.3A CN105322969B (en) 2015-12-09 2015-12-09 The method and device of data compression and decompression

Publications (2)

Publication Number Publication Date
CN105322969A CN105322969A (en) 2016-02-10
CN105322969B true CN105322969B (en) 2019-06-18

Family

ID=55249664

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201510907676.3A Active CN105322969B (en) 2015-12-09 2015-12-09 The method and device of data compression and decompression

Country Status (2)

Country Link
CN (1) CN105322969B (en)
WO (1) WO2017097071A1 (en)

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105322969B (en) * 2015-12-09 2019-06-18 北京奇虎科技有限公司 The method and device of data compression and decompression
CN108011952B (en) * 2017-12-01 2021-06-18 北京奇艺世纪科技有限公司 Method and device for acquiring compression dictionary
CN107994907B (en) * 2017-12-01 2021-05-28 北京奇艺世纪科技有限公司 Method and device for generating compression dictionary
CN110505655A (en) * 2018-09-10 2019-11-26 深圳市文鼎创数据科技有限公司 Data command processing method, storage medium and bluetooth shield
CN109683968B (en) * 2018-12-18 2022-03-29 北京东土军悦科技有限公司 Switch quick start method, switch and storage medium
CN111831211A (en) * 2019-04-19 2020-10-27 阿里巴巴集团控股有限公司 Data transmission method, device, equipment and storage medium
CN110430144A (en) * 2019-09-09 2019-11-08 联想(北京)有限公司 Data processing method, device, electronic equipment and medium
CN111277274A (en) * 2020-01-13 2020-06-12 平安国际智慧城市科技股份有限公司 Data compression method, device, equipment and storage medium
CN111526151A (en) * 2020-04-28 2020-08-11 网易（杭州）网络有限公司 Data transmission method and device, electronic equipment and storage medium
CN112965934A (en) * 2021-02-04 2021-06-15 北京高因科技有限公司 Log compression storage method and electronic device
CN115905168B (en) * 2022-11-15 2023-11-07 本原数据(北京)信息技术有限公司 Self-adaptive compression method and device based on database, equipment and storage medium
CN115858450B (en) * 2023-02-24 2023-05-05 深圳华龙讯达信息技术股份有限公司 Signal transmission system of high-adaptability CPU (Central processing Unit)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103326732A (en) * 2013-05-10 2013-09-25 华为技术有限公司 Method for packing data, method for unpacking data, coder and decoder
CN104468044A (en) * 2014-12-05 2015-03-25 北京国双科技有限公司 Data compression method and device applied to network transmission
CN104753540A (en) * 2015-03-05 2015-07-01 华为技术有限公司 Data compression method, data decompression method and device

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4732250B2 (en) * 2006-06-14 2011-07-27 キヤノン株式会社 Information processing apparatus, control method, and computer program
CN104462524A (en) * 2014-12-24 2015-03-25 福建江夏学院 Data compression storage method for Internet of Things
CN105322969B (en) * 2015-12-09 2019-06-18 北京奇虎科技有限公司 The method and device of data compression and decompression

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103326732A (en) * 2013-05-10 2013-09-25 华为技术有限公司 Method for packing data, method for unpacking data, coder and decoder
CN104468044A (en) * 2014-12-05 2015-03-25 北京国双科技有限公司 Data compression method and device applied to network transmission
CN104753540A (en) * 2015-03-05 2015-07-01 华为技术有限公司 Data compression method, data decompression method and device

Also Published As

Publication number Publication date
WO2017097071A1 (en) 2017-06-15
CN105322969A (en) 2016-02-10

Similar Documents

Publication Publication Date Title
CN105322969B (en) The method and device of data compression and decompression
CN104994128B (en) A kind of identification of data encoding type and code-transferring method and device
CN104899204B (en) Data storage method and device
CN110995273B (en) Data compression method, device, equipment and medium for power database
CN109299265A (en) Potential reflux user screening technique, device and electronic equipment
CN111107423A (en) Video service playing card pause identification method and device
CN108647329A (en) Processing method, device and the computer readable storage medium of user behavior data
CN113742174B (en) Cloud mobile phone application monitoring method and device, electronic equipment and storage medium
CN106250397B (en) User behavior characteristic analysis method and device
TW201438407A (en) Data transmission device, data transmission method, and program
CN108287874B (en) DB2 database management method and device
CN110032432B (en) Example compression method and device and example decompression method and device
JP2006287518A (en) Compression rule generation method, compression communication device, and program
CN111124851A (en) Method, system, terminal and storage medium for processing historical performance data of storage device
CN116384240A (en) Server energy consumption prediction method, device and storage medium
WO2018077059A1 (en) Barcode identification method and apparatus
CN109902083A (en) Method, apparatus, computer storage medium and the terminal of a kind of pair of mark processing
CN107306199B (en) Network element data playback method and device
CN114610792A (en) Data processing method, device and system and industrial equipment
CN111063347B (en) Real-time voice recognition method, server and client
CN106533715A (en) Data analysis method and system
CN112015726A (en) User activity prediction method, system and readable storage medium
CN115190083B (en) Data transmission method and device applied to intelligent equipment
CN117573469A (en) Log analysis method, device, equipment and medium
CN115757049B (en) Multi-service module log recording method, system, electronic equipment and storage medium

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right

Effective date of registration: 20220718

Address after: Room 801, 8th floor, No. 104, floors 1-19, building 2, yard 6, Jiuxianqiao Road, Chaoyang District, Beijing 100015

Patentee after: BEIJING QIHOO TECHNOLOGY Co.,Ltd.

Address before: 100088 room 112, block D, 28 new street, new street, Xicheng District, Beijing (Desheng Park)

Patentee before: BEIJING QIHOO TECHNOLOGY Co.,Ltd.

Patentee before: Qizhi software (Beijing) Co.,Ltd.

TR01 Transfer of patent right