CN104954280A - Data message processing method and device - Google Patents
Data message processing method and device Download PDFInfo
- Publication number
- CN104954280A CN104954280A CN201510196654.0A CN201510196654A CN104954280A CN 104954280 A CN104954280 A CN 104954280A CN 201510196654 A CN201510196654 A CN 201510196654A CN 104954280 A CN104954280 A CN 104954280A
- Authority
- CN
- China
- Prior art keywords
- data message
- territory
- information
- packet
- sequence
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 13
- 238000000034 method Methods 0.000 claims description 74
- 238000007906 compression Methods 0.000 claims description 19
- 238000005457 optimization Methods 0.000 claims description 19
- 238000004891 communication Methods 0.000 description 3
- 238000005538 encapsulation Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000000712 assembly Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 230000000875 corresponding Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005755 formation reaction Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/10—Flow control; Congestion control
- H04L47/38—Flow control; Congestion control by adapting coding or compression rate
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/50—Queue scheduling
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/565—Conversion or adaptation of application format or content
- H04L67/5651—Reducing the amount or size of exchanged application data
Abstract
The invention discloses a data message processing method and device. The method can comprise that data packets of an original data message are processed according to domains, the same domains of the original data message are gathered in the same set, and size information of each set is recorded; key domains are selected from the data package of the data message, the sets to which the key domains belong are ordered, and ordering information is recorded; sets except the sets to which the key domains belong are ordered according to the ordering information; and the size information, the ordering information and new data messages of the different sets after ordering are obtained. The method and device can improve the data message processing efficiency.
Description
Technical field
The present invention relates to the communications field, particularly relate to a kind of data message processing method and device.
Background technology
At present, in data message treatment technology, data message can comprise multiple packet, and is arrange in units of packet in the structure of data message, and namely multiple territories of each packet are all arranged together.Respectively each territory of each packet is processed with regard to needing when processing data message like this, but under many circumstances, the content in territory identical in multiple packets of data message may be identical or there is certain regularity.Such as: data message comprises packet 1, packet 2 and packet 3, wherein, each packet all comprises territory A, territory B and territory C, and wherein, packet 1, packet 2 are identical with the content of the territory A of packet 3.So respectively each territory is processed, separately process will be carried out to the content repeated or to the content that there is certain regularity, thus cause the problem that the efficiency comparison of data message process is low.Visible, current data message structure easily causes the efficiency of data message process low.
Summary of the invention
The invention provides a kind of data message processing method and device, the efficiency of data message process can be improved.
First aspect, a kind of data message processing method provided by the invention, comprising:
Carry out a point territory process to the packet of former data message, wherein, described point of territory process is used for the identical territory in described former data message to be divided in identity set, and records the size information of each set;
From the packet of described data message, select key, and the set at described key place is sorted, and record ordering information;
Sort according to the set of described sequencing information to the place, foreign lands except described key in described data message;
Obtain the new data message of each set after comprising described size information, described sequencing information and sequence.
In the first possible implementation of first aspect, described method also comprises:
Adjust the territory of each packet of described former data message, wherein, described adjustment comprises following at least one item:
A territory is split at least two territories, the order between territory is adjusted and at least two territories are merged into a territory;
The described packet to former data message carries out a point territory process, comprising:
A point territory process is carried out to the packet of the former data message after described adjustment.
In conjunction with the first possible implementation of first aspect or first aspect, in the implementation that the second of first aspect is possible, described according to described sequencing information the set at place, foreign lands except described key in described data message sorted after, described method also comprises:
Code optimization process is carried out to the set after sequence;
Described acquisition comprises the new data message of each set after described size information, described sequencing information and sequence, comprising:
Obtain the new data message comprising each set after described size information, described sequencing information and described code optimization process.
In conjunction with the first possible implementation of first aspect or first aspect, in the third possible implementation of first aspect, described acquisition comprises new data message of each set after described size information, described sequencing information and sequence, comprising:
Each set after described size information, described sequencing information and sequence is encapsulated, to obtain new data message.
In conjunction with the first possible implementation of first aspect or first aspect, in the 4th kind of possible implementation of first aspect, described method also comprises:
Each set after sequence is compressed;
Described acquisition comprises the new data message of each set after described size information, described sequencing information and sequence, comprising:
Each set after described size information, described sequencing information and described compression is encapsulated, to obtain new data message.
Second aspect, the invention provides a kind of data message processing unit, comprising: point territory unit, the first sequencing unit, the second sequencing unit and acquiring unit, wherein:
Described point of territory unit, for carrying out a point territory process to the packet of former data message, wherein, described point of territory process is used for the identical territory in described former data message to be divided in identity set, and records the size information of each set;
Described first sequencing unit, for selecting key in the packet from described data message, and sorts to the set at described key place, and record ordering information;
Described second sequencing unit, for sorting according to the set of described sequencing information to the place, foreign lands except described key in described data message;
Described acquiring unit, for obtaining the new data message of each set after comprising described size information, described sequencing information and sequence.
In the first possible implementation of second aspect, described device also comprises:
Adjustment unit, the territory for each packet to described former data message adjusts, and wherein, described adjustment comprises following at least one item:
A territory is split at least two territories, the order between territory is adjusted and at least two territories are merged into a territory;
Described point of territory unit is used for carrying out a point territory process to the packet of the former data message after described adjustment.
In conjunction with the first possible implementation of second aspect or second aspect, in the implementation that the second of second aspect is possible, described device also comprises:
Coding unit, for carrying out code optimization process to the set after sequence;
Described acquiring unit comprises new data message of each set after described size information, described sequencing information and described code optimization process for obtaining.
In conjunction with the first possible implementation of second aspect or second aspect, in the third possible implementation of second aspect, described acquiring unit is used for each set after by described size information, described sequencing information and sequence and encapsulates, to obtain new data message.
In conjunction with the first possible implementation of second aspect or second aspect, in the 4th kind of possible implementation of second aspect, described device also comprises:
Compression unit, for compressing each set after sequence;
Described acquiring unit is used for each set after described size information, described sequencing information and described compression to encapsulate, to obtain new data message.
In technique scheme, carry out a point territory process to the packet of former data message, wherein, described point of territory process is used for the identical territory in described former data message to be divided in identity set, and records the size information of each set; From the packet of described data message, select key, and the set at described key place is sorted, and record ordering information; Sort according to the set of described sequencing information to the place, foreign lands except described key in described data message; Obtain the new data message of each set after comprising described size information, described sequencing information and sequence.The content repeated in the data structure of such new data message or to the content that there is certain regularity be divided to set in, like this process this data message time just can improve treatment effeciency.
Accompanying drawing explanation
In order to be illustrated more clearly in the embodiment of the present invention or technical scheme of the prior art, be briefly described to the accompanying drawing used required in embodiment or description of the prior art below, apparently, accompanying drawing in the following describes is only some embodiments of the present invention, for those of ordinary skill in the art, under the prerequisite not paying creative work, other accompanying drawing can also be obtained according to these accompanying drawings.
Fig. 1 is the schematic flow sheet of a kind of data message processing method that the embodiment of the present invention provides;
Fig. 2 is the example schematic of a kind of data message that the embodiment of the present invention provides;
Fig. 3 is the example schematic of a kind of data sorting that the embodiment of the present invention provides;
Fig. 4 is the example schematic of a kind of data message sequence that the embodiment of the present invention provides;
Fig. 5 is the schematic flow sheet of the another kind of data message processing method that the embodiment of the present invention provides;
Fig. 6 is the example schematic of a kind of data message adjustment that the embodiment of the present invention provides;
Fig. 7 is the example schematic of the another kind of data message adjustment that the embodiment of the present invention provides;
Fig. 8 is the parsing schematic diagram in the not regular length territory of a kind of data message that the embodiment of the present invention provides;
Fig. 9 is the example schematic of a kind of data encoding optimization process that the embodiment of the present invention provides;
Figure 10 is the example schematic of a kind of data message encapsulation that the embodiment of the present invention provides;
Figure 11 is the example schematic of a kind of data message compression that the embodiment of the present invention provides;
Figure 12 is the schematic flow sheet of the another kind of data message processing method that the embodiment of the present invention provides;
Figure 13 is the structural representation of a kind of data message processing unit that the embodiment of the present invention provides;
Figure 14 is the structural representation of the another kind of data message processing unit that the embodiment of the present invention provides;
Figure 15 is the structural representation of the another kind of data message processing unit that the embodiment of the present invention provides;
Figure 16 is the structural representation of the another kind of data message processing unit that the embodiment of the present invention provides;
Figure 17 is the structural representation of the another kind of data message processing unit that the embodiment of the present invention provides.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, be clearly and completely described the technical scheme in the embodiment of the present invention, obviously, described embodiment is only the present invention's part embodiment, instead of whole embodiments.Based on the embodiment in the present invention, those of ordinary skill in the art, not making the every other embodiment obtained under creative work prerequisite, belong to the scope of protection of the invention.
Refer to Fig. 1, Fig. 1 is the schematic flow sheet of a kind of data message processing method that the embodiment of the present invention provides, and as shown in Figure 1, comprises the following steps:
101, carry out a point territory process to the packet of former data message, wherein, described point of territory process is used for the identical territory in described former data message to be divided in identity set, and records the size information of each set.
Former data message in the present embodiment can comprise multiple packet, and in addition, packet can be the packet with certain data structure, such as: Packet.In addition, each packet can comprise multiple territory, such as: each packet can comprise territory A, territory B, territory C and territory D, as shown in Figure 2, former packet can comprise packet 0, packet 1, packet 2 and packet 3, and packet 0 comprises: territory A0, territory B0, territory C0 and territory D0, and packet 1 comprises: territory A1, territory B1, territory C1 and territory D1, packet 2 comprises: territory A2, territory B2, territory C2 and territory D2, and packet 3 comprises: territory A3, territory B3, territory C3 and territory D3.Just territory A0, territory A1, territory A2 and territory A3 can be divided in a set by step 101 like this, and territory B0, territory B1, territory B2 and territory B3 are divided in another set etc., territory identical in former data message so just can be realized to be divided in identity set.
In addition, above-mentioned size information can record the size information of each set, or records the size information etc. in each territory in each set.
102, from the packet of described data message, select key, and the set at described key place is sorted, and record ordering information.
Wherein, above-mentioned key can be one or more key, in addition, when being multiple key, first can carry out one deck sequence by one of them, carrying out two layers of sequence according to second again if identical, such as: as shown in Figure 3, key comprises IP address field and Traffic Domain, just first can carry out ground floor sequence according to IP address field by step 102, carries out second layer sequence again to identical according to Traffic Domain.
In addition, above-mentioned selection key can be preset a kind of selective rule, such as: in selection packet, most important territory is as key, or user specifies important territory as key etc. in advance, is not construed as limiting this.In addition, ordering rule can be use merge sort, or increases progressively rule, or supports ordering rules such as locally increasing progressively or locally successively decrease, or can also be user-defined ordering rule etc.
Such as: as shown in Figure 4, select territory B as key, the order of sequence is B3, territory B1, territory B0 and territory B2.
103, sort according to the set of described sequencing information to the place, foreign lands except described key in described data message.
The sequence of step 103 can be the sequence using above-mentioned steps 102, such as: as shown in Figure 4, the order that step 102 sorts is B3, territory B1, territory B0 and territory B2, such step 103 just can sort to three set of other shown in Fig. 4, thus the sequence obtained as shown in Figure 4, each set is all sorted with the order of 3,1,0 to 2.
104, the new data message of each set after comprising described size information, described sequencing information and sequence is obtained.
The new data message generated as shown in Figure 4 just can be realized by above-mentioned steps, territory identical in this data message is arranged on together, and carry out sorting according to certain ordering rule, so just can realize former data message can by arranged together for the content of some repetitions, such as: as shown in Figure 3, identical IP address arrangement together, or by arranged together for the content that there is certain rule, as shown in Figure 3, close IP address arrangement together.Like this when processing this new data message, just can process together the content of the content repeated or the certain rule of existence, thus the efficiency of data message process can be improved.
Optionally, said method can be applied to the equipment possessing arbitrarily data message process, such as: server, people's computer (Personal Computer, PC), notebook computer, panel computer, mobile phone, electronic reader, remote controller, mobile unit, Web TV, wearable device etc. have the smart machine of data Storage and Processing ability.
In addition, above-mentioned data message can be the message possessing arbitrarily specific data structure, and wherein, the message of this specific data structure can be to developer or the known data structure message of user, such as: list message or network message etc.
In technique scheme, carry out a point territory process to the packet of former data message, wherein, described point of territory process is used for the identical territory in described former data message to be divided in identity set, and records the size information of each set; From the packet of described data message, select key, and the set at described key place is sorted, and record ordering information; Sort according to the set of described sequencing information to the place, foreign lands except described key in described data message; Obtain the new data message of each set after comprising described size information, described sequencing information and sequence.The content repeated in the data structure of such new data message or to the content that there is certain regularity be divided to set in, like this process this data message time just can improve treatment effeciency.
Refer to Fig. 5, Fig. 5 is the schematic flow sheet of the another kind of data message processing method that the embodiment of the present invention provides, and as shown in Figure 5, comprises the following steps:
501, the territory of each packet of former data message is adjusted.
Wherein, described adjustment can comprise following at least one item:
A territory is split at least two territories, the order between territory is adjusted and at least two territories are merged into a territory.
Such as: identifying each packet in former data message, and after the information in each packet structure and each territory, just can adjust the territory of each packet of described former data message.Wherein, here can be that part territory is adjusted, such as: as shown in Figure 6, former data message comprises timestamp territory, and in timestamp territory, often there is a part of changeless feature, 2E 85C93A as shown in Figure 6, in addition, the random nature of another part of this timestamp is stronger, rear 2 bytes as shown in Figure 6.Such above-mentioned steps 501 just can as shown in Figure 6, and the forward part in this timestamp territory is split as an independent territory, in this territory, the content of all packets is all identical, another part is divided into the territory that another is independent.So just can be arranged together by changeless territory as shown in Figure 6 by this method, thus the Entropy Changes of usage data is little, so, when processing this data message, will be very high to the efficiency of the contents processing of this repetition.Such as: when compressing, these identical territories are more prone to produce and mate, thus promote compression efficiency and the compression ratio of data.
Such as: can also be put together in the territory of identical rule, to excavate regularity and the relevance of data.Such as: as shown in Figure 7, each packet comprises field of constants, so just these two territories can be adjusted to the field of constants that formation one is larger together, or as shown in Figure 7, be adjusted the position of one of them field of constants, to increase the relevance of data.
Such as: multiple territory can also be merged into a larger territory, this multiple territory can be the feature itself possessing High relevancy, the matching length more can grown when dictionary compression after merging, thus improve compression ratio.Such as: for the packet comprising protocol number and port numbers two territories, protocol number and these two territories of port numbers can be merged into a territory.Again such as: in data message shown in Fig. 7, name and these two territories, IP address can be merged into a territory, a territory can also be merged in movable and these two territories, website.
Just can be improved the repeated Sum fanction of data message by above-mentioned steps, make the Entropy Changes of data little, be more suitable for compression like this, and during process, efficiency is higher.
502, carry out a point territory process to the packet of the former data message after described adjustment, wherein, described point of territory process is used for the identical territory in described former data message to be divided in identity set, and records the size information of each set.
In the present embodiment, above-mentioned being divided in identical territory can be understood as, using identical territory as a subfile with in set.
In addition, in above-mentioned former data message, the territory of packet can be the territory of regular length, or the not territory of regular length, wherein, the territory of regular length can not obtain its length information by resolving the territory of regular length, such as: as shown in Figure 8, packet comprises IP1 length field and IP1 territory, wherein, the length of IP1 length field is 1 byte, and its function is for representing when IP1 territory is that IP1 length of field is 4 bytes when the content in IP1 territory is the IP address of IPV4, when IP1 territory is that IP1 length of field is 16 bytes when the content in IP1 territory is the IP address of IPV6.
503, from the packet of described data message, select key, and the set at described key place is sorted, and record ordering information.
In the present embodiment, above-mentioned key can be the territory possessing information separated feature, such as: the territory such as protocol number, port numbers, IP address.
In addition, above-mentioned sequencing information can be sequence dictionary.
504, sort according to the set of described sequencing information to the place, foreign lands except described key in described data message.
505, the new data message of each set after comprising described size information, described sequencing information and sequence is obtained.
In the present embodiment, after step 504, described method can also comprise the steps:
Code optimization process is carried out to the set after sequence;
Step 505 can comprise:
Obtain the new data message comprising each set after described size information, described sequencing information and described code optimization process.
Wherein, above-mentioned code optimization process can be carry out code optimization process to some or all of set, and wherein, code optimization process can adopt LRU and delta to encode.
Wherein, delta coded treatment can be that the next data of user deduct data, is applicable to the data of local order, the key of use of such as sorting or timestamp etc.
Wherein, LRU coding can be applicable to the sparse or local value of standby numeric distribution and to distribute the set in territory of sparse feature, such as: the set in the territories such as protocol number, IP address.LRU uses a LRU record sheet, this table is for recording most recently used data value, next numerical value is detected at every turn, if there is identical numerical value in LRU record sheet, then replace with the index (index) in LRU record sheet, otherwise increase by represents that the index inserted is before numerical value.In addition, LRU record sheet can refresh in real time, also only can record fixing empirical value.Such as: as shown in Figure 9, former 16bit territory size, after LRU coding, if found in LRU record sheet, just becomes 2bit.
In the present embodiment, step 505 can comprise:
Each set after described size information, described sequencing information and sequence is encapsulated, to obtain new data message.
Wherein, above-mentioned size information and sequencing information can be the stems that continuous print is placed on new data message, such as: as shown in Figure 10 a, it should be noted that, in the present embodiment, data message can be understood as a file, and set can be understood as subfile, and the subfile size in Figure 10 just represents the size information of this subfile.In addition, in the present embodiment, all right discrete placement, such as: as shown in fig. lob.In addition, in the present embodiment, the packaged type of new data message is not construed as limiting, as long as the packaged type that can resolve can.
In the present embodiment, said method can also comprise the steps:
Each set after sequence is compressed;
Step 505 can comprise:
Each set after described size information, described sequencing information and described compression is encapsulated, to obtain new data message.
This execution mode, can realize first compressing and encapsulate, specifically as shown in fig. lla, point data for territory sequence can be compressed, wherein, be compressed in Figure 11 C and represent.
In addition, in the present embodiment, can also as shown in figure lib, after first each set after described size information, described sequencing information and sequence being encapsulated, then compress.
It should be noted that, in the present embodiment, said method can be applied to the scene supported lossless data compression or damage data compression, in addition, said method can also be applied to any scene needing to promote data redundancy and regularity, such as: need the scene arranged data in database scene etc.
In the present embodiment, the basis of the embodiment shown in Fig. 1 adds the execution mode of plurality of optional, and the efficiency improving data message process can be realized.
Refer to Figure 12, Figure 12 is the schematic flow sheet of the another kind of data message processing method that the embodiment of the present invention provides, and as shown in figure 12, comprises the following steps:
1201, each packet in list message is identified, and the information in packet structure each territory interior.
1202, part territory is adjusted.
Wherein, described adjustment can comprise following at least one item:
A territory is split at least two territories, the order between territory is adjusted and at least two territories are merged into a territory;
1203, a point territory is carried out to the data after adjustment, obtain multiple subfile.
Wherein, the identical territory in data message is divided in same subfile, and records the size information of each subfile.
1204, corresponding to key subfile sorts.
1205, record ordering dictionary.
Wherein, above-mentioned sequence dictionary can comprise sequencing information.
1206, all subfiles are reset according to sequence dictionary.
Wherein, the sequence of all subfiles can be sort according to above-mentioned sequence dictionary.
1207, code optimization process is carried out to parton file.
1208, all subfiles, all subfile size information, sequence dictionary information are encapsulated.
Can be specifically to coding priority treatment after all subfile size information of all subfiles, sequence dictionary information encapsulate.
1209, the file after encapsulation is compressed.
This embodiment, can improve the compression ratio of data.
Be apparatus of the present invention embodiment below, the method that apparatus of the present invention embodiment realizes for performing the inventive method embodiment one to three, for convenience of explanation, illustrate only the part relevant to the embodiment of the present invention, concrete ins and outs do not disclose, and please refer to the embodiment of the present invention one, embodiment two and embodiment three.
Refer to Figure 13, Figure 13 is the structural representation of a kind of data message processing unit that the embodiment of the present invention provides, and comprising: point territory unit 131, first sequencing unit 132, second sequencing unit 133 and acquiring unit 134, wherein:
Divide territory unit 131, for carrying out a point territory process to the packet of former data message, wherein, described point of territory process is used for the identical territory in described former data message to be divided in identity set, and records the size information of each set.
First sequencing unit 132, for selecting key in the packet from described data message, and sorts to the set at described key place, and record ordering information.
Second sequencing unit 133, for sorting according to the set of described sequencing information to the place, foreign lands except described key in described data message.
Acquiring unit 134, for obtaining the new data message of each set after comprising described size information, described sequencing information and sequence.
Optionally, said apparatus can be applied to the equipment possessing arbitrarily data message process, such as: server, PC, notebook computer, panel computer, mobile phone, electronic reader, remote controller, mobile unit, Web TV, wearable device etc. have the smart machine of data Storage and Processing ability.
In addition, above-mentioned data message can be the message possessing arbitrarily specific data structure, and wherein, the message of this specific data structure can to developer or the known data structure message of user, such as: list message or network message etc.
In technique scheme, carry out a point territory process to the packet of former data message, wherein, described point of territory process is used for the identical territory in described former data message to be divided in identity set, and records the size information of each set; From the packet of described data message, select key, and the set at described key place is sorted, and record ordering information; Sort according to the set of described sequencing information to the place, foreign lands except described key in described data message; Obtain the new data message of each set after comprising described size information, described sequencing information and sequence.The content repeated in the data structure of such new data message or to the content that there is certain regularity be divided to set in, like this process this data message time just can improve treatment effeciency.
Refer to Figure 14, Figure 14 is the structural representation of the another kind of data message processing unit that the embodiment of the present invention provides, and comprising: adjustment unit 141, point territory unit 142, first sequencing unit 143, second sequencing unit 144 and acquiring unit 145, wherein:
Adjustment unit 141, the territory for each packet to described former data message adjusts.
Wherein, described adjustment comprises following at least one item:
A territory is split at least two territories, the order between territory is adjusted and at least two territories are merged into a territory;
Divide territory unit 142, for carrying out a point territory process to the packet of the former data message after described adjustment, wherein, described point of territory process is used for the identical territory in described former data message to be divided in identity set, and records the size information of each set.
First sequencing unit 143, for selecting key in the packet from described data message, and sorts to the set at described key place, and record ordering information.
Second sequencing unit 144, for sorting according to the set of described sequencing information to the place, foreign lands except described key in described data message.
Acquiring unit 145, for obtaining the new data message of each set after comprising described size information, described sequencing information and sequence.
Optionally, as shown in figure 15, said apparatus can also comprise:
Coding unit 146, for carrying out code optimization process to the set after sequence;
Acquiring unit 145 may be used for obtaining the new data message comprising each set after described size information, described sequencing information and described code optimization process.
Optionally, acquiring unit 145 may be used for described size information, described sequencing information and each set after sorting to encapsulate, to obtain new data message.
Optionally, as shown in figure 16, said apparatus can also comprise:
Compression unit 147, for compressing each set after sequence;
Acquiring unit 145 may be used for each set after by described size information, described sequencing information and described compression and encapsulates, to obtain new data message.
In the present embodiment, the basis of the embodiment shown in Figure 13 adds the execution mode of plurality of optional, and the efficiency improving data message process can be realized.
Refer to Figure 17, Figure 17 is the structural representation of the another kind of data message processing unit that the embodiment of the present invention provides, as shown in figure 17, this device comprises: at least one processor 171, such as CPU, at least one network interface 174 or other user interfaces 173, memory 175, at least one communication bus 172.Communication bus 172 is for realizing the connection communication between these assemblies.This computing node 170 optionally comprises user interface 173, comprises display, keyboard or pointing device (such as, mouse, trace ball (trackball), touch-sensitive plate or touch sensitive display screen).Memory 175 may comprise high-speed RAM memory, still may comprise non-labile memory (non-volatile memory), such as at least one magnetic disc store.Memory 175 optionally can comprise at least one and be positioned at storage device away from aforementioned processor 171.
In some embodiments, memory 175 stores following element, executable module or data structure, or their subset, or their superset:
Operating system 1751, comprises various system program, for realizing various basic business and processing hardware based task;
Application program module 1752, comprises various application program, for realizing various applied business.
In embodiments of the present invention, by call memory 175 store program or instruction, processor 171 for:
Carry out a point territory process to the packet of former data message, wherein, described point of territory process is used for the identical territory in described former data message to be divided in identity set, and records the size information of each set;
From the packet of described data message, select key, and the set at described key place is sorted, and record ordering information;
Sort according to the set of described sequencing information to the place, foreign lands except described key in described data message;
Obtain the new data message of each set after comprising described size information, described sequencing information and sequence.
Optionally, processor 171 can also be used for:
Adjust the territory of each packet of described former data message, wherein, described adjustment comprises following at least one item:
A territory is split at least two territories, the order between territory is adjusted and at least two territories are merged into a territory;
Optionally, the packet to former data message that processor 171 performs carries out a point territory handling procedure, can comprise:
A point territory process is carried out to the packet of the former data message after described adjustment.
Optionally, processor 171 perform according to described sequencing information to described data message in sort except the set at place, foreign lands of described key after, can also be used for:
Code optimization process is carried out to the set after sequence;
Optionally, the acquisition that processor 171 performs comprises the new data message procedure of each set after described size information, described sequencing information and sequence, can comprise:
Obtain the new data message comprising each set after described size information, described sequencing information and described code optimization process.
Optionally, the acquisition that processor 171 performs comprises the program of the new data message of each set after described size information, described sequencing information and sequence, can comprise:
Each set after described size information, described sequencing information and sequence is encapsulated, to obtain new data message.
Optionally, processor 171 can also be used for:
Each set after sequence is compressed;
Optionally, the acquisition that processor 171 performs comprises the program of the new data message of each set after described size information, described sequencing information and sequence, can comprise:
Each set after described size information, described sequencing information and described compression is encapsulated, to obtain new data message.
In the present embodiment, carry out a point territory process to the packet of former data message, wherein, described point of territory process is used for the identical territory in described former data message to be divided in identity set, and records the size information of each set; From the packet of described data message, select key, and the set at described key place is sorted, and record ordering information; Sort according to the set of described sequencing information to the place, foreign lands except described key in described data message; Obtain the new data message of each set after comprising described size information, described sequencing information and sequence.The content repeated in the data structure of such new data message or to the content that there is certain regularity be divided to set in, like this process this data message time just can improve treatment effeciency.
One of ordinary skill in the art will appreciate that all or part of flow process realized in above-described embodiment method, that the hardware that can carry out instruction relevant by computer program has come, described program can be stored in a computer read/write memory medium, this program, when performing, can comprise the flow process of the embodiment as above-mentioned each side method.Wherein, described storage medium can be magnetic disc, CD, read-only store-memory body (Read-Only Memory, ROM) or random access memory (Random Access Memory is called for short RAM) etc.
Above disclosedly be only present pre-ferred embodiments, certainly can not limit the interest field of the present invention with this, therefore according to the equivalent variations that the claims in the present invention are done, still belong to the scope that the present invention is contained.
Claims (10)
1. a data message processing method, is characterized in that, comprising:
Carry out a point territory process to the packet of former data message, wherein, described point of territory process is used for the identical territory in described former data message to be divided in identity set, and records the size information of each set;
From the packet of described data message, select key, and the set at described key place is sorted, and record ordering information;
Sort according to the set of described sequencing information to the place, foreign lands except described key in described data message;
Obtain the new data message of each set after comprising described size information, described sequencing information and sequence.
2. the method for claim 1, is characterized in that, described method also comprises:
Adjust the territory of each packet of described former data message, wherein, described adjustment comprises following at least one item:
A territory is split at least two territories, the order between territory is adjusted and at least two territories are merged into a territory;
The described packet to former data message carries out a point territory process, comprising:
A point territory process is carried out to the packet of the former data message after described adjustment.
3. method as claimed in claim 1 or 2, is characterized in that, described according to described sequencing information in described data message except after the set at the place, foreign lands of described key sorts, described method also comprises:
Code optimization process is carried out to the set after sequence;
Described acquisition comprises the new data message of each set after described size information, described sequencing information and sequence, comprising:
Obtain the new data message comprising each set after described size information, described sequencing information and described code optimization process.
4. method as claimed in claim 1 or 2, is characterized in that, described acquisition comprises the new data message of each set after described size information, described sequencing information and sequence, comprising:
Each set after described size information, described sequencing information and sequence is encapsulated, to obtain new data message.
5. method as claimed in claim 1 or 2, it is characterized in that, described method also comprises:
Each set after sequence is compressed;
Described acquisition comprises the new data message of each set after described size information, described sequencing information and sequence, comprising:
Each set after described size information, described sequencing information and described compression is encapsulated, to obtain new data message.
6. a data message processing unit, is characterized in that, comprising: point territory unit, the first sequencing unit, the second sequencing unit and acquiring unit, wherein:
Described point of territory unit, for carrying out a point territory process to the packet of former data message, wherein, described point of territory process is used for the identical territory in described former data message to be divided in identity set, and records the size information of each set;
Described first sequencing unit, for selecting key in the packet from described data message, and sorts to the set at described key place, and record ordering information;
Described second sequencing unit, for sorting according to the set of described sequencing information to the place, foreign lands except described key in described data message;
Described acquiring unit, for obtaining the new data message of each set after comprising described size information, described sequencing information and sequence.
7. device as claimed in claim 6, it is characterized in that, described device also comprises:
Adjustment unit, the territory for each packet to described former data message adjusts, and wherein, described adjustment comprises following at least one item:
A territory is split at least two territories, the order between territory is adjusted and at least two territories are merged into a territory;
Described point of territory unit is used for carrying out a point territory process to the packet of the former data message after described adjustment.
8. device as claimed in claims 6 or 7, it is characterized in that, described device also comprises:
Coding unit, for carrying out code optimization process to the set after sequence;
Described acquiring unit comprises new data message of each set after described size information, described sequencing information and described code optimization process for obtaining.
9. device as claimed in claims 6 or 7, is characterized in that, described acquiring unit is used for each set after by described size information, described sequencing information and sequence and encapsulates, to obtain new data message.
10. device as claimed in claims 6 or 7, it is characterized in that, described device also comprises:
Compression unit, for compressing each set after sequence;
Described acquiring unit is used for each set after described size information, described sequencing information and described compression to encapsulate, to obtain new data message.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510196654.0A CN104954280B (en) | 2015-04-23 | 2015-04-23 | A kind of data message processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510196654.0A CN104954280B (en) | 2015-04-23 | 2015-04-23 | A kind of data message processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104954280A true CN104954280A (en) | 2015-09-30 |
| CN104954280B CN104954280B (en) | 2018-10-09 |
Family
ID=54168642
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510196654.0A Active CN104954280B (en) | 2015-04-23 | 2015-04-23 | A kind of data message processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104954280B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106789695A (en) * | 2016-11-24 | 2017-05-31 | 杭州迪普科技股份有限公司 | Message processing method and device |
| CN107800653A (en) * | 2016-09-06 | 2018-03-13 | 中兴通讯股份有限公司 | Message compression method and device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101286895A (en) * | 2008-05-22 | 2008-10-15 | 上海交通大学 | Dynamic configurable data monitoring system and method for distributed network |
| US20100005147A1 (en) * | 2008-06-17 | 2010-01-07 | Johnson Iii William Kimble | Ordered message processing |
| CN103401850A (en) * | 2013-07-19 | 2013-11-20 | 北京星网锐捷网络技术有限公司 | Message filtering method and device |
| CN104267937A (en) * | 2014-09-16 | 2015-01-07 | 中国工商银行股份有限公司 | Online small-batch data processing method and online small-batch data processing system |
-
2015
- 2015-04-23 CN CN201510196654.0A patent/CN104954280B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101286895A (en) * | 2008-05-22 | 2008-10-15 | 上海交通大学 | Dynamic configurable data monitoring system and method for distributed network |
| US20100005147A1 (en) * | 2008-06-17 | 2010-01-07 | Johnson Iii William Kimble | Ordered message processing |
| CN103401850A (en) * | 2013-07-19 | 2013-11-20 | 北京星网锐捷网络技术有限公司 | Message filtering method and device |
| CN104267937A (en) * | 2014-09-16 | 2015-01-07 | 中国工商银行股份有限公司 | Online small-batch data processing method and online small-batch data processing system |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107800653A (en) * | 2016-09-06 | 2018-03-13 | 中兴通讯股份有限公司 | Message compression method and device |
| CN107800653B (en) * | 2016-09-06 | 2021-09-03 | 中兴通讯股份有限公司 | Message compression method and device |
| CN106789695A (en) * | 2016-11-24 | 2017-05-31 | 杭州迪普科技股份有限公司 | Message processing method and device |
| CN106789695B (en) * | 2016-11-24 | 2020-04-03 | 杭州迪普科技股份有限公司 | Message processing method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104954280B (en) | 2018-10-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108365967B (en) | Method, system, terminal and computer readable storage medium for dynamically configuring communication parameters | |
| US10701150B2 (en) | Method and apparatus for storing network data | |
| US20120150877A1 (en) | Efficient database compression | |
| CN102111331B (en) | Matching method based on hash table and adopting mask five-element rule | |
| CN107977371A (en) | Requesting method, display methods and the client of page elements | |
| CN103678293A (en) | Data storage method and device | |
| CN108197324B (en) | Method and apparatus for storing data | |
| CN108989205B (en) | Identity identification and routing data generation method and device and server | |
| CN111368520A (en) | Editing method and device for electronic forms | |
| CN111311326A (en) | User behavior real-time multidimensional analysis method and device and storage medium | |
| CN102904918A (en) | Method and system for processing display of network media information | |
| CN105550179A (en) | Webpage collection method and browser plug-in | |
| CN104954280A (en) | Data message processing method and device | |
| CN111355816B (en) | Server selection method, device, equipment and distributed service system | |
| CN105245622A (en) | Configurable data processing method and system and mobile platform | |
| CN104572981A (en) | Web page caching method and mobile terminal device | |
| CN108762637A (en) | Control method, apparatus and system | |
| CN106897280A (en) | Data query method and device | |
| CN105117403A (en) | Log data fragmentation and query method and apparatus | |
| CN112114849B (en) | Resource updating method and device | |
| CN104484174A (en) | Processing method and processing device for compressed file with RAR (Roshal A Rchive) format | |
| CN114327857A (en) | Operation data processing method and device, computer equipment and storage medium | |
| US10659076B1 (en) | Reducing the amount of data stored in a sequence of data blocks by combining deduplication and compression | |
| US20200204688A1 (en) | Picture book sharing method and apparatus and system using the same | |
| CN108984221B (en) | Method and device for acquiring multi-platform user behavior logs |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20211227 Address after: 450046 Floor 9, building 1, Zhengshang Boya Plaza, Longzihu wisdom Island, Zhengdong New Area, Zhengzhou City, Henan Province Patentee after: Super fusion Digital Technology Co.,Ltd. Address before: 518129 Bantian HUAWEI headquarters office building, Longgang District, Guangdong, Shenzhen Patentee before: HUAWEI TECHNOLOGIES Co.,Ltd. |
|
| TR01 | Transfer of patent right |