CN103546307A - Network flow storage method - Google Patents
Network flow storage method Download PDFInfo
- Publication number
- CN103546307A CN103546307A CN201210246855.3A CN201210246855A CN103546307A CN 103546307 A CN103546307 A CN 103546307A CN 201210246855 A CN201210246855 A CN 201210246855A CN 103546307 A CN103546307 A CN 103546307A
- Authority
- CN
- China
- Prior art keywords
- client
- server
- network flow
- packet
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
The invention provides a network flow storage method. The method includes the following steps: S1, initializing a Client list and a Server list; S2, finding network flow, corresponding to a captured new-coming data packet, in a flow list; S3, updating the Client list and the Server list. The network flow belonging to a same node is aggregated together quickly, so that a flow classification system can be assisted in digging relationship among the network flow deeply so as to meet challenges brought by a novel application protocol, and technical support can be provided for designing and realizing of a high-performance flow classification system and a content monitoring system in a high-speed network.
Description
Technical field
The invention belongs to traffic classification technical field in network technology, relate in particular to a kind of new network flow storage means.
Background technology
Because the development of network technology and the network bandwidth is rapid, the data traffic in network is also multiplied, and on high-speed backbone network, data traffic reached Gbit each second, even more than 10Gbit.The network traffics that constantly increase have proposed new challenge to traffic classification: the efficiency of traditional traffic classification system based on packet has been difficult to meet the needs of high-speed backbone monitoring.At high-speed wideband net environment, the infinite arrival of network data high speed, and uninterrupted, present mass data feature, and this locality cannot be stored.Therefore, rely on traditional traffic classification system effectiveness of packet capture-packet reduction-pattern matching to satisfy the demand.In addition,, along with the complexity day by day of network environment, increasing application layer protocol adopts cryptographic protocol enciphered data payload package.In this case, the difficulty of finding packet load key is increasing, finally causes the traffic classification technology based on packet of serious failure.
Be different from the traffic classification technology based on packet, the traffic classification technology of stream Network Based is conceived to network flow.Traditionally network flow is defined as to the have identical five-tuple set of packet of (< source address, destination address, source port, destination interface, agreement >).As a kind of data exchange ways, network flow has reflected the details of the intercommunication of Host behavior and main frame from a microcosmic point.
The supposed premise of the traffic classification technology of stream Network Based is that different agreement has its distinctive network flow statistical property, and the flow producing with this different agreement of classifying.Because this technology has been introduced a large amount of statistical informations as basic reference factor, so it has inevitably been attached to the method for machine learning in identification, expectation obtains better traffic classification performance.Machine learning method was introduced in traffic classification technology in 2004, and the statistical property having according to flow is classified to flow.For example, the distribution character of network flow duration, flows free time, and the inter-packet gap time, the information such as packet length, for traffic classification, are distinctive information.They can be carried out traffic classification by machine learning model utilization as the feature of discriminant.
In order to extract network flow statistical nature, need to set up the data structure of a pick up and store network flow, and according to the standard of network flow, from background traffic, extract and storage networking stream information.At present, the traffic classification system of nearly all stream Network Based all shows pick up and store network flow with stream.Stream table has adopted a kind of Hash table to add structure network flow under each packet in determining background traffic of chained list, and it is stored.When data are coated catch after, traffic classification system can utilize the five-tuple of this packet to calculate a hash value, and utilizes this hash value to find the information that whether has this packet map network stream in Hash table.If there is no, first of take that this packet is belonging network stream arrives packet, for it sets up a network flow record.Utilize Hash table storage networking stream, conflict is inevitable.Therefore,, when conflict occurs, system can be set up a chained list carry in the respective items of Hash table for the network flow of conflict.Utilize this stream table, the traffic classification system of stream Network Based is the affiliated network flow of corresponding each packet exactly, and extracts efficiently the statistical nature of single network stream.
Along with the development of network technology, new application layer protocol emerges in an endless stream.In order to increase network utilization, and antagonism traffic classification system, many emerging application layer protocols can be enabled a plurality of network flows simultaneously and complete a communication task.Wherein, each network flow is only responsible for a part for task.P2P agreement is an exemplary of this emerging application layer protocol.In order to realize better, quickly file-sharing, many P2P consultations become a plurality of by a file division, and utilize a plurality of network flows to share this document simultaneously; Another typical example is interactive protocol, and this agreement needs to carry out alternately with server in running.In order to raise the efficiency, most interactive protocols all can be deposited different interaction contents to different servers, and client can utilize a plurality of network flows to realize the mutual fast of information simultaneously.This novel application layer protocol has proposed new challenge to the traffic classification system of stream Network Based: first, this agreement is used a plurality of network flows to complete same communication task simultaneously, the knowledge of extracting from single network stream and utilizing is reduced, affected the recognition performance of categorizing system; Secondly, the traffic classification system of current stream Network Based is conceived to single network stream, and the all-network stream that is difficult to this agreement to produce is classified.
In order to address the above problem, the challenge that reply new application layer protocol brings, the traffic classification technology of increasing stream Network Based starts to utilize many network flows feature.This novel network flow feature attempts, from many network flows angle, to find the relationship characteristic between a plurality of network flows, to realize accurate, the complete classification of P2P, interactive protocol flow.Yet current stream list structure is but difficult to extract many network flows relationship characteristic: stream table is used a kind of flat structure storage networking stream, and network flow is evenly distributed in Hash table.May not there is not any relation in the network flow with identical hash value, and belong to the hash value possibility difference of the network flow of same agreement, so we are difficult to the relation between judgement network flow.
From many network flows angle, find the relation between a plurality of network flows, extract the relationship characteristic between many network flows, can help the traffic classification system of stream Network Based to realize accurate, the complete classification of the novel protocol traffics such as P2P, interactive protocol.Yet current stream list structure is conceived to single network stream, use a kind of flat structure storage networking stream, be difficult to extract the relationship characteristic between a plurality of network flows.
Summary of the invention
(1) technical problem that will solve
Technical problem to be solved by this invention is: how a kind of new network flow pick up and store method is provided, can fast and effeciently extract the relationship characteristic between many network flows, to help traffic classification system to tackle better the challenge that new application layer protocol brings.
(2) technical scheme
In order to address the above problem, the invention provides a kind of network flow storage means, comprise step: S1. initialization Client table and Server table; S2. search the network flow that newly enters packet correspondence in stream table capturing; S3. upgrade Client table and Sever table.
Preferably, step S1 comprises: S1.1 is that size of Client initialization is n
chash table; S1.2 is that size of Sever initialization is n
shash table.
Preferably, for the initialized Hash table of Client is that a size is n
csequence list, a plurality of network flows that each list item is initiated for storing client.
Preferably, for the initialized Hash table of Server is that a size is n
ssequence list, a plurality of network flows that each list item receives for storing server.
Preferably, step S2 comprises: S2.1 catches one and newly enters packet; The hash value h1 of S2.2 calculated data bag forward five-tuple (< source address, destination address, source port, destination interface, agreement >); S2.3 utilizes hash value h1 to search in stream table whether have corresponding network flow, if exist, the direction of mark current data packet is from client to server, and performs step S3, if do not exist, performs step S2.3; The hash value h2 of the reverse five-tuple of S2.4 calculated data bag (< destination address, source address, destination interface, source port, agreement >); S2.5 utilizes hash value h2 to search in stream table whether have corresponding network flow, if exist, the direction of mark current data packet is from server to client end, and performs step S3, if do not exist, performs step S2.5; S2.6 utilizes hash value h1 for a network flow record of this packet establishment, and the direction of mark current data packet is first packet of the network flow from client to server, execution step S3.
Preferably, step S3 comprises: the hash value h3 of S3.1 computing client end IP address and the hash value h4 of server ip address; S3.2 judges whether packet is first packet of this network flow, if so, performs step S3.3, if not, perform step S3.4; S3.3 increases new network flow information in Client table and Server table; S3.4 usage data bag upgrades respective items in Client table and Server table; S3.5 returns to execution step S2.
Preferably, step S3.3 comprises: S3.31 utilizes hash value h3 to search at Client table the information that whether has this client, if do not exist, for this client creates corresponding record in Client table; S3.32 utilizes hash value h4 to search at Server table the information that whether has this server, if do not exist, for this server creates corresponding record in Server table.
Preferably, when creating client records, if list item corresponding to h3 taken by other clients, use chained list that client-side information is mounted to after the list item that h3 is corresponding.
Preferably, when creating server record, if list item corresponding to h4 taken by other servers, use chained list that server info is mounted to after the list item that h4 is corresponding.
Preferably, step S3.4 comprises: S3.41 judges whether packet is first packet of this network flow, if so, performs step S3.42, if not, perform step S3.43; S3.42 increases network flow information in the respective items of Client table and Sever table; S3.43 utilize that packet upgrades that Client table and Server show to the network flow information in corresponding.
(3) beneficial effect
Method of the present invention, on the basis of existing stream table, increases by two Hash tables for storing the network flow of the sending and receiving of client and server node.After packet is hunted down, first search its network flow corresponding in stream table, two the Hash tables of information updating that then utilize network flow and packet to provide.The method is by condensing together the network flow that belongs to same node rapidly, can help the traffic classification system degree of depth to excavate the relation between network flow, the challenge that reply new application layer protocol brings, can provide technical support for the Design and implementation of high-performance traffic classification system, content monitoring system in express network.
Accompanying drawing explanation
Further describe the present invention with reference to the accompanying drawings and in conjunction with example.Wherein:
Fig. 1 is according to the key step flow chart of the network flow storage means of the embodiment of the present invention.
Fig. 2 is the concrete implementation step flow chart according to the network flow storage means of the embodiment of the present invention.
Fig. 3 upgrades schematic diagram according to the Client table of the embodiment of the present invention and Server table.
Embodiment
Below in conjunction with drawings and Examples, the specific embodiment of the present invention is described in further detail.Following examples are used for illustrating the present invention, but are not used for limiting the scope of the invention.
Current stream list structure is conceived to single network stream, uses a kind of flat structure storage networking stream, is difficult to extract the relationship characteristic between a plurality of network flows, makes the traffic classification system of stream Network Based be difficult to tackle the challenge that new application layer protocol brings.For this problem, the present invention proposes a kind of new network flow storage means.The method, on the basis of existing stream table, increases by two Hash tables for storing the network flow of the sending and receiving of client and server node.After packet is hunted down, first search its network flow corresponding in stream table, two the Hash tables of information updating that then utilize network flow and packet to provide.By rapidly the network flow that belongs to same node being condensed together, the method can help the traffic classification system degree of depth to excavate the relation between network flow, the challenge that reply new application layer protocol brings.
As shown in Figure 1-Figure 3, according to a kind of new network flow storage means provided by the invention, it comprises the following steps:
S1 initialization Client table and Server table;
Wherein, step S1 further comprises:
S1.1 be size of Client initialization be nc(for example: Hash 4096) table;
Wherein, in step S1.1,
For the initialized Hash table of Client is a sequence list that size is nc, a plurality of network flows that each list item is initiated for storing a client;
Wherein, in step S1.2,
For the initialized Hash table of Server is a sequence list that size is ns, a plurality of network flows that each list item receives for storing a server;
Wherein, before step S1, also comprise the step of setting up for the described stream table of storage networking stream information.
S1.2 be size of Sever initialization be ns(for example: Hash 4096) table;
S2 searches the network flow that newly enters packet correspondence in stream table capturing;
Wherein, step S2 further comprises:
S2.1 catches one and newly enters packet;
Wherein, in step S2.1,
The packet of catching comprises Transmission Control Protocol and udp data bag.
The hash value h1 of S2.2 calculated data bag forward five-tuple (< source address, destination address, source port, destination interface, agreement >);
S2.3 utilizes hash value h1 to search in stream table whether have corresponding network flow, if exist, the direction of mark current data packet is from client to server, and performs step S3, if do not exist, performs step S2.3;
The hash value h2 of the reverse five-tuple of S2.4 calculated data bag (< destination address, source address, destination interface, source port, agreement >);
S2.5 utilizes hash value h2 to search in stream table whether have corresponding network flow, if exist, the direction of mark current data packet is from server to client end, and performs step S3, if do not exist, performs step S2.5;
S2.6 utilizes hash value h1 for a network flow record of this packet establishment, and the direction of mark current data packet is first packet of the network flow from client to server, execution step S3;
S3 upgrades Client table and Sever table;
As shown in Figure 2, captive packet is first for upgrading stream table, and the network flow information after upgrading in stream table is used for upgrading Client table and Server table;
Wherein, step S3 further comprises:
The hash value h3 of S3.1 computing client end IP address and the hash value h4 of server ip address;
S3.2 judges whether packet is first packet of this network flow, if so, performs step S3.3, if not, perform step S3.4;
S3.3 increases new network flow information in Client table and Server table;
Wherein, step S3.3 further comprises:
S3.31 utilizes hash value h3 to search at Client table the information that whether has this client, if do not exist, for this client creates corresponding record in Client table;
Wherein, in step S3.31,
When creating client records, if list item corresponding to h3 taken by other clients, use chained list that client-side information is mounted to after the list item that h3 is corresponding;
S3.32 utilizes hash value h4 to search at Server table the information that whether has this server, if do not exist, for this server creates corresponding record in Server table;
Wherein, in step S3.32,
When creating server record, if list item corresponding to h4 taken by other servers, use chained list that server info is mounted to after the list item that h4 is corresponding;
S3.4 usage data bag upgrades respective items in Client table and Server table;
Wherein, step S3.4 further comprises:
S3.41 judges whether packet is first packet of this network flow, if so, performs step S3.42, if not, perform step S3.43;
S3.42 increases network flow information in the respective items of Client table and Sever table;
S3.43 utilize that packet upgrades that Client table and Server show to the network flow information in corresponding;
S3.5 returns to execution step S2.
Description of the invention provides for example with for the purpose of describing, and is not exhaustively or limit the invention to disclosed form.Many modifications and variations are obvious for the ordinary skill in the art.Selecting and describing embodiment is for better explanation principle of the present invention and practical application, thereby and makes those of ordinary skill in the art can understand the various embodiment with various modifications that the present invention's design is suitable for special-purpose.
Claims (10)
1. a network flow storage means, is characterized in that, comprises step:
S1. initialization Client table and Server show;
S2. search the network flow that newly enters packet correspondence in stream table capturing;
S3. upgrade Client table and Sever table.
2. the method for claim 1, is characterized in that, step S1 comprises:
S1.1 is that size of Client initialization is n
chash table;
S1.2 is that size of Sever initialization is n
shash table.
3. method as claimed in claim 1 or 2, is characterized in that:
For the initialized Hash table of Client is that a size is n
csequence list, a plurality of network flows that each list item is initiated for storing client.
4. method as claimed in claim 1 or 2, is characterized in that:
For the initialized Hash table of Server is that a size is n
ssequence list, a plurality of network flows that each list item receives for storing server.
5. the method for claim 1, is characterized in that, step S2 comprises:
S2.1 catches one and newly enters packet;
The hash value h1 of S2.2 calculated data bag forward five-tuple (< source address, destination address, source port, destination interface, agreement >);
S2.3 utilizes hash value h1 to search in stream table whether have corresponding network flow, if exist, the direction of mark current data packet is from client to server, and performs step S3, if do not exist, performs step S2.3;
The hash value h2 of the reverse five-tuple of S2.4 calculated data bag (< destination address, source address, destination interface, source port, agreement >);
S2.5 utilizes hash value h2 to search in stream table whether have corresponding network flow, if exist, the direction of mark current data packet is from server to client end, and performs step S3, if do not exist, performs step S2.5;
S2.6 utilizes hash value h1 for a network flow record of this packet establishment, and the direction of mark current data packet is first packet of the network flow from client to server, execution step S3.
6. the method for claim 1, is characterized in that, step S3 comprises:
The hash value h3 of S3.1 computing client end IP address and the hash value h4 of server ip address;
S3.2 judges whether packet is first packet of this network flow, if so, performs step S3.3, if not, perform step S3.4;
S3.3 increases new network flow information in Client table and Server table;
S3.4 usage data bag upgrades respective items in Client table and Server table;
S3.5 returns to execution step S2.
7. method as claimed in claim 6, is characterized in that, step S3.3 comprises:
S3.31 utilizes hash value h3 to search at Client table the information that whether has this client, if do not exist, for this client creates corresponding record in Client table;
S3.32 utilizes hash value h4 to search at Server table the information that whether has this server, if do not exist, for this server creates corresponding record in Server table.
8. the method as described in claim 6 or 7, is characterized in that:
When creating client records, if list item corresponding to h3 taken by other clients, use chained list that client-side information is mounted to after the list item that h3 is corresponding.
9. the method as described in claim 6 or 7, is characterized in that:
When creating server record, if list item corresponding to h4 taken by other servers, use chained list that server info is mounted to after the list item that h4 is corresponding.
10. method as claimed in claim 6, is characterized in that, step S3.4 comprises:
S3.41 judges whether packet is first packet of this network flow, if so, performs step S3.42, if not, perform step S3.43;
S3.42 increases network flow information in the respective items of Client table and Sever table;
S3.43 utilize that packet upgrades that Client table and Server show to the network flow information in corresponding.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210246855.3A CN103546307B (en) | 2012-07-16 | 2012-07-16 | Network flow storage method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210246855.3A CN103546307B (en) | 2012-07-16 | 2012-07-16 | Network flow storage method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103546307A true CN103546307A (en) | 2014-01-29 |
| CN103546307B CN103546307B (en) | 2016-12-21 |
Family
ID=49969383
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210246855.3A Active CN103546307B (en) | 2012-07-16 | 2012-07-16 | Network flow storage method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103546307B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107248939A (en) * | 2017-05-26 | 2017-10-13 | 中国人民解放军理工大学 | Network flow high-speed associative method based on hash memories |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101729413A (en) * | 2009-11-06 | 2010-06-09 | 清华大学 | Multi-service processing system and method based on ATCA |
| CN102468987A (en) * | 2010-11-08 | 2012-05-23 | 清华大学 | NetFlow characteristic vector extraction method |
-
2012
- 2012-07-16 CN CN201210246855.3A patent/CN103546307B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101729413A (en) * | 2009-11-06 | 2010-06-09 | 清华大学 | Multi-service processing system and method based on ATCA |
| CN102468987A (en) * | 2010-11-08 | 2012-05-23 | 清华大学 | NetFlow characteristic vector extraction method |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107248939A (en) * | 2017-05-26 | 2017-10-13 | 中国人民解放军理工大学 | Network flow high-speed associative method based on hash memories |
| CN107248939B (en) * | 2017-05-26 | 2020-07-31 | 中国人民解放军理工大学 | Network flow high-speed correlation method based on hash memory |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103546307B (en) | 2016-12-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103051725B (en) | Application and identification method, data digging method, Apparatus and system | |
| CN102724063B (en) | Log acquisition server and packet delivery, Log Clustering method and network | |
| CN108833166B (en) | Edge cloud message forwarding method and system and network message forwarding method and system | |
| CN102970242B (en) | Method for achieving load balancing | |
| Huang et al. | Software-defined QoS provisioning for fog computing advanced wireless sensor networks | |
| US20080209053A1 (en) | HTTP-Based Peer-to-Peer Framework | |
| CN102724317A (en) | Network data flow classification method and device | |
| CN102148854B (en) | Method and device for identifying peer-to-peer (P2P) shared flows | |
| WO2021017930A1 (en) | Message forwarding | |
| CN102468987B (en) | NetFlow characteristic vector extraction method | |
| CN110572274A (en) | named data network method for optimizing deployment and management of edge computing nodes | |
| CN103581044A (en) | Flow statistic method and device | |
| CN101184000A (en) | Packet sampling and application signature based internet application flux identifying method | |
| CN103024085A (en) | System and method for processing P2P (peer-to-peer) node request | |
| CN103001964A (en) | Cache acceleration method under local area network environment | |
| CN101599897A (en) | A kind of peer-to-peer network flow control methods that detects based on application layer | |
| CN103281211A (en) | Large-scale network node grouping management system and management method | |
| CN104994016A (en) | Method and apparatus for packet classification | |
| CN102571946A (en) | Realization method of protocol identification and control system based on P2P (peer-to-peer network) | |
| CN103746768B (en) | A kind of recognition methods of packet and equipment | |
| CN103457976A (en) | Data downloading method and system | |
| CN101668035B (en) | Method for recognizing various P2P-TV application video flows in real time | |
| Bashir et al. | Classifying P2P activity in Netflow records: A case study on BitTorrent | |
| CN103546307A (en) | Network flow storage method | |
| Li et al. | High performance flow feature extraction with multi-core processors |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |