CN102170373A - Method and device for monitoring network flow - Google Patents
Method and device for monitoring network flow Download PDFInfo
- Publication number
- CN102170373A CN102170373A CN2010101134698A CN201010113469A CN102170373A CN 102170373 A CN102170373 A CN 102170373A CN 2010101134698 A CN2010101134698 A CN 2010101134698A CN 201010113469 A CN201010113469 A CN 201010113469A CN 102170373 A CN102170373 A CN 102170373A
- Authority
- CN
- China
- Prior art keywords
- flow
- network address
- network
- untreated
- sequence
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Abstract
The present invention discloses a method and a device for monitoring network flow and the method and the device improve the time efficiency of statistical analysis of network flow original data. The technical scheme comprises that the method carries out ordering of top N websites having greatest flows in network. The method comprises carrying out ordering of random N websites in the network according to flows of the websites to form an initial sequence of a currently processed sequence; comparing the flow of a next unprocessed website with a website having the smallest flow in the currently processed sequence; and discarding the unprocessed website if the flow of the unprocessed website is smaller than the flow of the website having the smallest flow in the currently processed sequence, or inserting the unprocessed website into a proper position in the currently processed sequence according to the flow of the unprocessed website, and at the same time discarding the website having the smallest flow to update the currently processed sequence, and repeating the above step until all networks have been processed.
Description
Technical field
The present invention relates to IP network flow monitoring and analytical technology, relate in particular to the technology that the flow to each network address in the network sorts.
Background technology
Packet header sampling is one of main traffic monitoring technology of IP backbone, and this technology is a kind of workable, technology that cost performance is good.IETF has released the IPFIX standard at this application specific, but because the flow collection data volume is generally all very big, the ageing requirement that adds network flow monitoring is than higher, so the performance issue that adopts software to realize is more outstanding, the TOP N when being mainly reflected in the buffer memory of image data and fast access, initial data statistical analysis (that is: N maximum or minimum value) sorts these two aspects.
From the principle of various full sort algorithms as can be known, bubble sort, heapsort, Straight Selection Sort can obtain TOP N maximum (or minimum) value under the situation of not finishing whole orderings; But because the bubbling method is lower for the ordering efficient of a random number series, so improve TOP N ordering techniques at heapsort, Straight Selection Sort.In addition, though Straight Insertion Sort also need just can obtain TOP N maximum (minimum) value finishing under the situation of whole orderings,, can meet the demands equally if its algorithm is once improved.
In the IP network flow monitoring, need to understand the traffic statistics situation of internet, applications, purpose website, source IP address, procotol or the like each dimension usually.For instance, need the flow distribution of understanding in network from which IP address (source IP address) usually, which IP address of whereabouts (purpose IP address), the flow distribution situation and the ranking of each source IP address or purpose IP address.But the quantity maximum of IP address can reach 2
32Individual, so flow is carried out in all IP addresses, and to carry out rank be unpractical, and in most of the cases, only need to understand a collection of IP address to the web influence maximum, it is the earlier IP address of network traffics rank, usually said TOP N rank that Here it is, N value usually between 10～100, the big again or littler practical application meaning that all lost.
In computerized algorithm, traditional all kinds of sort algorithms also can satisfy this demand of this network flow statistic in theory, but exist very big performance deficiency in actual use, and main cause is that traditional sort algorithm is full sort algorithm.In the traffic monitoring of IP address, traditional full sort algorithm means and will the flow that the extremely huge IP address of quantity in the network produces be sorted one by one, and what need in fact to pay close attention to only is the IP address of N before the wherein very little flow rank, but for the mass data after the N position sort in fact at all need not, the result who brings expends a large amount of quality time in vain.
Summary of the invention
The objective of the invention is to address the above problem, a kind of network traffics monitoring method is provided, improved the efficient of network traffics initial data statistical analysis on ageing.
Another object of the present invention is to provide a kind of network traffics monitoring device.
Technical scheme of the present invention is: the present invention has disclosed a kind of network traffics monitoring method, and the network address of top n flow maximum in the network is sorted, and wherein N is a natural number, and this monitoring method comprises:
The size of any N network address in the network according to flow sorted, constitute initiation sequence when the pre-treatment sequence;
With the untreated network address of the next one and in the pre-treatment sequence network address of flow minimum carry out the comparison of flow size;
If the flow of this untreated network address is less than the flow that deserves the network address of flow minimum in the pre-treatment sequence, then abandon this untreated network address, otherwise, this untreated network address is inserted into the correct position that deserves in the pre-treatment sequence according to its flow size, the network address that abandons this flow minimum is simultaneously deserved the pre-treatment sequence to upgrade, and returns previous step and all handles until all network address suddenly.
According to an embodiment of network traffics monitoring method of the present invention, the span of N is between 10～100.
According to an embodiment of network traffics monitoring method of the present invention, this any N network address is N the network address that receives at first in the network flow monitoring process.
According to an embodiment of network traffics monitoring method of the present invention, it is to adopt binary chop that untreated network address is inserted into the correct position that deserves in the pre-treatment sequence according to its flow size.
The present invention has also disclosed a kind of network traffics monitoring device, and the network address of top n flow maximum in the network is sorted, and wherein N is a natural number, and this monitoring device comprises:
The flow monitoring module, the flow of each network address in the monitoring network;
Order module connects this flow monitoring module, and the size of any N network address in the network according to flow sorted, and constitutes the initiation sequence when the pre-treatment sequence;
Comparison module connects this order module and this flow monitoring module, will carry out the comparison of flow size from the next untreated network address of this flow monitoring module output with from the network address of flow minimum when the pre-treatment sequence of this order module;
The sequence update module, its input connects this comparison module, output connects this order module, if the flow of this untreated network address is less than the flow that deserves the network address of flow minimum in the pre-treatment sequence, then abandon this untreated network address, otherwise, this untreated network address is inserted into the correct position that deserves in the pre-treatment sequence according to its flow size, the network address that abandons this flow minimum is simultaneously deserved the pre-treatment sequence to upgrade;
Wherein this comparison module and this sequence update module are that all network address in network of moving are repeatedly all handled.
According to an embodiment of network traffics monitoring device of the present invention, the span of N is between 10～100.
According to an embodiment of network traffics monitoring device of the present invention, this any N network address is N the network address that receives at first in the network flow monitoring process.
According to an embodiment of network traffics monitoring device of the present invention, in this sequence update module untreated network address being inserted into the correct position that deserves in the pre-treatment sequence according to its flow size is to adopt binary chop.
The present invention contrasts prior art following beneficial effect: the present invention is by sorting any N network address in the network earlier according to its flow size, again with the untreated network address of the next one and before this in N network address the network address of flow minimum carry out flow-rate ratio, if untreated network address then directly abandons untreated network address less than the network address flow of minimum in N the network address, otherwise the network address that this network address is inserted into suitable position and abandons the flow minimum is upgraded this sequence.With traditional sort method the network address flow is carried out the technological means of statistical monitoring in the contrast prior art, the present invention can save more time and resource.
Description of drawings
Fig. 1 is the flow chart of the embodiment of network traffics monitoring method of the present invention.
Fig. 2 is a sequence schematic diagram in the network traffics monitoring method example of the present invention.
Fig. 3 is the schematic diagram that in the network traffics monitoring method example of the present invention sequence is upgraded.
Fig. 4 is the schematic diagram of the embodiment of network traffics monitoring device of the present invention.
Embodiment
The invention will be further described below in conjunction with drawings and Examples.
The embodiment of network traffics monitoring method
Fig. 1 shows the embodiment of network traffics monitoring method of the present invention.Seeing also Fig. 1, is the detailed description to each step of the network traffics monitoring method of present embodiment below.
Step S10: the flow that from network, obtains N network address.
The N here is a natural number, and size can be decided as required, is located between 10～100 usually.Here following the example of of N network address can be arbitrarily, normally the N that receives at first in a network flow monitoring process network address.For convenience, N is set at 10, this 1 network address is shown D1, D2 respectively ... D10.
Step S11: this N network address is sorted according to the flow size, constitute initiation sequence when the pre-treatment sequence.
Can be ordering from small to large, also can be ordering from big to small.The sort method here can be any one known sort method.What constitute this moment works as the pre-treatment sequence as shown in Figure 2, and as can be seen from Figure 2, this is that a flow value is according to the chained list of arranging from big to small.This chained list shown in Figure 2 can be regarded the initial value when the pre-treatment sequence as.
Step S12: obtain next untreated network address.May be defined as Dx in the present embodiment for convenience of description.
Step S13: the network address of this untreated network address and the flow minimum in the pre-treatment sequence is carried out the comparison of flow size.If untreated network address flow then enters step S14 less than the network address of flow minimum, otherwise enter step S15.
From the above, be exactly the size that compares Dx and D7.If Dx then enters step S14 less than D7, otherwise enter step S15.
Step S14: directly abandon untreated network address.Return step S12 then
Step S15: untreated network address is inserted into correct position in the pre-treatment sequence according to the flow size.
This insertion method can adopt any one known insertion method, for example can adopt binary chop algorithm efficiently.
For example, the network address that needs to insert is D11, supposes D11 less than D4 and greater than D8, then D11 is inserted between D4 and the D8, at this time works as the pre-treatment sequence and has become 11, owing to only need 10, can eliminate the network address D7 of flow minimum, as shown in Figure 3.
Step S16: the network address that abandons the flow minimum in the pre-treatment sequence.Return step S12 then.
Above-mentioned step S15 and S16 are exactly in order to upgrade when the pre-treatment sequence.A loop structure is arranged in the method for present embodiment, and the condition of jumping out circulation is that all network address that need handle are all finished dealing with.
The embodiment of network traffics monitoring device
Fig. 4 shows the embodiment of network traffics monitoring device of the present invention.See also Fig. 4, the network traffics monitoring device of present embodiment comprises flow monitoring module 10, order module 20, comparison module 30 and sequence update module 40.Annexation between them is: the output of flow monitoring module 10 connects order module 20 and comparison module 30 respectively.The output of order module 20 connects comparison module 30.The output catenation sequence update module 40 of comparison module 30.The output of sequence update module 40 connects order module 20.
The flow of each network address in flow monitoring module 10 monitoring networks.Order module 20 is obtained the flow of N network address from flow monitoring module 10, and this N network address is sorted according to the flow size, constitutes the initial value when the pre-treatment sequence.The N here is a natural number, and size can be decided as required, is located between 10～100 usually.Here following the example of of N network address can be arbitrarily, normally the N that receives at first in a network flow monitoring process network address.
In sequence update module 40,, then abandon this untreated network address if comparative result is the flow of untreated network address flow less than the network address of flow minimum in the pre-treatment sequence.Otherwise, this untreated network address is inserted into the correct position in the pre-treatment sequence that order module 20 is finished according to its flow size, abandon the network address of flow minimum in the pre-treatment sequence simultaneously, thereby finish the renewal of working as the pre-treatment sequence in the order module 20.In the process of inserting untreated network address, can be any one known insertion algorithm, can adopt binary chop algorithm more efficiently.
In actual applications, traffic statistics monitoring method of the present invention is starkly lower than traditional dependence Straight Insertion Sort and the traffic statistics finished monitoring on time and resource consumption.
The foregoing description provides to those of ordinary skills and realizes or use of the present invention; those of ordinary skills can be under the situation that does not break away from invention thought of the present invention; the foregoing description is made various modifications or variation; thereby protection scope of the present invention do not limit by the foregoing description, and should be the maximum magnitude that meets the inventive features that claims mention.
Claims (8)
1. network traffics monitoring method sorts to the network address of top n flow maximum in the network, and wherein N is a natural number, and this monitoring method comprises:
The size of any N network address in the network according to flow sorted, constitute initiation sequence when the pre-treatment sequence;
With the untreated network address of the next one and in the pre-treatment sequence network address of flow minimum carry out the comparison of flow size;
If the flow of this untreated network address is less than the flow that deserves the network address of flow minimum in the pre-treatment sequence, then abandon this untreated network address, otherwise, this untreated network address is inserted into the correct position that deserves in the pre-treatment sequence according to its flow size, the network address that abandons this flow minimum is simultaneously deserved the pre-treatment sequence to upgrade, and returns previous step and all handles until all network address suddenly.
2. network traffics monitoring method according to claim 1 is characterized in that the span of N is between 10～100.
3. network traffics monitoring method according to claim 1 is characterized in that, this any N network address is N the network address that receives at first in the network flow monitoring process.
4. network traffics monitoring method according to claim 1 is characterized in that, it is to adopt binary chop that untreated network address is inserted into the correct position that deserves in the pre-treatment sequence according to its flow size.
5. network traffics monitoring device sorts to the network address of top n flow maximum in the network, and wherein N is a natural number, and this monitoring device comprises:
The flow monitoring module, the flow of each network address in the monitoring network;
Order module connects this flow monitoring module, and the size of any N network address in the network according to flow sorted, and constitutes the initiation sequence when the pre-treatment sequence;
Comparison module connects this order module and this flow monitoring module, will carry out the comparison of flow size from the next untreated network address of this flow monitoring module output with from the network address of flow minimum when the pre-treatment sequence of this order module;
The sequence update module, its input connects this comparison module, output connects this order module, if the flow of this untreated network address is less than the flow that deserves the network address of flow minimum in the pre-treatment sequence, then abandon this untreated network address, otherwise, this untreated network address is inserted into the correct position that deserves in the pre-treatment sequence according to its flow size, the network address that abandons this flow minimum is simultaneously deserved the pre-treatment sequence to upgrade;
Wherein this comparison module and this sequence update module are that all network address in network of moving are repeatedly all handled.
6. network traffics monitoring device according to claim 5 is characterized in that the span of N is between 10～100.
7. network traffics monitoring device according to claim 5 is characterized in that, this any N network address is N the network address that receives at first in the network flow monitoring process.
8. network traffics monitoring device according to claim 5 is characterized in that, in this sequence update module untreated network address being inserted into the correct position that deserves in the pre-treatment sequence according to its flow size is to adopt binary chop.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010101134698A CN102170373A (en) | 2010-02-25 | 2010-02-25 | Method and device for monitoring network flow |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010101134698A CN102170373A (en) | 2010-02-25 | 2010-02-25 | Method and device for monitoring network flow |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN102170373A true CN102170373A (en) | 2011-08-31 |
Family
ID=44491357
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2010101134698A Pending CN102170373A (en) | 2010-02-25 | 2010-02-25 | Method and device for monitoring network flow |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102170373A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104123311A (en) * | 2013-04-28 | 2014-10-29 | 腾讯科技（深圳）有限公司 | Data traffic reminding method and device |
| CN105187279A (en) * | 2015-09-28 | 2015-12-23 | 广东睿江科技有限公司 | Traffic statistical and real-time ranking method |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101505244A (en) * | 2009-03-27 | 2009-08-12 | 北京星网锐捷网络技术有限公司 | Bandwidth measurement method and apparatus |
| CN101621454A (en) * | 2008-06-30 | 2010-01-06 | 华为技术有限公司 | Passive optical network system, optical line terminal and optical network units |
-
2010
- 2010-02-25 CN CN2010101134698A patent/CN102170373A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101621454A (en) * | 2008-06-30 | 2010-01-06 | 华为技术有限公司 | Passive optical network system, optical line terminal and optical network units |
| CN101505244A (en) * | 2009-03-27 | 2009-08-12 | 北京星网锐捷网络技术有限公司 | Bandwidth measurement method and apparatus |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104123311A (en) * | 2013-04-28 | 2014-10-29 | 腾讯科技（深圳）有限公司 | Data traffic reminding method and device |
| CN104123311B (en) * | 2013-04-28 | 2019-01-29 | 腾讯科技（深圳）有限公司 | A kind of data traffic reminding method and device |
| CN105187279A (en) * | 2015-09-28 | 2015-12-23 | 广东睿江科技有限公司 | Traffic statistical and real-time ranking method |
| CN105187279B (en) * | 2015-09-28 | 2019-01-15 | 广东睿江云计算股份有限公司 | A kind of method of traffic statistics and real-time ranking |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20080112415A1 (en) | Network Processors and Pipeline Optimization Methods | |
| CN109803246B (en) | Random access and data transmission method based on grouping in large-scale MTC network | |
| CN103477601A (en) | Method and apparatus for network-friendly collaborative caching | |
| CN1910868A (en) | Method and device for controlling alignment buffer | |
| CN102314336A (en) | Data processing method and system | |
| CN111159002A (en) | Data edge acquisition method based on grouping, edge acquisition equipment and system | |
| CN102170373A (en) | Method and device for monitoring network flow | |
| CN104243348A (en) | Data processing method and device | |
| Nallusamy et al. | Decision Tree‐Based Entries Reduction scheme using multi‐match attributes to prevent flow table overflow in SDN environment | |
| CN102037689A (en) | Adaptive scheduler for communication systems apparatus, system and method | |
| CN103166912B (en) | The transmission method of a kind of packet, system | |
| CN104125146B (en) | A kind of method for processing business and device | |
| CN102316173A (en) | Method and device for aggregating network address | |
| CN107911484B (en) | Message processing method and device | |
| CN112612941B (en) | Financial security public opinion information crawling method and device | |
| CN112019589B (en) | Multi-level load balancing data packet processing method | |
| Benacer et al. | HPQ: A high capacity hybrid priority queue architecture for high-speed network switches | |
| CN102315979A (en) | Method and device for monitoring network flow | |
| CN105516994A (en) | Bandwidth distribution method and device | |
| Vanderbauwhede et al. | FPGA-accelerated Information Retrieval: High-efficiency document filtering | |
| CN113590322A (en) | Data processing method and device | |
| Elsayed et al. | On the Impact of Network Delays on Time-to-Live Caching | |
| CN103324640A (en) | Method and device for determining search result file, as well as equipment | |
| CN106776794B (en) | Mass data processing method and system | |
| CN106412899B (en) | A kind of network request method for saving mobile terminal flow |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C53 | Correction of patent of invention or patent application | ||
| CB03 | Change of inventor or designer information |
Inventor after: Zheng Wenyun Inventor before: Zhou Ye Inventor before: Yu Gang Inventor before: Wang Hua Inventor before: Fan Jing Inventor before: Zhan Hai Inventor before: Zheng Wenyun |
|
| C12 | Rejection of a patent application after its publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20110831 |