CN101626323A - Method and device for monitoring network data flow - Google Patents
Method and device for monitoring network data flow Download PDFInfo
- Publication number
- CN101626323A CN101626323A CN200910109034A CN200910109034A CN101626323A CN 101626323 A CN101626323 A CN 101626323A CN 200910109034 A CN200910109034 A CN 200910109034A CN 200910109034 A CN200910109034 A CN 200910109034A CN 101626323 A CN101626323 A CN 101626323A
- Authority
- CN
- China
- Prior art keywords
- message
- characteristic information
- record
- burin
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a method and a device for monitoring network data flow. Messages are sorted, extracted and stored through user settings, statistics can be flexibly carried out on the messages with different types, burden of a system is reduced, and network status is flexibly reflected. When the invention is adopted, characteristic information parameters concerned by users can be flexibly set, only set characteristic information is compared in statistical monitoring, statistics is carried out on different characteristic information with different records, and count incrementing is carried out on the same characteristic information. The statistics monitoring mechanism which can be configured can greatly simplify the workload of a network manager, and reduces the memory consumption of the system.
Description
Technical field
The present invention relates to a kind of technology of network data monitoring statistics.
Background technology
Fast-developing IP network technology impels a large amount of emerging commerce to use, serve and emerge in large numbers.Service that these are emerging and demand impel users that the network bandwidth, performance, service quality, fail safe are had higher requirement.The convergence-level switch need be done forwarding to a large amount of user's messages, reasonably distributes bandwidth, and it is particularly crucial that monitoring network stream seems.Classification by data traffic, analyze effectively checkout equipment state and user profile, according to these statistical informations, webmaster can be used for planning network and application resource and satisfy client's demand.Simultaneously, User Status be can detect, employed network of detailed client and application resource got access to; Realization is to the Real time identification and the classification of data flow; Prevent that DOS (Denial of Service denial of service) from attacking, virus and network worm etc.These data help to understand and the feedback security incident.
The communication of different kinds of business may be one group of packet that any station terminal equipment sends to another station terminal equipment in the network, and this group packet has in fact just constituted a data flow of certain business in the carrier network.If management system can be distinguished all streams that the whole network transmits, the size of the delivery time of each stream of accurate recording, the switch ports themselves that takies, transfer source/destination address and data flow, just can analyze and add up, and then calculate the baseline of proper communication and the exceptional communication flow of finding burst the flow and the flow direction of operator's all communications of the whole network.
Existing data-flow analysis adopts at random the mode of message up sending to come the phase-split network data flow mostly.So but exist this statistical of a large amount of messages to be difficult to the reflection actual network conditions on the network, the mode of this accounting message does not possess statistical property, is difficult to draw the loss that more complete analysis result can cause the Device memory resource simultaneously.What network engineers more paid close attention to is the message information that switch is transmitted in a period of time.Also lack a kind of technology that can carry out network data monitoring according to prior flexible configuration in the prior art,, also do not find the patent application of this respect through patent retrieval.
Summary of the invention
The technical problem that the present invention solves is the method and apparatus that has proposed a kind of network traffic data monitoring, has realized configurating filtered user of falling and unconcerned irrelevant message information according to the user.
The technical solution used in the present invention is:
A kind of network data flow quantity monitoring method comprises following processing procedure: the data message that receives is classified, determine the type of message; If data message is two, three-tier message, then message is carried out characteristic information and resolve; The characteristic information that parses is searched in database as the distinguishing identifier of data message class, if there is statistic record history in this data message class, then from the count value that increases this data message class; Otherwise, in database, produce a new record that reflects the counting situation of this data message class; Burin-in process is carried out in statistic record in the database.
Further, the characteristic information parsing to message also comprises following processing procedure: DSCP (differentiated services code points) value to three-tier message is shone upon processing, and the value after will shining upon is as a characteristic information of three-tier message.
Further, before being classified, message also comprises following processing procedure described: judge that the message that receives still wants dropping packets for the normal message of transmitting; Carry out discard processing for the message that will abandon; For the then requirement that is provided with of reading system of the message of normal forwarding, the message that need add up is set then to be changeed subsequent step and handles, otherwise directly E-Packets.
Preferably, the characteristic information of described two layer message comprises at least a following information: go out/inbound port, source MAC (media interviews control) address, target MAC (Media Access Control) address, VLAN (Virtual Local AreaNetwork, VLAN) ID, VLAN priority, ethernet type, forwarding state; The characteristic information of described three-tier message comprises at least a following information: go out/inbound port, source IP address, purpose IP address, source port number, destination slogan, three layer protocol types, message DSCP value, forwarding state.
Preferably, described burin-in process process specifically comprises: the record for never upgrading in the maximum idle time inside counting value that is provided with, worn out this record; Reach the record of preset threshold for count value, recorded information is sent to NM server.
A kind of network traffic data monitoring device comprises: sort module, be used for the data message that receives is classified, and determine the type of message; Parsing module is used for two, three-tier message carries out characteristic information and resolve, and obtains the characteristic information of message; Record processing module is used for searching at database as the distinguishing identifier of data message class according to the characteristic information that parses, for there being statistic record history, then from the count value that increases this data message class; Otherwise, in database, produce a new record that reflects the counting situation of this data message class; The burin-in process module is used for burin-in process is carried out in the statistic record of database.
Further, described parsing module also is used for the DSCP value of three-tier message is shone upon processing, and the value after will shining upon is as a characteristic information of three-tier message.
Preferably, the characteristic information of described two layer message comprises at least a following information: go out/inbound port, source MAC, target MAC (Media Access Control) address, VLAN ID, VLAN priority, ethernet type, forwarding state; The characteristic information of described three-tier message comprises at least a following information: go out/inbound port, source IP address, purpose IP address, source port number, destination slogan, three layer protocol types, message DSCP value, forwarding state.
Preferably, described burin-in process module specifically is used for following two classes record is carried out different burin-in process: for the record that never upgrades in the maximum idle time inside counting value that is provided with, described burin-in process module is worn out this record; Reach the record of preset threshold for count value, described burin-in process module sends to NM server with recorded information.
A kind of network data flow quantity monitoring method and the device that adopt the present invention to propose, the characteristic information parameter that the user is concerned about can be set flexibly, when statistical monitoring, only compare at the characteristic information value that is provided with, add up with different records for different characteristic information values.This configurable statistical monitoring mechanism that the present invention proposes can be simplified the workload of webmaster greatly, reduces the memory consumption of system.
Description of drawings
Fig. 1 is a network data flow quantity monitoring method process principle figure of the present invention;
Fig. 2 is a network traffic data monitoring device schematic diagram of the present invention;
Fig. 3 is a network data flow quantity monitoring method embodiment flow chart of the present invention.
Embodiment
Below in conjunction with accompanying drawing, describe the specific implementation of technical solution of the present invention in detail.
Be illustrated in figure 1 as network data flow quantity monitoring method flow process of the present invention, comprise following processing procedure:
1) packet for the treatment of monitoring and statistics carries out the message classification processing, determines the type of message.
2) be two layer message or three-tier message for determining message, message carried out characteristic information resolve, extract the characteristic information of message.
The characteristic information of described two layer message comprises at least a following information: go out/inbound port, source media access control address, purpose media access control address, VLAN ID, VLAN priority, ethernet type, forwarding state.
The characteristic information of described three-tier message comprises at least a following information: go out/inbound port, source IP address, purpose IP address, source port number, destination slogan, three layer protocol types, message differentiated services code points value, forwarding state
3) characteristic information is retrieved as the record of index information in database, and the record of new database more; The record of database upgrades and comprises: the count value that increases new record and record is from increasing.
4) record in the database is carried out burin-in process, aging record is sent to server or deleted.
Network traffic data monitoring device principle of the present invention as shown in Figure 2 comprises three basic handling modules: sort module, parsing module and record processing module in the device.Wherein sort module is used for message information is classified, and determines the type of message, and for example: type of message is two layer message or three-tier message; Parsing module is used for two, three-tier message carries out characteristic information and resolve, and obtains the characteristic information of message; Record processing module is used for searching at database as the distinguishing identifier of data message class according to the characteristic information that parses, and there is statistic record history in this data message class, and then the count value of this data message class is carried out from increasing; Otherwise, in database, produce a new record to write down the counting situation of this data message class.
In the device burin-in process module can also be set in addition, be used for burin-in process is carried out in the statistic record of database, its burin-in process process can be record for never upgrading in the maximum idle time inside counting value that is provided with, and this record is worn out; Reach the record of preset threshold for count value, recorded information is sent to NM server.
Be illustrated in figure 3 as network data flow quantity monitoring method embodiment flow chart of the present invention, processing procedure is as follows:
1) port of log-on data detection, and the reception message, the message that receives is done classification for the first time, judgement is normal message of transmitting or the message that will abandon, carry out discard processing for the message that will abandon, otherwise the inquiry system configuration determines whether this message needs to do statistical disposition;
2) data message of receiving and need adding up is carried out the classification second time, confirm its type of message; If determining type of message is two layer message or three-tier message, according to type of message two layers or three layers of characteristic information are extracted then;
3) if three-tier message can also detect user's configuration, judge whether to do the mapping of DSCP value; That has disposed the mapping option just is mapped as user configured mapping value to the DSCP value of original message, but does not change original message DSCP value;
4) according to user configuration information, the message characteristic information that step 1)-step 3) is extracted is done filtration, keep the characteristic information that the user was concerned about;
5) according to the message characteristic information of handling through step 4), searching database, judge whether to exist the record of this class message,, do not exist with regard to generating a new record and come this class message of statistical counting if exist then upgrade its count value (promptly count value being carried out from increasing);
6) when reaching the maximum idle time inside counting that the user is provided with, the record of certain message never is updated, when perhaps count value reaches maximum, this record will wear out so, then, sends to NM server according to user configuration information with record deletion or record.
According to above-mentioned handling process, lift the recording process of example contract quotation literary composition in database of two monitoring and statisticses below again.
One, monitoring and statistics two layer message
Suppose that NM server need monitor the situation of source MAC in the two layer message, target MAC (Media Access Control) address, VLAN ID, switch is analyzed for the value in the source MAC of transmitting each two layer message that comes, target MAC (Media Access Control) address, VLAN ID territory, as long as need do have in the characteristic information project of message of statistics one different, will in database, generate different records so and do statistics.
Two, monitoring and statistics OSPF message
The protocol fields of message is used for identifying the protocol type that message carries, and wherein 1 is that ICMP (Internet control message protocol) message, 6 is that TCP (transmission control protocol) message, 17 is that UDP (User Datagram Protoco (UDP)) message, 89 is OSPF (Open Shortest Path First ospf) message.Therefore for the situation of statistics OSPF message, it is 89 three-tier message that switch is just added up the IP protocol fields.Generating a record when promptly for the first time receiving the OSPF message in database, to be used to add up the IP protocol fields be 89 three-tier message, and the OSPF message of receiving afterwards then carries out from increasing the count value of this record.Equally, in some practical application, the client more needs to add up the protocol massages information of some route, and switch is just finished statistics according to the key value information of these messages so.
This configurable statistical information that the present invention adopts has been simplified the workload of NM server greatly, has reduced the memory consumption of system.The present invention is provided with by the user message is classified, extracts, stored, and can add up inhomogeneous message flexibly, has reduced the burden of system, has reacted network conditions flexibly.
Above description only is preferred embodiment of the present invention, and in order to restriction the present invention, within the spirit and principles in the present invention all, any modification of being done is not equal to replacement, improves etc., all should be included within protection scope of the present invention.
Claims (9)
1. a network data flow quantity monitoring method is characterized in that, described method comprises following processing procedure:
The data message that receives is classified, determine the type of message;
If data message is two, three-tier message, then message is carried out characteristic information and resolve;
The characteristic information that parses is searched in database as the distinguishing identifier of data message class, if there is statistic record history in this data message class, then from the count value that increases this data message class; Otherwise, in database, produce a new record that reflects the counting situation of this data message class;
Burin-in process is carried out in statistic record in the database.
2. network data flow quantity monitoring method according to claim 1, it is characterized in that, characteristic information parsing to message also comprises: the differentiated services code points value to three-tier message is shone upon processing, and the value after will shining upon is as a characteristic information of three-tier message.
3. network data flow quantity monitoring method according to claim 1 and 2, it is characterized in that the characteristic information of described two layer message comprises at least a following information: go out/inbound port, source media access control address, purpose media access control address, VLAN ID, VLAN priority, ethernet type, forwarding state; The characteristic information of described three-tier message comprises at least a following information: go out/inbound port, source IP address, purpose IP address, source port number, destination slogan, three layer protocol types, message differentiated services code points value, forwarding state.
4. network data flow quantity monitoring method according to claim 1 and 2 is characterized in that, described burin-in process process specifically comprises:
Record for never upgrading in the maximum idle time inside counting value that is provided with is worn out this record;
Reach the record of preset threshold for count value, recorded information is sent to NM server.
5. network data flow quantity monitoring method according to claim 1 and 2 is characterized in that, also comprises following processing procedure before message is classified described:
Judge that the message that receives is that the normal message of transmitting is still wanted dropping packets;
Carry out discard processing for the message that will abandon; For the then requirement that is provided with of reading system of the message of normal forwarding, the message that need add up is set then to be changeed subsequent step and handles, otherwise directly E-Packets.
6. a network traffic data monitoring device is characterized in that, described device comprises:
Sort module is used for the data message that receives is classified, and determines the type of message;
Parsing module is used for two, three-tier message carries out characteristic information and resolve, and obtains the characteristic information of message;
Record processing module is used for searching at database as the distinguishing identifier of data message class according to the characteristic information that parses, for there being statistic record history, then from the count value that increases this data message class; Otherwise, in database, produce a new record that reflects the counting situation of this data message class;
The burin-in process module is used for burin-in process is carried out in the statistic record of database.
7. network traffic data monitoring device according to claim 6 is characterized in that, described parsing module also is used for the differentiated services code points value of three-tier message is shone upon processing, and the value after will shining upon is as a characteristic information of three-tier message.
8. according to claim 6 or 7 described network traffic data monitoring devices, it is characterized in that the characteristic information of described two layer message comprises at least a following information: go out/inbound port, source media access control address, purpose media access control address, VLAN ID, VLAN priority, ethernet type, forwarding state; The characteristic information of described three-tier message comprises at least a following information: go out/inbound port, source IP address, purpose IP address, source port number, destination slogan, three layer protocol types, message differentiated services code points value, forwarding state.
9. according to claim 6 or 7 described network traffic data monitoring devices, it is characterized in that described burin-in process module specifically is used for following two classes record is carried out different burin-in process:
For the record that never upgrades in the maximum idle time inside counting value that is provided with, described burin-in process module is worn out this record;
Reach the record of preset threshold for count value, described burin-in process module sends to NM server with recorded information.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN200910109034A CN101626323A (en) | 2009-07-23 | 2009-07-23 | Method and device for monitoring network data flow |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN200910109034A CN101626323A (en) | 2009-07-23 | 2009-07-23 | Method and device for monitoring network data flow |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN101626323A true CN101626323A (en) | 2010-01-13 |
Family
ID=41522019
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN200910109034A Pending CN101626323A (en) | 2009-07-23 | 2009-07-23 | Method and device for monitoring network data flow |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101626323A (en) |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101982981A (en) * | 2010-11-12 | 2011-03-02 | 福州大学 | Classified detection device for digital television transport streams |
| CN102135932A (en) * | 2011-03-08 | 2011-07-27 | 浪潮(北京)电子信息产业有限公司 | Monitoring system and monitoring method thereof |
| CN102143044A (en) * | 2010-08-12 | 2011-08-03 | 华为技术有限公司 | System and method for processing access network |
| CN102231747A (en) * | 2011-07-18 | 2011-11-02 | 杭州华三通信技术有限公司 | Method and equipment for obtaining attack message |
| CN103580959A (en) * | 2013-11-15 | 2014-02-12 | 大连梯耐德网络技术有限公司 | Distributed statistical reporting implementation method |
| WO2014201906A1 (en) * | 2013-06-20 | 2014-12-24 | 中兴通讯股份有限公司 | Traffic statistics collection method and apparatus |
| CN105516100A (en) * | 2015-12-01 | 2016-04-20 | 北京浩瀚深度信息技术股份有限公司 | Message parsing method and system based on BS (Browser/Server) structure |
| WO2017211304A1 (en) * | 2016-06-07 | 2017-12-14 | 中兴通讯股份有限公司 | Software defined network-based counting method, device and system |
| CN107888456A (en) * | 2017-12-04 | 2018-04-06 | 北京百度网讯科技有限公司 | Method and device for monitoring port data volume |
| CN110380938A (en) * | 2019-08-07 | 2019-10-25 | 重庆金美通信有限责任公司 | A kind of multidimensional converged network monitoring method based on programmable model |
| CN111030931A (en) * | 2019-12-17 | 2020-04-17 | 苏州浪潮智能科技有限公司 | Method and equipment for forwarding priority label across network segments |
| CN111565125A (en) * | 2020-07-15 | 2020-08-21 | 成都数维通信技术有限公司 | Method for acquiring message passing through network traffic path |
| CN113114502A (en) * | 2020-05-11 | 2021-07-13 | 苏州乐米凡电气科技有限公司 | System for distributing user information ratio based on service quality |
| CN113132259A (en) * | 2019-12-31 | 2021-07-16 | 北京金山云网络技术有限公司 | Traffic data packet statistical method, device, equipment and storage medium |
-
2009
- 2009-07-23 CN CN200910109034A patent/CN101626323A/en active Pending
Cited By (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102143044A (en) * | 2010-08-12 | 2011-08-03 | 华为技术有限公司 | System and method for processing access network |
| CN102143044B (en) * | 2010-08-12 | 2013-10-02 | 华为技术有限公司 | System and method for processing access network |
| CN101982981A (en) * | 2010-11-12 | 2011-03-02 | 福州大学 | Classified detection device for digital television transport streams |
| CN102135932A (en) * | 2011-03-08 | 2011-07-27 | 浪潮(北京)电子信息产业有限公司 | Monitoring system and monitoring method thereof |
| CN102231747A (en) * | 2011-07-18 | 2011-11-02 | 杭州华三通信技术有限公司 | Method and equipment for obtaining attack message |
| US9887892B2 (en) | 2013-06-20 | 2018-02-06 | Xi'an Zhongxing New Software Co. Ltd. | Traffic statistics collection method and device |
| WO2014201906A1 (en) * | 2013-06-20 | 2014-12-24 | 中兴通讯股份有限公司 | Traffic statistics collection method and apparatus |
| CN103580959A (en) * | 2013-11-15 | 2014-02-12 | 大连梯耐德网络技术有限公司 | Distributed statistical reporting implementation method |
| CN103580959B (en) * | 2013-11-15 | 2017-04-26 | 大连梯耐德网络技术有限公司 | Distributed statistical reporting implementation method |
| CN105516100A (en) * | 2015-12-01 | 2016-04-20 | 北京浩瀚深度信息技术股份有限公司 | Message parsing method and system based on BS (Browser/Server) structure |
| CN105516100B (en) * | 2015-12-01 | 2018-06-05 | 北京浩瀚深度信息技术股份有限公司 | Message parsing method and system based on BS structures |
| WO2017211304A1 (en) * | 2016-06-07 | 2017-12-14 | 中兴通讯股份有限公司 | Software defined network-based counting method, device and system |
| CN107888456A (en) * | 2017-12-04 | 2018-04-06 | 北京百度网讯科技有限公司 | Method and device for monitoring port data volume |
| CN107888456B (en) * | 2017-12-04 | 2019-05-10 | 北京百度网讯科技有限公司 | Method and device for monitoring port data volume |
| CN110380938A (en) * | 2019-08-07 | 2019-10-25 | 重庆金美通信有限责任公司 | A kind of multidimensional converged network monitoring method based on programmable model |
| CN111030931A (en) * | 2019-12-17 | 2020-04-17 | 苏州浪潮智能科技有限公司 | Method and equipment for forwarding priority label across network segments |
| CN113132259A (en) * | 2019-12-31 | 2021-07-16 | 北京金山云网络技术有限公司 | Traffic data packet statistical method, device, equipment and storage medium |
| CN113132259B (en) * | 2019-12-31 | 2022-07-05 | 北京金山云网络技术有限公司 | Traffic data packet statistical method, device, equipment and storage medium |
| CN113114502A (en) * | 2020-05-11 | 2021-07-13 | 苏州乐米凡电气科技有限公司 | System for distributing user information ratio based on service quality |
| CN111565125A (en) * | 2020-07-15 | 2020-08-21 | 成都数维通信技术有限公司 | Method for acquiring message passing through network traffic path |
| CN111565125B (en) * | 2020-07-15 | 2020-10-09 | 成都数维通信技术有限公司 | Method for acquiring message passing through network traffic path |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101626323A (en) | Method and device for monitoring network data flow | |
| US9860154B2 (en) | Streaming method and system for processing network metadata | |
| EP1764951B1 (en) | Statistical trace-based method, apparatus, node and system for real-time traffic classification | |
| CN103314557B (en) | Network system, controller, switch, and traffic monitoring method | |
| EP3151470B1 (en) | Analytics for a distributed network | |
| CN104115463B (en) | For processing the streaming method and system of network metadata | |
| Phaal et al. | InMon corporation's sFlow: A method for monitoring traffic in switched and routed networks | |
| JP4341413B2 (en) | PACKET TRANSFER APPARATUS HAVING STATISTICS COLLECTION APPARATUS AND STATISTICS COLLECTION METHOD | |
| US7623466B2 (en) | Symmetric connection detection | |
| US8284665B1 (en) | Flow-based rate limiting | |
| US11546266B2 (en) | Correlating discarded network traffic with network policy events through augmented flow | |
| EP2482497B1 (en) | Data forwarding method, data processing method, system and device thereof | |
| US20130294449A1 (en) | Efficient application recognition in network traffic | |
| EP2887609B1 (en) | System and method for analyzing devices accessing a network | |
| CN101933290A (en) | Method for configuring acls on network device based on flow information | |
| US20100042565A1 (en) | Mezzazine in-depth data analysis facility | |
| CN103444132A (en) | Network system, and switching method | |
| CA2897664A1 (en) | An improved streaming method and system for processing network metadata | |
| KR101191251B1 (en) | 10 Gbps scalable flow generation and control, using dynamic classification with 3-level aggregation | |
| US8553539B2 (en) | Method and system for packet traffic congestion management | |
| US20160248652A1 (en) | System and method for classifying and managing applications over compressed or encrypted traffic | |
| WO2017070965A1 (en) | Data processing method based on software defined network and related device | |
| US7266088B1 (en) | Method of monitoring and formatting computer network data | |
| JP4246238B2 (en) | Traffic information distribution and collection method | |
| JP2008135871A (en) | Network monitoring system, network monitoring method, and network monitoring program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Open date: 20100113 |