CN110149248B - Method for rapidly counting and analyzing router flow - Google Patents
Method for rapidly counting and analyzing router flow Download PDFInfo
- Publication number
- CN110149248B CN110149248B CN201910492425.1A CN201910492425A CN110149248B CN 110149248 B CN110149248 B CN 110149248B CN 201910492425 A CN201910492425 A CN 201910492425A CN 110149248 B CN110149248 B CN 110149248B
- Authority
- CN
- China
- Prior art keywords
- data
- cache
- feature
- router
- memory storage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2255—Hash tables
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/04—Processing captured monitoring data, e.g. for logfile generation
- H04L43/045—Processing captured monitoring data, e.g. for logfile generation for graphical visualisation of monitoring data
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0876—Network utilisation, e.g. volume of load or congestion level
Abstract
The invention discloses a method for rapidly counting and analyzing router flow, which comprises the following steps: creating 2 memory storage areas corresponding to the kernel module of the router, wherein the 2 memory storage areas respectively correspond to the cache data of the previous period and the cache data of the current period; acquiring data packet information received and sent by a router based on a hook function of a built-in netfilter of the router, acquiring a source address, a destination address, a source port and a destination port of the data packet, performing hash function calculation to obtain a key, matching a memory storage area corresponding to an index of current cycle cache data, acquiring a red and black tree cache corresponding to the memory storage area, and acquiring a corresponding characteristic type from the red and black tree cache if the key can be matched; and after the characteristic data are matched, identifying and counting the data packet to obtain the flow of the router corresponding to the characteristic type. The invention realizes the rapid classification of the flow and counts the classified flow.
Description
Technical Field
The invention relates to the technical field of rapid router flow, in particular to a method for rapidly counting and analyzing router flow.
Background
The router processes network traffic of different applications and has different priorities, so that the application with high priority has higher data forwarding priority when the network is congested under the condition of limited bandwidth. Meanwhile, the router needs a traffic analysis function so as to provide a data analysis report for a user. The two flow-based analyses are both implemented on the basis of data flow application classification.
In the prior art, a kernel module developed based on netfilter is adopted for carrying out matching flow classification, but more technologies have great improvement space in matching efficiency and matching flexibility, and a matching mode is difficult to expand. The invention aims to realize a dynamically configurable quick matching method.
Disclosure of Invention
The invention provides a method for rapidly counting and analyzing router flow, which realizes that user configuration characteristic parameters including protocol types, ports, domain names and user-defined prefabricated characteristic data are dynamically imported, and the characteristic data only needs to be extracted once in the same period time, thereby realizing rapid matching. And an index mode based on periodic time is adopted to realize that the cache data in the kernel has cache effective time, so that the cache effective time is kept consistent with the periodic import characteristic data, and the classified flow is counted while the flow is classified quickly.
A method for fast statistical analysis of router traffic comprises the following steps:
1) creating 2 memory storage areas corresponding to a kernel module of the router, wherein the 2 memory storage areas correspond to the cache data of the previous period and the cache data of the current period respectively, the two memory storage areas are used for matching the cache data of the previous period and the cache data of the current period respectively, calculating an offset index based on time, and acquiring an index of the cache data of the previous period and an index of the cache data of the current period;
2) acquiring data packet information received and sent by a router based on a hook function of a built-in netfilter of the router, acquiring a source address, a destination address, a source port and a destination port of the data packet, performing hash function calculation to obtain a key, matching a memory storage area corresponding to an index of current cycle cache data acquired in step 1), acquiring a red and black tree cache corresponding to the memory storage area, loading feature data from a feature data file by the red and black tree cache, wherein the feature data comprises a feature type, and if the feature type can be matched, acquiring the feature type from the red and black tree cache, and entering step 4);
if not, matching the memory storage area corresponding to the index of the cache data of the previous period obtained in the step 1), obtaining the red-black tree cache corresponding to the memory storage area, if matching, updating the red-black tree cache corresponding to the memory storage area to the red-black tree cache corresponding to the memory storage area corresponding to the index of the cache data of the current period, and simultaneously updating the feature type from the loaded feature data file, entering the step 4), and if not, entering the step 3);
3) extracting features of data packet information received and sent by a router, matching the feature types with the feature data file loaded in the step 2), if the feature types are matched, obtaining a source address, a destination address, a source port and a destination port of the data packet, calculating a hash function to obtain a key, taking the matched feature type as a value, storing the key and the value into a red and black tree cache corresponding to a memory storage area corresponding to an index of cache data in the current period, and entering the step 4);
4) after the feature data are matched, the data packet is identified by the acquired feature type, the flow counter corresponding to the feature type is updated, data corresponding to the length of the data packet is added to the flow counter, and the flow of the router corresponding to the feature type is obtained through statistics.
And performing subsequent flow shaping processing, and periodically outputting the flow shaping processing to a specified file, wherein the subsequent flow shaping processing is used for visually obtaining a flow classification statistical result in a third-party visual display tool and providing data support for flow monitoring and flow shaping.
In step 1), the method of the present invention is applied to provider router equipment of openwrt, and the router is an openwrt router.
The index of the cache data in the previous period is [ current system time (second)/period value (second) +1 ]% 2, the current system time (second)/period value (second) is calculated by using INT (data type), and% 2 represents that 2 is left;
the index of the cache data in the current period is [% 2 of the current system time (second)/period value (second) ], the current system time (second)/period value (second) is calculated by using INT (data type), and% 2 represents that 2 is left;
in step 2), the packet information includes a source address, a destination address, a source port, and a destination port.
The feature type refers to types corresponding to different feature values defined in the feature data file, for example, level0_ UDP represents a type that a packet level of the UDP protocol is 0, and level0_ TCP represents a type that a packet level of the TCP protocol is 1.
In step 3), extracting features of the data packet information received and sent by the router, wherein the extracted features comprise a data packet protocol type, a destination address of the data packet, a destination port of the data packet, a domain name of http, a p2p download feature and an audio/video type.
And 4) adding data corresponding to the length of the data packet into a flow meter counter, counting, performing subsequent flow shaping treatment, and periodically outputting the data to a specified file.
The method for rapidly counting and analyzing the router flow adopts a self-defined kernel module mapping based on netfilter, is applied to router equipment, and comprises the following steps:
and carrying out flow statistics classification on the data packets, wherein each data packet is matched in a kernel statistics module mapping, the netfilter-based custom kernel module mapping is used for marking data of different grades, when the hook function of the netfilter of the data packet is handed to the kernel module mapping for matching, the mapping is periodically loaded with the latest user characteristic data and preset characteristic data, then characteristic matching is carried out, and the data is handed to the netfilter for further processing. Firstly, when the kernel module profiling registers, a plurality of storage spaces are pre-allocated and used for red and black tree caching stored based on time, and corresponding user-defined feature data and preset feature data are imported at the same time (and then, the corresponding user-defined feature data are periodically imported, so that dynamic modification is realized). Furthermore, a cache for pre-allocating a plurality of statistical characteristic data is required, and the cache is used for storing the traffic statistics of the corresponding characteristic Level, such as (Level0, Level1, Level2 … of TCP).
Next, a packet query is performed to determine whether the data has been identified as corresponding to the characteristic type. The retrieval is carried out by adopting a red-black tree with higher matching efficiency, the retrieval is carried out based on the source address, the destination address, the source port and the key obtained by calculating the destination port of the data packet, if the retrieval is carried out, the data packet is directly marked as the corresponding characteristic type, otherwise, the next step of matching is carried out.
The red and black tree cache adopts a pre-allocated memory block based on time, n storage spaces are pre-allocated, and the index of the memory block at the current moment is that the current system time (second)/period value (second)/n% n is based on time de-offset index, so that the imported cache data after the period time is ensured to be effective. Specifically, pre-allocation of 2 storage spaces can be set.
When the cache data information is not matched, feature data in the data packet needs to be extracted, wherein the extracting of the feature data of the data packet comprises: protocol type (tcp, udp, http), port. And if the protocol is http, a corresponding domain name address needs to be extracted, matching the acquired feature data with preset feature data by using an http method (get, put, post and the like), wherein the preset feature data comprises common p2p download features and common video website data features, and identifying a corresponding data type if the preset feature data is matched with the common p2p download features.
The user-defined feature data, including user-defined domain names, ports, protocols, are again matched. And if so, identifying the corresponding data type, and after the matching of the pre-set characteristic data, ensuring that the matching level of the characteristic data defined by the user is higher than the preset characteristic.
After the feature data is matched, the corresponding feature data information is stored in the cache, so that the data packet (source address, destination address, source port and destination port) of the type can be directly searched and hit in the cache when being matched next time, and the feature information does not need to be extracted again. And meanwhile, updating the flow of the characteristic data of the corresponding grade, and periodically outputting the corresponding flow data to a specified file according to a certain format for external statistical analysis.
Compared with the prior art, the invention has the following advantages:
the invention adopts a self-defined kernel module developed based on netfilter to realize dynamic import of user configuration characteristic parameters including protocol type, port, domain name and self-defined prefabricated characteristic data. Extensible matching user features are supported.
The invention adds cache matching based on the kernel module, ensures that the data packets of the same type (the source address, the destination address, the source port and the destination port are consistent) only need to extract the characteristic data once in the same cycle time, and realizes quick matching. The kernel module mapping adopts an index mode based on periodic time, so that cache data in a kernel has cache effective time, and the consistency of the cache effective time and the periodic import characteristic data is ensured.
The invention realizes the flow classification based on the kernel module and integrates the flow statistics, rapidly classifies the flow, simultaneously counts the classified flow, periodically outputs the counted flow to a specified file, and can be used for displaying the flow information by a third-party graphical tool.
Drawings
FIG. 1 is a schematic flow chart illustrating a method for fast statistical analysis of router traffic according to the present invention;
FIG. 2 provides a schematic diagram of the main components of an embodiment of the present invention;
FIG. 3 is a schematic flow chart of the fast statistical analysis of traffic according to the present invention;
fig. 4 is a third party traffic graph display diagram of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention more apparent, some embodiments of the present invention will be described in detail below with reference to the accompanying drawings.
As shown in fig. 1, a method for fast statistically analyzing router traffic includes the following steps:
1) creating 2 memory storage areas corresponding to a kernel module of an openwrt router, wherein the 2 memory storage areas correspond to cache data of a previous period and cache data of a current period respectively, the two memory storage areas are used for matching the cache data of the previous period and the cache data of the current period respectively, calculating an offset index based on time, and acquiring an index of the cache data of the previous period, wherein the index is [ current system time (second)/period (second) +1 ]% 2, the current system time (second)/period (second) is calculated by INT (data type), and% 2 represents that 2 is left;
acquiring an index of the cache data of the current period [ current system time (second)/period value (second) ]% 2, calculating the current system time (second)/period value (second) by using INT (data type), and indicating that 2 is left;
2) acquiring data packet information received and sent by a router based on a hook function of a built-in netfilter of the router, wherein the data packet information comprises a source address, a destination address, a source port and a destination port, acquiring the source address, the destination address, the source port and the destination port of the data packet, performing hash function calculation to obtain key, matching a memory storage area corresponding to an index of cache data in the current period obtained in the step 1), acquiring a red-black tree cache corresponding to the memory storage area, and if the key can be matched, acquiring a feature type (the feature type is a type corresponding to different feature values defined in a feature data file, for example, level0_ UDP represents a type with a UDP protocol packet level of 0, and level0_ TCP represents a type with a TCP protocol packet level of 1) from the red-black tree cache loaded with the feature data, and entering step 4);
if not, matching the memory storage area corresponding to the index of the cache data of the previous period obtained in the step 1), obtaining the red-black tree cache corresponding to the memory storage area, if matching, updating the red-black tree cache corresponding to the memory storage area to the red-black tree cache corresponding to the memory storage area corresponding to the index of the cache data of the current period, and simultaneously updating the feature type from the loaded feature data file, entering the step 4), and if not, entering the step 3);
3) extracting features of data packet information received and sent by a router, wherein the extracted features comprise a data packet protocol type, a destination address of the data packet, a destination port of the data packet, a domain name of http, a p2p download feature and an audio/video type, and are matched with the feature file matching feature type loaded in the step 2), if the extracted features are matched with the feature file matching feature type loaded in the step 2), obtaining a source address, the destination address, a source port and a destination port of the data packet, performing hash function calculation to obtain a key, taking the matched feature type as a value, storing the key and the value into a red and black tree cache corresponding to a memory storage area corresponding to an index of cache data in the current period, and entering the step 4);
4) after the feature data are matched, the data packet is identified by the acquired feature type, the flow counter corresponding to the feature type is updated, data corresponding to the length of the data packet is added to the flow counter, and the flow of the router corresponding to the feature type is obtained through statistics. And performing subsequent flow shaping processing, and periodically outputting the flow shaping processing to a specified file, wherein the subsequent flow shaping processing is used for visually obtaining a flow classification statistical result in a third-party visual display tool and providing data support for flow monitoring and flow shaping.
The method for rapidly counting and analyzing the router flow is applied to provider router equipment of openwrt, a user-defined kernel module mapping based on netfilter is adopted, and the method comprises the following steps:
1) the router device starts a kernel analysis module mapping by using iptables, and periodically loads characteristic data for identifying traffic, wherein the characteristic data comprises a characteristic data file prefabricated by a system and a characteristic data file defined by a user. The feature data will include a feature type level, a protocol type, and corresponding feature parameters. The corresponding characteristic data file format is as follows: (level0_ udp:22,53,5060,5070,5080), level2_ domains:. video.qq.com,. hd.sohu.com.cn,. video.sina.com.cn). The analysis module mapping is a kernel-level module developed based on netfilter, is self-loaded into a kernel through starting, and is started by iptables-t rule-i interface-j SHAPING, wherein the interface is a network card name to be analyzed.
2) Meanwhile, when the kernel analysis module is started, 2 corresponding cache blocks (a cache with expiration time is created and used for connection tracking, matched feature data are matched from the cache within a certain time, and the matching efficiency is improved) are created, and cache indexes loaded in the current period are obtained in each period and used for loading feature data stored in the red and black trees; obtaining a current period cache index for matching a hit connection cache of a current period; and acquiring the current last period cache index for matching the expired connection cache (the characteristic type needs to be updated). The cache block with the time offset acquires a cache index based on time, the index of the cache data in the previous period is acquired as [ current system time (second)/period value (second) +1 ]% 2, the current system time (second)/period value (second) is calculated by INT (data type), and% 2 represents that 2 is left; acquiring an index of the cache data of the current period [ current system time (second)/period value (second) ]% 2, calculating the current system time (second)/period value (second) by using INT (data type), and indicating that 2 is left;
3) the hook function based on netfilter is delivered to a kernel analysis module for data matching, before matching, whether the connection is matched with a characteristic data type or not is searched (a cache block in the period and a buffer block in the previous period are searched, the buffer block in the previous period is preferentially matched), if the connection is hit in the previous period, the corresponding characteristic type needs to be updated to the latest type, the connection of the data packet adopts a source address, a destination address, a source port and a destination port, hash is carried out on the data packet to obtain a key, the corresponding cache red and black tree is matched, once the connection is matched, the step 6 is carried out, and otherwise, the step 4 is carried out.
4) And extracting mirror image features of the data packet, wherein the extracted features mainly comprise protocol types, ports and sub-features (http domain names, p2p features and the like) of different protocols, if the sub-features are tls handshake packets of https, the corresponding domain name information is obtained through the handshake packets, and the matching with the feature data can be carried out through domain name suffixes. The Udp protocol matches built-in P2P feature types, including common P2P downloads (KaZaA, eDonkey/eMule/Kad, Gnutella, BitTorrent, DirectConnect, PPLive/PPStream, xunlei), the type matching feature data is obtained by analyzing corresponding data summary, and the corresponding P2P download data feature values are obtained by direct binary data matching summary. The UDP is used for classifying the key data of the video conference by matching the real-time data of the video conference type, and judging whether the real-time video or voice data exists or not by matching the rtp type of the UDP.
5) And if the characteristic data is matched in the last step, storing the corresponding characteristic type into the red and black tree of the cache block in the period time, wherein the key is obtained by hashing a source address, a destination address, a source port and a destination port of the data packet, and the value is the corresponding characteristic type.
6) After the feature data are matched, the data packet is identified, counted and marked by the obtained feature type, and the fwmark value can be used for subsequent flow shaping processing to realize different types of data forwarding priority processing. In addition, the flow counter corresponding to the characteristic type is updated, data corresponding to the length of the data packet is added to the counter, and the data is periodically output to a specified file, and the data is in a fixed format as follows: tcp _ level _0:45959, tcp _ level _1:14509, tcp _ level _2:45470, tcp _ level _3: 0; udp _ level _0:135021, udp _ level _1:123912, udp _ level _2:182028, udp _ level _3: 0; the data can be used for visually acquiring flow classification statistical results in a third-party visual display tool, and data support is provided for flow monitoring and flow shaping.
And loading the characteristic data by using the kernel module profiling, directly marking the corresponding characteristic data type if the cache matching connection with the effective time is successful, otherwise, extracting the characteristic, then matching the characteristic data with the loaded characteristic data, marking the corresponding data type if the matching is successful, and updating the counter of the corresponding data type.
The kernel module profiling is realized based on the netfilter interface, and feature data loading firstly needs to import a corresponding preset feature data file and then import user-defined feature data. And the imported feature data are stored in the corresponding feature cache and are used for matching by the feature data extraction and matching component, and then the feature data are loaded on the basis of cycle execution.
The cache with dead time creates corresponding 2 cache blocks (2 cache blocks are needed because the statistical information of the previous period needs to be stored) when the kernel module is started, and is used for storing the matched feature data of the red and black trees in different periods and the flow classification statistical information in different periods.
Feature data extraction and matching are performed when a data packet arrives, and the extracted features mainly include protocol types, ports and sub-features (http domain names, p2p features and the like) of different protocols. The extracted feature data is matched with the feature data in the feature data import component.
After the feature data are matched, the data packet is identified, counted and marked with an fwmark value, so that the method can be used for subsequent flow shaping processing and different types of data forwarding priority processing can be realized. In addition, the result of the flow statistics is periodically output to a specified file, so that the flow classification statistical result can be visually obtained in a third-party visual display tool, and data support is provided for flow monitoring and flow shaping.
Fig. 2 is a schematic diagram of a main component structure provided in an embodiment of the present invention, and relates to a kernel module profiling is a kernel module developed based on netfilter, where the kernel module mainly includes four components:
loading characteristic data; caching with failure time; extracting and matching feature data; flag, statistical data type.
When the kernel module profiling starts and registers to the netfilter, the four components are started, the feature data loading firstly needs to import the corresponding preset feature data file, and then imports the user-defined feature data. The corresponding characteristic data file format comprises: protocol type, feature level, port, domain name such as (level0_ udp:22,53,5060,5070,5080), (level2_ domains:. video.qq.com,. hd.sohu.com.cn,. video.sina.com.cn). And the imported feature data are stored in the corresponding feature cache and are used for matching by the feature data extraction and matching component, and then the component loads the feature data based on cycle execution.
The cache with dead time creates corresponding 2 cache blocks (2 cache blocks are needed because the statistical information of the previous period needs to be stored) when the kernel module is started, and is used for storing the matched feature data of the red and black trees in different periods and the flow classification statistical information in different periods.
Feature data extraction and matching are performed when a data packet arrives, and the extracted features mainly include protocol types, ports and sub-features (http domain names, p2p features and the like) of different protocols. The extracted feature data is matched with the feature data in the feature data import component.
After the feature data are matched, the data packet is identified, counted and marked with an fwmark value, so that the method can be used for subsequent flow shaping processing and different types of data forwarding priority processing can be realized. In addition, the result of the flow statistics is periodically output to a designated file according to a fixed format such as: tcp _ level _0:45959, tcp _ level _1:14509, tcp _ level _2:45470, tcp _ level _3: 0; udp _ level _0:135021, udp _ level _1:123912, udp _ level _2:182028, and udp _ level _3: 0. Therefore, the flow classification statistical result can be visually obtained in a third-party visual display tool, and data support is provided for flow monitoring and flow shaping.
Fig. 3 is a flow chart of fast statistical analysis of traffic in some examples provided by the examples of the present invention, and fig. 3 can more clearly understand the steps of statistical analysis of traffic.
Step 1, firstly, a kernel module periodically leads in a preset feature data file (/ var/mapping.conf) and a user-defined feature data file (/ etc/mapping.conf), and feature data needing to be matched are loaded.
And 2, handing the netfilter-based hook function to the kernel module for data matching, searching whether the connection is matched with the feature data type (searching in the period time and the last period time) before matching, and if the connection is hit in the cache in the last period, updating the corresponding feature type to the latest type. If the connection (source address, destination address, source port, destination port) of the packet has been matched step 7 is entered. Otherwise, go to step 3.
And 3, extracting characteristic data of the data packet, extracting protocol header information (skb _ transport _ header), data header information of tcp or udp, and continuously judging whether the http service exists or not if the tcp protocol exists, if the tcp protocol needs to further acquire detailed information such as a domain name and a method, if the tcp protocol exists, acquiring corresponding domain name information through a handshake packet, and matching the domain name information with the characteristic data through a domain name suffix. The Udp protocol matches built-in P2P feature types, including common P2P downloads (KaZaA, eDonkey/eMule/Kad, Gnutella, BitTorrent, DirectConnect, PPLive/PPStream, xunlei), the type matching feature data is obtained by analyzing corresponding data summaries, and direct binary matching summaries are adopted to obtain corresponding P2P download data feature values. In addition, UDP is used for matching real-time data of the video conference type and classifying key data of the video conference, and whether real-time video or voice data exists is judged by matching the rtp type of UDP.
And 4, matching feature data defined by a user, wherein the feature data comprises a domain name, a protocol, an ip and a port. The data are directly matched with the data packet characteristic data extracted in the previous step.
And 5, acquiring the red and black tree cache at the current moment, and performing offset indexing on the current system time (second) period value (second)/n% n based on time to realize the cache with time failure. And clearing the cache in the last period of the hit rule to avoid the last time of cache hit.
And 6, storing the corresponding characteristic data into the cached red and black trees in the period.
And 7, identifying the fwmark of the data packet as a mark value of a corresponding characteristic grade, and identifying the classified data for subsequent data processing of flow shaping and distinguishing different priorities.
And 8, updating the flow counter corresponding to the classified data type, adding data corresponding to the data packet length into the counter, and periodically outputting the data to a specified file for third-party flow graph display so as to provide data support for flow monitoring and flow shaping.
By loading the analyzed kernel module profiling, a specific characteristic parameter can be flexibly configured according to the behavior characteristics of the user, classification characteristics are defined, and in addition, data reading is carried out by using an html format period, so that data support can be provided for flow monitoring and flow shaping. As shown in fig. 4, a third party traffic graph is shown to provide data support for traffic monitoring and traffic shaping.
Claims (4)
1. A method for fast statistical analysis of router traffic is characterized by comprising the following steps:
1) creating 2 memory storage areas corresponding to a kernel module of the router, wherein the 2 memory storage areas correspond to the cache data of the previous period and the cache data of the current period respectively, the two memory storage areas are used for matching the cache data of the previous period and the cache data of the current period respectively, calculating an offset index based on time, and acquiring an index of the cache data of the previous period and an index of the cache data of the current period;
the index of the cache data in the previous period is [ current system time (second)/period value (second) +1 ]% 2, the current system time (second)/period value (second) is calculated by using an INT data type, and% 2 represents that 2 is left;
the index of the cache data in the current period is [ current system time (second)/period value (second) ]% 2, the current system time (second)/period value (second) is calculated by adopting an INT data type, and% 2 represents that 2 is left;
2) acquiring data packet information received and sent by a router based on a hook function of a built-in netfilter of the router, acquiring a source address, a destination address, a source port and a destination port of the data packet, performing hash function calculation to obtain a key, matching a memory storage area corresponding to an index of current cycle cache data acquired in step 1), acquiring a red and black tree cache corresponding to the memory storage area, loading feature data from a feature data file by the red and black tree cache, wherein the feature data comprises a feature type, and if the feature type can be matched, acquiring the feature type from the red and black tree cache, and entering step 4);
if not, matching the memory storage area corresponding to the index of the cache data of the previous period obtained in the step 1), obtaining the red-black tree cache corresponding to the memory storage area, if matching, updating the red-black tree cache corresponding to the memory storage area to the red-black tree cache corresponding to the memory storage area corresponding to the index of the cache data of the current period, and simultaneously updating the feature type from the loaded feature data file, entering the step 4), and if not, entering the step 3);
3) extracting features of data packet information received and sent by a router, matching the feature types with the feature data file loaded in the step 2), if the feature types are matched, obtaining a source address, a destination address, a source port and a destination port of the data packet, calculating a hash function to obtain a key, taking the matched feature type as a value, storing the key and the value into a red and black tree cache corresponding to a memory storage area corresponding to an index of cache data in the current period, and entering the step 4);
4) after the feature data are matched, the data packet is identified by the acquired feature type, the flow counter corresponding to the feature type is updated, data corresponding to the length of the data packet are added into the flow counter, and the statistics is carried out to obtain the flow of the router corresponding to the feature type.
2. The method for fast statistically analyzing router traffic as claimed in claim 1, wherein in step 1), the router is an openwrt router.
3. The method as claimed in claim 1, wherein in step 2), the packet information includes a source address, a destination address, a source port, and a destination port.
4. The method for rapidly statistically analyzing the router traffic according to claim 1, wherein in step 3), the feature extraction is performed on the packet information received and sent by the router, and the extracted features include a packet protocol type, a destination address of the packet, a destination port of the packet, a domain name of http, a p2p download feature, and an audio-video type.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910492425.1A CN110149248B (en) | 2019-06-06 | 2019-06-06 | Method for rapidly counting and analyzing router flow |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910492425.1A CN110149248B (en) | 2019-06-06 | 2019-06-06 | Method for rapidly counting and analyzing router flow |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110149248A CN110149248A (en) | 2019-08-20 |
| CN110149248B true CN110149248B (en) | 2020-03-03 |
Family
ID=67590780
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910492425.1A Active CN110149248B (en) | 2019-06-06 | 2019-06-06 | Method for rapidly counting and analyzing router flow |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110149248B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112261060B (en) * | 2020-10-30 | 2023-04-07 | 四川创智联恒科技有限公司 | Repeated data packet detection method for reliable communication transmission |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105704036A (en) * | 2014-11-27 | 2016-06-22 | 华为技术有限公司 | Message forwarding method, apparatus, and system |
| CN105812277A (en) * | 2014-12-31 | 2016-07-27 | 中国电信股份有限公司 | Access request control method, access request control system and communication equipment |
| CN108418847A (en) * | 2017-02-09 | 2018-08-17 | 中国移动通信集团甘肃有限公司 | A kind of network traffic cache system, method and device |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7464181B2 (en) * | 2003-09-11 | 2008-12-09 | International Business Machines Corporation | Method for caching lookups based upon TCP traffic flow characteristics |
| CN100444579C (en) * | 2005-01-18 | 2008-12-17 | 北京大学 | Method of implementing quick network message distribution based on adaptive cache mechanism |
| CN101202652B (en) * | 2006-12-15 | 2011-05-04 | 北京大学 | Device for classifying and recognizing network application flow quantity and method thereof |
| CN102611626B (en) * | 2012-03-30 | 2014-11-26 | 北京英诺威尔科技股份有限公司 | System and method for analyzing network flow |
| KR101365496B1 (en) * | 2012-07-06 | 2014-03-12 | 한국외국어대학교 연구산학협력단 | Adaptive Traffic Buffering Method and System in IP Networks |
| CN103763154B (en) * | 2014-01-11 | 2018-02-23 | 浪潮电子信息产业股份有限公司 | A kind of network flow detection method |
-
2019
- 2019-06-06 CN CN201910492425.1A patent/CN110149248B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105704036A (en) * | 2014-11-27 | 2016-06-22 | 华为技术有限公司 | Message forwarding method, apparatus, and system |
| CN105812277A (en) * | 2014-12-31 | 2016-07-27 | 中国电信股份有限公司 | Access request control method, access request control system and communication equipment |
| CN108418847A (en) * | 2017-02-09 | 2018-08-17 | 中国移动通信集团甘肃有限公司 | A kind of network traffic cache system, method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110149248A (en) | 2019-08-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10547674B2 (en) | Methods and systems for network flow analysis | |
| CN112039904A (en) | Network traffic analysis and file extraction system and method | |
| US9210090B1 (en) | Efficient storage and flexible retrieval of full packets captured from network traffic | |
| US20120182891A1 (en) | Packet analysis system and method using hadoop based parallel computation | |
| US10009239B2 (en) | Method and apparatus of estimating conversation in a distributed netflow environment | |
| JP2007336512A (en) | Statistical information collecting system, and apparatus thereof | |
| CN108667770B (en) | Website vulnerability testing method, server and system | |
| US8909808B2 (en) | Redundancy elimination for web caching | |
| CN113590910B (en) | Network traffic retrieval method and device | |
| CN102739457A (en) | Network flow recognition system and method based on DPI (Deep Packet Inspection) and SVM (Support Vector Machine) technology | |
| US11722587B2 (en) | Network recorders with computer data packet truncation | |
| CN107181605B (en) | Message detection method and system, content extraction device and flow matching device | |
| CN112565262A (en) | Flow data processing method, system, network equipment and storage medium | |
| CN110149248B (en) | Method for rapidly counting and analyzing router flow | |
| CN111224894A (en) | Traffic collection marking method and system for iOS device | |
| CN1878141A (en) | Network control apparatus and control method | |
| CN112929376A (en) | Flow data processing method and device, computer equipment and storage medium | |
| CN111182072A (en) | Application identification method and device of session request and computer equipment | |
| KR20100024723A (en) | System and method for analyzing alternative internet traffic using routing based on policy | |
| KR100608541B1 (en) | An apparatus for capturing Internet ProtocolIP packet with sampling and signature searching function, and a method thereof | |
| CN111741127B (en) | Communication connection blocking method and device, electronic equipment and storage medium | |
| CN106506400B (en) | data stream identification method and outlet device | |
| CN112688924A (en) | Network protocol analysis system | |
| CN113315678A (en) | Encrypted TCP (Transmission control protocol) traffic acquisition method and device | |
| CN112822289A (en) | Data uploading and reading method, system, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |