CN110149248A - A kind of method of express statistic analysis router traffic - Google Patents
A kind of method of express statistic analysis router traffic Download PDFInfo
- Publication number
- CN110149248A CN110149248A CN201910492425.1A CN201910492425A CN110149248A CN 110149248 A CN110149248 A CN 110149248A CN 201910492425 A CN201910492425 A CN 201910492425A CN 110149248 A CN110149248 A CN 110149248A
- Authority
- CN
- China
- Prior art keywords
- data
- characteristic
- router
- type
- memory storage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2255—Hash tables
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/04—Processing captured monitoring data, e.g. for logfile generation
- H04L43/045—Processing captured monitoring data, e.g. for logfile generation for graphical visualisation of monitoring data
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0876—Network utilisation, e.g. volume of load or congestion level
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Software Systems (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Environmental & Geological Engineering (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a kind of methods of express statistic analysis router traffic, comprising: creates corresponding 2 memory storage areas in the kernel module of router, 2 memory storage areas respectively correspond the data cached of the data cached and current period in a period；The Hook Function of built-in netfilter based on router gets the packet information that router sends and receivees, it gets the source address of the data packet, destination address, source port, destination port and carries out hash function and key is calculated, match memory storage area corresponding to the data cached index of current period, obtain the corresponding red black tree caching of the memory storage area, if can match, corresponding characteristic type is obtained from red black tree caching；After characteristic matching, which is identified, counts, obtains the flow of router character pair type.The present invention realizes Fast Classification flow simultaneously, also counts the flow of classification.
Description
Technical field
The present invention relates to router traffic rapid technology fields, and in particular to a kind of express statistic analysis router traffic
Method.
Background technique
There are priority differences for the network flow of router processing different application, high to ensure under limited bandwidth conditions
Priority is applied in network congestion, and higher data forwarding priority level is possessed.Router needs flow analysis function simultaneously
Can, to be supplied to Users'Data Analysis report.Analysis of the two above based on flow, with data traffic application class
On the basis of realize.
The prior art, which is used, also has a presence based on what the kernel module that netfilter develop match traffic classification, but
More technology is in matching efficiency and matches the space that is significantly improved in flexibility, it is difficult to extend match pattern.Purport of the present invention
Has the fast matching method of dynamic and configurable in realization.
Summary of the invention
The present invention provides a kind of method of express statistic analysis router traffic, realize that dynamic imports user configuration feature
Parameter includes protocol type, port, domain name and customized prefabricated characteristic, only needs to extract within the same period time
Characteristic realizes Rapid matching.And the indexing model based on the periodical time is used, it realizes data cached in kernel
Has caching effective time, to guarantee that cache validity period is consistent with periodically importing characteristic, to realize quick
Sorting flow simultaneously, also counts the flow of classification.
A kind of method of express statistic analysis router traffic, comprising the following steps:
1) corresponding 2 memory storage areas are created in the kernel module of router, 2 memory storage areas respectively correspond
The data cached and current period in one period it is data cached, two memory storage areas are respectively used to match the slow of a period
Deposit data and current period it is data cached, the calculating of offset index is carried out based on the time, obtained the data cached of a upper period
Index and current period data cached index；
2) Hook Function of the built-in netfilter based on router gets the data packet letter that router sends and receivees
Breath, gets the source address of the data packet, destination address, source port, destination port and carries out hash function key is calculated,
Memory storage area corresponding to the data cached index of the current period obtained with step 1), obtains the memory storage area pair
The red black tree caching answered, red black tree caching are loaded with characteristic from characteristic data file, and characteristic includes characteristic type, such as
Fruit can match, and obtain characteristic type from red black tree caching, enter step 4)；
If it does not match, matching step 1) memory storage corresponding to the obtained data cached index in a upper period
Region obtains the corresponding red black tree caching of the memory storage area, if it does, the corresponding red black tree of the memory storage area is slow
It deposits and updates red black tree caching corresponding to memory storage area corresponding to the data cached index of current period, while from loading
Characteristic data file in update characteristic type, enter step 4), if mismatch, enter step 3)；
3) feature extraction is carried out to the packet information that router sends and receivees, with the characteristic being loaded into step 2)
File matching characteristic type matching gets source address, the destination address, source port, destination of the data packet if be matched to
Mouthful carrying out hash function is calculated key, and using the characteristic type being matched to as value, key and value are stored to working as
In the corresponding red black tree caching of memory storage area corresponding to data cached index of preceding period, enter step 4)；
4) after characteristic matching, the characteristic type that will acquire is identified the data packet, character pair type
Flowmeter counter update, increase the data of corresponding data packet length into counter of flowmeter, it is corresponding to obtain router for statistics
The flow of characteristic type.
Subsequent traffic shaping processing, and be periodically output in specified file, in third party's visual presentation tool
Traffic classification statistical result intuitively is got, provides data supporting for traffic monitoring and traffic shaping.
In step 1), the method for the present invention is applied to provider's router device of openwrt, and the router is
Openwrt router.
The data cached index in a upper period=[present system time (second)/periodic quantity (second)+1] %2, current system
Time (second)/periodic quantity (second) is calculated using INT (data type), and %2 indicates to carry out remainder to 2；
The data cached index of current period=[present system time (second)/periodic quantity (second)] %2, when current system
Between (second)/periodic quantity (second) using INT (data type) calculate, %2 indicate to 2 carry out remainders；
In step 2), the packet information includes source address, destination address, source port, destination port.
Characteristic type refers to type corresponding to different characteristic value defined in characteristic data file, such as level0_udp table
It is shown as the type that the data packet grade of udp protocol is 0, level0_tcp is expressed as the class that the data packet grade of Transmission Control Protocol is 1
Type.
In step 3), feature extraction is carried out to the packet information that router sends and receivees, the feature of extraction includes number
According to packet protocol type, the destination address of data packet, the destination port of data packet, the domain name of http, p2p download features, audio-video
Type.
In step 4), increases the data of corresponding data packet length into counter of flowmeter, count, at subsequent traffic shaping
Reason, and be periodically output in specified file, the prior art can be used, for intuitively obtaining in third party's visual presentation tool
Traffic classification statistical result is got, provides data supporting for traffic monitoring and traffic shaping.
The method of express statistic analysis router traffic of the present invention uses the customized kernel module based on netfilter
Shaping is applied to router device, this method comprises:
Data packet is carried out in traffic statistics classification, each data packet carries out in interior nuclear statistics module shaping
Match, the data of different brackets type are marked in the customized kernel module shaping based on netfilter, when data packet quilt
The Hook Function of netfilter transfers to kernel module shaping to be matched, and it is special that shaping periodically loads newest user
Data and preset feature data are levied, then carries out characteristic matching, then transfer to netfilter to be further processed data.It is first
First kernel module shaping will pre-allocate multiple memory spaces when registering, the red black tree caching for being stored based on the time, together
When import corresponding user's defined feature data and preset characteristic and (will periodically import corresponding user later to determine
Adopted characteristic, to realize that dynamic can be modified).Furthermore need to pre-allocate the caching of multiple statistical nature data, which uses
In storage character pair grade traffic statistics as a result, such as (Level0 of TCP, Level1, Level2 ...).
Secondly data packet inquiry is carried out, determines whether the data have been previously identified as character pair type.The retrieval uses
The higher red black tree of matching efficiency carries out, the source address based on data packet, destination address, source port, and destination port calculates gained
To key retrieved, directly marked if retrieving the data packet be corresponding characteristic type, otherwise carry out in next step
Match.
Red black tree caching uses time-based predistribution memory block, pre-allocates n memory space, the memory at current time
Block index is then: present system time (second)/periodic quantity (second)/n%n goes offset index based on the time, thus when guaranteeing the period
Between after import it is data cached effectively.2 memory spaces of specific settable predistribution.
When being not matched to data cached information, then needs to extract the characteristic in data packet, extract the data packet
Characteristic includes: protocol type (tcp, udp, http), port.It then needs to extract corresponding domain name if it is the agreement of http
Address, http method (get, put, post etc.), is matched by the characteristic got with preset characteristic, in advance
The characteristic of system includes common p2p download features, and common video website data characteristics is identified if being matched to
Corresponding data type.
User-defined characteristic, including user-defined domain name, port, agreement are matched again.If being matched to
It is identified corresponding data type, which guarantees user-defined characteristic after the matching of prefabricated characteristic
It is higher than preset features with grade.
After matching characteristic data by corresponding characteristic information preservation into caching, the data packet of such the type
(source address, destination address, source port, destination port) will be in matching next time, and directly retrieval hit in the buffer does not need again
Secondary extraction characteristic information.The flow of the characteristic of corresponding grade is updated simultaneously, corresponding data on flows will be according to one
The formula that fixes periodically is output in specified file, is used for external statistical analysis.
Compared with prior art, the present invention has the advantage that
The present invention uses the customized kernel module developed based on netfilter, realizes that dynamic imports user configuration feature
Parameter includes protocol type, port, domain name and customized prefabricated characteristic.Support expansible matching user personality.
The present invention is based on kernel module be added cache match, guarantee same type of data packet (source address, destination address,
Source port, destination port are consistent) only need to extract a characteristic within the same period time, realize Rapid matching.Kernel
Module shaping use the indexing model based on the periodical time, realize kernel in it is data cached have caching effective time,
It is consistent to guarantee to cache validity period with periodically importing characteristic.
The present invention is based on kernel module realize traffic classification and traffic statistics in one, Fast Classification flow simultaneously,
The flow of classification is counted, is periodically output in specified file, can be used for third party's graphical tools and show flow letter
Breath.
Detailed description of the invention
Fig. 1 is the flow diagram for the method that express statistic of the invention analyzes router traffic；
Fig. 2 provides main component structure diagram for the embodiment of the present invention；
Fig. 3 is the flow diagram that express statistic provided by the invention analyzes flow；
Fig. 4 is that third party's flow figure of the invention shows schematic diagram.
Specific embodiment
In order to make the object, technical scheme and advantages of the embodiment of the invention clearer, below in conjunction with attached in the present invention
Figure, is described in detail some embodiments of the present invention.
As shown in Figure 1, a kind of method of express statistic analysis router traffic, comprising the following steps:
1) corresponding 2 memory storage areas, 2 memory storage areas point are created in the kernel module of openwrt router
Not Dui Ying a upper period data cached and current period it is data cached, two memory storage areas are respectively used to match one
The data cached and current period in period it is data cached, the calculating of offset index is carried out based on the time, obtained upper period
Data cached index=[present system time (second)/periodic quantity (second)+1] %2, present system time (second)/periodic quantity
(second) is calculated using INT (data type), and %2 indicates to carry out remainder to 2；
Data cached index=[present system time (second)/periodic quantity (second)] %2 of current period is obtained, it is current to be
Time (second)/periodic quantity (second) is united using INT (data type) calculating, %2 indicates to carry out remainders to 2；
2) Hook Function of the built-in netfilter based on router gets the data packet letter that router sends and receivees
Breath, packet information includes source address, destination address, source port, destination port, gets source address, the purpose of the data packet
Address, source port, destination port carry out hash function and key, matching step 1 are calculated) obtained current period is data cached
Index corresponding to memory storage area, obtain the memory storage area corresponding red black tree caching, if it is possible to match, from
Being loaded with acquisition characteristic type in the red black tree caching of characteristic, (characteristic type is characterized the spy of difference defined in data file
Type corresponding to value indicative, as level0_udp is expressed as the type that the data packet grade of udp protocol is 0, level0_tcp table
It is shown as the type that the data packet grade of Transmission Control Protocol is 1), it enters step 4)；
If it does not match, matching step 1) memory storage corresponding to the obtained data cached index in a upper period
Region obtains the corresponding red black tree caching of the memory storage area, if it does, the corresponding red black tree of the memory storage area is slow
It deposits and updates red black tree caching corresponding to memory storage area corresponding to the data cached index of current period, while from loading
Characteristic data file in update characteristic type, enter step 4), if mismatch, enter step 3)；
3) feature extraction is carried out to the packet information that router sends and receivees, the feature of extraction includes data pack protocol
The domain name of type, the destination address of data packet, the destination port of data packet, http, the type of p2p download features, audio-video, with
The characteristic data file matching characteristic type matching being loaded into step 2), if be matched to, get the data packet source address,
Destination address, source port, destination port carry out hash function key is calculated, using the characteristic type being matched to as
Value stores key and value to the corresponding red and black of memory storage area corresponding to the data cached index of current period
In tree caching, enter step 4)；
4) after characteristic matching, the characteristic type that will acquire is identified the data packet, character pair type
Flowmeter counter update, increase the data of corresponding data packet length into counter of flowmeter, it is corresponding to obtain router for statistics
The flow of characteristic type.Subsequent traffic shaping processing, and be periodically output in specified file, it is visualized for third party
Traffic classification statistical result is intuitively got in tool, provides data supporting for traffic monitoring and traffic shaping.
The method of express statistic analysis router traffic of the present invention is applied to provider's router device of openwrt, adopts
With the customized kernel module shaping based on netfilter, method comprising steps of
1) router device starts kernel analysis module shaping using iptables, will periodically be loaded into for identification
The characteristic of flow, this feature data include the prefabricated characteristic data file of system and user-defined characteristic text
Part.Characteristic will include characteristic type grade, protocol type and corresponding characteristic parameter.Character pair data file lattice
Formula is such as: (level0_udp:22,53,5060,5070,5080), level2_domains:*.video.qq.com, *
.hd.sohu.com.cn,*.video.sina.com.cn).Analysis module shaping is the kernel developed based on netfilter
Grade module is opened through booting bootstrap loading into kernel by iptables-t mangle-i interface-j SHAPING
Dynamic, wherein interface is the Adapter Name for needing to analyze.
2) corresponding 2 cache blocks (cachings with the out-of-service time, for connecting will be created in the starting of interior kernel analysis module simultaneously
Tracking is connect, the characteristic matched will be matched from caching within a certain period of time, improve matching efficiency), in each period
Property in will acquire current period load caching index, for load red black tree storage characteristic；It is slow to obtain current period
Index is deposited, for matching the Connection Cache of current period hit；Current upper period caching index is obtained, for matching
Through expired Connection Cache (needing to update characteristic type).Cache blocks with time migration are to carry out obtaining caching rope based on the time
Draw, obtained data cached index=[present system time (second)/periodic quantity (second)+1] %2 in a upper period, current system
Time (second)/periodic quantity (second) is calculated using INT (data type), and %2 indicates to carry out remainder to 2；Obtain the slow of current period
The index of deposit data=[present system time (second)/periodic quantity (second)] %2, present system time (second)/periodic quantity (second) are adopted
It is calculated with INT (data type), %2 indicates to carry out remainder to 2；
3) Hook Function based on netfilter transfers to kernel analysis module to carry out Data Matching, is examined before matching
Whether the rope connection has been matched feature data types, and (in this period cache blocks and buffer stopper is examined on a upper period
Rope, a period buffer stopper on priority match), it hit and caches if it is a upper period, then need to update character pair type to most
New type, the connection of the data packet use source address, destination address, source port, and destination port carries out Hash and obtains key,
With corresponding caching red black tree, 6 are entered step if being matched to, otherwise enters step 4.
4) data packet mirror image features will be extracted, the feature of extraction mainly has protocol type, port, the son of different agreement
Feature (domain name of http, p2p feature etc.) then obtains corresponding domain name by the handshake packet if it is the tls handshake packet of https
Information can carry out being matched with characteristic by domain suffix.The agreement of Udp will match built-in P2P characteristic type, packet
Include common P2P downloading (KaZaA, eDonkey/eMule/Kad, Gnutella, BitTorrent, DirectConnect,
PPLive/PPStream, xunlei), the type matching characteristic data are obtained by analyzing corresponding Data Summary, are used
Be that straight binary Data Matching summarizes to obtain corresponding P2P downloading data characteristic value.UDP will match the reality of video conference type
When data, for the critical data for video conference of classifying, the rtp type by matching udp judge whether real-time class video or
Voice data.
If 5) previous step is matched to characteristic, corresponding characteristic type is saved to the cache blocks in this cycle time
Red black tree in, wherein key is source data packet address, destination address, source port, and destination port carries out obtained by Hash, and value is
Corresponding characteristic type.
6) after characteristic matching, the characteristic type that will acquire is identified the data packet, counts, label
Fwmark value can be used for subsequent traffic shaping processing, realize the different types of different processing of data forwarding priority.Furthermore
The flowmeter counter of character pair type updates, and increases the data of corresponding data packet length into its counter, and periodically defeated
Out into specified file, according to fixed format such as: tcp_level_0:45959, tcp_level_1:14509, tcp_level_
2:45470,tcp_level_3:0；udp_level_0:135021,udp_level_1:123912,udp_level_2:
182028,udp_level_3:0；These data can be used in third party's visual presentation tool intuitively getting flow point
Class statistical result provides data supporting for traffic monitoring and traffic shaping.
Kernel module shaping carries out characteristic loading, if having the cache match successful connection of effective time, directly
Line identifier character pair data type is tapped into, feature extraction is otherwise carried out, is then matched with the characteristic of loading, is matched
To corresponding data type is then marked, and update the counter of corresponding data type.
Interface based on netfilter realizes kernel module shaping, and characteristic is loaded into preset firstly the need of that will correspond to
Characteristic data file import, then import user-defined characteristic.It imports characteristic and is stored in corresponding feature caching
In, matching component is extracted for characteristic and is matched, and hereafter will be executed characteristic based on the period and is loaded into.
Creation is corresponded into 2 cache blocks (due to needing to save upper one when being buffered in kernel module starting with the out-of-service time
The statistical information in period thus need 2 cache blocks), for the characteristic that different period red black tree storages is matched to, and
Traffic classification statistical information in different cycles.
Characteristic is extracted and matches and will execute the data packet feature extraction, the feature master of extraction when data packet reaches
There are protocol type, port, the subcharacter (domain name of http, p2p feature etc.) of different agreement.The characteristic extracted will be with
The characteristic that characteristic imports in component is matched.
The data packet will be identified, be counted after characteristic matching, and marked fwmark value, can be used for rear afterflow rate
Shape correction realizes the different types of different processing of data forwarding priority.In addition, the result of traffic statistics will be periodical
It is output in specified file, can intuitively get traffic classification statistics knot in third party's visual presentation tool in this way
Fruit provides data supporting for traffic monitoring and traffic shaping.
Fig. 2 provides main component structure diagram for the embodiment of the present invention, be related to kernel module shaping be based on
The kernel module of netfilter exploitation, which mainly includes four components:
Characteristic is loaded into；Caching with the out-of-service time；Characteristic is extracted and is matched；Label, statistical data type.
When kernel module shaping starting is registered to netfilter, above four components will be started, characteristic is loaded into
It is imported firstly the need of preset characteristic data file will be corresponded to, then imports user-defined characteristic.Character pair data text
Part format includes: protocol type, feature level, port, domain name such as (level0_udp:22,53,5060,5070,5080),
(level2_domains:*.video.qq.com,*.hd.sohu.com.cn,*.video.sina.com.cn).Import feature
Data are stored in corresponding feature caching, are extracted matching component for characteristic and are matched, hereafter the component will be based on
Period executes characteristic and is loaded into.
Creation is corresponded into 2 cache blocks (due to needing to save upper one when being buffered in kernel module starting with the out-of-service time
The statistical information in period thus need 2 cache blocks), for the characteristic that different period red black tree storages is matched to, and
Traffic classification statistical information in different cycles.
Characteristic is extracted and matches and will execute the data packet feature extraction, the feature master of extraction when data packet reaches
There are protocol type, port, the subcharacter (domain name of http, p2p feature etc.) of different agreement.The characteristic extracted will be with
The characteristic that characteristic imports in component is matched.
The data packet will be identified, be counted after characteristic matching, and marked fwmark value, can be used for rear afterflow rate
Shape correction realizes the different types of different processing of data forwarding priority.In addition, the result of traffic statistics will be periodical
It is output in specified file, according to fixed format such as: tcp_level_0:45959, tcp_level_1:14509, tcp_
level_2:45470,tcp_level_3:0；udp_level_0:135021,udp_level_1:123912,udp_level_
2:182028,udp_level_3:0.Traffic classification system can be intuitively got in third party's visual presentation tool in this way
Meter is as a result, provide data supporting for traffic monitoring and traffic shaping.
Fig. 3 is the flow chart that express statistic analyzes flow in some examples of present example provided, and Fig. 3 can be relatively clear
Clear understanding statisticallys analyze the step of flow.
Step 1 kernel module first periodically will import preset feature data file (/var/shapging.conf), and
User-defined characteristic data file (/etc/shaping.conf), load need matched characteristic.
Step 2 transfers to this kernel module to carry out Data Matching based on the Hook Function of netfilter, is examined before matching
Whether the rope connection has been matched feature data types (being retrieved within this cycle time with a upper cycle time), such as
Fruit is upper period hit caching, then needs to update character pair type to newest type.If connection (the source of the data packet
Address, destination address, source port, destination port) it has been matched to, enter step 7.Otherwise 3 are entered step.
Step 3 extracts the characteristic of the data packet, extracts protocol header information (skb_transport_header),
The data packet header information of tcp or udp is further continued for judging whether (http service) if tcp agreement, further if it is needing
The details such as domain name and method are obtained, if it is the tls handshake packet of https, then corresponding domain name are obtained by the handshake packet
Information can carry out being matched with characteristic by domain suffix.The agreement of Udp will match built-in P2P characteristic type, packet
Include common P2P downloading (KaZaA, eDonkey/eMule/Kad, Gnutella, BitTorrent, DirectConnect,
PPLive/PPStream, xunlei), the type matching characteristic data are obtained by analyzing corresponding Data Summary, are used
Be that direct binary match summarizes to obtain corresponding P2P downloading data characteristic value.Furthermore UDP will match the reality of video conference type
When data, for the critical data for video conference of classifying, the rtp type by matching udp judge whether real-time class video or
Voice data.
Step 4 matches user-defined characteristic, and this feature data include domain name, agreement, ip, port.These data
Directly matched in the data packet characteristic that previous step is extracted.
Step 5 obtains the red black tree caching at current time, when present system time (second) * periodic quantity (second)/n%n is based on
Between carry out offset index, realize the caching to fail with the time.The caching for clearing up this hit rule upper period, avoided last time from caching
It is hit.
Step 6 saves corresponding characteristic into the red black tree of the caching in this cycle time.
The fwmark that step 7 identifies the data packet is the mark value of character pair grade, is carried out to the data after classification
Mark distinguishes the data processing of different priorities for subsequent traffic shaping.
The flowmeter counter of the data type of the corresponding classification of step 8 pair updates, and the data for increasing corresponding data packet length arrive
It in its counter, and is periodically output in specified file, is shown for third party's flow figure, be that traffic monitoring and flow are whole
Shape provides data supporting.
, can be according to user oneself behavioural characteristic by the kernel module shaping of load analysis, it can be with flexible configuration one
A specific characteristic parameter, is defined characteristic of division, is in addition read out data using the html format period, can be flow
Monitoring and traffic shaping provide data supporting.It is that traffic monitoring and flow are whole as shown in figure 4, being shown for third party's flow figure
Shape provides data supporting.
Claims (6)
1. a kind of method of express statistic analysis router traffic, which comprises the following steps:
1) corresponding 2 memory storage areas are created in the kernel module of router, 2 memory storage areas respectively correspond one week
The data cached and current period of phase it is data cached, two memory storage areas are respectively used to match the caching number in a period
According to data cached with current period, the calculating of offset index is carried out based on the time, obtains the data cached rope in a upper period
Draw the data cached index with current period；
2) Hook Function of the built-in netfilter based on router gets the packet information that router sends and receivees,
It gets the source address of the data packet, destination address, source port, destination port and carries out hash function and key, matching step is calculated
Memory storage area corresponding to the data cached index of the rapid current period 1) obtained, it is corresponding to obtain the memory storage area
Red black tree caching, red black tree caching are loaded with characteristic from characteristic data file, and characteristic includes characteristic type, if energy
Enough matchings obtain characteristic type from red black tree caching, enter step 4)；
If it does not match, matching step 1) memory storage area corresponding to the obtained data cached index in a upper period
Domain obtains the corresponding red black tree caching of the memory storage area, if it does, the corresponding red black tree caching of the memory storage area
Update the corresponding red black tree caching of memory storage area corresponding to the data cached index of current period, while from loading
Characteristic type is updated in characteristic data file, is entered step 4), if mismatched, is entered step 3)；
3) feature extraction is carried out to the packet information that router sends and receivees, with the characteristic data file being loaded into step 2)
Matching characteristic type matching, if be matched to, get the source address of the data packet, destination address, source port, destination port into
Key is calculated in row hash function, and using the characteristic type being matched to as value, key and value were stored to current week
In the corresponding red black tree caching of memory storage area corresponding to phase data cached index, enter step 4)；
4) after characteristic matching, the characteristic type that will acquire is identified the data packet, the stream of character pair type
Batching counter updates, and increases the data of corresponding data packet length into counter of flowmeter, and statistics obtains router character pair
The flow of type.
2. the method for express statistic analysis router traffic according to claim 1, which is characterized in that in step 1), institute
The router stated is openwrt router.
3. the method for express statistic analysis router traffic according to claim 1, which is characterized in that in step 1), on
The data cached index in one period=[present system time (second)/periodic quantity (second)+1] %2, present system time (second)/
Periodic quantity (second) is calculated using INT data type, and %2 indicates to carry out remainder to 2.
4. the method for express statistic analysis router traffic according to claim 1, which is characterized in that in step 1), when
The data cached index in preceding period=[present system time (second)/periodic quantity (second)] %2, present system time (second)/week
Time value (second) is calculated using INT data type, and %2 indicates to carry out remainder to 2.
5. the method for express statistic analysis router traffic according to claim 1, which is characterized in that in step 2), institute
The packet information stated includes source address, destination address, source port, destination port.
6. the method for express statistic analysis router traffic according to claim 1, which is characterized in that right in step 3)
The packet information that router sends and receivees carries out feature extraction, and the feature of extraction includes data pack protocol type, data packet
Destination address, the destination port of data packet, the domain name of http, p2p download features, audio-video type.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910492425.1A CN110149248B (en) | 2019-06-06 | 2019-06-06 | Method for rapidly counting and analyzing router flow |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910492425.1A CN110149248B (en) | 2019-06-06 | 2019-06-06 | Method for rapidly counting and analyzing router flow |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110149248A true CN110149248A (en) | 2019-08-20 |
| CN110149248B CN110149248B (en) | 2020-03-03 |
Family
ID=67590780
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910492425.1A Active CN110149248B (en) | 2019-06-06 | 2019-06-06 | Method for rapidly counting and analyzing router flow |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110149248B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112261060A (en) * | 2020-10-30 | 2021-01-22 | 四川创智联恒科技有限公司 | Repeated data packet detection method for reliable communication transmission |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050060428A1 (en) * | 2003-09-11 | 2005-03-17 | International Business Machines Corporation | Apparatus and method for caching lookups based upon TCP traffic flow characteristics |
| CN1809019A (en) * | 2005-01-18 | 2006-07-26 | 北京大学 | Method of implementing quick network message distribution based on adaptive cache mechanism |
| CN101202652A (en) * | 2006-12-15 | 2008-06-18 | 北京大学 | Device for classifying and recognizing network application flow quantity and method thereof |
| CN102611626A (en) * | 2012-03-30 | 2012-07-25 | 北京英诺威尔科技股份有限公司 | System and method for analyzing network flow |
| KR101365496B1 (en) * | 2012-07-06 | 2014-03-12 | 한국외국어대학교 연구산학협력단 | Adaptive Traffic Buffering Method and System in IP Networks |
| CN103763154A (en) * | 2014-01-11 | 2014-04-30 | 浪潮电子信息产业股份有限公司 | Network flow detection method |
| CN105704036A (en) * | 2014-11-27 | 2016-06-22 | 华为技术有限公司 | Message forwarding method, apparatus, and system |
| CN105812277A (en) * | 2014-12-31 | 2016-07-27 | 中国电信股份有限公司 | Access request control method, access request control system and communication equipment |
| CN108418847A (en) * | 2017-02-09 | 2018-08-17 | 中国移动通信集团甘肃有限公司 | A kind of network traffic cache system, method and device |
-
2019
- 2019-06-06 CN CN201910492425.1A patent/CN110149248B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050060428A1 (en) * | 2003-09-11 | 2005-03-17 | International Business Machines Corporation | Apparatus and method for caching lookups based upon TCP traffic flow characteristics |
| CN1809019A (en) * | 2005-01-18 | 2006-07-26 | 北京大学 | Method of implementing quick network message distribution based on adaptive cache mechanism |
| CN101202652A (en) * | 2006-12-15 | 2008-06-18 | 北京大学 | Device for classifying and recognizing network application flow quantity and method thereof |
| CN102611626A (en) * | 2012-03-30 | 2012-07-25 | 北京英诺威尔科技股份有限公司 | System and method for analyzing network flow |
| KR101365496B1 (en) * | 2012-07-06 | 2014-03-12 | 한국외국어대학교 연구산학협력단 | Adaptive Traffic Buffering Method and System in IP Networks |
| CN103763154A (en) * | 2014-01-11 | 2014-04-30 | 浪潮电子信息产业股份有限公司 | Network flow detection method |
| CN105704036A (en) * | 2014-11-27 | 2016-06-22 | 华为技术有限公司 | Message forwarding method, apparatus, and system |
| CN105812277A (en) * | 2014-12-31 | 2016-07-27 | 中国电信股份有限公司 | Access request control method, access request control system and communication equipment |
| CN108418847A (en) * | 2017-02-09 | 2018-08-17 | 中国移动通信集团甘肃有限公司 | A kind of network traffic cache system, method and device |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112261060A (en) * | 2020-10-30 | 2021-01-22 | 四川创智联恒科技有限公司 | Repeated data packet detection method for reliable communication transmission |
| CN112261060B (en) * | 2020-10-30 | 2023-04-07 | 四川创智联恒科技有限公司 | Repeated data packet detection method for reliable communication transmission |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110149248B (en) | 2020-03-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20210152445A1 (en) | Aggregation of select network traffic statistics | |
| CN104488231B (en) | Method, apparatus and system for selectively monitoring flow | |
| JP6217839B2 (en) | Packet processing apparatus, packet processing method and program | |
| Imbrenda et al. | Analyzing cacheable traffic in isp access networks for micro cdn applications via content-centric networking | |
| US20120182891A1 (en) | Packet analysis system and method using hadoop based parallel computation | |
| CN105681125A (en) | Method for counting traffic of virtual machine extranet of cloud platform | |
| CN104378234A (en) | Cross-data-center data transmission processing method and system | |
| CN106874319A (en) | The distributed statistical method and device of click volume | |
| US10491606B2 (en) | Method and apparatus for providing website authentication data for search engine | |
| CN111294849A (en) | Method and device for judging poor Internet access quality of LTE (Long term evolution) user | |
| CN106331172A (en) | Method and device for detecting resources for content distribution network | |
| CN110149248A (en) | A kind of method of express statistic analysis router traffic | |
| CN110266603B (en) | System and method for analyzing network flow of identity authentication service based on HTTP (hyper text transport protocol) | |
| KR100681000B1 (en) | Apparatus and method for measuring per-flow information of traffic | |
| CN101127690A (en) | Identification method for next generation of network service traffic | |
| US9749840B1 (en) | Generating and analyzing call detail records for various uses of mobile network resources | |
| CN106326280B (en) | Data processing method, device and system | |
| Park et al. | Performance improvement of payload signature-based traffic classification system using application traffic temporal locality | |
| KR102423039B1 (en) | Real-time packet data storing method and apparatus for mass network monitoring | |
| Trevisan et al. | WHAT: A big data approach for accounting of modern web services | |
| CN107612831B (en) | Transmission method and device for data message of access source station | |
| Aouini et al. | Towards understanding residential internet traffic: From packets to services | |
| CN115499230A (en) | Network attack detection method and device, equipment and storage medium | |
| EP2605480B1 (en) | Apparatus and method for HTTP analysis | |
| CN108540471A (en) | Mobile application clustering network flow method, computer readable storage medium and terminal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |