CN1822567A - Multi-domain Network Packet Classification Method Based on Network Flow - Google Patents

Abstract

Present invention relates to network filtering and monitoring technology field. It includes following steps: receiving reached network package, collecting network package head information, statistics and normalization network flow property, computing element obtaining network package classifying structure according to configured rule congregation and network flow statistical property, network package classifying unit obtaining network package head information and classifying network package through sorter data structure obtained by calculating unit, transmitting unit transmitting network package in output queue according to classifying result. Present invention is realized based on microprocessor universal platform or network processing unit special platform, combined network flow dynamic statistics property and rule aggregative static structure property, optimizing network package classification method, raising 80-400 per cent average classifying rate than current both abroad and home same class of method, and reducing memory requirements by 30-600 per cent.

Description

Multi-domain Network Packet Classification Method Based on Network Flow

technical field

The invention relates to the technical field of network filtering and monitoring.

Background technique

In many policy-based network filtering and monitoring applications such as layer 4 switches, state inspection firewalls, QoS routers, and load balancing, multi-domain network packet classification methods are the key components and core technologies of system performance. Compared with traditional layer-2 switching and layer-3 routing, multi-domain network packet classification not only checks and processes Ethernet and IP headers, but also checks and processes headers of higher-level network protocols such as TCP and UDP. The multi-domain network packet classification problem is an example of the best matching filter rule problem. For the common classification of network packets below the fourth layer, there are 7 domains that can be selected as domains for filtering rules: source/destination network layer address (32 bits each), source/destination transport layer port (16 bits each), service type (8 bits), the protocol field (8 bits) and the transport layer protocol flag (8 bits), a total of 120 bits (bits). In fact, filtering rules do not involve all these seven domains, and most network applications are limited to five domains (source/destination network layer address, source/destination transport layer port and protocol domain). The method proposed by the invention is applicable to the multi-domain network packet classification problem of any dimension.

In recent years, multi-domain network packet classification methods have attracted much attention in the international academic and industrial circles. At present, high-end policy routers above Gigabit class all use ASIC/FPGA and other dedicated chip solutions. Due to the long development cycle of equipment based on dedicated chips, large volume and power consumption, and high difficulty in updating, the cost-effectiveness of such products is very low, which limits the widespread use of high-performance network packet classification equipment. Therefore, the research and development of new high-performance network packet classification methods, combined with the latest software and hardware platforms, has become the only way to promote high-performance network packet classification equipment. Researchers from companies such as Stanford University, University of California, San Diego, Cisco, Lucent, and ServGate have conducted many studies and experiments in this area, and proposed a series of solutions to network packet classification problems, mainly including two types of methods: decision-based Tree structure search method HiCuts ^{[P.Gupta and N. McKeown, "Packet classification using hierarchical intelligent cuttings," Proc.Hot Interconnects, 1999]} and HyperCuts ^{[F.Baboescu, and G. Varghese," Packet classification Using Multidimensional Cutting, "Proc.ACM SIGCOMM, 2003]} , and RFC based on querying multi-field tables ^{[P. Gupta and N.McKeown, "Packet classification on multiple fields," Proc.ACM SIGCOMM 99, 1999]} and HSM ^{[Bo Xu, Dongyi Jiang , Jun Li, "HSM: A Fast Packet Classification Algorithm", The IEEE 19th International Conference on Advanced Information Networking and Applications (AINA), Taiwan, 2005]} method. These two types of methods use the structural characteristics of the classification rule set from different angles, and eliminate the redundancy of the search space through a variety of heuristic algorithms to improve the speed of network packet classification.

However, the existing solutions only utilize the characteristics of the classification rules themselves in the design, and do not take into account the statistical characteristics of network traffic. In the practical application of network packet classification, the following problems usually occur: the vast majority of network packets only match the rules in a certain subset of the rule set, and there are quite a few rules that only match a very small number of network packets. The reason for this problem is the strictness of network packet classification rules, and corresponding classification policies must be set for each business requirement or security requirement. For example, in order to prevent a certain type of worm virus from the external network from invading the internal network, a policy targeting the worm port will appear in the firewall policy to filter and prevent network packets with the characteristics of the worm from the external network from intruding into the internal network. However, the activity of worms is likely to be only an isolated phenomenon in a certain period of time. Therefore, in most of the normal operation of the network, only a very small number of network packets will match this rule. Since the formulation of the rules is often not optimized for a specific network, and it is difficult to get timely updates, a considerable part of the rules in the rule base that usually exists in the network packet classification device can rarely be matched, that is, Some rules rarely participate in the actual network packet classification process when the network is running normally, but the existence of these rules itself greatly increases the complexity of the network packet classification problem. Several existing network packet classification methods fail to take into account the reality related to the statistical characteristics of network traffic, so they cannot solve the problem of network packet classification itself caused by useless rules (rules that rarely match) under specific traffic conditions. Therefore, the average transmission rate of the network cannot be further improved.

Contents of the invention

The purpose of the present invention is to provide a multi-domain network packet classification method based on network traffic statistics characteristics with high average classification rate and less memory occupation.

The present invention is characterized in that: the method is realized based on a microprocessing general-purpose platform or a network processor-specific platform, and has the following steps successively:

Step 1. The network packet accepting unit accepts the input network packet:

After the network card of this unit receives the network packet that arrives and parses the network packet header information, then output the content of the network packet header information in the cache queue of the dynamic buffer memory of this unit;

Step 2. The sampling unit performs network traffic characteristics statistics on the network packet header information output by the network packet receiving unit based on a fixed sampling interval, and performs normalization processing on the statistical information according to the update time to obtain the prior distribution of the traffic. The method described in step 2 contains the following steps in turn:

Step 2.1. Sampling:

The processor in the sampling unit extracts a five-tuple packet header information as a sampling sample from every N continuously arriving network packets in the input network packet header information queue according to the set sampling interval Ns, and each sampling sample is is a data group consisting of the following five fields:

32-bit source IP address, represented by int32 sIP, int represents an integer type, the same below;

32-bit target IP address, represented by int32 dIP;

16-bit source port, represented by int16 sPort;

16-bit destination port, represented by int16 dPort;

8-bit protocol, represented by int8 protocol;

Step 2.2. Record statistics:

The statistics include:

The number of times that the source IP address of each class B network segment appears in the sampling sample is represented by int32 sIP[65536]. The class B network segment refers to the first 16 bits of the 32-bit IP address, the same below;

The number of times that the target IP address of each Class B network segment occurs in the sampling sample is represented by int32 dIP[65536];

The number of times each source port appears in the sampling sample, represented by int32 sPort[65536];

The number of times each target port appears in the sampling sample, represented by int32 dPort[65536];

The number of times each protocol appears in the sampling sample is represented by int32 protocol[256];

Step 2.3. Normalization:

When the data structure of the classifier is updated, the prior distribution is obtained by normalizing the statistical data structure, and the data structure of the prior distribution is the following two-dimensional array:

float Prior[i][j],

The Prior[i][j] is a normalized statistic for each domain, where i=0, 1, 2, 3, 4, respectively corresponding to the source IP address, target IP address, source port, target port and protocol field; j=0,...,65535;

For source IP address:

$\mathrm{Prior}\left[0\right]\left[j\right]=\frac{\mathrm{Stats}.\mathrm{sIP}\left[j\right]}{\underset{k=0}{\overset{65535}{\mathrm{\Σ}}}\mathrm{Stats}.\mathrm{sIP}\left[k\right]},$ Stats means statistics, the same below;

For target IP address:

$\mathrm{<span\; class="notranslate">\; Prior</span>}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; Stats</span>}<span\; class="notranslate">\; .</span>\mathrm{<span\; class="notranslate">\; DIP</span>}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; ]</span>}{\underset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; 65535</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\mathrm{<span\; class="notranslate">\; Stats</span>}<span\; class="notranslate">\; .</span>\mathrm{<span\; class="notranslate">\; DIP</span>}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; ]</span>}<span\; class="notranslate">\; ,</span>$

For source port:

$\mathrm{<span\; class="notranslate">\; Prior</span>}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; Stats</span>}<span\; class="notranslate">\; .</span>\mathrm{<span\; class="notranslate">\; sPort</span>}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; ]</span>}{\underset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; 65535</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\mathrm{<span\; class="notranslate">\; Stats</span>}<span\; class="notranslate">\; .</span>\mathrm{<span\; class="notranslate">\; sPort</span>}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; ]</span>}<span\; class="notranslate">\; ,</span>$

For destination port:

$\mathrm{<span\; class="notranslate">\; Prior</span>}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; Stats</span>}<span\; class="notranslate">\; .</span>\mathrm{<span\; class="notranslate">\; dPort</span>}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; ]</span>}{\underset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; 65535</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\mathrm{<span\; class="notranslate">\; Stats</span>}<span\; class="notranslate">\; .</span>\mathrm{<span\; class="notranslate">\; dPort</span>}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; ]</span>}<span\; class="notranslate">\; ,</span>$

For protocol fields:

$\mathrm{Prior}\left[4\right]\left[j\right]=\frac{\mathrm{Stats}.\mathrm{protocol}\left[j\right]}{\underset{k=0}{\overset{65535}{\mathrm{\&Sigma;}}}\mathrm{Stats}.\mathrm{protocol}\left[k\right]},$ When 256≤j≤65535, Prior[4][j]=0;

Step 3. Classification rules are set in the calculation unit, and then the updated classifier data structure and the normalized rule set are output by the statistical unit, and the normalization refers to using the rules of the generation mask or wildcards with the five-tuple Interval description:

The calculation unit contains a processor and a high-speed storage device, the prior distribution information of the processor related to network traffic statistics is that the input end is connected with the corresponding output end of the sampling unit, and the processor is also provided with a rule collection input;

The data structure of the classifier is a decision tree structure and a linear table is used to store rule subsets on the leaf nodes of the decision tree; the calculation unit divides the search space step by step when updating the classification data structure, and continuously Narrow down the search scope and the number of rules, program the search of the entire rule set to the search of the subset of the rule set, when the number of rules in the searched subset is less than the set threshold, by using a linear search method for the linear table The current subset is searched to obtain the final classification result. The threshold is represented by Thresh, which is set to 1-10;

The method for updating the classifier data structure contains the following steps in turn:

Step 3.1. Initialization:

Set the search space as U, which is the possible value space of the barracuda in the multi-domain network packet header information. The initial search space of the five-tuple is: [0, 2 ³² -1], [0, 2 ³² - 1], [0, 2 ¹⁶ -1], [0, 2 ¹⁶ -1], [0, 2 ⁸ -1];

The division of the setting space, that is, dividing the value range into a set of multiple sub-ranges on one or more specified domains, then a division of the current search space is obtained;

Step 3.2. input rule sets and system parameters to the calculation unit by computer peripheral equipment, the system parameters include update cycle, sampling interval and Thresh in the classifier data structure, adjustable parameters such as spfac _min and spfac _max ;

A rule in the rule set is represented by R and contains:

The source IP address described by two 32-bit integers, expressed as int32 sIP[2];

The destination IP address described by two 32-bit integers, expressed as int32 dIP[2];

The source port described by two 16-bit integers, denoted as int16 sPort[2];

Destination port described by two 16-bit integers, denoted as int16 dPort[2];

The protocol interval described by two 8-bit integers is expressed as int8 protocol[2];

Rule priority represented by int32 priority;

The classification result represented by int8 action;

Step 3.3. Set the following decision tree node data structure in the computing unit, i.e. the data structure of the classifier, each node includes:

divide the dimensions,

number of divisions,

Node information, the node information value of the internal node is 0, the node information value of the leaf node is the number of rules, represented by 1~Thresh, Thresh is the threshold value of the number of rules of the leaf node;

Jump pointer: for an internal node, the pointer points to the head node of the lower node; for a leaf node, the pointer points to a linear table storing the rule ID of the corresponding rule subset;

Step 3.4. Create a root node and assign all rules to the root node, which is represented by v _root ;

Step 3.5. Send the root node into the current queue Q ₁ ;

Step 3.6. Send node v _c from the current queue;

Step 3.7. Judging whether the number of rules in the node v _c is greater than the threshold T (ie Thresh);

Step 3.8.

If: the number of rules in the node _vc is greater than (or equal to) the threshold T, then set the node _vc as a leaf node;

If: the number of rules in the node v _c is less than the threshold T, then execute the next step;

Step 3.9. According to the prior probability of the current node _vc and the number N of rules in the rule subset corresponding to the node, use any one of the following two space allocation functions to allocate memory space to the child nodes of the node _vc :

The first:

$\mathrm{<span\; class="notranslate">\; SpaceAlloc</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; spfac</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; *</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; spfa</span>}{\mathrm{<span\; class="notranslate">\; c</span>}}_{\mathrm{<span\; class="notranslate">\; min</span>}}<span\; class="notranslate">\; +</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; P</span>}}_{{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}}^{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; min</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; P</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; K</span>}<span\; class="notranslate">\; )</span>$

in, $P_{v_c}^l=\frac{1}{f}\underset{i=0}{\overset{f}{\mathrm{\&Sigma;}}}\underset{j=a_l}{\overset{b_l}{\mathrm{\&Sigma;}}}\mathrm{Prior}\left[i\right]\left[j\right]$ is the prior distribution probability of the flow of node v _c ;

When max(P ^l )-min(P ^l )>0: K=(spfac _max -spfac _min )/(max(P ^l )-min(P ^l ))

When max(P ^l )-min(P ^l )=0: K=0,

in,

N is the number of rules of the current node;

l represents the depth of the node of the decision tree;

[a _i , b _i ] is the interval represented by the search space corresponding to node v _c in domain i;

max(P ^l ) and min(P ^l ) are the maximum and minimum prior probabilities of all nodes in layer l of the decision tree;

spfac _min and spfac _max are used to limit the value range of spfac(v _c ), generally set spfac _min = 1, spfac _max = 4; F = 5, indicating 5 domains;

The second type:

$\mathrm{<span\; class="notranslate">\; SpaceAlloc</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; spfac</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; BOUND</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; P</span>}}_{{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}}^{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; *</span>{\mathrm{<span\; class="notranslate">\; D.</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; spfa</span>}{\mathrm{<span\; class="notranslate">\; c</span>}}_{\mathrm{<span\; class="notranslate">\; avg</span>}}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; spfa</span>}{\mathrm{<span\; class="notranslate">\; c</span>}}_{\mathrm{<span\; class="notranslate">\; min</span>}}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; spfa</span>}{\mathrm{<span\; class="notranslate">\; c</span>}}_{\mathrm{<span\; class="notranslate">\; max</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>$

$\mathrm{<span\; class="notranslate">\; BOUND</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\left\{\begin{array}{cc}\mathrm{<span\; class="notranslate">\; a</span>}& \mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; c</span>}\\ \mathrm{<span\; class="notranslate">\; b</span>}& \mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; <</span>\mathrm{<span\; class="notranslate">\; b</span>}\\ \mathrm{<span\; class="notranslate">\; c</span>}& \mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; ></span>\mathrm{<span\; class="notranslate">\; c</span>}\end{array}\right.$

$\mathrm{<span\; class="notranslate">\; spfa</span>}{\mathrm{<span\; class="notranslate">\; c</span>}}_{\mathrm{<span\; class="notranslate">\; avg</span>}}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; spfac</span>}}_{\mathrm{<span\; class="notranslate">\; min</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; spfac</span>}}_{\mathrm{<span\; class="notranslate">\; max</span>}}<span\; class="notranslate">\; )</span>$

Among them, D ^l represents the number of all nodes of the decision tree in layer l;

Step 3.10. Determination of segmentation dimension and number of divisions:

First use the following formula to calculate the memory usage of node v,

$\mathrm{<span\; class="notranslate">\; sm</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; numCuts</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\mathrm{<span\; class="notranslate">\; numRules</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; numCuts</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; )</span>$

Where f is the dimension to be split; v _i is a child node of v; numRules(v) is the number of rules belonging to the node. By maximizing the memory estimation function sm(v, f) under the constraint of sm(v, f)≤SpaceAlloc(v), the optimal number of partitions numCuts(v, f) on the f domain can be selected;

After the optimal number of divisions on each domain is determined, the search space of domain f can be evenly divided into numCuts(v, f) parts, f=1,...,F, assuming that each subspace falls into N _i rules , i=1,...,numCuts(v, f), then we choose to make: $\frac{1}{\mathrm{numCuts}(v,f)}\underset{i=1}{\overset{\mathrm{numCuts}(v,f)}{\mathrm{\&Sigma;}}}{N}_i$ The minimized f is used as the division dimension of node v;

Step 4. Network packet classification and classifier data structure update:

Step 4.1. Obtain the network packet header information from the DRAM as the processor used for network packet classification, that is, the five-tuple information of the network packet header;

Step 4.2. Read the current tree node v _curr to determine the segmentation dimension and the number of segmentations;

Step 4.3. divide the current search space according to the division domain and division times given in the step 4.2, then determine the subspace of each domain after division according to the numerical value of the corresponding domain in the network packet header;

Step 4.4. According to the relative position of the subspace in the step 4.3 in the search space, determine the sequence number of each subspace after the division, and then sequentially read the child nodes of the current tree node, and determine the corresponding child nodes of each domain after the division;

Step 4.5. If the child node finally obtained in step 4.4 is a leaf node, then go to step 4.6; otherwise, update the current search space to the subspace, and update the current tree node to the child node, repeat steps 4.2～ 4.4;

Step 4.6. Sequentially read the list of rules stored in the child nodes described in step 4.4, and perform range matching on each domain, and the rule that matches all domains of the network packet header and has the highest priority is the linear search result, namely The final matching rule of the net packet header;

Step 4.7. Perform operations on the network packet according to the processing method of the rules obtained by the linear search.

The network packet classification method based on network traffic statistics fully combines the statistical characteristics of network traffic (dynamic characteristics) and the structural characteristics of rule sets (static characteristics), and proposes a set of design methods for network packet classification systems suitable for various complex network environments. This method performs periodic sampling, normalization and prior transformation on network traffic of different types and protocols, and introduces various statistical characteristics of traffic into the data structure of the network packet classifier in the form of prior probability. , so as to achieve a reasonable ratio of space (memory requirements) and time (classification rate), further optimize the network packet classification method, and improve the performance of network packet classification equipment in different network environments and various network applications. The experimental results of the system simulation all show that the present invention has an 80%-400% increase in the average classification rate index compared with the currently recognized best network packet classification method at home and abroad, but has a 30%-600% improvement in memory requirements. reduce. More importantly, the network packet classification method based on the statistical characteristics of network traffic can adapt to different rule sets and network traffic, and it is much more stable than other classification methods in various network environments and network applications. Therefore, the network packet classification device has good flexibility and practicability.

Table 1 compares this method with RFC ^{[P.Gupta and N.McKeown, "Packet classification on multiple fields," Proc.ACM SIGCOMM 99, 1999]} , ABV ^{[F.Baboesccu and G.Varghese, Scalable Packet Classification.Proc. ACM SIGCOMM, 2001]} and HiCuts ^{[P.Gupta and N. McKeown, "Packet classification using hierarchical intelligent cuttings," Proc.Interconnects, 1999] The} memory usage performance of the method. In all rule sets, the method shows obvious advantages. In terms of search time, this method is also on the same order of magnitude as the current best algorithms.

Table 1 Comparison of spatial performance between this method and RFC, ABV and HiCuts methods (unit: KB) RFCs ABV HiCuts This method FW1FW2CR1CR2SN1 8169109662,220∞ ^* 6.234.81,0773,1572,435 281291004, 235789 2357602,031473

Note: FW1 and FW2 in the table are real firewall rule sets,

CR1 and CR2 are real core router rule sets, and SN1 is a synthetic rule set.

This method further mines the data structure characteristics of the network control rule set, and uses a variety of heuristic methods at different levels and stages of the classification problem, which can solve the problem that the performance of a single heuristic algorithm is greatly affected by the change of the data structure to a certain extent. In order to reduce the complexity of the network packet classification method in the worst case. The introduction of the traffic statistics feature is to improve the average classification efficiency of the packet classification method. Traffic statistics can be regarded as a learning process for a specific network structure and network characteristics. Introducing the results of traffic statistics into the heuristic method in the form of prior probability can make the results obtained by this method have the characteristics of Bayesian classifiers, so that they can be adaptively adjusted according to the specific network structure and network traffic characteristics, and improve Average classification efficiency.

At present, high-end policy routers above Gigabit class all use ASIC/FPGA and other dedicated chip solutions. Due to the long development cycle of equipment based on dedicated chips, large volume and power consumption, and high difficulty in updating, the cost-effectiveness of such products is very low, which limits the widespread use of high-performance network packet classification equipment. And the multi-domain network packet classification method proposed by the present invention can be realized on multiple platforms, including a general-purpose platform based on a microprocessor CPU and a special-purpose platform based on a network processor NPU, which can ensure good performance and be applicable to different networks. Therefore, the entire software and hardware system can be provided to manufacturers as the core module of the multi-domain network packet classification method to improve the performance of network packet classification equipment based on general-purpose processor platforms, and greatly reduce the production cost of high-end policy routing and firewall products. Thereby promoting and accelerating the implementation and operation of the next generation Internet.

Description of drawings

Fig. 1. An example of a decision tree structure diagram according to the present invention. in:

————→Indicates *next pointer

………… Indicates sequential storage

Figure 2. Schematic block diagram of the system according to the invention.

Figure 3. Flow chart of the classification method program.

Figure 4. Flowchart of network packet classification program.

Detailed ways

Below in conjunction with Fig. 2, introduce the hardware structure of the present invention:

A receiving unit

Main tasks: receive network packets, parse network packet headers (five domains), and cache network packets, header information, and network packet content into the processing queue.

Related equipment: network card, cache (DRAM)

Input and output: input and arrive at the network packet from the network card, and output the packet header information network packet content to the cache queue.

B sampling unit

The main task: according to the sampling interval to make statistics on the header information of the arriving network packets, and normalize the statistical information according to the update time, and provide the flow prior distribution for the calculation unit. (Note: The sampling interval is defined as sampling every N arriving network packets, that is, each sampling records the packet header information of one of the N consecutive arriving network packets, and N usually takes 1 to 1000, depending on the throughput of the device ;Update time is defined as the moment when the rule base is modified or when the update cycle arrives, where the update cycle is usually set to one day or one week, depending on the change of network traffic characteristics.)

Related equipment: hard disk, processor.

Input and output: read the network packet header information from the cache (according to the sampling interval), and output the traffic prior distribution to the computing unit (according to the update time).

C computing unit

Main task: set classification rules and update classifier data structure.

Related equipment: processor (CPU), high-speed storage device (SRAM or Cache)

Input and output: input the prior distribution of network traffic statistics from the sampling unit, input the rule set from the control unit; output the classifier data structure and the normalized rule set to the classification unit. (Note: The data structure of the classifier is a decision tree structure and there is a linear table on the leaf nodes to store a subset of rules; the normalization of rules refers to describing the rules with masks or wildcards with a unified five-element interval.)

Taxon D

Main task: high-speed network packet classification. According to the header information (five-tuple) of the network packet, the classification result is obtained by searching through the classifier.

Related equipment: processor (CPU), high-speed storage device (SRAM or Cache)

Input and output: For the network packet classification process, the classification unit obtains the network packet header information from the input buffer queue, and sends the classification result to the output unit; for the data update process, the classification unit reads the new classifier data structure from the computing unit and A new collection of rules.

E control unit

Main task: Set classification rules and system adjustable parameters (such as update cycle, sampling interval, and adjustable parameters in the classifier data structure, etc.).

Related equipment: computer peripherals, monitors, keyboards, mice, etc.

Input and output: the administrator inputs rule sets and system parameters to the control unit through peripheral devices; the control unit outputs rule sets and system parameters to the computing unit.

F sending unit

Main task: Send network packets, and send network packets in the output queue according to the classification results.

Related equipment: network card, cache (DRAM)

Input and output: read the classification results from the computing unit, and decide to send or discard the corresponding network packets in the output buffer queue.

Then, describe the software method that the present invention adopts in detail:

The software method mainly includes three parts: the statistical method of network traffic characteristics; the updating method of classifier data structure and the searching method of network packets. According to the order of system construction, the three methods are described in detail respectively:

The statistical method of A network traffic characteristics is carried out in the sampling unit, which is divided into three steps in turn:

A1 sampling:

According to the sampling interval Ns, a packet header information (five-tuple) is extracted from every Ns continuously arriving network packets in the input queue as a sampling sample. The data structure of each sampling sample is:

Header

{

int32sIP, dIP; //32-bit source and destination addresses

int16 sPort, dPort; //16-bit source port and destination port

int8 protocol; //8-bit protocol

};

A2 records:

Statistics include the number of occurrences of the source/destination address of each class B network segment (the first 16 bits of the 32bit IP address) in the sample, the number of occurrences of each source/destination port, and the number of occurrences of the protocol. The statistics data structure is:

Stats

{

int32sIP[65536], dIP[65536];//Make statistics on each class B network segment

int32sPort[65536], dPort[65536];//Statistics for each port

int32protocol[256];//Statistics for each protocol

};

For example: if a sample is {166.111.0.1 (destination address), 162.105.0.1 (source address), 80 (destination port), 3000 (source port), 17 (protocol)}, the corresponding statistics changes as follows:

Stats.dIP[42607]++; //166*256+111=42607

Stats.sIP[41577]++; //162*256+105=41577

Stats.dPort[80]++;

Stats.sPort[3000]++;

Stats.protocol[17]++;

A3 normalization:

When the data structure of the classifier is updated, the Stats structure of the statistics is normalized to obtain the prior distribution Prior, and its data structure is a two-dimensional array:

float Prior[5][65536];

Among them, Prior[i][j] (i=0,...,4; j=0,...,65535) is the normalized statistics of each domain, where i=0,...,4 correspond to source address, target There are five fields including address, source port, destination port, and protocol. For example, Prior[3][80] indicates the prior probability that the target port is 80, and the calculation method is as follows:

$\mathrm{<span\; class="notranslate">\; Prior</span>}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; 80</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; Stats</span>}<span\; class="notranslate">\; .</span>\mathrm{<span\; class="notranslate">\; dPort</span>}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; 80</span>}<span\; class="notranslate">\; ]</span>}{\underset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; 65535</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\mathrm{<span\; class="notranslate">\; Stats</span>}<span\; class="notranslate">\; .</span>\mathrm{<span\; class="notranslate">\; dPort</span>}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; ]</span>}$

Note: For the prior Prior[4][j] of the protocol domain, when j>=256, Prior[4][j]=0.

B classifier data structure update method, carried out in the calculation unit:

The main body of the classifier data structure is a decision tree. Through the step-by-step division of the search space by the decision tree, the search range and the number of rules are continuously reduced, and the search for the entire rule set is changed to a search for its subset. When the number of rules in the search subset is less than the threshold Thresh (Thresh=8-10), use the linear search method to search the current subset to obtain the final classification result.

B1 definition:

Search space U: all possible value spaces of the multi-area network packet header. For quintuple classification, the initial search space is {[0, 2 ³² -1], [0, 2 ³² -1], [0, 2 ¹⁶ -1], [0, 2 ¹⁶ -1], [0, 2 ⁸ -1]};

Space division: Divide the value range into a set of multiple sub-ranges on one or more specified domains, and then obtain a division of the current search space. For example, for the initial search space, if the first domain (source address) is bisected, two search subspaces U1={[0,2 ³¹ -1], [0,2 ³² -1], [0, 2 ¹⁶ -1], [0, 2 ¹⁶ -1], [0, 2 ⁸ -1]} and U2={[2 ³¹ , 2 ³² -1], [0, 2 ³² -1], [0, 2 ¹⁶ -1], [0, 2 ¹⁶ -1], [0, 2 ⁸ -1]}.

Input rule set R: each input rule includes the scope expression of the five domains, the rule priority (when a network packet matches multiple rules, the rule with the highest priority is taken) and the classification result. The set of rules is marked R, and each rule is marked R _i . The rule data structure is defined as follows:

Rule{

int32sIP[2], dIP[2]; // Each address range is described by two 32-bit integers

int16sPort[2], dPort[2]; // Each port range is described by two 16-bit integers

int8 protocol[2];//Each protocol interval is described by two 8-bit integers

int32priority;//rule priority

int8 action;//classification result

};

Decision tree node data structure:

The data structure of each node of the decision tree is as follows:

TREENODE {

unsigned chardimToCut;//division dimension

unsigned charnumCuts;//number of divisions

char nodeInfo;//Node information, the internal node is 0, and the leaf node is the number of rules (1~T)

void *next;//Jump pointer, for the internal node, the pointer points to the head node of the lower node;

//For leaf nodes, the pointer points to the linear table of rule IDs

};

The decision tree structure diagram is shown in Figure 1.

B2 space allocation function:

First give the space allocation function in HiCuts:

     SpaceAlloc(v)＝N*spfac

Among them, v represents the current node, N is the number of rules contained in the current node, and spfac is the space factor, which is a constant, usually set to 1~4. The larger the spfac, the more memory space is available, that is, the more detailed the decomposition of the current search space can be.

Two different forms of space allocation functions can be used in this method, and the functions of these two forms are given below:

Form I:

$\mathrm{<span\; class="notranslate">\; SpaceAlloc</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; spfac</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; *</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; spfa</span>}{\mathrm{<span\; class="notranslate">\; c</span>}}_{\mathrm{<span\; class="notranslate">\; min</span>}}<span\; class="notranslate">\; +</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; v</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; min</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; P</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; K</span>}<span\; class="notranslate">\; )</span>$

${\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; v</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; f</span>}}\underset{\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; f</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; f</span>}}}{\overset{{\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; f</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\mathrm{<span\; class="notranslate">\; Prior</span>}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ]</span>$

When max(P ^l )-min(P ^l )>0: K=(spfac _max -spfac _min )/(max(P ^l )-min(P ^l ))

When max(P ^l )-min(P ^l )=0: K=0,

Among them, N is the number of rules of the current node; l represents the depth of the node of the decision tree; [a _f , b _f ] is the interval represented by the search space corresponding to the node v (v represents a decision tree node) in the f domain ; P _v ^l is called the prior probability of node v; max(P ^l ) and min(P ^l ) are the maximum and minimum prior probability of all nodes in the first layer of the decision tree; spfac _min and spfac _max are used to limit spfac( The value range of v) is generally set to 1~4.

Form II:

$\mathrm{<span\; class="notranslate">\; SpaceAlloc</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; spfac</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span>$

$<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; BOUND</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; v</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; *</span>{\mathrm{<span\; class="notranslate">\; D.</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; spfa</span>}{\mathrm{<span\; class="notranslate">\; c</span>}}_{\mathrm{<span\; class="notranslate">\; avg</span>}}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; spfa</span>}{\mathrm{<span\; class="notranslate">\; c</span>}}_{\mathrm{<span\; class="notranslate">\; min</span>}}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; spfa</span>}{\mathrm{<span\; class="notranslate">\; c</span>}}_{\mathrm{<span\; class="notranslate">\; max</span>}}<span\; class="notranslate">\; )</span>$

$\mathrm{<span\; class="notranslate">\; BOUND</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\left\{\begin{array}{cc}\mathrm{<span\; class="notranslate">\; a</span>}& \mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; c</span>}\\ \mathrm{<span\; class="notranslate">\; b</span>}& \mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; <</span>\mathrm{<span\; class="notranslate">\; b</span>}\\ \mathrm{<span\; class="notranslate">\; c</span>}& \mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; ></span>\mathrm{<span\; class="notranslate">\; c</span>}\end{array}\right.$

$\mathrm{<span\; class="notranslate">\; spfa</span>}{\mathrm{<span\; class="notranslate">\; c</span>}}_{\mathrm{<span\; class="notranslate">\; avg</span>}}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; spfac</span>}}_{\mathrm{<span\; class="notranslate">\; min</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; spfa</span>}{\mathrm{<span\; class="notranslate">\; c</span>}}_{\mathrm{<span\; class="notranslate">\; max</span>}}<span\; class="notranslate">\; )</span>$

Among them, D ^l represents the number of all nodes of the decision tree in layer l.

It can be seen from the above two definitions that this method is different from HiCuts, the space factor spfac is no longer a constant, but is replaced by the function spfac(v). Both forms of space allocation functions are proportional to the number of rules of the current node, but the value of spfac(v) is constrained in the interval [spfac _min , spfac _max ] in different ways.

B3 Choice of segmentation dimension and number of segmentation

Once the space allocation function determines the available memory space, this method and HiCuts determine the number of splits and the split dimension in the same way: first determine the maximum split for all possible split dimensions (that is, the five fields in the quintuple) times, and then according to the dimension criterion

First estimate possible memory usage:

$\mathrm{<span\; class="notranslate">\; sm</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; numCuts</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\mathrm{<span\; class="notranslate">\; numRules</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; numCuts</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; )</span>$

Where f is the dimension to be split; v _i is a child node of v; numRules(v) is the number of rules belonging to the node. By maximizing the memory estimation function sm(v, f) under the constraint of sm(v, f)≤SpaceAlloc(v), the optimal division times numCuts(v, f) on the f domain can be selected.

After the optimal number of divisions on each domain is determined, the search space of domain f can be evenly divided into numCuts(v, f) parts (f=1,...,F), assuming that each subspace falls into N _i rule (i=1, ..., numCuts(v, f)), then we choose to make:

$\frac{1}{\mathrm{numCuts}(v,f)}\underset{i=1}{\overset{\mathrm{numCuts}(v,f)}{\mathrm{\&Sigma;}}}{N}_i$ The minimized f is used as the partition dimension of node v

The C net bag search method is carried out in the net bag classification unit, and the concrete steps are as follows:

Step 1: Read the H quintuple data of the network packet header, for example: {source address=166.111.8.28, destination address=162.105.38.12, source port=2000, destination port=80, protocol=17}. Initialize the current search The space S _curr is the global search space, for example: S _curr = {{0, 2 ³² -1}, {0, 2 ³² -1}, {0, ^{2 16} -1}, {0, 2 ¹⁶ -1}, {0, 2 ⁸ -1}}.

Step 2: Read the current tree node v _curr (the first node is the root node v _root ), and determine the division domain DimToCut and division times NumToCut.

Step 3: Divide the current search space S _curr according to DimToCut and NumToCut. For example: choose to divide the target port domain, and the number of divisions is 64 times, then the search space is divided into 64 subspaces in the target port domain, and the first subspace The spaces are {{0, 2 ³² -1}, {0, ^{2 32} -1}, {0, 2 ¹⁶ -1}, {0, 1023}, {0, 2 ⁸ -1}} and the second one is {{0, ^{2 32} -1}, {0, 2 ³² -1}, {0, ^{2 16} -1}, {1024, 2047}, {0, 2 ⁸ -1}}, ..., the 64th is {{0, 2 ³² -1}, {0, 2 ³² -1}, {0, 2 ¹⁶ -1}, {64512, 65535}, {0, ^{2 8} -1}}. Then determine the subspace S _next to which it belongs according to the value of the corresponding field in the header H of the network packet. For example: the destination port field in H is 80, since 0<=80<=1023, the network packet belongs to the first subspace {{0, 2 ³² -1}, {0, 2 ³² -1}, {0, 2 ¹⁶ -1}, {0, 1023}, {0, 2 ⁸ -1}}.

Step 4: determine the subspace sequence number of H according to the relative position of S _next in S _curr (the subspace sequence number of 80 is 1), read the child nodes of the current node in sequence, and determine the corresponding child node v _{next of} H (that is, the current node's first child).

Step 5: If v _next is a leaf node, go to step 6; otherwise, set S _crr =S _next , and v _curr =v _next , and repeat steps 2 to 4.

Step 6: Read the rule list RuleList stored in v _next , and perform a linear search for the rules in the list for H (that is, perform range matching on each domain), and match all domains of H and have the highest priority rule (set as R _opt ) is the linear search result, that is, the final corresponding rule of H.

Step 7: Perform operations on the network package to which H belongs according to the Action attribute of R _opt . Such as discarding or forwarding, etc.

Note: Linear table lookup--after arriving at the leaf node, read the rule ID list and the number of rules stored in the leaf node, and use this to read the rules one by one, and compare the area with the header information of the network packet. Since the rule ID represents the priority of the rule, and the rule ID is stored in the leaf node according to the priority from high to low, it is only necessary to return the first matched rule, for example, net packet {source address=166.111.8.28, target Address=162.105.38.12, source port=2000, destination port=80, protocol=17} matches the rule {166.111. ^* , 162.106. ^* , any, 80, TCP}.

in conclusion

This method further mines the data structure characteristics of the network control rule set, and uses a variety of heuristic methods at different levels and stages of the classification problem, which can solve the problem that the performance of a single heuristic algorithm is greatly affected by the change of the data structure to a certain extent. In order to reduce the complexity of the network packet classification method in the worst case. The introduction of the traffic statistics feature is to improve the average classification efficiency of the packet classification method. Traffic statistics can be regarded as a learning process for a specific network structure and network characteristics. Introducing the results of traffic statistics into the heuristic method in the form of prior probability can make the results obtained by this method have the characteristics of Bayesian classifiers, so that they can be adaptively adjusted according to the specific network structure and network traffic characteristics, and improve Average classification efficiency.

Application prospects

At present, high-end policy routers above Gigabit class all use ASIC/FPGA and other dedicated chip solutions. Due to the long development cycle of equipment based on dedicated chips, large volume and power consumption, and high difficulty in updating, the cost-effectiveness of such products is very low, which limits the widespread use of high-performance network packet classification equipment. The multi-domain network packet classification method proposed in this patent can be implemented on a variety of platforms, including a general-purpose platform based on a microprocessor CPU and a dedicated platform based on a network processor NPU, which can ensure good performance and be applicable to different network applications. Therefore, the entire software and hardware system can be provided to manufacturers as the core module of the multi-domain network packet classification method to improve the performance of network packet classification equipment based on general-purpose processor platforms, and greatly reduce the production cost of high-end policy routing and firewall products. Thereby promoting and accelerating the implementation and operation of the next generation Internet.

Claims (1)

1, the multi-domain net packet classifying method of flow Network Based is characterized in that: this method is based on that little processing general-purpose platform or network processing unit dedicated platform realize, has following steps successively:

Step 1. net bag is accepted the unit and is accepted input net bag:

After the network interface card of this unit is accepted the net bag that arrives and is resolved net bag header packet information, the content of output net bag header packet information in the buffer queue of the dynamic buffering memory of this unit again;

Step 2. sampling unit foundation is the fixed sampling interval network packet header that the net bag connects the output of bag unit to be done the network traffics statistics of features, and the information of being added up is done the prior distribution that obtains flow after the normalized according to updated time, the described method of step 2 contains following steps successively:

Step 2.1. sampling:

Processor in the described sampling unit is according to extracting a five-tuple header packet information as a sample in every N of the sampling interval Ns that sets from the network packet header formation of the input net bag that arrives continuously, each sampling sample be one by following five data sets that the territory is formed:

32 potential source IP addresses represent that with int32 sIP int represents integer type, down together;

32 target ip address are represented with int32 dIP;

16 potential source ports are represented with int16 sPort;

16 target ports are represented with int16 dPort;

8 bit protocols are represented with int8 protocol;

Step 2.2. records statistics:

Described statistic comprises:

The number of times that each category-B network segment source IP address occurs in described sample is used int32 sIP[65536] expression, the described category-B network segment is meant preceding 16 of 32 IP addresses, down together;

The number of times that each category-B network segment target ip address occurs in described sample is used int32 dIP[65536] expression;

The number of times that each source port occurs in described sample is used int32 sPort[65536] expression;

The number of times that each target port occurs in described sample is used int32 dPort[65536] expression;

The number of times that each agreement occurs in described sample is used int32 protocol[256] expression;

Step 2.3. normalization:

When the classifier data topology update, the statistic data structure to be carried out normalization obtain prior distribution, the data structure of this prior distribution is following two-dimensional array:

float Prior[i][j]，

Described Prior[i] [j] be each territory normalization statistic, wherein, and i=0,1,2,3,4, once distinguish corresponding source IP address, target ip address, source port, target port and protocol domain; J=0 ..., 65535;

For source IP address:

$\mathrm{Prior}\left[0\right]\left[j\right]=\frac{\mathrm{Stats}.\mathrm{sIP}\left[j\right]}{\underset{k=0}{\overset{65535}{\mathrm{\&Sigma;}}}\mathrm{Stats}.\mathrm{sIP}\left[k\right]},$ Stats represents statistic, down together;

For target ip address:

$\mathrm{Prior}\left[1\right]\left[j\right]=\frac{\mathrm{Stats}.\mathrm{dIP}\left[j\right]}{\underset{k=0}{\overset{65535}{\mathrm{\&Sigma;}}}\mathrm{Stats}.\mathrm{dIP}\left[k\right]},$

For source port:

$\mathrm{Prior}\left[2\right]\left[j\right]=\frac{\mathrm{Stats}.\mathrm{sPort}\left[j\right]}{\underset{k=0}{\overset{65535}{\mathrm{\&Sigma;}}}\mathrm{Stats}.\mathrm{sPort}\left[k\right]},$

For target port:

$\mathrm{Prior}\left[3\right]\left[j\right]=\frac{\mathrm{Stats}.\mathrm{dPort}\left[j\right]}{\underset{k=0}{\overset{65535}{\mathrm{\&Sigma;}}}\mathrm{Stats}.\mathrm{dPort}\left[k\right]},$

For protocol domain:

$\mathrm{Prior}\left[4\right]\left[j\right]=\frac{\mathrm{Stats}.\mathrm{protocol}\left[j\right]}{\underset{k=0}{\overset{65535}{\mathrm{\&Sigma;}}}\mathrm{Stats}.\mathrm{protocol}\left[k\right]},$ When 256≤j≤65535, Prior[4] [j]=0;

Step 3. is provided with classifying rules in computing unit, classifier data structure and normalized regular collection after upgrading by statistic unit output again, and described standardization is meant that the rule of handle friendly mask of generation or asterisk wildcard is with describing between described five-tuple location:

Described computing unit includes a processor and a high speed storing equipment, and the prior distribution information of the related network traffic statistics of this processor is that input links to each other with the corresponding output end of described sampling unit, and this processor also is provided with a regular collection input;

The data structure of described grader is a decision tree structure and comes the storage rule subclass by a linear list on the leaf node of this decision tree; Described computing unit is divided step by step to the search volume when upgrading the grouped data structure, constantly dwindle hunting zone and regular number, the search of whole regular collection is programmed to the search of this regular collection subclass, when the regular number in the searched subclass during less than setting threshold, by adopting linear search technique that current subclass is searched for to obtain final classification results to described linear list, this threshold value is represented with Thresh, is made as 1～10;

Described classifier data topology update method contains step successively:

Step 3.1. initialization:

The setting search space is U, i.e. the possible value space of the mullet of multiple domain net bag header packet information, and the initial ranging space of described five-tuple is followed successively by: [0,2 ³²-1], [0,2 ³²-1], [0,2 ¹⁶-1], [0,2 ¹⁶-1], [0,2 ⁸-1];

The division of setting space promptly is divided into span the set of a plurality of subranges on one or more territory of formulating, then obtain a division to the current search space;

To described computing unit input rule set and system parameters, described system parameters comprises the update cycle to step 3.2., the Thresh in sampling interval and the described classifier data structure, spfac by computer peripheral _MinAnd spfac _MaxDeng adjustable parameter;

A rule in the described regular collection is represented with R, contains:

Source IP address with two 32 integers are described is expressed as int32 sIP[2];

Target ip address with two 32 integers are described is expressed as int32 dIP[2];

Source port with two 16 integers are described is expressed as int16 sPort[2];

Target port with two 16 integers are described is expressed as int16 dPort[2];

Agreement interval with two 8 integers are described is expressed as int8 protocol[2];

The regular priority of representing with int32 priority;

The classification results of representing with int8 action;

Step 3.3. is set as follows described decision tree nodes data structure in described computing unit, i.e. the data structure of grader, and each node comprises:

Partition dimension,

Divide number of times,

Nodal information, the nodal information value of internal node is 0, and the nodal information value of leaf node is regular number, represents with 1～Thresh, and Thresh is the threshold value of leaf node rule number;

Jump cursor: for internal node, the head node of this pointed lower level node; For leaf node, this pointed is deposited the linear list of the rule ID of rule of correspondence subclass;

Step 3.4. creates root node, and strictly all rules all is assigned to root node, root node v _RootExpression;

Step 3.5. sends into current queue Q to described root node ₁

Step 3.6. sends node v from current queue _c

Step 3.7. judges this node v _cIn regular number whether greater than threshold value T (being Thresh);

Step 3.8.

If: node v _cIn regular number greater than (or equaling) threshold value T, then this node v _cBe set to leaf node;

If: node v _cIn regular number less than threshold value T, then carry out next step;

Step 3.9. is according to present node v _cPrior probability and the pairing regular subclass of this node in regular number N adopt any node v that gives in following two kinds of allocation of space functions _cChild node storage allocation space:

First kind:

$\mathrm{SpaceAlloc}\left(v_c\right)=N*\mathrm{spfac}\left(v_c\right)=N*(\mathrm{spfa}{c}_{\min }+(P_{v_c}^l-\min \left(P^l\right)*K)$

Wherein, $P_{v_c}^l=\frac{1}{F}\underset{i=0}{\overset{F}{\mathrm{\&Sigma;}}}\underset{j=a_i}{\overset{b_i}{\mathrm{\&Sigma;}}}\mathrm{Prior}\left[i\right]\left[j\right]$ Be node v _cThe prior distribution probability of flow;

As max (P ^l)-min (P ^l)＞0 o'clock: K=(spfac _Max-spfac _Min)/(max (P ^l)-min (P ^l))

As max (P ^l)-min (P ^l)=0 o'clock: K=0,

Wherein,

N is the regular number of present node;

L represents the degree of depth of the node of decision tree;

[a _i, b _i] be node v _cThe interval of corresponding search volume representative in the i territory;

Max (P ^l) and min (P ^l) be the minimum and maximum prior probability of all node of decision tree l layer;

Spfac _MinAnd spfac _MaxBe used for limiting spfac (v _c) span, generally be made as spfac _Min=1, spfac _Max=4; F=5 represents 5 territories;

Second kind:

$\mathrm{SpaceAlloc}\left(v_c\right)=N*\mathrm{spfac}\left(v_c\right)=N*\mathrm{BOUND}(P_{v_c}^l*D^l*\mathrm{spfa}{c}_{\mathrm{avg}},{\mathrm{spfac}}_{\min },{\mathrm{spfac}}_{\max }),$

$\mathrm{BOUND}(a,b,c)=\left\{\begin{array}{cc}a& b\&le;a\&le;c\\ b& a<b\\ c& a>c\end{array}\right.$

${\mathrm{spfac}}_{\mathrm{avg}}=\frac{1}{2}({\mathrm{spfac}}_{\min }+{\mathrm{spfac}}_{\max })$

Wherein, D ^lThe expression decision tree is in the number of all node of l layer;

Determining of step 3.10. cutting dimension and division number of times:

At first the internal memory with following formula computing node v uses,

$\mathrm{sm}(v,f)=\underset{i=1}{\overset{\mathrm{numCuts}(v,f)}{\mathrm{\&Sigma;}}}\mathrm{numRules}\left(v_i\right)+\mathrm{numCuts}(v,f)$

Wherein f is by the dimension of cutting; v _iIt is the child node of v; NumRules (v) is the regular number that belongs to this node.By sm (v, f)≤SpaceAlloc (constraint v) down maximization internal memory evaluation function sm (v, f) can be chosen in optimal dividing frequency n umCuts on the f territory (v, f);

After having determined the optimal dividing number of times on each territory, just the search volume in f territory evenly can be divided into numCuts (v, f) part, f=1 ..., F supposes that each subspace falls into N _iIndividual rule, i=1 ..., numCuts (v, f), we select to make so:

$\frac{1}{\mathrm{numCuts}(v,f)}\underset{i=1}{\overset{\mathrm{numCuts}(v,f)}{\mathrm{\&Sigma;}}}{N}_i$ Minimized f is as the partition dimension of node v;

Bag classification of step 4. net and classifier data topology update:

Step 4.1. obtains net bag header packet information, i.e. the five-tuple information in net bag packet header as the processor of net bag classification usefulness from described DRAM;

Step 4.2. reads current tree node v _Curr, determine cutting dimension and cutting number of times;

Step 4.3. divides the current search space according to dividing domain that provides in the step 4.2 and division number of times, the subspace in each territory after determining to divide according to the numerical value of corresponding field in the net bag packet header again;

Step 4.4. determine to divide back subspace sequence number under each according to the relative position of the subspace in the step 4.3 in the search volume, and order reads the child node of described current tree node again, determines to divide the pairing child node in each territory, back;

Step 4.5. then changes step 4.6 over to if the child node that step 4.4 finally obtains is a leaf node; Otherwise, be described subspace with the current search spatial update, and current tree node be updated to described child node, repeating step 4.2～4.4;

The list of rules of depositing in the child node described in the step 4.6. order read step 4.4, and each territory carried out commensurate in scope, the rule of all mating with net bag all territories, packet header and having a limit priority is the linear search result, i.e. the rule of the final coupling in net bag packet header;

Step 4.7. carries out the operation to this net bag according to the processing mode of the resulting rule of linear search.

Images