CN1822567A - Multi-domain net packet classifying method based on network flow - Google Patents

Abstract

Present invention relates to network filtering and monitoring technology field. It includes following steps: receiving reached network package, collecting network package head information, statistics and normalization network flow property, computing element obtaining network package classifying structure according to configured rule congregation and network flow statistical property, network package classifying unit obtaining network package head information and classifying network package through sorter data structure obtained by calculating unit, transmitting unit transmitting network package in output queue according to classifying result. Present invention is realized based on microprocessor universal platform or network processing unit special platform, combined network flow dynamic statistics property and rule aggregative static structure property, optimizing network package classification method, raising 80-400 per cent average classifying rate than current both abroad and home same class of method, and reducing memory requirements by 30-600 per cent.

Description

The multi-domain net packet classifying method of flow Network Based

Technical field

The present invention relates to network filtering and monitoring technique field.

Background technology

At the 4th layer switch, state-inspection firewall, in the multinomial network filtering and monitoring application based on strategy such as QoS router and load balancing, multi-domain net packet classifying method is the key components and the core technology place of systematic function.Second layer exchange here and traditional is compared with the 3rd layer of route, and Ethernet and IP packet header are not only checked and handled in the classification of multiple domain net bag, and checks and handle the more packet header of upper layer network agreement such as TCP and UDP.Multiple domain net bag classification problem is an example of optimum Match filtering rule problem.For the common net bag classification below four layers, have the territory that 7 territories can be selected to filtering rule: source/purpose network layer address (each 32), source/purpose transport layer port (each 16), COS (8), protocol domain (8) and transport layer protocol sign (8) amount to 120 (bit).In fact, filtering rule does not relate to all these 7 territories, and most network applications are limited in (source/purpose network layer address, source/purpose transport layer port and protocol domain) on five territories.The method that the present invention proposes is applicable to the multiple domain net bag classification problem of any dimension.

Academia and industrial quarters receive much attention multi-domain net packet classifying method in the world always in recent years.The above high-end policy router equipment of gigabit level all adopts special chip solutions such as ASIC/FPGA at present.Because long based on the equipment development cycle of special chip, the volume power consumption is big, upgrade the difficulty height, so the efficiency-cost ratio of this series products is very low, limited being extensive use of of high-performance net bag sorting device.Therefore, research and development novel high-performance net packet classifying method, and, become the only way of promoting high-performance net bag sorting device in conjunction with current up-to-date hardware and software platform.The researcher of companies such as Stanford Univ USA, branch school, Santiago, University of California, Cisco, Lucent and ServGate has carried out many researchs and experiment in this respect, proposed the scheme of a series of solution net bag classification problems, mainly comprised two class methods: based on the lookup method HiCuts of decision tree structure ^{[P.Gupta and N. McKeown, " Packet classification using hierarchical intelligent cuttings, " Proc.Hot Interconnects, 1999]}And HyperCuts ^{[F.Baboescu, and G. Varghese, " Packet classification Using Multidimensional Cutting, " Proc.ACM SIGCOMM, 2003]}, and based on the RFC that inquires about the multiple domain table ^{[P. Gupta and N.McKeown, " Packet classification on multiple fields, " Proc.ACM SIGCOMM 99,1999]}And HSM ^{[Bo Xu, Dongyi Jiang, Jun Li, " HSM:A Fast Packet Classification Algorithm ", The IEEE 19th International Conference on Advanced Information Networking and Applications (AINA), Taiwan, 2005]}Method.These two class methods are utilized the architectural characteristic of classifying rules set from different perspectives, eliminate the redundancy of search volume by multiple heuritic approach, improve net bag classification speed.

Yet existing solution has only been utilized the characteristic of classifying rules itself in design, does not consider the statistical property of network traffics.In the practical application of net bag classification, usually following problem can appear: most net bags only with certain subclass of regular collection in rule match, and have considerable rule only to have the net bag of minute quantity to match.The reason that causes occurring this problem is the tightness of net bag classifying rules, be necessary for each business demand or safety requirements the respective classified strategy is set.Invade Intranet such as worm-type virus, will occur strategy in the firewall policy, filter and also stop the net bag intrusion Intranet that has this worm characteristic in the outer net at this worm port for certain type of taking precautions against outer net.Therefore but the activity of worm only is the some phenomena of certain specific period probably, in the time of the normal operation of network most, only has few net bag and this rule match.Because the formulation of rule often is not optimized at particular network, be difficult to simultaneously be upgraded timely, therefore the rule base that is present in usually in the net bag sorting device all has sizable a part of rule seldom can be matched, that is this part rule seldom participates in actual net bag assorting process when the situation of the normal operation of network, but existence of these rules itself have increased the complexity of net bag classification problem greatly.Existing a few class net packet classifying method is owing to fail to consider this reality relevant with the network flow statistic characteristic, give the net bag complexity that classification itself brings so can't solve network useless rule (rule that seldom matches) under particular flow rate, thereby can't further improve the average transmission rate of network.

Summary of the invention

The object of the present invention is to provide the multi-domain net packet classifying method of the few traffic statistics characteristic Network Based of a kind of average classification speed height and EMS memory occupation.

The invention is characterized in: this method is based on that little processing general-purpose platform or network processing unit dedicated platform realize, has following steps successively:

Step 1. net bag is accepted the unit and is accepted input net bag:

After the network interface card of this unit is accepted the net bag that arrives and is resolved net bag header packet information, the content of output net bag header packet information in the buffer queue of the dynamic buffering memory of this unit again;

Step 2. sampling unit foundation is the fixed sampling interval network packet header that the net bag connects the output of bag unit to be done the network traffics statistics of features, and the information of being added up is done the prior distribution that obtains flow after the normalized according to updated time, the described method of step 2 contains following steps successively:

Step 2.1. sampling:

Processor in the described sampling unit is according to extracting a five-tuple header packet information as a sample in every N of the sampling interval Ns that sets from the network packet header formation of the input net bag that arrives continuously, each sampling sample be one by following five data sets that the territory is formed:

32 potential source IP addresses represent that with int32 sIP int represents integer type, down together;

32 target ip address are represented with int32 dIP;

16 potential source ports are represented with int16 sPort;

16 target ports are represented with int16 dPort;

8 bit protocols are represented with int8 protocol;

Step 2.2. records statistics:

Described statistic comprises:

The number of times that each category-B network segment source IP address occurs in described sample is used int32 sIP[65536] expression, the described category-B network segment is meant preceding 16 of 32 IP addresses, down together;

The number of times that each category-B network segment target ip address occurs in described sample is used int32 dIP[65536] expression;

The number of times that each source port occurs in described sample is used int32 sPort[65536] expression;

The number of times that each target port occurs in described sample is used int32 dPort[65536] expression;

The number of times that each agreement occurs in described sample is used int32 protocol[256] expression;

Step 2.3. normalization:

When the classifier data topology update, the statistic data structure to be carried out normalization obtain prior distribution, the data structure of this prior distribution is following two-dimensional array:

float Prior[i][j]，

Described Prior[i] [j] be each territory normalization statistic, wherein, and i=0,1,2,3,4, once distinguish corresponding source IP address, target ip address, source port, target port and protocol domain; J=0 ..., 65535;

For source IP address:

$\mathrm{Prior}\left[0\right]\left[j\right]=\frac{\mathrm{Stats}.\mathrm{sIP}\left[j\right]}{\underset{k=0}{\overset{65535}{\mathrm{\Σ}}}\mathrm{Stats}.\mathrm{sIP}\left[k\right]},$ Stats represents statistic, down together;

For target ip address:

$\mathrm{Prior}\left[1\right]\left[j\right]=\frac{\mathrm{Stats}.\mathrm{dIP}\left[j\right]}{\underset{k=0}{\overset{65535}{\mathrm{\Σ}}}\mathrm{Stats}.\mathrm{dIP}\left[k\right]},$

For source port:

$\mathrm{Prior}\left[2\right]\left[j\right]=\frac{\mathrm{Stats}.\mathrm{sPort}\left[j\right]}{\underset{k=0}{\overset{65535}{\mathrm{\Σ}}}\mathrm{Stats}.\mathrm{sPort}\left[k\right]},$

For target port:

$\mathrm{Prior}\left[3\right]\left[j\right]=\frac{\mathrm{Stats}.\mathrm{dPort}\left[j\right]}{\underset{k=0}{\overset{65535}{\mathrm{\Σ}}}\mathrm{Stats}.\mathrm{dPort}\left[k\right]},$

For protocol domain:

$\mathrm{Prior}\left[4\right]\left[j\right]=\frac{\mathrm{Stats}.\mathrm{protocol}\left[j\right]}{\underset{k=0}{\overset{65535}{\mathrm{\Σ}}}\mathrm{Stats}.\mathrm{protocol}\left[k\right]},$ When 256≤j≤65535, Prior[4] [j]=0;

Step 3. is provided with classifying rules in computing unit, classifier data structure and normalized regular collection after upgrading by statistic unit output again, and described standardization is meant that the rule of handle friendly mask of generation or asterisk wildcard is with describing between described five-tuple location:

Described computing unit includes a processor and a high speed storing equipment, and the prior distribution information of the related network traffic statistics of this processor is that input links to each other with the corresponding output end of described sampling unit, and this processor also is provided with a regular collection input;

The data structure of described grader is a decision tree structure and comes the storage rule subclass by a linear list on the leaf node of this decision tree; Described computing unit is divided step by step to the search volume when upgrading the grouped data structure, constantly dwindle hunting zone and regular number, the search of whole regular collection is programmed to the search of this regular collection subclass, when the regular number in the searched subclass during less than setting threshold, by adopting linear search technique that current subclass is searched for to obtain final classification results to described linear list, this threshold value is represented with Thresh, is made as 1～10;

Described classifier data topology update method contains step successively:

Step 3.1. initialization:

The setting search space is U, i.e. the possible value space of the mullet of multiple domain net bag header packet information, and the initial ranging space of described five-tuple is followed successively by: [0,2 ³²-1], [0,2 ³²-1], [0,2 ¹⁶-1], [0,2 ¹⁶-1], [0,2 ⁸-1];

The division of setting space promptly is divided into span the set of a plurality of subranges on one or more territory of formulating, then obtain a division to the current search space;

To described computing unit input rule set and system parameters, described system parameters comprises the update cycle to step 3.2., the Thresh in sampling interval and the described classifier data structure, spfac by computer peripheral _MinAnd spfac _MaxDeng adjustable parameter;

A rule in the described regular collection is represented with R, contains:

Source IP address with two 32 integers are described is expressed as int32 sIP[2];

Target ip address with two 32 integers are described is expressed as int32 dIP[2];

Source port with two 16 integers are described is expressed as int16 sPort[2];

Target port with two 16 integers are described is expressed as int16 dPort[2];

Agreement interval with two 8 integers are described is expressed as int8 protocol[2];

The regular priority of representing with int32 priority;

The classification results of representing with int8 action;

Step 3.3. is set as follows described decision tree nodes data structure in described computing unit, i.e. the data structure of grader, and each node comprises:

Partition dimension,

Divide number of times,

Nodal information, the nodal information value of internal node is 0, and the nodal information value of leaf node is regular number, represents with 1～Thresh, and Thresh is the threshold value of leaf node rule number;

Jump cursor: for internal node, the head node of this pointed lower level node; For leaf node, this pointed is deposited the linear list of the rule ID of rule of correspondence subclass;

Step 3.4. creates root node, and strictly all rules all is assigned to root node, root node v _RootExpression;

Step 3.5. sends into current queue Q to described root node ₁

Step 3.6. sends node v from current queue _c

Step 3.7. judges this node v _cIn regular number whether greater than threshold value T (being Thresh);

Step 3.8.

If: node v _cIn regular number greater than (or equaling) threshold value T, then this node v _cBe set to leaf node;

If: node v _cIn regular number less than threshold value T, then carry out next step;

Step 3.9. is according to present node v _cPrior probability and the pairing regular subclass of this node in regular number N adopt any node v that gives in following two kinds of allocation of space functions _cChild node storage allocation space:

First kind:

$\mathrm{SpaceAlloc}\left(v_c\right)=N*\mathrm{spfac}\left(v_c\right)=N*(\mathrm{spfa}{c}_{\min }+(P_{v_c}^l-\min \left(P^l\right)*K)$

Wherein, $P_{v_c}^l=\frac{1}{F}\underset{i=0}{\overset{F}{\mathrm{\Σ}}}\underset{j=a_l}{\overset{b_l}{\mathrm{\Σ}}}\mathrm{Prior}\left[i\right]\left[j\right]$ Be node v _cThe prior distribution probability of flow;

As max (P ^l)-min (P ^l)＞0 o'clock: K=(spfac _Max-spfac _Min)/(max (P ^l)-min (P ^l))

As max (P ^l)-min (P ^l)=0 o'clock: K=0,

Wherein,

N is the regular number of present node;

L represents the degree of depth of the node of decision tree;

[a _i, b _i] be node v _cThe interval of corresponding search volume representative in the i territory;

Max (P ^l) and min (P ^l) be the minimum and maximum prior probability of all node of decision tree l layer;

Spfac _MinAnd spfac _MaxBe used for limiting spfac (v _c) span, generally be made as spfac _Min=1, spfac _Max=4; F=5 represents 5 territories;

Second kind:

$\mathrm{SpaceAlloc}\left(v_c\right)=N*\mathrm{spfac}\left(v_c\right)=N*\mathrm{BOUND}(P_{v_c}^l*D^l*\mathrm{spfa}{c}_{\mathrm{avg}},\mathrm{spfa}{c}_{\min },\mathrm{spfa}{c}_{\max }),$

$\mathrm{BOUND}(a,b,c)=\left\{\begin{array}{cc}a& b\&le;a\&le;c\\ b& a<b\\ c& a>c\end{array}\right.$

$\mathrm{spfa}{c}_{\mathrm{avg}}=\frac{1}{2}({\mathrm{spfac}}_{\min }+{\mathrm{spfac}}_{\max })$

Wherein, D ^lThe expression decision tree is in the number of all node of l layer;

Determining of step 3.10. cutting dimension and division number of times:

At first the internal memory with following formula computing node v uses,

$\mathrm{sm}(v,f)=\underset{i=1}{\overset{\mathrm{numCuts}(v,f)}{\mathrm{\&Sigma;}}}\mathrm{numRules}\left(v_i\right)+\mathrm{numCuts}(v,f)$

Wherein f is by the dimension of cutting; v _iIt is the child node of v; NumRules (v) is the regular number that belongs to this node.By sm (v, f)≤SpaceAlloc (constraint v) down maximization internal memory evaluation function sm (v, f) can be chosen in optimal dividing frequency n umCuts on the f territory (v, f);

After having determined the optimal dividing number of times on each territory, just the search volume in f territory evenly can be divided into numCuts (v, f) part, f=1 ..., F supposes that each subspace falls into N _iIndividual rule, i=1 ..., numCuts (v, f), we select to make so: $\frac{1}{\mathrm{numCuts}(v,f)}\underset{i=1}{\overset{\mathrm{numCuts}(v,f)}{\mathrm{\&Sigma;}}}{N}_i$ Minimized f is as the partition dimension of node v;

Bag classification of step 4. net and classifier data topology update:

Step 4.1. obtains net bag header packet information, i.e. the five-tuple information in net bag packet header as the processor of net bag classification usefulness from described DRAM;

Step 4.2. reads current tree node v _Curr, determine cutting dimension and cutting number of times;

Step 4.3. divides the current search space according to dividing domain that provides in the step 4.2 and division number of times, the subspace in each territory after determining to divide according to the numerical value of corresponding field in the net bag packet header again;

Step 4.4. determine to divide back subspace sequence number under each according to the relative position of the subspace in the step 4.3 in the search volume, and order reads the child node of described current tree node again, determines to divide the pairing child node in each territory, back;

Step 4.5. then changes step 4.6 over to if the child node that step 4.4 finally obtains is a leaf node; Otherwise, be described subspace with the current search spatial update, and current tree node be updated to described child node, repeating step 4.2～4.4;

The list of rules of depositing in the child node described in the step 4.6. order read step 4.4, and each territory carried out commensurate in scope, the rule of all mating with net bag all territories, packet header and having a limit priority is the linear search result, i.e. the rule of the final coupling in net bag packet header;

Step 4.7. carries out the operation to this net bag according to the processing mode of the resulting rule of linear search.

The net packet classifying method of traffic statistics Network Based fully combines the statistical property (dynamic characteristic) of network traffics and the architectural characteristic (static characteristic) of regular collection, proposes the net bag categorizing system method for designing that a whole set of is applicable to various complex network environments.This method is carried out periodic samples, normalization and priori to the flow of the network under dissimilar, the various protocols and is transformed, go in the data structure of various statistical properties with flow with the form introducing net bag grader of prior probability, thereby reach the proportioning of rational space (memory requirements) and time (classification speed), further optimize net packet classifying method, improve net bag sorting device under different network environments and the performance of multiple network in using.The experimental result of system emulation shows that all the present invention has 80%～400% raising than current best net packet classifying method of generally acknowledging both at home and abroad on average classification speed index, and 30%～600% reduction is arranged in memory requirements.Even more important a bit is, the net packet classifying method of traffic statistics characteristic Network Based can adapt to different regular collections and network traffics voluntarily, in multiple different network environments and network application, show the stability that is better than other sorting techniques greatly, thereby make net bag sorting device have good flexibility and practicality.

Table 1 has compared this method and RFC ^{[P.Gupta and N.McKeown, " Packet classification on multiple fields, " Proc.ACM SIGCOMM 99,1999]}, ABV ^{[F.Baboesccu and G.Varghese, Scalable Packet Classification.Proc.ACM SIGCOMM, 2001]}And HiCuts ^{[P.Gupta and N. McKeown, " Packet classification using hierarchical intelligent cuttings, " Proc.Interconnects, 1999]}The EMS memory occupation performance of method.In all regular collections, this method has all showed remarkable advantages.Aspect search time, this method also with current best algorithm on the same order of magnitude.

The space performance of table 1 this method and RFC, ABV and HiCuts method is (unit: KB) relatively

	RFC	ABV	HiCuts	This method
					FW1 FW2 CR1 CR2 SN1	816 910 966 2,220 ∞ *	6.2 34.8 1,077 3,157 2,435	28 129 100 4,235 789	23 57 60 2,031 473

Annotate: FW1 and FW2 are true firewall rule set in the table,

CR1 and CR2 are true core router regular collection, and SN1 is the composition rule set.

This method is by further excavating the data structure feature of network control regular collection, different aspects and different phase in classification problem are used in combination multiple heuristic, can solve the problem that single heuritic approach performance is brought more greatly by the data structure variable effect to a certain extent, thereby the complexity of net packet classifying method under worst case reduced.The introducing of traffic statistics characteristic then is in order to improve the average classification effectiveness of packet classification method.Traffic statistics can be regarded as the learning process to particular network structure and network characteristic.The result of traffic statistics is introduced in the heuristic with the form of prior probability, the result that this method is drawn has the Bayes classifier characteristic, thereby can carry out the self adaptation adjusting according to particular network structure and network traffics characteristic, improve average classification effectiveness.

The above high-end policy router equipment of gigabit level all adopts special chip solutions such as ASIC/FPGA at present.Because long based on the equipment development cycle of special chip, the volume power consumption is big, upgrade the difficulty height, so the efficiency-cost ratio of this series products is very low, limited being extensive use of of high-performance net bag sorting device.And the multi-domain net packet classifying method that the present invention proposes both can have been realized on kinds of platform, comprise based on the general-purpose platform of microprocessor CPU and the dedicated platform of processor NPU Network Based, can guarantee good performance and the adaptability that heterogeneous networks is used again, therefore a whole set of software and hardware system nucleus module that can be used as multi-domain net packet classifying method offers manufacturer to improve the performance based on the net bag sorting device of common treatment applicator platform, significantly reduce the production cost of high-end tactful route and firewall product, thereby promote and quicken the enforcement and the operation of Next Generation Internet.

Description of drawings

Fig. 1. decision tree structure illustrated example of the present invention.Wherein:

------→ expression * next pointer

The order of representation storage

Fig. 2. the theory diagram of system of the present invention.

Fig. 3. the sorting technique program flow chart.

Fig. 4. net bag sort program FB(flow block).

Embodiment

Below in conjunction with Fig. 2, introduce hardware configuration of the present invention:

The A receiving element

Main task: receive the net bag, resolve net bag packet header (five territories), and with net bag and header packet information and net bag content caching to handling in the formation.

Relevant device: network interface card, buffer memory (DRAM)

Input and output: arrive net bag, output packet header net bag content in buffer queue from the network interface card input.

The B sampling unit

Main task: the net bag header packet information that arrives is added up according to the sampling interval, and statistical information is carried out normalized, for computing unit provides the flow prior distribution according to updated time.(annotate: the sampling interval is defined as once samples to every N arrival net bag, promptly each sample record N continuous wherein some header packet information of net bag that arrive, and N gets 1～1000 usually, decides on device throughput; Updated time is defined as the moment that moment that rule base is modified or update cycle arrive, and the update cycle here was set to one day or a week usually, and view network discharge characteristic changes and decides.)

Relevant device: hard disk, processor.

Input and output: from buffer memory, read network packet header (according to the sampling interval), to computing unit output flow prior distribution (according to updated time).

The C computing unit

Main task: classifying rules is set and upgrades the classifier data structure.

Relevant device: processor (CPU), high speed storing equipment (SRAM or Cache)

Input and output:, gather from the control unit input rule from the prior distribution of sampling unit fan-in network traffic statistics; To taxon output category device data structure and normalized regular collection.(annotate: the data structure of grader is decision tree structure and linear list storage rule subclass is arranged on leaf node; The standardization of rule is meant to be described the rule that has mask or asterisk wildcard with five yuan of unified intervals.)

The D taxon

Main task: High Speed Network bag classification.According to net bag header packet information (five-tuple), obtain classification results by the grader search.

Relevant device: processor (CPU), high speed storing equipment (SRAM or Cache)

Input and output: for net bag assorting process, taxon is obtained net bag header packet information from the input buffer queue, and classification results is delivered to output unit; For data updating process, taxon is read in new classifier data structure and new regular collection from computing unit.

The E control unit

Main task: classifying rules and system adjustable parameter (as the update cycle, the adjustable parameter in sampling interval and the classifier data structure etc.) are set.

Relevant device: computer peripheral, monitor, keyboard, mouse etc.

Input and output: the keeper gathers and system parameters to the control unit input rule by ancillary equipment; Control unit exports regular collection and system parameters to computing unit.

The F transmitting element

Main task: send the net bag, according to the net bag in the classification results transmission output queue.

Relevant device: network interface card, buffer memory (DRAM)

Input and output: from computing unit, read classification results, and determine to send or abandon corresponding net bag in the output buffers formation with this.

Then, describe software approach of the present invention in detail:

Software approach mainly comprises three partial contents: network traffics statistics of features method; Classifier data topology update method and net bag lookup method.According to the system constructing order, respectively these three methods are described in detail:

A network traffics statistics of features method is carried out in sampling unit, divides three steps successively:

The A1 sampling:

According to sampling interval Ns, from input rank, extract a header packet information (five-tuple) as a sample in every Ns net bag that arrives continuously.The data structure of each sample is:

Header

{

Int32sIP, dIP; // 32 potential source address and destination addresses

Int16 sPort, dPort; // 16 potential source port and target ports

Int8 protocol; // 8 bit protocols

}；

The A2 record:

Statistic comprises the occurrence number of each category-B network segment (the preceding 16bit of 32bitIP address) source/destination address in sample, the occurrence number and the agreement occurrence number of each source/target port.The statistic data structure is:

Stats

{

Int32sIP[65536], dIP[65536]; // each category-B network segment is added up

Int32sPort[65536], dPort[65536]; // each port is added up

Int32protocol[256]; // each agreement is added up

}；

For example: if certain sample is that { 166.111.0.1 (destination address), 162.105.0.1 (source address), 80 (target ports), 3000 (source ports), 17 (agreements) } then corresponding statistic is changed to:

Stats.dIP[42607]++；//166*256+111＝42607

Stats.sIP[41577]++；//162*256+105＝41577

Stats.dPort[80]++；

Stats.sPort[3000]++；

Stats.protocol[17]++；

A3 normalization:

When the classifier data topology update, statistic Stats structure to be carried out normalization obtain prior distribution Prior, its data structure is a two-dimensional array:

float Prior[5][65536]；

Prior[i wherein] [j] (i=0 ..., 4; J=0 ..., 65535) and be the normalization statistic in each territory, i=0 wherein ..., five territories such as the corresponding source address of 4 difference, destination address, source port, target port, agreement.Prior[3 for example] [80] expression target port is 80 prior probability, computational methods are as follows:

$\mathrm{Prior}\left[3\right]\left[80\right]=\frac{\mathrm{Stats}.\mathrm{dPort}\left[80\right]}{\underset{k=0}{\overset{65535}{\mathrm{\&Sigma;}}}\mathrm{Stats}.\mathrm{dPort}\left[k\right]}$

Annotate: for protocol domain priori Prior[4] [j], Prior[4 when j＞=256] [j]=0.

B classifier data topology update method, in computing unit, carry out:

The classifier data main structure body is a decision tree, by the step by step division of decision tree to the search volume, constantly dwindles hunting zone and regular number, will become the search to its subclass to the search of whole regular collection.When the regular number in the search subset less than threshold value Thresh the time (Thresh=8～10), adopt the method for linear search that current subclass is searched for, obtain final classification results.

The B1 definition:

Search volume U: all possible value space in multiple domain net bag packet header.For the five-tuple classification, the initial ranging space is { [0,2 ³²-1], [0,2 ³²-1], [0,2 ¹⁶-1], [0,2 ¹⁶-1], [0,2 ⁸-1] };

Spatial division: on one or more territory of appointment, span is divided into the set of a plurality of subranges, then obtains a division to the current search space.For example,, then obtain two search subspace U1={[0,2 if on first territory (source address), halve to the initial ranging space ³¹-1], [0,2 ³²-1], [0,2 ¹⁶-1], [0,2 ¹⁶-1], [0,2 ⁸-1] } and U2={[2 ³¹, 2 ³²-1], [0,2 ³²-1], [0,2 ¹⁶-1], [0,2 ¹⁶-1], [0,2 ⁸-1] }.

Input rule set R: each bar rule of input comprises the scope statement in five territories and regular priority (getting the highest rule of priority during a plurality of rule of net bag coupling) and classification results.Regular collection is labeled as R, and each bar rule is labeled as R _iThe regular data organization definition is as follows:

Rule{

Int32sIP[2], dIP[2]; // each address section is described with two 32 integers

Int16sPort[2], dPort[2]; Describe with two 16 integers between // each ports zone

Int8 protocol[2]; // each agreement is interval to be described with two 8 integers

Int32priority; // regular priority

Int8 action; // classification results

}；

The decision tree nodes data structure:

The data structure of each node of decision tree is as follows:

TREENODE{

Unsigned chardimToCut; // partition dimension

Unsigned charnumCuts; // division number of times

Char nodeInfo; // nodal information, internal node are 0, and leaf node is regular number (1～T)

Void * next; // jump cursor is for the head node of this pointed lower level node of internal node;

// for the linear list of this pointed rule ID of leaf node

}；

Decision tree structure figure sees Fig. 1.

B2 allocation of space function:

At first provide the allocation of space function among the HiCuts:

SpaceAlloc(v)＝N*spfac

Wherein, v represents present node, and N is the regular number that comprises in the present node, and spfac is a steric factor, is a constant, is made as 1～4 usually.Spfac is big more, and the available memory space that means is many more, promptly can be careful more to the decomposition in current search space.

Can adopt two kinds of multi-form allocation of space functions in this method, provide the function of these two kinds of forms below respectively:

Form I:

$\mathrm{SpaceAlloc}\left(v\right)=N*\mathrm{spfac}\left(v\right)=N*(\mathrm{spfa}{c}_{\min }+(P_v^l-\min \left(P^l\right)*K)$

$P_v^l=\frac{1}{F}\underset{f=0}{\overset{F}{\mathrm{\&Sigma;}}}\underset{i=a_f}{\overset{b_f}{\mathrm{\&Sigma;}}}\mathrm{Prior}\left[f\right]\left[i\right]$

As max (P ^l)-min (P ^l)＞0 o'clock: K=(spfac _Max-spfac _Min)/(max (P ^l)-min (P ^l))

As max (P ^l)-min (P ^l)=0 o'clock: K=0,

Wherein, N is the regular number of present node; L represents the degree of depth of the node of decision tree; [a _f, b _f] be the interval of the corresponding search volume representative in the f territory of node v (v represents a decision tree nodes); P _v ^lThe prior probability that is called node v; Max (P ^l) and min (P ^l) be the minimum and maximum prior probability of all node of decision tree l layer; Spfac _MinAnd spfac _Max(span v) generally is made as 1～4 to be used for limiting spfac.

Form II:

$\mathrm{SpaceAlloc}\left(v\right)=N*\mathrm{spfac}\left(v\right)$

$=N*\mathrm{BOUND}(P_v^l*D^l*\mathrm{spfa}{c}_{\mathrm{avg}},\mathrm{spfa}{c}_{\min },\mathrm{spfa}{c}_{\max })$

$\mathrm{BOUND}(a,b,c)=\left\{\begin{array}{cc}a& b\&le;a\&le;c\\ b& a<b\\ c& a>c\end{array}\right.$

$\mathrm{spfa}{c}_{\mathrm{avg}}=\frac{1}{2}({\mathrm{spfac}}_{\min }+\mathrm{spfa}{c}_{\max })$

Wherein, D ^lThe expression decision tree is in the number of all node of l layer.

From above two kinds of definition as can be seen, this method is different with HiCuts, and steric factor spfac no longer is a constant, but (v) replaces with function spfac.The allocation of space function of two kinds of forms all is directly proportional with the regular number of present node, and just (value v) constrains in interval [spfac with spfac by different modes _Min, spfac _Max] in.

The selection of B3 cutting dimension and cutting number of times

In case the allocation of space function has been determined available memory space, this method and HiCuts adopt and determine cutting number of times and cutting dimension in a like fashion: at first determine maximum cutting number of times for all possible cutting dimension (being five territories in the five-tuple), then according to the dimension criterion

Estimate that at first possible internal memory uses:

$\mathrm{sm}(v,f)=\underset{i=1}{\overset{\mathrm{numCuts}(v,f)}{\mathrm{\&Sigma;}}}\mathrm{numRules}\left(v_i\right)+\mathrm{numCuts}(v,f)$

Wherein f is by the dimension of cutting; v _iIt is the child node of v; NumRules (v) is the regular number that belongs to this node.By sm (v, f)≤SpaceAlloc (constraint v) down maximization internal memory evaluation function sm (v, f) can be chosen in optimal dividing frequency n umCuts on the f territory (v, f).

After having determined the optimal dividing number of times on each territory, just the search volume in f territory evenly can be divided into numCuts (v, f) part (f=1 ..., F), suppose that each subspace falls into N _iIndividual rule (i=1 ..., numCuts (v, f)), we select to make so:

$\frac{1}{\mathrm{numCuts}(v,f)}\underset{i=1}{\overset{\mathrm{numCuts}(v,f)}{\mathrm{\&Sigma;}}}{N}_i$ Minimized f is as the partition dimension of node v

C net bag lookup method carries out in net bag taxon, and concrete steps are as follows:

The first step: read net bag packet header H five-tuple data, for example: { source address=166.111.8.28, destination address=162.105.38.12, source port=2000, target port=80, agreement=17}. initialization current search space S _CurrBe the global search space, for example: S _Curr={ { 0,2 ³²-1}, { 0,2 ³²-1}, { 0,2 ¹⁶-1}, { 0,2 ¹⁶-1}, { 0,2 ⁸-1}}.

Second step: read current tree node v _Curr(first node is root node v _Root), determine dividing domain DimToCut and divide times N umToCut.

The 3rd step: according to DimToCut and NumToCut to the current search space S _CurrDivide. for example: select to divide the target port territory, dividing number of times is 64 times, and then the search volume is divided into 64 sub spaces in the target port territory, and wherein first subspace is { { 0,2 ³²-1}, { 0,2 ³²-1}, { 0,2 ¹⁶-1}, and 0,1023}, { 0,2 ⁸-1}}, second is { { 0,2 ³²-1}, { 0,2 ³²-1}, { 0,2 ¹⁶-1}, and 1024,2047}, { 0,2 ⁸-1}} ..., the 64th is { { 0,2 ³²-1}, { 0,2 ³²-1}, { 0,2 ¹⁶-1}, and 64512,65535}, { 0,2 ⁸-1}}.Determine subspace S under it according to corresponding field numerical value among the net bag packet header H then _NextFor example: the target port territory is 80 among the H, because 0＜=80＜=1023, then this net bag belongs to the 1st sub spaces { { 0,2 ³²-1}, { 0,2 ³²-1}, { 0,2 ¹⁶-1}, and 0,1023}, { 0,2 ⁸-1}}.

The 4th step: according to S _NextAt S _CurrIn relative position determine subspace sequence number (subspace, 80 place sequence number is 1) under the H, order reads the child node of present node, determines the child node v of H correspondence _Next(being the 1st child node of present node).

The 5th step: if v _NextBe leaf node, forwarded for the 6th step so to; Otherwise establish S _Crr=S _Next, and v _Curr=v _Next, and repeated for second to four step.

The 6th step: read v _NextIn the list of rules RuleList that deposits, and H carried out the linear search (promptly commensurate in scope being carried out in each territory) of rule in this tabulation, all mate with all territories of H and rule with limit priority (is made as R _Opt) for the linear search result, i.e. the final corresponding rule of H.

The 7th step: according to R _OptThe Action attribute carry out operation to net bag under the H.For example abandon or forwarding etc.

Annotate: linear list is searched--and arrive after the leaf node, read the rule ID tabulation and the regular number of leaf node storage, and read rule one by one, carry out the zone comparison with net bag header packet information with this.Because rule ID delegate rules priority, and rule ID is stored in the leaf node according to priority from high in the end, therefore only need return first rule that is matched gets final product, for example net wraps { source address=166.111.8.28, destination address=162.105.38.12, source port=2000, target port=80, agreement=17} and rule { 166.111. ^*, 162.106. ^*, any, 80, TCP} coupling.

Conclusion

This method is by further excavating the data structure feature of network control regular collection, different aspects and different phase in classification problem are used in combination multiple heuristic, can solve the problem that single heuritic approach performance is brought more greatly by the data structure variable effect to a certain extent, thereby the complexity of net packet classifying method under worst case reduced.The introducing of traffic statistics characteristic then is in order to improve the average classification effectiveness of packet classification method.Traffic statistics can be regarded as the learning process to particular network structure and network characteristic.The result of traffic statistics is introduced in the heuristic with the form of prior probability, the result that this method is drawn has the Bayes classifier characteristic, thereby can carry out the self adaptation adjusting according to particular network structure and network traffics characteristic, improve average classification effectiveness.

Application prospect

The above high-end policy router equipment of gigabit level all adopts special chip solutions such as ASIC/FPGA at present.Because long based on the equipment development cycle of special chip, the volume power consumption is big, upgrade the difficulty height, so the efficiency-cost ratio of this series products is very low, limited being extensive use of of high-performance net bag sorting device.And the multi-domain net packet classifying method that this patent proposes both can have been realized on kinds of platform, comprise based on the general-purpose platform of microprocessor CPU and the dedicated platform of processor NPU Network Based, have and to guarantee good performance and the adaptability that heterogeneous networks is used, therefore a whole set of software and hardware system nucleus module that can be used as multi-domain net packet classifying method offers manufacturer to improve the performance based on the net bag sorting device of common treatment applicator platform, significantly reduce the production cost of high-end tactful route and firewall product, thereby promote and quicken the enforcement and the operation of Next Generation Internet.

Claims (1)

1, the multi-domain net packet classifying method of flow Network Based is characterized in that: this method is based on that little processing general-purpose platform or network processing unit dedicated platform realize, has following steps successively:

Step 1. net bag is accepted the unit and is accepted input net bag:

After the network interface card of this unit is accepted the net bag that arrives and is resolved net bag header packet information, the content of output net bag header packet information in the buffer queue of the dynamic buffering memory of this unit again;

Step 2. sampling unit foundation is the fixed sampling interval network packet header that the net bag connects the output of bag unit to be done the network traffics statistics of features, and the information of being added up is done the prior distribution that obtains flow after the normalized according to updated time, the described method of step 2 contains following steps successively:

Step 2.1. sampling:

Processor in the described sampling unit is according to extracting a five-tuple header packet information as a sample in every N of the sampling interval Ns that sets from the network packet header formation of the input net bag that arrives continuously, each sampling sample be one by following five data sets that the territory is formed:

32 potential source IP addresses represent that with int32 sIP int represents integer type, down together;

32 target ip address are represented with int32 dIP;

16 potential source ports are represented with int16 sPort;

16 target ports are represented with int16 dPort;

8 bit protocols are represented with int8 protocol;

Step 2.2. records statistics:

Described statistic comprises:

The number of times that each category-B network segment source IP address occurs in described sample is used int32 sIP[65536] expression, the described category-B network segment is meant preceding 16 of 32 IP addresses, down together;

The number of times that each category-B network segment target ip address occurs in described sample is used int32 dIP[65536] expression;

The number of times that each source port occurs in described sample is used int32 sPort[65536] expression;

The number of times that each target port occurs in described sample is used int32 dPort[65536] expression;

The number of times that each agreement occurs in described sample is used int32 protocol[256] expression;

Step 2.3. normalization:

When the classifier data topology update, the statistic data structure to be carried out normalization obtain prior distribution, the data structure of this prior distribution is following two-dimensional array:

float Prior[i][j]，

Described Prior[i] [j] be each territory normalization statistic, wherein, and i=0,1,2,3,4, once distinguish corresponding source IP address, target ip address, source port, target port and protocol domain; J=0 ..., 65535;

For source IP address:

$\mathrm{Prior}\left[0\right]\left[j\right]=\frac{\mathrm{Stats}.\mathrm{sIP}\left[j\right]}{\underset{k=0}{\overset{65535}{\mathrm{\&Sigma;}}}\mathrm{Stats}.\mathrm{sIP}\left[k\right]},$ Stats represents statistic, down together;

For target ip address:

$\mathrm{Prior}\left[1\right]\left[j\right]=\frac{\mathrm{Stats}.\mathrm{dIP}\left[j\right]}{\underset{k=0}{\overset{65535}{\mathrm{\&Sigma;}}}\mathrm{Stats}.\mathrm{dIP}\left[k\right]},$

For source port:

$\mathrm{Prior}\left[2\right]\left[j\right]=\frac{\mathrm{Stats}.\mathrm{sPort}\left[j\right]}{\underset{k=0}{\overset{65535}{\mathrm{\&Sigma;}}}\mathrm{Stats}.\mathrm{sPort}\left[k\right]},$

For target port:

$\mathrm{Prior}\left[3\right]\left[j\right]=\frac{\mathrm{Stats}.\mathrm{dPort}\left[j\right]}{\underset{k=0}{\overset{65535}{\mathrm{\&Sigma;}}}\mathrm{Stats}.\mathrm{dPort}\left[k\right]},$

For protocol domain:

$\mathrm{Prior}\left[4\right]\left[j\right]=\frac{\mathrm{Stats}.\mathrm{protocol}\left[j\right]}{\underset{k=0}{\overset{65535}{\mathrm{\&Sigma;}}}\mathrm{Stats}.\mathrm{protocol}\left[k\right]},$ When 256≤j≤65535, Prior[4] [j]=0;

Step 3. is provided with classifying rules in computing unit, classifier data structure and normalized regular collection after upgrading by statistic unit output again, and described standardization is meant that the rule of handle friendly mask of generation or asterisk wildcard is with describing between described five-tuple location:

Described computing unit includes a processor and a high speed storing equipment, and the prior distribution information of the related network traffic statistics of this processor is that input links to each other with the corresponding output end of described sampling unit, and this processor also is provided with a regular collection input;

The data structure of described grader is a decision tree structure and comes the storage rule subclass by a linear list on the leaf node of this decision tree; Described computing unit is divided step by step to the search volume when upgrading the grouped data structure, constantly dwindle hunting zone and regular number, the search of whole regular collection is programmed to the search of this regular collection subclass, when the regular number in the searched subclass during less than setting threshold, by adopting linear search technique that current subclass is searched for to obtain final classification results to described linear list, this threshold value is represented with Thresh, is made as 1～10;

Described classifier data topology update method contains step successively:

Step 3.1. initialization:

The setting search space is U, i.e. the possible value space of the mullet of multiple domain net bag header packet information, and the initial ranging space of described five-tuple is followed successively by: [0,2 ³²-1], [0,2 ³²-1], [0,2 ¹⁶-1], [0,2 ¹⁶-1], [0,2 ⁸-1];

The division of setting space promptly is divided into span the set of a plurality of subranges on one or more territory of formulating, then obtain a division to the current search space;

To described computing unit input rule set and system parameters, described system parameters comprises the update cycle to step 3.2., the Thresh in sampling interval and the described classifier data structure, spfac by computer peripheral _MinAnd spfac _MaxDeng adjustable parameter;

A rule in the described regular collection is represented with R, contains:

Source IP address with two 32 integers are described is expressed as int32 sIP[2];

Target ip address with two 32 integers are described is expressed as int32 dIP[2];

Source port with two 16 integers are described is expressed as int16 sPort[2];

Target port with two 16 integers are described is expressed as int16 dPort[2];

Agreement interval with two 8 integers are described is expressed as int8 protocol[2];

The regular priority of representing with int32 priority;

The classification results of representing with int8 action;

Step 3.3. is set as follows described decision tree nodes data structure in described computing unit, i.e. the data structure of grader, and each node comprises:

Partition dimension,

Divide number of times,

Nodal information, the nodal information value of internal node is 0, and the nodal information value of leaf node is regular number, represents with 1～Thresh, and Thresh is the threshold value of leaf node rule number;

Jump cursor: for internal node, the head node of this pointed lower level node; For leaf node, this pointed is deposited the linear list of the rule ID of rule of correspondence subclass;

Step 3.4. creates root node, and strictly all rules all is assigned to root node, root node v _RootExpression;

Step 3.5. sends into current queue Q to described root node ₁

Step 3.6. sends node v from current queue _c

Step 3.7. judges this node v _cIn regular number whether greater than threshold value T (being Thresh);

Step 3.8.

If: node v _cIn regular number greater than (or equaling) threshold value T, then this node v _cBe set to leaf node;

If: node v _cIn regular number less than threshold value T, then carry out next step;

Step 3.9. is according to present node v _cPrior probability and the pairing regular subclass of this node in regular number N adopt any node v that gives in following two kinds of allocation of space functions _cChild node storage allocation space:

First kind:

$\mathrm{SpaceAlloc}\left(v_c\right)=N*\mathrm{spfac}\left(v_c\right)=N*(\mathrm{spfa}{c}_{\min }+(P_{v_c}^l-\min \left(P^l\right)*K)$

Wherein, $P_{v_c}^l=\frac{1}{F}\underset{i=0}{\overset{F}{\mathrm{\&Sigma;}}}\underset{j=a_i}{\overset{b_i}{\mathrm{\&Sigma;}}}\mathrm{Prior}\left[i\right]\left[j\right]$ Be node v _cThe prior distribution probability of flow;

As max (P ^l)-min (P ^l)＞0 o'clock: K=(spfac _Max-spfac _Min)/(max (P ^l)-min (P ^l))

As max (P ^l)-min (P ^l)=0 o'clock: K=0,

Wherein,

N is the regular number of present node;

L represents the degree of depth of the node of decision tree;

[a _i, b _i] be node v _cThe interval of corresponding search volume representative in the i territory;

Max (P ^l) and min (P ^l) be the minimum and maximum prior probability of all node of decision tree l layer;

Spfac _MinAnd spfac _MaxBe used for limiting spfac (v _c) span, generally be made as spfac _Min=1, spfac _Max=4; F=5 represents 5 territories;

Second kind:

$\mathrm{SpaceAlloc}\left(v_c\right)=N*\mathrm{spfac}\left(v_c\right)=N*\mathrm{BOUND}(P_{v_c}^l*D^l*\mathrm{spfa}{c}_{\mathrm{avg}},{\mathrm{spfac}}_{\min },{\mathrm{spfac}}_{\max }),$

$\mathrm{BOUND}(a,b,c)=\left\{\begin{array}{cc}a& b\&le;a\&le;c\\ b& a<b\\ c& a>c\end{array}\right.$

${\mathrm{spfac}}_{\mathrm{avg}}=\frac{1}{2}({\mathrm{spfac}}_{\min }+{\mathrm{spfac}}_{\max })$

Wherein, D ^lThe expression decision tree is in the number of all node of l layer;

Determining of step 3.10. cutting dimension and division number of times:

At first the internal memory with following formula computing node v uses,

$\mathrm{sm}(v,f)=\underset{i=1}{\overset{\mathrm{numCuts}(v,f)}{\mathrm{\&Sigma;}}}\mathrm{numRules}\left(v_i\right)+\mathrm{numCuts}(v,f)$

Wherein f is by the dimension of cutting; v _iIt is the child node of v; NumRules (v) is the regular number that belongs to this node.By sm (v, f)≤SpaceAlloc (constraint v) down maximization internal memory evaluation function sm (v, f) can be chosen in optimal dividing frequency n umCuts on the f territory (v, f);

After having determined the optimal dividing number of times on each territory, just the search volume in f territory evenly can be divided into numCuts (v, f) part, f=1 ..., F supposes that each subspace falls into N _iIndividual rule, i=1 ..., numCuts (v, f), we select to make so:

$\frac{1}{\mathrm{numCuts}(v,f)}\underset{i=1}{\overset{\mathrm{numCuts}(v,f)}{\mathrm{\&Sigma;}}}{N}_i$ Minimized f is as the partition dimension of node v;

Bag classification of step 4. net and classifier data topology update:

Step 4.1. obtains net bag header packet information, i.e. the five-tuple information in net bag packet header as the processor of net bag classification usefulness from described DRAM;

Step 4.2. reads current tree node v _Curr, determine cutting dimension and cutting number of times;

Step 4.3. divides the current search space according to dividing domain that provides in the step 4.2 and division number of times, the subspace in each territory after determining to divide according to the numerical value of corresponding field in the net bag packet header again;

Step 4.4. determine to divide back subspace sequence number under each according to the relative position of the subspace in the step 4.3 in the search volume, and order reads the child node of described current tree node again, determines to divide the pairing child node in each territory, back;

Step 4.5. then changes step 4.6 over to if the child node that step 4.4 finally obtains is a leaf node; Otherwise, be described subspace with the current search spatial update, and current tree node be updated to described child node, repeating step 4.2～4.4;

The list of rules of depositing in the child node described in the step 4.6. order read step 4.4, and each territory carried out commensurate in scope, the rule of all mating with net bag all territories, packet header and having a limit priority is the linear search result, i.e. the rule of the final coupling in net bag packet header;

Step 4.7. carries out the operation to this net bag according to the processing mode of the resulting rule of linear search.

Images