CN1287570C - High speed filtering and stream dividing method for keeping connection features - Google Patents
High speed filtering and stream dividing method for keeping connection features Download PDFInfo
- Publication number
- CN1287570C CN1287570C CNB2004100171891A CN200410017189A CN1287570C CN 1287570 C CN1287570 C CN 1287570C CN B2004100171891 A CNB2004100171891 A CN B2004100171891A CN 200410017189 A CN200410017189 A CN 200410017189A CN 1287570 C CN1287570 C CN 1287570C
- Authority
- CN
- China
- Prior art keywords
- packet
- rule
- address
- filtration
- port
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Abstract
The present invention relates to a high-speed filtering and stream splitting method for maintaining connection features, which is used in the technical field of network information technology. Firstly, the present invention sets up an output port classification rule and a filtering and stream splitting rule, and then the data filtering and stream splitting treatment is carried out, IP data packets are extracted from original data packets received from networks through the protocol analysis, the relevant data for post-treatment is partitioned to each output port group according to the set-up filtering and stream splitting rule, the Hash operation is carried out for the address of a data packet partitioned to each group and the port information, the modular operation is carried our between the Hash value and the total port number of ports of the group, and the result is the output port sequence number corresponding to the data packet in the group. The present invention can effectively meet the requirements of forwarding mass data streams at high speed and clustering the mass data streams when an intrusion detection system, a flow rate statistical system, a content audit system, a load balancing system and other application systems are constructed under the condition of high-speed backbone networks.
Description
Technical field
What the present invention relates to is a kind of broadband networks high flow capacity data processing method, and particularly a kind of high-rate fitration shunt method that is applicable to the maintenance connection performance of broadband networks data is used for network information technology field.
Background technology
Nearly 2 years, the construction of China's backbone network and metropolitan area network reached suitable scale, and the bandwidth of backbone network is usually more than 2.5G.The network security product of present stage such as fire compartment wall, intruding detection system, network security audit etc. are that can the assurance system make correct effectively the analysis and the primary prerequisite of judging to the security incident that takes place in the network to the handling property of network packet.There is significant limitation in existing safety product for the mass data processing demand of 2.5G, the above broadband network of 10G:
1, rely on CPU to realize the processing of network data based on products such as the IDS of unicomputer mode, auditing systems, but there is the restriction on quantity and the speed in parts such as internal memory, CPU, can not unrestrictedly support the processing demands of big data quantity.Therefore, this type of safety product can't be directly used in the above express network of 2.5G.
2, the load-balancing device based on seven layer-switching technologies is present ideal style at the high flow capacity network data processing with combining of all kinds of IDS, auditing system.But present load balance process mechanism is to preserve the address information that TCP connects with the form of connection table, and along with the expansion of network size and complicated, the connection table will be exponential increase with the expansion of network size.In addition, also exist a large amount of useless falsenesses to connect on network, the continuous expansion of connection table data scale also can strengthen the resource consumption in the process of tabling look-up, even adopt high performance NPU also to be difficult to adapt to the processing demands that super large connects data.
3, the connection table searches problem fast and has limited the raising of data-handling capacity.Employing software realizes that data retrieval speed is slower; Based on the implementation of hardware such as relevance memory (CAM), carrying cost is too high, can not satisfy the storage of unlimited huge connection table and searches fast.
The Secure Application switch FireProof of Israel Radware company can do the load balancing of flow to many fire compartment wall FireWall, VPN, IDS.Used multiple traditional load-balancing algorithm among the FireProof, as repeating query algorithm, weighting minimal user method, the minimum byte method of weighting, weighting minimal data bag method.In order to guarantee to handle consistent through the load-balancing algorithm data result to the bi-directional data bag that belongs to same connection, in FireProof, need to follow the tracks of and write down each and connect two-way data message, guarantee that by detecting these information the subsequent communications data can obtain identical load balance process.Network for high data traffic, FireProof adopts traditional Hash table method, for being connected in the Hash table, each sets up a record, when the next message of this connection arrives, from the Hash table, can obtain OPADD and port, message is passed to former selected fire compartment wall this connection distribution.When connection termination or overtime, FireProof is with information deletion from the Hash table of this connection.This product weak point is: in order to guarantee belonging to the bi-directional data bag uniformity of treatment of same connection, must in this product every linkage record information relevant with processing, along with the continuous expansion of network size, the storage of huge day by day link information and the data processing demand that searches problem and can't satisfy the high-speed wideband network.
The broadband development of the Internet makes network traffics increase rapidly, in the face of the mass data on core net, the backbone network, only can satisfy the network security product of the data-handling capacity of 100,000,000 networks, gigabit networking, owing to can't carry out effective message collection and processing to the data of high flow capacity, cause data acquisition disappearance and occur failing to report, become the main bottleneck that these safety products are used in the telecommunications backbone network field.
Summary of the invention
Purpose of the present invention mainly is at these defectives of products such as fire compartment wall, intrusion detection, security audit mass data processing mechanism in the high-speed backbone network is used, proposes a kind of high-rate fitration shunt method that keeps connection performance.The branch flow algorithm that it is handled based on Hash according to the combination of user-defined filtration shunting rule, will be from the raw data packets that network receives to the significant data in real time of Business Processing effectively, and keep that being forwarded to corresponding rear end cluster treatment system carries out further Business Processing with connecting, solve effectively under the high-speed backbone environment, when making up intrusion detection, traffic statistics, content auditing, network monitor, load balancing and other application system, mass data flow is carried out transmitting at a high speed and the demand of cluster processing.
The present invention is achieved by the following technical solutions, at first is the output port rule of classification to be set and to filter the shunting rule.The configuration of output port rule of classification is at first each output port to be divided into groups according to the Business Processing type of the backend application system of correspondence, and then determines the allocation proportion of each port processing data packets flow in this group according to the disposal ability of the back-end system of each port correspondence in the group.The configuration of filtering the shunting rule is according to IP address information or special field different packets to be divided in the middle of each grouping.The processing procedure of data filter shunting is at first to pass through protocal analysis, from the raw data packets that network receives, extract the IP packet, from a large amount of initial data, filter and the irrelevant part of subsequent treatment according to the filtration shunting rule that sets, related data is divided in the different output port groupings.The address and the port information of packet that will be divided into then in each group carries out the Hash computing, the port sum delivery that hash value is comprised with this grouping again, and the result who obtains is exactly the output port sequence number of this packet correspondence in affiliated grouping.
Below the present invention is further illustrated, particular content is as follows:
1. rule is provided with: at first rule will be set before initial data is filtered the shunting processing.Rule is divided into two parts, and a part is an output port packet configuration rule, and another part is to filter the shunting rule.
■ output port packet configuration rule
The advantage of output port being carried out packet transaction is for the complement filter rule, the output port that corresponding Business Processing demand is close is with the centralized and unified processing of mode of grouping, separate the mode of independent process with each port and compare, the processing procedure of packet mode is more simple efficient.Following several principles is mainly followed in the grouping of output port:
(1) grouping of output port at first according to the business demand of back-end processing system, is divided into groups according to different Business Processing types.But because system may have overlappingly respectively in the filtering rule attribute mutually with a plurality of systems, therefore same output port can appear in the different groupings.
(2) in each packets inner, consider that there is unbalanced situation in each system processing power, can realize balanced the distribution in order to guarantee data processing task, same output port can repeatedly appear in same group, promptly determines the allocation proportion of each port data packet flow in this group in grouping according to the data-handling capacity of the back-end system of each port correspondence.
(3) in the group each output port be distributed in the proof load equilibrium time, must guarantee that also all packets of same TCP connection (two-way) must be forwarded on the same output port, so that rear end gathering and reduce to received data.
■ filters the shunting rule
The setting of filtering the shunting rule comprises dual mode: a kind of is IP address and port information with packet,, rule is arranged in the filtration shunting rule list based on address information as direct monitored object as source IP address, source IP mask, source port, purpose IP address, purpose IP mask, destination interface based on these IP address informations.Another kind is based on the regular set-up mode of special field information, and promptly the user only need be provided with the side-play amount of special field, this field, the length of this field, the matching content and the corresponding branch Flow Behavior that filters of this field.
It is as follows to shunt regular tableau format based on the filtration of the IP address of packet and port information:
| Regular number | Source IP address | Source IP mask | Source port | Purpose IP address | Purpose IP mask | Destination interface | Filter and divide Flow Behavior |
It is as follows to shunt regular tableau format based on the filtration of special field information:
| Regular number | The fields offset amount | Field length | Matching content | Filter and divide Flow Behavior |
Wherein:
Regular number: the unique identification sequence number that each is regular
Source/purpose IP address: the source end of data packet transmission and the IP address value of destination
Source/purpose IP mask: the source end of data packet transmission and the subnet mask of destination IP field
Source/destination interface: the source end of data packet transmission and the port numbers of destination
Fields offset amount: the side-play amount of the special field that the needs that begin to calculate from the IP packet content mate
Field length: the length that needs the special field of coupling
Matching content: the matching value of special field, these special field can comprise as the receiver in URL address, the Email mail and addresser's address field.
Filter and divide Flow Behavior: " 0 " is represented this data packet discarding; " 1 " expression is forwarded to group 1 with this packet; " 2 " expression is forwarded to group 2 with this packet; " N " expression is forwarded to group N with this datagram; " 1 " represents that this rule is invalid, is applicable to that certain bar rule is not easy to delete immediately but current situation about cannot use again.All fields except that filtering branch Flow Behavior field all are 0 in the shunting rule list if filter, and represent that then this rule is applicable to all packets.
Filter the setting of shunting rule and will shunt the foundation of handling as filtration after this.As the packet that receives from network and one during based on the filtration shunting rule match of the IP address of packet and port information, packet will divide Flow Behavior to be forwarded to corresponding grouping according to the filtration that this rule is set; If when from the packet that network receives, comprising the data identical with the special field value that sets in advance, the IP address information of this packet will be extracted, divide Flow Behavior together with the corresponding filtration that sets in advance, form a dynamic filtration shunting rule, write based on the IP address of packet and the filtration of port information and shunt in the rule list based on IP address and port information.All subsequent packet that TCP/UDP under this packet and this packet connects all will divide Flow Behavior to be forwarded to corresponding grouping according to the filtration that this rule is set.The filtration shunting rule that forms based on the feature field coupling has aging characteristic, after this connects end, shunt this corresponding in rule list rule also with deleted based on the IP address of packet and the filtration of port information, for the packet that newly receives, to regenerate new dynamic filtration shunting rule equally according to the process of above-mentioned special field coupling based on IP address and port information.
2. the data filter shunting is handled: after rule sets, just can begin that the raw data packets on the network is filtered shunting and handle.This data handling procedure comprises following link:
(1) initialization: will filter the shunting rule and import internal memory, and, then rule will be set in the CAM system if adopt the CAM technology to realize.System can generate based on the IP address of packet and the filtration of port information and shunt rule list and shunt two tables of rule list based on the filtration of special field information according to the content of user's setting in " shunting rule list based on the IP address of packet and the filtration of port information " and " based on the filtration shunting rule list of special field information ".Wherein, comprise the static rule that is provided with and dynamic regular two parts of setting based on the IP address of packet and the filtration shunting rule list of port information, the static rule that is provided with is the content that the user sets, the dynamic rule of the setting Rule content that to be system dynamically generate when the filtration shunting rule match of carrying out based on special field information sees hereinafter the process about filtration treatment for details.Because it is ageing that the dynamic rule of setting has, what therefore import to internal memory increases " dynamic/static state " and " time-out count " two fields based on the IP address of packet and the filtration shunting rule list of port information, and form is:
| Regular number | IP address, source | Source IP mask | Source port | Purpose IP address | Purpose IP mask | Destination interface | Filter and divide Flow Behavior | Dynamically/static state | Overtime (counting |
(2) intercepting raw data packets: the packet on the intercept network, carry out protocal analysis, extract the IP packet according to the Internet protocol data packet format.
(3) filtration treatment:
At first, system with the IP packet that obtains according to mating of setting in advance based on the IP address of packet and the filtration shunting rule list of port information, if the match is successful, then, packet is divided in the different output port groupings according to " filter and divide Flow Behavior " of being provided with in the rule list.If the rule that matches belongs to dynamic setting rule, and packet belongs to the connection end packet, then remove this and dynamically set rule, otherwise, with the time-out count of this rule clear 0.For surpassing the situation that count value is finished dealing with not yet, this rule of dynamically setting is also with deleted.
After this, for failing and shunting the packet of rule match success in the rule list based on the IP address of packet and the filtration of port information, mate again with based on the rule that is provided with in the filtration of the special field information shunting rule list, if the match is successful, the filtration of setting in the filtration shunting rule according to this special field information divides Flow Behavior that packet is divided in the different output port groupings, and according to the source IP of this packet, the filtration of purpose IP address and this rule divides Flow Behavior to generate a dynamic filtration shunting rule based on the IP address of packet and port information to join in the filtration shunting rule list based on the IP address of packet and port information, by mating, will extract the follow-up data bag that connects under this packet with this new dynamic programming that forms.Be complementary if packet is failed and shunt rule based on the filtration of special field information, then abandon this packet.
Because what dynamically generate is at matching being connected of special field content based on the IP address of packet and the filtration shunting rule of port information, therefore can not produce huge dynamic programming quantity, also can not be subjected to the super large linking number influence that similar SYNFlooding (spreading unchecked synchronously) attacks, common CAM technology can satisfy the demands.
(4) shunting is handled: dividing flow algorithm is that high low level step-by-step XOR is carried out in the source IP address and the purpose IP address that are divided into each packet in each grouping, if TCP/UDP packet, then again TCP/UDP port numbers and operation result are carried out XOR once more, finally obtain an operation result HASH value.With the output port sum delivery that is comprised in this operation result HASH value and this grouping, the result who obtains is exactly the output port sequence number of this packet correspondence in affiliated grouping again.
Adopt the principle of this minute flow algorithm to be: to determine can all packets from network with the IP address and belong to the packet that carries out exchanges data between some source hosts and the destination host, and the characteristics of XOR are operation result and the sequence independence of importing data, therefore can guarantee that the IP address value that computing obtains with port numbers of the bi-directional data bag in the same connection is identical, the XOR of step-by-step simultaneously has certain discrete feature, substantially can guarantee that different connections obtains different end values after XOR is handled, therefore again by with grouping in output port sum delivery after, just can guarantee and different packets balancedly can either be assigned on the different output ports, the packet that belongs to same connection can be exported from identical port again, can not kept the shunting of the load balancing under the connection situation to handle again thereby realize not relying on the connection table.
The present invention has substantive distinguishing features and marked improvement: (1) utilizes and filters the preprocess method that the shunting rule is carried out the coarseness filtration to raw data packets, can significantly reduce the data processing load of rear end; In addition, can also mate, can shunt processing the follow-up data bag that connects under the packet that mates special content to packet content.(2) can shunt according to type of data packet, can shunt according to flow proportional same type data packets again.(3) shunting that utilizes the hash algorithm delivery to realize is handled, not needing to rely on the connection table can keep connecting equally, solve connection table that traditional load-balancing algorithm exists and be difficult to the deficiency safeguarding and store too greatly, and only need promptly can directly determine output port according to the result behind the delivery, avoided traditional algorithm to retrieve problems such as the search efficiency that causes is low one by one to the data in the connection table, guarantee higher recall precision, thereby can satisfy the demand of mass data flow no maximum concurrent processing on the high-speed backbone.(3) this method is fit to adopt specialized hardware technology such as field programmable gate array (FPGA) and Content Addressable Memory (CAM) chip to realize, can reach high cost performance, guarantees the linear speed processing demands at the high-speed backbone network.If limit output port quantity in one group and be 2 power power, then can adopt step-by-step " with " mode realize, be more suitable for hardware and realize, further raise the efficiency and reduce cost.
Description of drawings
Fig. 1 system schematic of the invention process
Embodiment
As shown in Figure 1, in core backbone network environment, insert many intrusion detection devices, audit equipment and network monitor equipment the operation situation of network system is carried out security monitoring.These safety means extract need the mass data in core backbone and self handle the foundation of required information as back-end analysis and differentiation.Adopt the technical scheme that proposes among the present invention, can satisfy the Large Volume Data forwarding processor and can't satisfy the demand that classification of Data is transmitted processing.Set forth the specific embodiment of technical solution of the present invention below.
(1) output port packet configuration rule is set: suppose that High Speed Network filters the shunting access platform and have 8 output ports, port-mark number is respectively: 0,1,2,3,4,5,6,7, and these ports are each safety monitoring device of opposite rear end respectively.According to the type of service processing demands of each safety monitoring device and data-handling capacity separately, each port is divided into four groupings:
| Packet number | The output port identification number |
| 1 | 3；2；0；2 |
| 2 | 1；4 |
| 3 | 5；7；5；6；7 |
| 4 | 0；4 |
Be divided in the same grouping port explanation they will handle the packet of same type, its middle port 2 is dividing into groups to occur twice in 1, the back-end processing equipment that port 2 correspondences are described will receive and handle in this output port 1/2nd data volume; Port 4 is divided in the middle of 2 and 4 two groupings, illustrates that port 4 will be responsible for transmitting the data from these two groupings.
(2) filtration shunting rule is set: filtering the regular setting of shunting is in order to realize the data filter shunting of coarseness, will to filter out the useless data of back-end processing, the rear end required data of each treatment facility being divided in the port grouping of appointment.The example of filtering the setting of shunting rule is as follows:
Shunt rule list based on the IP address of packet and the filtration of port information:
| Regular number | Source IP address | Source IP mask | Purpose IP address | Purpose IP mask | Source port number | The destination slogan | Filter and divide Flow Behavior | Dynamically/static state | Time-out count |
| 0 | 0.0.0.0 | 0.0.0.0 | 0.0.0.0 | 0.0.0.0 | 0 | 21 | 1 | Static | / |
| 1 | 0.0.0.0 | 0.0.0.0. | 0.0.0.0 | 0.0.0.0 | 5050 | 0 | 0 | Static | / |
| 2 | 61.125.2.1 | 255.255.255.255 | 0.0.0.0 | 0.0.0.0 | 0 | 0 | 2 | Static | / |
| 3 | 0.0.0.0 | 0.0.0.0 | 61.125.2.1 | 255.255.255.255 | 0 | 0 | 3 | Static | / |
| 4 | 61.125.34.3 | 255.255.255.255 | 0.0.0.0 | 0.0.0.0 | 0 | 0 | 2 | Static | / |
| 5 | 0.0.0.0 | 0.0.0.0 | 0.0.0.0 | 0.0.0.0 | 80 | 0 | 1 | Static | / |
| 6 | 61.125.3.8 | 255.255.255.0 | 0.0.0.0 | 0.0.0.0 | 90 | 0 | 4 | Static | / |
Filtration shunting rule list based on special field information:
| Regular number | The fields offset amount | Field length | Matching content | Filter and divide Flow Behavior |
| 301 | 42 | 15 | xy@yahoo.com.cn | 3 |
(3) packet that is truncated to from the core backbone network is carried out protocal analysis, the IP packet is extracted.Suppose and intercept following two packets:
Packet a is by source address 61.125.3.8, source IP mask 255.255.255.0, and source port 90 is sent to destination address 10.10.25.30, and purpose IP mask 255.255.255.0, destination interface are 1290.
Packet b is based on the packet that is truncated to when smtp protocol sends mail, source address is 10.10.19.131, source IP mask 255.255.255.0, source port is 1664, destination address is 216.136.173.18, purpose IP mask 255.255.255.0, destination interface are 25, and addresser's addresses of items of mail is xy@yahoo.com.cn.
(4) each IP packet is mated with each rule of filtering in the shunting rule list respectively, filter to divide Flow Behavior be 0 rule if match, with this data packet discarding, all the other packets that remain are carried out mark with packet number, indicate the group result of each packet.If some packets can not mate with any rule, then with this data packet discarding.
Can draw from the rule of above-mentioned definition, be divided into group 1 data and be: the destination slogan is that 21 packet, source port number are 80 packet; The data that are divided into group 2 are: secondary IP address is that 61.125.2.1, IP mask are that packet and the secondary IP address that 255.255.255.255 sends is that 61.125.34.3, IP mask are the packet that 255.255.255.255 sends; The data that are divided into group 3 are: being sent to the IP address is 61.125.2.1, the IP mask is the packet of 255.255.255.255, and all packets that comprise the affiliated connection of xy@yahoo.com.cn content in 15 bytes beginning of the 0X42 byte behind packet IP packet header; The data that are divided into group 4 are: secondary IP address is that 261.125.3.8, IP mask are that 255.255.255.0, port numbers are 90 packets that send.In addition, if after rule match is finished, identify each packet with packet number.
According to the situation of supposition in (3) processing procedure: the rule that is provided with in the IP address information of packet a and the filtration shunting rule list based on the IP address of packet and port information is mated one by one to packet a, matching result is found the satisfied rule 6 of the IP address information of this packet, so packet a divides Flow Behavior to be divided into grouping 4 processing according to the filtration that is provided with in the rule 6.
According to the situation of supposition in (3) processing procedure: the rule that is provided with in the address information among the packet b and the filtration shunting rule list based on the IP address of packet and port information is mated one by one to packet b, matching result finds that the match is successful with any rule, be that the value " xy@yahoo.com.cn " of 15 bytes after the 0X42 byte extracts and mates one by one based on the rule in the filtration shunting rule list of special field information again with fields offset amount in this packet, matching result satisfies rule 301.Divide Flow Behavior according to the filtration that is provided with in the rule 301, packet b is divided into grouping 3, and extract IP address information in this packet, divide Flow Behavior together with the filtration that is provided with in the rule 301, generating regular joining based on the IP address of packet and the filtration of port information of new dynamic filtration shunting shunts in the rule list, that is:
| Regular number | Source IP address | Source IP mask | Purpose IP address | Purpose IP mask | Source port number | The destination slogan | Filter and divide Flow Behavior | Dynamically/static state | Time-out count |
| 7 | 10.10.19.131 | 255.255.255.0 | 216.136.173.18 | 255.255.255.0 | 1664 | 25 | 3 | Dynamically | 200 |
(5) to being divided into the data of each grouping, shunt processing according to the branch flow algorithm that proposes among the present invention.The source IP address 61.125.3.8 of packet a and the high low level step-by-step of purpose IP address 10.10.25.30 XOR, the end value that obtains is carried out the step-by-step XOR with the XOR result of source port number 90 and destination slogan 1290 again, the result who finally obtains and the port in the grouping 4 total delivery, that is: { (0X3D7D_0X0A0A) _ (0X0308_0X191E) _ (0X005A_0X050A) } mod 2={0X3777_0X1A16_0X0550}mod 2=1
Operation result is 1, shows that this packet a should be from the 2nd the port output of dividing into groups 4, i.e. port 4 outputs.
The source IP address 10.10.19.131 of packet b and the high low byte step-by-step of purpose IP address 216.136.173.18 XOR, the end value that obtains is carried out the step-by-step XOR with the XOR result of source port number 1664 and destination slogan 25 again, result who finally obtains and the total delivery of the port in the grouping 3, that is:
{(0X0A0A_0XD888)_(0X1383_0XAD12)_(0X0680_0X0019)}mod5＝{0XD282_0XBE91_0X0699}＝4
Operation result is 4, shows that this packet b should be from the 5th the port output of dividing into groups 3, i.e. port 7 outputs.
(6) subsequent packet that connects under packet a, the b is received in supposition, because the IP address of affiliated same connection packet is identical with port, therefore the result who obtains through above-mentioned calculating process is also identical, still exports from identical port so guaranteed all subsequent packet of identical connection.
(7) receive the end packet that connects under the packet b, then remove the rule 7 that dynamically arranges.
If the high-rate fitration shunt method that does not rely on the connection table that the present invention proposes adopts the CAM technology to realize, insert for 12 road 2.5G POS, possess the data access capability of 30G, satisfy the linear speed data access demand of high-speed backbone network.
Claims (6)
1, a kind of high-rate fitration shunt method that keeps connection performance, it is characterized in that, the output port rule of classification at first is set and filters the shunting rule, the configuration of output port rule of classification is at first each output port to be divided into groups according to the Business Processing type of the backend application system of correspondence, and then determine that according to the disposal ability of back-end system of each port correspondence in the group allocation proportion of each port processing data packets flow in this group, the configuration of filtering the shunting rule are according to IP address information or special field packet to be divided in the middle of each divides into groups; Carry out the data filter shunting again and handle, by protocal analysis, from the raw data packets that network receives, extract the IP packet earlier, will be divided in each output port grouping with the subsequent treatment related data according to the filtration shunting rule that sets; The address and the port information of packet that will be divided into then in each group carries out the Hash computing, the port sum delivery that hash value is comprised with this grouping again, and the result who obtains is exactly the output port sequence number of this packet correspondence in affiliated grouping.
2, the high-rate fitration shunt method of maintenance connection performance according to claim 1 is characterized in that, following principle is followed in the configuration of described output port grouping:
(1) grouping of output port is at first according to the business demand of back-end processing system, divide into groups according to the Business Processing type, filtering respectively when a system and a plurality of system has mutually in the regular attribute of shunting when overlapping, and same output port can appear in the plural grouping;
(2) in each packets inner, realize balanced the distribution in order to guarantee data processing task, same output port can repeatedly appear in same group, promptly determines the allocation proportion of each port data packet flow in this group in grouping according to the data-handling capacity of the back-end system of each port correspondence;
(3) in the group each output port be distributed in the proof load equilibrium time, guarantees that all two-way packets of same TCP connection must be forwarded on the same output port, be convenient to rear end gathering and reduce to received data.
3, the high-rate fitration shunt method of maintenance connection performance according to claim 1 is characterized in that, the setting of described filtration shunting rule comprises dual mode:
A kind of be with source IP address, source IP mask, source port, purpose IP address, purpose IP mask, destination interface IP address and the port information of packet as direct monitored object, based on these IP address informations rule is arranged in the filtration shunting rule list based on address information;
Another kind is based on the regular set-up mode of special field information, and promptly the user only need be provided with the side-play amount of special field, this field, the length of this field, the matching content and the corresponding branch Flow Behavior that filters of this field;
It is as follows to shunt regular tableau format based on the filtration of the IP address of packet and port information:
Regular number Source IP address Source IP mask Source port Purpose IP address Purpose IP mask Destination interface Filter and divide Flow Behavior
It is as follows to shunt regular tableau format based on the filtration of special field information:
Regular number The fields offset amount Field length Matching content Filter and divide Flow Behavior
Wherein:
Regular number: the unique identification sequence number that each is regular;
Source/purpose IP address: the source end of data packet transmission and the IP address value of destination;
Source/purpose IP mask: the source end of data packet transmission and the subnet mask of destination IP field;
Source/destination interface: the source end of data packet transmission and the port numbers of destination;
Fields offset amount: the side-play amount of the special field that the needs that begin to calculate from the IP packet content mate;
Field length: the length that needs the special field of coupling;
Matching content: the matching value of special field, these special field comprise: the receiver in URL address, the Email mail and addresser's address field;
Filter and divide Flow Behavior: " 0 " is represented this data packet discarding, " 1 " expression is forwarded to group 1 with this packet, " 2 " expression is forwarded to group 2 with this packet, " N " expression is forwarded to group N with this packet, " 1 " expression suspends uses this rule, all fields except that filtering branch Flow Behavior field all are 0 in the shunting rule list if filter, and represent that then this rule is applicable to all packets.
4, according to the high-rate fitration shunt method of claim 1 or 3 described maintenance connection performances, it is characterized in that, filter the setting of shunting rule and will shunt the foundation of handling as filtration after this, as the packet that receives from network and one during based on the filtration shunting rule match of the IP address of packet and port information, packet will divide Flow Behavior to be forwarded to corresponding grouping according to the filtration that this rule is set; If when from the packet that network receives, comprising the data identical with the special field value that sets in advance, the IP address information of this packet will be extracted, divide Flow Behavior together with the corresponding filtration that sets in advance, form a dynamic filtration shunting rule based on IP address and port information, write based on the IP address of packet and the filtration of port information and shunt in the rule list, all subsequent packet that TCP/UDP under this packet and this packet connects all will divide Flow Behavior to be forwarded to corresponding grouping according to the filtration that this rule is set, the filtration shunting rule that forms based on the feature field coupling has aging characteristic, after this connects end, shunt this corresponding in rule list rule also with deleted based on the IP address of packet and the filtration of port information, for the packet that newly receives, to regenerate new dynamic filtration shunting rule equally according to the process of above-mentioned special field coupling based on IP address and port information.
5, the high-rate fitration shunt method of maintenance connection performance according to claim 1 is characterized in that, described data filter shunting is handled, and comprises following link:
(1) initialization: will filter the shunting rule and import internal memory, if adopt the CAM technology to realize, then rule is set in the CAM system, system can be according to the content of user's setting in " based on the IP address of packet and the filtration shunting rule list of port information " and " based on the filtration shunting rule list of special field information ", generation is shunted rule list and is shunted two tables of rule list based on the filtration of special field information based on the IP address of packet and the filtration of port information, wherein, comprise the static rule that is provided with and dynamic regular two parts of setting based on the IP address of packet and the filtration shunting rule list of port information, the static rule that is provided with is the content that the user sets, dynamically the rule of the setting Rule content that to be system dynamically generate when the filtration shunting rule match of carrying out based on special field information;
(2) intercepting raw data packets: the packet on the intercept network, carry out protocal analysis, extract the IP packet according to the Internet protocol data packet format;
(3) filtration treatment:
At first, system with the IP packet that obtains according to mating of setting in advance based on the IP address of packet and the filtration shunting rule list of port information, if the match is successful, then according to " filter and divide Flow Behavior " of being provided with in the rule list, packet is divided in each output port grouping, if the rule that matches belongs to dynamic setting rule, and packet belongs to the connection end packet, then remove this and dynamically set rule, otherwise, the time-out count of this rule is clear 0, and for surpassing count value also in situation about handling, this rule of dynamically setting is also with deleted;
For with packet based on rule match failure in the filtration shunting rule list of the IP address of packet and port information, mate again with based on the rule that is provided with in the filtration of the special field information shunting rule list, if the match is successful, the filtration of setting in the filtration shunting rule according to this special field information divides Flow Behavior that packet is divided in the corresponding output port grouping, and according to the source IP of this packet, the filtration of purpose IP address and this rule divides Flow Behavior to generate a dynamic filtration shunting rule based on the IP address of packet and port information to join in the filtration shunting rule list based on the IP address of packet and port information, by mating with this new dynamic programming that forms, will extract the follow-up data bag that connects under this packet, if packet is failed with the filtration shunting rule match based on special field information, then abandon this packet;
(4) shunting is handled: adopt source IP address and the purpose IP address of dividing flow algorithm will be divided into each packet in each grouping to carry out high low level step-by-step XOR, if TCP/UDP packet, then again TCP/UDP port numbers and operation result are carried out XOR once more, finally obtain an operation result HASH value, with the output port sum delivery that is comprised in this operation result HASH value and this grouping, the result who obtains is exactly the output port sequence number of this packet correspondence in affiliated grouping again.
6, the high-rate fitration shunt method of maintenance connection performance according to claim 5, it is characterized in that, because dynamically the rule of setting has ageing, what therefore import to internal memory increases " dynamic/static state " and " time-out count " two fields based on the IP address of packet and the filtration shunting rule list of port information, and form is:
Regular number IP address, source Source IP mask Source port Purpose IP address Purpose IP mask Destination interface Filter and divide Flow Behavior Dynamically/static state Time-out count
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB2004100171891A CN1287570C (en) | 2004-03-25 | 2004-03-25 | High speed filtering and stream dividing method for keeping connection features |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB2004100171891A CN1287570C (en) | 2004-03-25 | 2004-03-25 | High speed filtering and stream dividing method for keeping connection features |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN1564547A CN1564547A (en) | 2005-01-12 |
| CN1287570C true CN1287570C (en) | 2006-11-29 |
Family
ID=34478825
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNB2004100171891A Expired - Fee Related CN1287570C (en) | 2004-03-25 | 2004-03-25 | High speed filtering and stream dividing method for keeping connection features |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN1287570C (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9807642B2 (en) | 2012-12-28 | 2017-10-31 | Huawei Technologies Co., Ltd. | Traffic distribution method, device, and system |
Families Citing this family (45)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN100426773C (en) * | 2005-11-30 | 2008-10-15 | 中兴通讯股份有限公司 | Method for equalizing port flow while multiple-MAC-port inter-connecting |
| CN100459524C (en) * | 2006-04-25 | 2009-02-04 | 中国移动通信集团公司 | Media stream shunting system and method |
| CN100561937C (en) * | 2006-06-15 | 2009-11-18 | 华为技术有限公司 | A kind of method and device of realizing network flow load sharing |
| CN101141670B (en) * | 2006-09-05 | 2011-08-24 | 中兴通讯股份有限公司 | Message distributing system and method |
| CN101217455B (en) * | 2007-01-05 | 2011-07-27 | 上海复旦光华信息科技股份有限公司 | A secure content filtering shunt based on the integration of useful connecting data |
| EP2321934B1 (en) * | 2008-09-12 | 2014-06-25 | Hewlett-Packard Development Company, L.P. | System and device for distributed packet flow inspection and processing |
| CN101789884B (en) * | 2009-01-23 | 2012-03-28 | 英业达股份有限公司 | Load balancing method for network intrusion detection |
| CN101896011A (en) * | 2009-05-18 | 2010-11-24 | 大唐移动通信设备有限公司 | Information filtering equipment and method |
| CN101896010A (en) * | 2009-05-18 | 2010-11-24 | 大唐移动通信设备有限公司 | Equipment and method for filtering information |
| CN101699802B (en) * | 2009-10-23 | 2012-02-29 | 北京锐安科技有限公司 | Method for branching mass data |
| US8782787B2 (en) | 2009-10-28 | 2014-07-15 | Hewlett-Packard Development Company, L.P. | Distributed packet flow inspection and processing |
| CN101764741B (en) * | 2009-11-27 | 2012-06-06 | 上海恒为信息科技有限公司 | Filtering and shunting device and method supporting multi-service function |
| CN102209019B (en) * | 2010-03-30 | 2015-09-16 | 杭州华三通信技术有限公司 | A kind of load-balancing method based on message payload and load-balancing device |
| CN101909003A (en) * | 2010-07-07 | 2010-12-08 | 南京烽火星空通信发展有限公司 | Line speed shunt equipment and method |
| CN102143082A (en) * | 2010-07-07 | 2011-08-03 | 南京烽火星空通信发展有限公司 | Line speed shunt device and method under uneven processing capability |
| US8711703B2 (en) | 2010-10-29 | 2014-04-29 | Telefonaktiebolaget L M Ericsson (Publ) | Load balancing in shortest-path-bridging networks |
| CN102868628B (en) * | 2011-07-06 | 2016-03-02 | 阿里巴巴集团控股有限公司 | Flow segmentation, device and system |
| US9450870B2 (en) * | 2011-11-10 | 2016-09-20 | Brocade Communications Systems, Inc. | System and method for flow management in software-defined networks |
| CN102495764A (en) * | 2011-12-06 | 2012-06-13 | 曙光信息产业股份有限公司 | Method and device for realizing data distribution |
| CN102387160B (en) * | 2011-12-13 | 2014-10-22 | 曙光信息产业(北京)有限公司 | System and method based on IP message quintuple filtering strategy |
| CN102523163A (en) * | 2011-12-19 | 2012-06-27 | 曙光信息产业(北京)有限公司 | Flow managing equipment and method supporting a plurality of flow division modes |
| CN102497385B (en) * | 2011-12-31 | 2015-09-16 | 曙光信息产业股份有限公司 | A kind of network traffics auditing method and auditing system |
| CN102780771A (en) * | 2012-07-12 | 2012-11-14 | 深圳市同洲电子股份有限公司 | Service transmission method, device and equipment |
| CN103780601A (en) * | 2012-10-17 | 2014-05-07 | 北京力控华康科技有限公司 | Method for automatically establishing Ethernet communication safety rules |
| CN102932270A (en) * | 2012-11-27 | 2013-02-13 | 无锡城市云计算中心有限公司 | Load balancing method and device supporting network security service |
| CN103281246A (en) * | 2013-05-20 | 2013-09-04 | 华为技术有限公司 | Message processing method and network equipment |
| CN103414611B (en) * | 2013-08-21 | 2016-04-20 | 宁波成电泰克电子信息技术发展有限公司 | A kind of flow statistical method of high speed laod network equalizing system |
| CN103841096A (en) * | 2013-09-05 | 2014-06-04 | 北京科能腾达信息技术股份有限公司 | Intrusion detection method with matching algorithm automatically adjusted |
| CN103685221A (en) * | 2013-09-05 | 2014-03-26 | 北京科能腾达信息技术股份有限公司 | A network invasion detection method |
| CN103685222A (en) * | 2013-09-05 | 2014-03-26 | 北京科能腾达信息技术股份有限公司 | A data matching detection method based on a determinacy finite state automation |
| CN103491069A (en) * | 2013-09-05 | 2014-01-01 | 北京科能腾达信息技术股份有限公司 | Filtering method for network data package |
| CN103685224A (en) * | 2013-09-05 | 2014-03-26 | 北京安博达通科技有限责任公司 | A network invasion detection method |
| CN103701783B (en) * | 2013-12-17 | 2017-01-11 | 沈阳觉醒软件有限公司 | Preprocessing unit, data processing system consisting of same, and processing method |
| CN104539549B (en) * | 2014-12-30 | 2018-01-02 | 天津市锦标科技有限公司 | A kind of data message processing method based on high density network flow |
| CN106302236A (en) * | 2015-05-27 | 2017-01-04 | 国家计算机网络与信息安全管理中心 | A kind of method of data distribution and access device |
| CN105681317A (en) * | 2016-02-03 | 2016-06-15 | 国网智能电网研究院 | Novel business and database auditing engine |
| CN105704059A (en) * | 2016-03-31 | 2016-06-22 | 北京百卓网络技术有限公司 | Load balancing method and load balancing system |
| CN107196837B (en) * | 2017-06-16 | 2020-06-16 | 四川省农业科学院服务中心 | Multi-data service comprehensive networking method based on VLAN division application |
| CN108322405A (en) * | 2018-03-21 | 2018-07-24 | 山东超越数控电子股份有限公司 | A kind of flow equalization method and device based on data |
| CN108809730B (en) * | 2018-06-25 | 2021-11-19 | 创新先进技术有限公司 | Method and device for controlling flow switching of machine room |
| CN109639592B (en) * | 2018-12-11 | 2023-01-06 | 武汉奥浦信息技术有限公司 | Rapid data analysis method and device based on ten-gigabit traffic |
| CN111478822B (en) * | 2019-02-21 | 2022-11-04 | 上海多算科技股份有限公司 | Efficient filtering method for cluster application network traffic |
| CN110708211B (en) * | 2019-08-30 | 2021-09-17 | 上海唯链信息科技有限公司 | Network flow testing method and system |
| CN111030998B (en) * | 2019-11-15 | 2021-10-01 | 中国人民解放军战略支援部队信息工程大学 | Configurable protocol analysis method and system |
| CN114374622B (en) * | 2021-12-31 | 2023-12-19 | 恒安嘉新（北京）科技股份公司 | Shunting method based on fusion shunting equipment and fusion shunting equipment |
-
2004
- 2004-03-25 CN CNB2004100171891A patent/CN1287570C/en not_active Expired - Fee Related
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9807642B2 (en) | 2012-12-28 | 2017-10-31 | Huawei Technologies Co., Ltd. | Traffic distribution method, device, and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN1564547A (en) | 2005-01-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN1287570C (en) | High speed filtering and stream dividing method for keeping connection features | |
| US7436830B2 (en) | Method and apparatus for wire-speed application layer classification of upstream and downstream data packets | |
| US20040073671A1 (en) | Method and apparatus for filtering packets using a dedicated processor | |
| CN103491069A (en) | Filtering method for network data package | |
| CN103685224A (en) | A network invasion detection method | |
| CN103841096A (en) | Intrusion detection method with matching algorithm automatically adjusted | |
| CN1941716A (en) | Method, device and system for accounting application flow | |
| CN1759574A (en) | Method and device for the classification and redirection of data packets in a heterogeneous network | |
| CN1416239A (en) | Method for switching in virtual local area network of the access network with mixed optical fiber and coaxial line | |
| CN1545254A (en) | A method of fast data packet filtering | |
| CN103685221A (en) | A network invasion detection method | |
| CN106972985A (en) | Accelerate the method and DPI equipment of the processing of DPI device datas and forwarding | |
| CN101465760A (en) | Method and system for detecting abnegation service aggression | |
| US20030033531A1 (en) | System and method for string filtering | |
| CN103685222A (en) | A data matching detection method based on a determinacy finite state automation | |
| CN101764741A (en) | Filtering and shunting device and method supporting multi-service function | |
| CN106850547A (en) | A kind of data restoration method and system based on http protocol | |
| CN206962832U (en) | Network data auditing system based on FPGA high-performance capture cards | |
| CN1741504A (en) | Flow controlling method based on application and network equipment for making applied flow control | |
| CN1992595A (en) | Terminal and related computer implemented method for detecting malicious data for computer network | |
| CN103457824A (en) | Message processing method and device | |
| CN1592215A (en) | Method for partitioned document recombination and service distribution | |
| CN1610335A (en) | Safety filtering current shunt of exchange structure based on network processor and CPU array | |
| CN103179109A (en) | Secondary session query function based filtering and distribution device and method thereof | |
| CN1490991A (en) | Method for virtual Ethernet adapter card |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| C17 | Cessation of patent right | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20061129 Termination date: 20130325 |