CN101114991B - Method for implementing Ethernet based data flow high speed comparison - Google Patents
Method for implementing Ethernet based data flow high speed comparison Download PDFInfo
- Publication number
- CN101114991B CN101114991B CN200610103723XA CN200610103723A CN101114991B CN 101114991 B CN101114991 B CN 101114991B CN 200610103723X A CN200610103723X A CN 200610103723XA CN 200610103723 A CN200610103723 A CN 200610103723A CN 101114991 B CN101114991 B CN 101114991B
- Authority
- CN
- China
- Prior art keywords
- data
- content
- comparison
- module
- ethernet
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn - After Issue
Links
Images
Abstract
The invention discloses a method which releases data stream high speed comparison on the base of Ethernet network. The method comprises that: a data package analytic module, a role comparison module, a content reduction module, a content comparison module and a comparison result process module are established, and the local management instructions are transported by the CPU through the PCI to conduct the local process; analysis on the Ethernet network data package is done, and the TCP data package is recombined according to the TCP agreement; the role comparison to the analyzed data is conducted and according to the comparison result, the data is distributed to the content reduction module and the content comparison module; content reduction is conducted to the data which is distributed to the content reduction module and then the data is transported to the content comparison module; content comparison to the reduced data and the data which is distributed to the content comparison module are conducted; corresponding process is conducted according to the reduction result. The invention reduces the load of CPU and greatly improves the processing speed of the network data in the system, thus satisfying the increasing needs of network data scream and the role base. .
Description
Technical field
The present invention relates to network safety filed, particularly a kind of method based on the comparison of Ethernet realization data flow high speed is specifically related to a kind of method of comparing based on the Ethernet realization data flow and the high speed of extensive rule base.
The present invention use but be not limited to network flow virus filtration, network flow content characteristic search, invasion inspection technology and application, intrusion prevention technology and use, in the Ethernet fields such as the access control of network special applications stream, statistics.
Background technology
Along with being the network development of main flow with the ethernet technology, content safety begins to become one of key issue in the present information security, and the safe practice of content-based comparison will be the another development trend of current network safety prevention.The search of the virus filtration of network data flow, content characteristic, invasion inspection technology, intrusion prevention be technological, all be used the network security technology field to technology such as the access control of network special applications stream, statistics.In these were used, all requisite rule comparison and the content that needs to accomplish based on the Ethernet data bag compared.
Adopt CPU to accomplish a kind of general solution that the pattern of comparing based on the rule comparison and the content of Ethernet data bag becomes above-mentioned application.Under this pattern, comparison of the rule of all Ethernet data bags and content comparison operation are all accomplished by CPU (Central Processing Unit, central processing unit); CPU need carry out multi-process simultaneously to be handled, and each process contents processing has nothing in common with each other, and resolves comprising packet; The rule comparison; Reduction of data, content is than equity, and various resource consumptions have all concentrated on the CPU.
Above-mentioned solution is at the low discharge network or only to contain under the rule base of minority rule be feasible, but along with the raising of network performance, network traffic data and rule base increase day by day, and the disposal ability of the CPU deficiency that just seems gradually begins to become the bottleneck of various application.
Summary of the invention
In order to overcome the problem that exists in the prior art, the present invention proposes a kind of sharing out the work and helping one another based on multimode and carries out the method that the Ethernet data bag is compared at a high speed.
The rule comparison that the present invention carries out whole Ethernet data bag with content comparison operate and break up a plurality of modules and handle, adopt each intermodule to cooperate each other, the mode of operation of bulk flow waterline.
The present invention specifically is achieved in that
A kind of method based on the comparison of Ethernet realization data flow high speed comprises the steps:
Step 1, set up packet parsing module, regular comparing module, content recovery module, content comparing module and comparison result processing module, by CPU through PCI channel transfer local management instruction carrying out local management;
Step 2, carry out the parsing of Ethernet data bag, abide by Transmission Control Protocol, tcp data Bao Zuoliu is recombinated;
Step 3, to the data after resolving, carry out rule and compare, be diverted to content recovery module and content comparing module according to comparison result;
Step 4, the data that are diverted to the content recovery module are carried out content reduction, after the reduction, output to the content comparing module;
Step 5, data after the reduction and the rule base that is assigned to the data based correspondence of content comparing module are carried out the content comparison;
Step 6, according to the content comparison result, handle accordingly, that is, abandon data flow or let data flow normally send CPU to through CPU or with data flow.
Said step 1-6 adopts the order of continuous productive process to carry out, and back one packet when the last data bag executes the processing entering next step of current step, gets into current step, and the like.
In the said step 2,
The parsing of said Ethernet data bag comprises:
(1) according to the Ethernet data packet format from type of data packet angular area divided data bag, handle two layers, three layers and four layers of Ethernet data packet header;
(2) assigned address from the packet application layer data takes out data content, according to the data content that takes out data stream is carried out the differentiation of application type;
Said to tcp data Bao Zuoliu reorganization, be the tcp data bag to be carried out the order reorganization according to the continuous mode of tcp data bag stem sequence number.
In the said step 4,
When carrying out the content reduction, if the data flow that exists data restoring module not support then is diverted to CPU and carries out the content reduction.
In the said step 5,
Said content comparison adopts parallel data are flowed of a plurality of comparison engine to compare;
Said comparison engine adopts the mode that compares one by one with comparison rules to compare.
In the said step 6,
According to the call number that is prestored into the comparison result processing module, utilize the call number of hitting that the content comparing module imports into, compare out data flow corresponding processing mode, handle.
Adopt the method for the invention; The comparison of rule that whole Ethernet data bag is carried out with content comparison operate and break up a plurality of modules and handle; Intermodule is cooperated each other; Adopt the mode of pile line operation, guarantee that processing data packets is the shortest blanking time, thereby improved the rule comparison and the content comparison speed of gigabit wire speed.Reduce the burden of CPU, improved the speed of system handles network data flow greatly, satisfied the growing networking data flow and the requirement of rule base.
Description of drawings
Fig. 1 is for realizing the structure chart of the method for the invention;
Fig. 2 is the flow chart of the method for the invention.
Embodiment
Below in conjunction with accompanying drawing the embodiment of the method for the invention is carried out detailed description.
The method of the invention specifically comprises:
The first step is set up Ethernet data bag parsing module, regular comparing module, content recovery module, content comparing module and comparison result processing module in advance, is instructed through PCI channel transfer local management by CPU and carries out the management to each module of system;
Second step, at Ethernet data bag parsing module, accomplish the analytical capabilities and the TCP stream recombination function of Ethernet data bag, specific as follows:
1, according to the Ethernet data packet format from type of data packet angular area divided data bag, handle two layers, three layers and four layers of Ethernet data packet header;
2, take out data content from the assigned address of packet application layer data, according to the data content that takes out data stream is carried out the differentiation of application type, for example mail data stream etc. is applicable to the application layer data of packet is resolved;
3, abide by Transmission Control Protocol packet is done the stream reorganization, the tcp data bag is carried out the order reorganization, prevent that tcp data stream is out of order according to the mode that tcp data bag stem sequence number is continuous;
4, packet of finishing dealing with and resolving information are sent to regular comparing module in the lump, carry out rule-based triage operator.
The 3rd step, in regular comparing module,, data are carried out the rule comparison according to the characteristic that from packet, parses agreement, the network address, the network port and application layer data, according to comparison result packet is shunted processing, processing mode is following:
1, is diverted to the content comparing module and carries out the content comparison;
2, be diverted to the content recovery module and carry out the content reduction, after the reduction, output to the content comparing module.
Result according to the rule comparison becomes packed data stream, mail data stream, encoded data stream, clear data stream etc. with data flow classification.
In the 4th step, in the content recovery module, carry out the content reduction; Carry out the content reduction for needs, but the data flow that the content recovery module can not be supported is diverted to CPU; Utilize the CPU processing more flexible,, carry out the content reduction the expansion engine of CPU as the content reduction.Accomplish the content restoring operation to data flow in the content recovery module, for example mail data flows, encoded data stream.CPU carries out the complex data stream restoring operation as the expansion engine of content reduction, for example packed data stream.
CPU utilizes each module of PCI passage and system to communicate in addition, realizes local management.Specifically be meant; CPU utilizes the local management instruction, and between the modules such as Ethernet data bag parsing module, regular comparing module, content recovery module, content comparing module and comparison result processing module, carries out the transmitting-receiving of packet; Accomplish the transmitting-receiving of local management instruction, realize local management.
In the 5th step, in the content comparing module, the data flow that is assigned to the content comparing module after handling through the data flow after the content reduction with through shunting is done content-based comparison.
Adopt the mode of the parallel comparison of a plurality of comparison engine that data flow is done content-based comparison, each comparison engine adopts the mode that compares one by one with comparison rules to compare, and the comparison elapsed time can calculate by following mode:
When the comparison engine number was 2, the comparison elapsed time was 1/2 of single engine;
When the comparison engine number was 4, the comparison elapsed time was 1/4 of single engine;
When the comparison engine number was n, the comparison elapsed time was the 1/n of single engine;
Can reach content comparison speed at a high speed with this.
Whether hit according to comparison at last, the output comparison result, said comparison result comprises and hits call number.
In the 6th step, in the comparison result processing module,, handle accordingly according to the content comparison result.
In the comparison result processing module; Each call number corresponding processing mode that prestores (abandon data flow or let data flow normally send CPU to) through CPU or with data flow; The call number of hitting that utilization is imported into from the content comparing module is compared, and obtains the processing mode (abandon data flow or let data flow normally send CPU to through CPU or with data flow) of data flow.
Above six steps all be to adopt pile line operation; Promptly second packet needn't wait for that six steps of first packet just do first step operation after finishing dealing with; But in the intact first step of first processing data packets; When getting into for second step, second bag just can get into first step operation, and the back by that analogy.Can guarantee that like this processing data packets is the shortest blanking time, thereby realize the rule comparison and the content comparison of gigabit wire speed.
Utilize method provided by the present invention, can carry out plural serial stage, to support more massive rule comparison and content comparison.
Claims (6)
1. the method based on the comparison of Ethernet realization data flow high speed is characterized in that, comprises the steps:
Step 1, set up packet parsing module, regular comparing module, content recovery module, content comparing module and comparison result processing module, by CPU through PCI channel transfer local management instruction carrying out local management;
Step 2, carry out the parsing of Ethernet data bag, abide by Transmission Control Protocol, tcp data Bao Zuoliu is recombinated according to the mode that tcp data bag stem sequence number is continuous;
Step 3, basis parse the characteristic of agreement, the network address, the network port and application layer data from packet, data are carried out the rule comparison, are diverted to content recovery module and content comparing module according to comparison result;
Step 4, the data that are diverted to the content recovery module are carried out content reduction, after the reduction, output to the content comparing module;
Step 5, data after the reduction and the rule base that is assigned to the data based correspondence of content comparing module are carried out the content comparison;
Step 6, according to the content comparison result, handle accordingly, that is, abandon data flow or let data flow normally send CPU to through CPU or with data flow.
2. the method based on the comparison of Ethernet realization data flow high speed as claimed in claim 1 is characterized in that:
Said step 1-6 adopts the order of continuous productive process to carry out, and back one packet when the last data bag executes the processing entering next step of current step, gets into current step, and the like.
According to claim 1 or claim 2 realize it is characterized in that the method for data flow high speed comparison based on Ethernet:
In the said step 2,
The parsing of said Ethernet data bag comprises:
(1) according to the Ethernet data packet format from type of data packet angular area divided data bag, handle two layers, three layers and four layers of Ethernet data packet header;
(2) assigned address from the packet application layer data takes out data content, according to the data content that takes out data stream is carried out the differentiation of application type;
Said to tcp data Bao Zuoliu reorganization, be the tcp data bag to be carried out the order reorganization according to the continuous mode of tcp data bag stem sequence number.
According to claim 1 or claim 2 realize it is characterized in that the method for data flow high speed comparison based on Ethernet:
In the said step 4,
When carrying out the content reduction, if the data flow that exists the content recovery module not support then is diverted to CPU and carries out the content reduction.
According to claim 1 or claim 2 realize it is characterized in that the method for data flow high speed comparison based on Ethernet:
In the said step 5,
Said content comparison adopts parallel data are flowed of a plurality of comparison engine to compare;
Said comparison engine adopts the mode that compares one by one with comparison rules to compare.
According to claim 1 or claim 2 realize it is characterized in that the method for data flow high speed comparison based on Ethernet:
In the said step 6,
According to the call number that is prestored into the comparison result processing module, utilize the call number of hitting that the content comparing module imports into, compare out data flow corresponding processing mode, handle.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN200610103723XA CN101114991B (en) | 2006-07-27 | 2006-07-27 | Method for implementing Ethernet based data flow high speed comparison |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN200610103723XA CN101114991B (en) | 2006-07-27 | 2006-07-27 | Method for implementing Ethernet based data flow high speed comparison |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101114991A CN101114991A (en) | 2008-01-30 |
| CN101114991B true CN101114991B (en) | 2012-06-13 |
Family
ID=39023120
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN200610103723XA Withdrawn - After Issue CN101114991B (en) | 2006-07-27 | 2006-07-27 | Method for implementing Ethernet based data flow high speed comparison |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101114991B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102761466B (en) * | 2011-04-25 | 2015-04-15 | 中国科学院空间科学与应用研究中心 | IEEE (Institute of Electrical and Electronics Engineers) 1394 bus data record processing system and method |
| CN103731364B (en) * | 2014-01-16 | 2017-02-01 | 赛特斯信息科技股份有限公司 | X86 platform based method for achieving trillion traffic rapid packaging |
| CN107454276B (en) * | 2016-06-01 | 2021-07-27 | 中兴通讯股份有限公司 | User terminal equipment, data forwarding method thereof and communication system |
| CN106549969B (en) * | 2016-11-21 | 2019-10-22 | 英赛克科技（北京）有限公司 | Data filtering method and device |
| CN106656853B (en) * | 2016-12-27 | 2019-08-16 | 盛科网络（苏州）有限公司 | The method and device of traffic flow information is extracted under Ethernet chip low-delay mode |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN2443562Y (en) * | 2000-12-28 | 2001-08-15 | 深圳市浔宝网络技术有限公司 | High-side router IP9000 |
| CN1558625A (en) * | 2004-02-10 | 2004-12-29 | 北京锐安科技有限公司 | Method and apparatus for real time replacing internet data |
-
2006
- 2006-07-27 CN CN200610103723XA patent/CN101114991B/en not_active Withdrawn - After Issue
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN2443562Y (en) * | 2000-12-28 | 2001-08-15 | 深圳市浔宝网络技术有限公司 | High-side router IP9000 |
| CN1558625A (en) * | 2004-02-10 | 2004-12-29 | 北京锐安科技有限公司 | Method and apparatus for real time replacing internet data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101114991A (en) | 2008-01-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107959690A (en) | DDoS attack cross-layer cooperative defense method based on software defined network | |
| CN104348716B (en) | A kind of message processing method and equipment | |
| CN1972240A (en) | Fast package filter processing method and its apparatus | |
| EP1158729B1 (en) | Stackable lookup engines | |
| CN101114991B (en) | Method for implementing Ethernet based data flow high speed comparison | |
| US20030231625A1 (en) | Selective header field dispatch in a network processing system | |
| CN101599963B (en) | Suspected network threat information screener and screening and processing method | |
| US8839405B2 (en) | Intelligent PHY with security detection for ethernet networks | |
| CN104394090B (en) | A kind of use DPI carries out the SDN controllers of network flow classification to packet | |
| EP2767064B1 (en) | Intelligent connectors integrating magnetic modular jacks and intelligent physical layer devices | |
| CN101141390A (en) | Novel self-defining ethernet out-of-band data packet filtering method and device | |
| US8131841B2 (en) | Method and apparatus for detecting predefined signatures in packet payload | |
| CN101051891A (en) | Method and device for safety strategy uniformly treatment in safety gateway | |
| CN101052046A (en) | Anti-virus method and device for fire-proof wall | |
| US20050190697A1 (en) | Transmission control system using link aggregation | |
| CN103475653A (en) | Method for detecting network data package | |
| Bremler-Barr et al. | CompactDFA: Scalable pattern matching using longest prefix match solutions | |
| CN101242362A (en) | Find key value generation device and method | |
| CN103746920B (en) | A kind of method that data transfer is realized based on gateway | |
| CN102497297A (en) | System and method for realizing deep packet inspection technology based on multi-core and multi-thread | |
| CN106534048A (en) | Method of preventing SDN denial of service attack, switch and system | |
| CN114398174A (en) | Dynamic energy-saving scheduling method and system for data center network in SDN environment | |
| CN102014065A (en) | Method for analyzing packet headers, header analysis preprocessing device and network processor | |
| CN107749826A (en) | A kind of data packet forwarding method and system | |
| CN101217486B (en) | A mobile Internet data load allocation method based on network processor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| ASS | Succession or assignment of patent right |
Owner name: BEIJING ZUOJIANG TECHNOLOGY CO., LTD. Free format text: FORMER OWNER: BEIJING NANSHAN BRIDGE IMFORMATION TECHNOLOGY CO., LTD. Effective date: 20120328 |
|
| C41 | Transfer of patent application or patent right or utility model | ||
| COR | Change of bibliographic data |
Free format text: CORRECT: ADDRESS; FROM: 100083 HAIDIAN, BEIJING TO: 100142 HAIDIAN, BEIJING |
|
| TA01 | Transfer of patent application right |
Effective date of registration: 20120328 Address after: 100142 room 511, Xinzhou business building, 58 Fu Cheng Road, Beijing, Haidian District Applicant after: Beijing Zuojiang Technology Co., Ltd. Address before: 100083 quantum core block 27, Haidian District, Beijing, 1501, Zhichun Road Applicant before: Beijing Nanshan Bridge Imformation Technology Co., Ltd. |
|
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| AV01 | Patent right actively abandoned |
Granted publication date: 20120613 Effective date of abandoning: 20161208 |
|
| C20 | Patent right or utility model deemed to be abandoned or is abandoned |