CN102594624A - Method for efficiently capturing network data packets at high speed based on field programmable gate array (FPGA) - Google Patents
Method for efficiently capturing network data packets at high speed based on field programmable gate array (FPGA) Download PDFInfo
- Publication number
- CN102594624A CN102594624A CN2012100585459A CN201210058545A CN102594624A CN 102594624 A CN102594624 A CN 102594624A CN 2012100585459 A CN2012100585459 A CN 2012100585459A CN 201210058545 A CN201210058545 A CN 201210058545A CN 102594624 A CN102594624 A CN 102594624A
- Authority
- CN
- China
- Prior art keywords
- rule
- comparison
- application
- packet
- parameter
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention relates to a method for capturing data packets at a linear speed from a high-speed data network system according to the setting of a user based on a field programmable gate array (FPGA). The method comprises four parts, namely rule making, network data packet resolving, rule comparison and rule application. Because unlimited number of flexible comparison rules and application rules are matched and compared with each parameter resolved from network data, various complex applications can be furthest satisfied; and because the method is implemented by adopting the FPGA or an application specific integrated circuit (ASIC), a plurality of comparison rules and application rules can be used at the same time, and the application requirement for capturing various data packets interested by the user at the linear speed from the high-speed Ethernet is met.
Description
[technical field]
The present invention relates to network test technical fields such as local area network (LAN), wide area network, refer in particular to a kind of condition according to user's setting that realizes based on FPGA is caught packet from data network system method.
[background technology]
Today; The Internet has become necessary part in our daily life; Email, ecommerce, search engine, video are shared, SNS community etc. use in a large number and universal let us relies on the Internet more; But enjoy the Internet to we bring convenient the time; Can Virtual network operator and user be concerned about several problems: this network security? How is its performance? Just in case what if break down? How does service quality guarantee? And the solution of these a series of problems all depends on network test, depends on the packet that extraction is relevant with it from network system and analyzes.But broadband network system development is swift and violent, and 10,000,000,000 have popularized, and 100,000 million (100G) have begun to use, and 400,000 million (400G) research and develop, its development even surmounted the speed of " Moore's Law " prediction.And traditional be provided with specified bytes and bag template based on MCU or PC to catch the method that packet analyzes still be that complexity all can not satisfy the demands from speed:
See from speed, after network speed reaches 10G, the PC of present main flow or the disposal ability that MCU has not had high speed like this, 40G and 100G just more need not mention.
See that from complexity unification of three nets and increasing application make network system become increasingly complex, traditional packet snapping method is because the restriction of MCU and PC disposal ability can not support a plurality of Wire speed packet that are provided with under the condition to handle.We just can only support limited several simple overanxious condition by the sniffer packet capturing software of known, and this is for modern complicated network system, no matter is network security analysis or fault location, all is far from being enough.
So industry presses for a kind of ability and supports the complicated linear speed network packet catching method in the express network system of 10G and more speed that imposes a condition down.
[summary of the invention]
To the deficiency of prior art, the invention provides a kind of hardware implementation method of coming based on scale programmable logic device (FPGA).The characteristics of FPGA are to use very flexible; Can programme to FPGA as software through the VerilogHDL language and realize the function that the user needs; Has simultaneously the high speed of hardware circuit again; The advantage can multidiameter delay handled has well overcome the deficiency of the disposal ability that CPU brings owing to the restriction of thread.Utilizing this method can support complicacy to impose a condition down catches out with network packet with linear speed in 10G and above express network system.
In order to achieve the above object, the present invention has adopted following technical scheme:
The present invention adopts the hardware programming implementation based on FPGA, also can change into by ASIC and realizing.
The present invention includes 4 modular assemblies:
Build a Rulemaking module in the FPGA, the formulation of rule comprises comparison rule and two parts of application rule:
The width of comparison rule is N bit (the N span is 2 to 64); Each comparison rule comprises sign, uses skew; Mask; Six parameters of maximum and minimum value, comparison rule can have M (span of M is 1 to 64), and six parameters of the width of each comparison rule and other can be different.
The implication of six parameters of each comparison rule is respectively: whether sign is used to indicate this comparison rule to use, and 1 expression is used, and 0 expression is not used; Use this comparison rule of expression and be which part of the packet that is applied to parse; The original position of skew expression rule application; Mask is used for expression and whether shields corresponding rule position, and 1 expression does not shield, and comparison is not participated in 0 expression shielding; Maximum is represented qualified maximum; Minimum value is represented qualified minimum value.
Each application rule comprises sign, uses, and three parameters of comparison rule indication are formed, and application rule can have X (the X span is 1 to 32), and the parameter of each application rule all can be different.
The implication of three parameters of each application rule is respectively: whether sign is used for indication should rule use; Use expression and use this regular purposes, like packet capturing, statistics, statistics with histogram etc.; The bit wide of comparison rule indication is identical with the number M of comparison rule, and each bit position is corresponding with a rule, is used for indicating this rule whether to participate in the comparison of application rule.
Building a network packet in the FPGA resolves; Be used for the Ethernet data bag that receives is resolved, protocol-dependent each parameter of lead code, mac source address, MAC destination address, IP source address, IP destination address, type, length, agreement and IP of each network packet is exported with register.
Build a regular comparison module in the FPGA, be used for each parameter utilization comparison rule of the packet that parses from network packet is compared, then comparative result is exported.
Build a rule application module in the FPGA, be used for mating the indication that the dateout bag is caught from the comparative result of regular comparison module output and each parameter of rule application.
A kind of high-speed and high-efficiency network packet catching method of realizing based on FPGA of the present invention; Because adopted comparison rule that number flexibly do not limit and application rule and carried out matching ratio from each parameter that network data parses; So can satisfy the application of various complicacies to greatest extent; Owing to adopted FPGA or ASIC to realize; So can a plurality of comparison rule use simultaneously with application rule, satisfied from the high speed data network system linear speed catch the application demand of various user's interest bags.
[description of drawings]
Fig. 1 is the high-speed and high-efficiency network packet catching method hardware system structure sketch map of realizing based on FPGA in the embodiment of the invention.
Fig. 2 is the high-speed and high-efficiency network packet catching method logical process sketch map of realizing based on FPGA in the embodiment of the invention.
Fig. 3 is the high-speed and high-efficiency network packet catching method comparison rule data format sketch map of realizing based on FPGA in the embodiment of the invention.
Fig. 4 is the high-speed and high-efficiency network packet catching method comparison rule data format parameter sketch map of realizing based on FPGA in the embodiment of the invention.
Fig. 5 is the high-speed and high-efficiency network packet catching method application rule data format sketch map of realizing based on FPGA in the embodiment of the invention.
Fig. 6 is the high-speed and high-efficiency network packet catching method application rule data format parameter sketch map of realizing based on FPGA in the embodiment of the invention.
Fig. 7 is 10 the network packet examples of high-speed and high-efficiency network packet catching method that realize based on FPGA in the embodiment of the invention.
[embodiment]
Below in conjunction with accompanying drawing and embodiment, technical scheme of the present invention is done further elaboration.
As shown in Figure 1; The cpu i/f module participate in rule relatively with the formulation of rule application, then FPGA receive through Network Interface Module that Ethernet data bag from equipment under test or tested network system compares through rule and rule application after the packet capture result is exported.
Simple in order to describe, only adopt 3 different comparison rule in this example, application process is identical when regular greater than 3.First comparison rule is that 32bit is wide, and second comparison rule is that 16bit is wide, and the 3rd comparison rule is that 8bit is wide, concrete data format such as Fig. 3, and concrete parameter is set to: for flags parameters, we are set at 1 at this; For application parameter, all rules all adopt same coded system, and our coded system is in this example: 0 expression lead code and delimiter, 1 expression target MAC (Media Access Control) address; 2 expression source MACs, 3 expression purpose IP addresses, 4 expression source IP addresss, 5 expression types; 6 expression 802.2LLC, 7 expression 802.2SNAP, the version number of 8 expression IPV4,9 expression header length; 10 expression COSs, 11 expression IPV4 total lengths, 12 expression IPV4 signs ... in this example; The value that we set is respectively: 1,2,3; For offset parameter, our value is respectively 0,1,2; For maximum, at inferior FEFEFEFE, FEFE, the FE of being set at respectively; For minimum value, be set at FEFEFE00 respectively, FEF0,10; If the comparison rule tables of data after the good value is as shown in Figure 4, its implication is following:
Comparison rule 1 (32bit) is used, and is applied to the comparison of target MAC (Media Access Control) address, begins to choose 32bit from highest order and participates in relatively, and every this 32bit satisfies more than or equal to FEFEFE00, all thinks smaller or equal to the packet of FEFEFEFE to meet comparison condition.
Comparison rule 2 (16bit) is used; Be applied to the comparison of source MAC; Choose 16bit since second byte (from high to low) highest order and participate in relatively, every this 16bit satisfies more than or equal to FEF0, all thinks smaller or equal to the packet of FEFE to meet comparison condition.
Comparison rule 3 (8bit) is used, and is applied to the comparison of purpose IP address, chooses 8bit since the 3rd byte (from high to low) highest order and participates in relatively, and every this 8bit satisfies more than or equal to 10, all thinks smaller or equal to the packet of FE to meet comparison condition.
Equally, simple in order to describe, only adopt 2 different application rules in this example, its data format is as shown in Figure 5, and method for using is identical during greater than two application rules.Concrete parameter is set to: for flags parameters, value is 1 in this example; For application parameter, all rules all adopt same coded system, and our coded system is in this example: 0 expression storage; Total number statistics is carried out in 1 expression, 2 expression statistics with histogram, and 3 represent storage and carry out total number statistics; 4 expression storage and statistics with histogram; 5 expression storages, total number are added up and statistics with histogram, number statistics and statistics with histogram that 6 expressions are total ... value in this example is respectively: 0,1; For the comparison rule indication parameter, value is 3 ' b101 and 3 ' b011 respectively in this example; If the application rule data format table after the good value is as shown in Figure 6, its implication is following:
Application rule 2 is used, and is applied to carry out total number statistics, and it is promptly eligible to satisfy regular comparison condition 1,2,3 simultaneously.
In this example 10 network packet are caught analysis, because three rules in this example are only to source MAC, target MAC (Media Access Control) address, source IP address compares, so other packet informations have not just listed, and the information of 10 network packet is seen Fig. 7.
Concrete realization is as shown in Figure 2, and logic realization was divided into for 4 steps:
The first step: the data that Rulemaking module parses cpu i/f module is brought, give comparison rule module and application rule modules respectively with comparison rule data format as shown in Figure 4 that parses and application rule data format as shown in Figure 6.
Second step: the network packet parsing module will be from the source MAC the network packet that Network Interface Module is come, target MAC (Media Access Control) address, source IP address, lead code and delimiter; Source IP address,, type, 802.2LLC; 802.2SNAP, the version number of IPV4, IPV4 header length, IPV4 COS; The IPV4 total length, information analysis such as IPV4 sign come out, and give the comparison rule module after adding flag.
The 3rd step: regular comparison module compares after the information of receiving the network packet parsing module, such as first bag target MAC (Media Access Control) address is: FEFEFEF01111; Source MAC is: F0FEFE555555; Purpose IP address is: 555515EEEEEE, and comparison rule 1 (32bit) is applied to the comparison of target MAC (Media Access Control) address; Begin to choose 32bit from highest order and be FEFEFEF0; And this value just is in more than or equal to FEFEFE00, within the scope smaller or equal to FEFEFEFE, thereby thinks that regular 1 matees; Comparison rule 2 (16bit) is applied to the comparison of source MAC, choose 16bit since second byte (from high to low) highest order and be FEFE, and this value just is in more than or equal to FEF0, within the scope smaller or equal to FEFE, thereby also confirms as coupling; Comparison rule 3 (8bit) is applied to the comparison of purpose IP address, and the value of choosing 8bit since the 3rd byte (from high to low) highest order is 15, and this value just satisfies more than or equal to 10, smaller or equal to the condition of FE, thereby thinks coupling, like this
The comparison match result of the output of first packet is 3 ' b111; Same calculating can get,
The comparison match result of the output of second packet is 3 ' b111;
The comparison match result of the output of the 3rd packet is 3 ' b011;
The comparison match result of the output of the 4th packet is 3 ' b101;
The comparison match result of the output of the 5th packet is 3 ' b111;
The comparison match result of the output of the 6th packet is 3 ' b111;
The comparison match result of the output of the 7th packet is 3 ' b111;
The comparison match result of the output of the 8th packet is 3 ' b111;
The comparison match result of the output of the 9th packet is 3 ' b110;
The comparison match result of the output of the tenth packet is 3 ' b110.
The four Steps Rule application module judges according to the flags parameters of rule application data format whether this applications effective earlier behind the comparison match result who receives the output of regular comparison module, then with comparison match result and comparison rule indicate carry out step-by-step and; Again with step-by-step and the indication of result and comparison rule compare, if equate, then expression meets corresponding rule condition; Operate according to application then, in this example, after the comparison match result 3 ' b111 of first data comes; With the comparison rule of first application rule indication 3 ' b101 carry out step-by-step with, the result who draws is 3 ' b101, equates with the comparison rule indication; Thereby expression meets the application corresponding rule condition; Indicate to carry out application corresponding operation 0 (storage), it also is qualified in like manner also can calculating second application rule, therefore also need carry out application corresponding operation 1 (carrying out total number statistics); Like this
The result that finally catches of the output of first packet is 2 ' b11, need carry out application corresponding operation 0 (storage) and 1 (carrying out total number statistics); Same calculating can get,
The result that finally catches of the output of second packet is 2 ' b11, need carry out application corresponding operation 0 (storage) and 1 (carrying out total number statistics);
The result that finally catches of the output of the 3rd packet is 2 ' b01, only need carry out total number statistics;
The result that finally catches of the output of the 4th packet is 2 ' b11, need store and total number statistics;
The result that finally catches of the output of the 5th packet is 2 ' b11, need store and total number statistics;
The result that finally catches of the output of the 6th packet is 2 ' b11, need store and total number statistics;
The result that finally catches of the output of the 7th packet is 2 ' b11, need store and total number statistics;
The result that finally catches of the output of the 8th packet is 2 ' b11, need store and total number statistics;
The result that finally catches of the output of the 9th packet is 2 ' b00, need abandon, and does not deal with;
The result that finally catches of the output of the tenth packet is 2 ' b00, need abandon, and does not deal with.
More than combine the accompanying drawing specific embodiments of the invention to be described; But these explanations can not be understood that to have limited scope of the present invention; Protection scope of the present invention is limited the claims of enclosing, and any change on claim of the present invention basis all is protection scope of the present invention.
Claims (10)
1. high-speed and high-efficiency network packet catching method of realizing based on FPGA; Comprise Rulemaking; Network packet is resolved, rule relatively with four part compositions of rule application, it is characterized in that regular comparison module compares the rule of the data based Rulemaking module that parses from the network packet parsing module; Rule application module is carried out application processes to the result who relatively comes out then, and whether the output network data portion should captively indicate.
2. Rulemaking according to claim 1 is characterized in that comprising the formulation of comparison rule and two parts of formulation of application rule.
3. the formulation of comparison rule according to claim 2, the width that it is characterized in that comparison rule are N bit (the N span is 2 to 64), and each comparison rule comprises sign; Use; Skew, mask, six parameters of maximum and minimum value; Comparison rule can have M (span of M is 1 to 64), and six parameters of the width of each comparison rule and other can be different.
4. sign according to claim 3 is used, skew, and mask, six parameters of maximum and minimum value is characterized in that sign is used to indicate this comparison rule whether to use; Use this comparison rule of expression and be the concrete parameter of the packet that is applied to parse; The original position of skew expression rule application; Mask is used for expression and whether shields corresponding rule position; Maximum is represented qualified maximum; Minimum value is represented qualified minimum value.
5. the concrete parameter of packet according to claim 4 is characterized in that protocol-dependent each parameter of lead code, mac source address, MAC destination address, IP source address, IP destination address, type, length, agreement and IP of character network packet.
6. the formulation of application rule according to claim 2; It is characterized in that each application rule comprises sign, use that three parameters of comparison rule indication are formed; Application rule can have X (the X span is 1 to 32), and the parameter of each application rule all can be different.
7. sign according to claim 6 is used, and three parameters of comparison rule indication is characterized in that sign is used for indication and should rule whether uses; Use expression and use this regular purposes, like storage, statistics, statistics with histogram; The bit wide of comparison rule indication is identical with the number M of comparison rule, and each bit position is corresponding with a rule, is used to indicate this rule whether to participate in the comparison of application rule.
8. network packet according to claim 1 is resolved; It is characterized in that the Ethernet data bag that is used for receiving resolves, protocol-dependent each parameter of lead code, mac source address, MAC destination address, IP source address, IP destination address, type, length, agreement and IP of each network packet is exported with register.
9. rule according to claim 1 relatively is characterized in that being used for each parameter utilization comparison rule of the packet that parses from network packet is compared, and then the comparison match result is exported.
10. rule application according to claim 1 is characterized in that being used for to carrying out matching ratio, the indication that the dateout bag is caught from the comparative result of regular comparison module output and each parameter of rule application.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2012100585459A CN102594624A (en) | 2012-03-06 | 2012-03-06 | Method for efficiently capturing network data packets at high speed based on field programmable gate array (FPGA) |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2012100585459A CN102594624A (en) | 2012-03-06 | 2012-03-06 | Method for efficiently capturing network data packets at high speed based on field programmable gate array (FPGA) |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN102594624A true CN102594624A (en) | 2012-07-18 |
Family
ID=46482844
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2012100585459A Pending CN102594624A (en) | 2012-03-06 | 2012-03-06 | Method for efficiently capturing network data packets at high speed based on field programmable gate array (FPGA) |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102594624A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103516773A (en) * | 2012-12-26 | 2014-01-15 | 深圳市友讯达科技发展有限公司 | Method, device and system for transmitting synchronous data |
| CN112383835A (en) * | 2020-11-02 | 2021-02-19 | 四川天邑康和通信股份有限公司 | Network switching and network packet capturing method based on intelligent set top box |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030225841A1 (en) * | 2002-05-31 | 2003-12-04 | Sang-Hern Song | System and method for preventing spam mails |
| CN101286896A (en) * | 2008-06-05 | 2008-10-15 | 上海交通大学 | IPSec VPN protocol drastic detecting method based on flows |
| CN101040279B (en) * | 2004-12-21 | 2010-04-28 | 中兴通讯股份有限公司 | System and method for filter rubbish e-mails faced to connection |
| CN102075318A (en) * | 2010-12-28 | 2011-05-25 | 重庆邮电大学 | FPGA-based multi-channel data packet monitoring and timestamp capture system and method |
-
2012
- 2012-03-06 CN CN2012100585459A patent/CN102594624A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030225841A1 (en) * | 2002-05-31 | 2003-12-04 | Sang-Hern Song | System and method for preventing spam mails |
| CN101040279B (en) * | 2004-12-21 | 2010-04-28 | 中兴通讯股份有限公司 | System and method for filter rubbish e-mails faced to connection |
| CN101286896A (en) * | 2008-06-05 | 2008-10-15 | 上海交通大学 | IPSec VPN protocol drastic detecting method based on flows |
| CN102075318A (en) * | 2010-12-28 | 2011-05-25 | 重庆邮电大学 | FPGA-based multi-channel data packet monitoring and timestamp capture system and method |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103516773A (en) * | 2012-12-26 | 2014-01-15 | 深圳市友讯达科技发展有限公司 | Method, device and system for transmitting synchronous data |
| CN112383835A (en) * | 2020-11-02 | 2021-02-19 | 四川天邑康和通信股份有限公司 | Network switching and network packet capturing method based on intelligent set top box |
| CN112383835B (en) * | 2020-11-02 | 2022-04-26 | 四川天邑康和通信股份有限公司 | Network switching and network packet capturing method based on intelligent set top box |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110287163B (en) | Method, device, equipment and medium for collecting and analyzing security log | |
| DE112011103561T5 (en) | Network processor and method for accelerating data packet parsing | |
| Benáček et al. | P4-To-VHDL: Automatic generation of high-speed input and output network blocks | |
| CN112217805B (en) | Multi-mode protocol adaptation method for power distribution Internet of things | |
| CN114285781B (en) | SRV6 service flow statistics method, device, electronic equipment and medium | |
| CN107483341B (en) | Method and device for rapidly forwarding firewall-crossing messages | |
| CN108809752B (en) | Adaptive monitoring method and device for network traffic, NPB (network provider node B) equipment and medium | |
| CN206922798U (en) | A kind of Multi-protocol converter, data transmitting equipment and communication system | |
| CN108650178B (en) | Service message processing method, device and system | |
| CN102594624A (en) | Method for efficiently capturing network data packets at high speed based on field programmable gate array (FPGA) | |
| CN102710491B (en) | The method and apparatus that the lossless real-time line rate of the PATRICIA trees aided in using PCAP type filters and hardware is filtered | |
| CN111917753A (en) | Modbus TCP message analysis method based on bit field | |
| CN102970189A (en) | Method and system for network data analysis based on application layer data | |
| Zazo et al. | Automated synthesis of FPGA-based packet filters for 100 Gbps network monitoring applications | |
| CN113347258A (en) | Method and system for data acquisition, monitoring and analysis under cloud flow | |
| CN105429901A (en) | Uplink data package forwarding method and device, and downlink data package forwarding method and device | |
| CN102497319B (en) | System and method for realizing single packet matching by utilizing automaton | |
| US10432582B2 (en) | Technologies for scalable local addressing in high-performance network fabrics | |
| CN110933001B (en) | Basic processing unit structure of extensible reconfigurable switch packet parser | |
| CN108123872B (en) | Traffic classification and forwarding method and system for power Internet of things | |
| WO2022176035A1 (en) | Conversion device, conversion method, and conversion program | |
| US10355985B2 (en) | Network property verification | |
| CN113206807B (en) | Method for information processing, electronic device, and computer storage medium | |
| CN208638363U (en) | A kind of Network records analytical equipment applied to substation network port | |
| CN107734055A (en) | A kind of data exchange monitoring system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| DD01 | Delivery of document by public notice | ||
| DD01 | Delivery of document by public notice |
Addressee: Shanghai Naxuan Electronic Technology Co., Ltd. Document name: Notification that Application Deemed to be Withdrawn |
|
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20120718 |