CN103685221A - A network invasion detection method - Google Patents
A network invasion detection method Download PDFInfo
- Publication number
- CN103685221A CN103685221A CN201310398922.8A CN201310398922A CN103685221A CN 103685221 A CN103685221 A CN 103685221A CN 201310398922 A CN201310398922 A CN 201310398922A CN 103685221 A CN103685221 A CN 103685221A
- Authority
- CN
- China
- Prior art keywords
- packet
- rule
- data
- state
- port
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Abstract
The invention brings forward a network invasion detection method. The method comprises the following steps: 1) data packets in a network are captured; 2) full protocol stack parsing analyzing is carried out on the captured data packets; and 3), a regular expression rule set is compiled into a DFA state conversion table through a determinacy finite state machine, and the DFA state conversion table is compressed; 4) the data which has been subjected to the parsing analyzing in the step 2 is written into the DFA state conversion table after the compression, and the matching is carried out; and 5) matching results are output. According to the invention, through the full protocol stack parsing analyzing carried out on the captured data packets, the decoding speed is substantially raised; requirements for real time decoding are realized; and requirements for an inner storage of a server are reduced. Furthermore, according to the invention, high speed message matching is realized through a DFA state transfer table, and accurate detection and defense can be carried out on existing threats, and high-grade escape techniques can be accurately identified and eliminated.
Description
Technical field
The present invention relates to network security technology field, relate in particular to a kind of network inbreak detection method.
Background technology
Network invasion monitoring is as one of current topmost Active Network Security measure, it is connected and is identified and respond by the hostile network in cyber-net resource, effectively supplement and perfect safety measures such as access control, data encryption, fire compartment wall, virus prevention, the integrality that has improved information security foundation structure, has become link indispensable in information system security solution.
Senior reclusion technology (AET, Advanced Evasion Technique), the reclusion stacking network strength (cyber-force) of the attacking case that is penetrated into the computer attack of various countries' political struggle is slightly shown in not freshly, the Bank of Korea's computer network fault, the New York Times of the U.S. and the attack that Wall Street Journal is subject to that occur are recently enough to illustrate this situation.Obviously hacker's attack means and the variation that matter has occurred ability, according to the report of Garter, from over 2011, the ability of cyber-defence has lagged behind the means of attack far away.And senior reclusion technology (AET) is certainly the technical barrier of headache particularly to IDS/IPS manufacturer, the test (4.15 chapters and sections part) that has increased separately AET the up-to-date IPS testing standard < < NSS_Labs_ips group test methodology v6.2 > > announcing from NSS Lab can be found out the attention degree to AET.
Fire compartment wall and IPS are the safety guarantee equipment of core in network, and fire compartment wall carries out the filtration of data conventionally according to data stream port, address, agreement etc., and IPS further carries out the depth detection of packet.For real understanding and Sampling network packet, the agreement that IPS needs deep understanding data flow to adopt.If on surface, the protocol format of the saturating data flow of exhaustive analysis is just enough, but fact proved really not so.As far back as 1998, from Tim Newsham and the Thomas Ptacek of Secure Network company, delivered about how penetrating technical article < < insertion, reclusion and the Denial of Service attack of IDS/IPS: avoid network invasion monitoring > >.Nearly 2 years, domestic relevant research, the Xu Jinwei researcher of Headquarters of the General Staff research institute once delivered too much piece of writing article with regard to AET.Conventional AET means have: character string is obscured, four kinds of the violations of encryption and tunneling technique, fragment technology and agreement.
For senior reclusion, attack and should consider new interception pattern, simple feature database match pattern no longer can reach interception object completely, and therefore, the present invention will propose a kind of brand-new network inbreak detection method, and the method will improve the coefficient of safety of network greatly.
Summary of the invention
In order to overcome the defect of prior art, the object of the invention is to propose a kind of network inbreak detection method that can improve network security coefficient.
For achieving the above object, network inbreak detection method of the present invention, it comprises following concrete steps:
1) catch the packet in network;
2) caught packet is carried out to full protocol stack parsing;
3) by deterministic finite state machine, regular expression rule set is compiled into DFA state transition table, DFA state transition table is compressed;
4) data after step 2 is resolved write DFA state transition table after compression and do and mate;
5) output matching result.
Further, the concrete grammar of described step 2 comprises:
1) protocol resolver carries out initialization, and the regularity loading after compiling is expressed collection;
2) read data packet, and it is carried out to packet transaction;
3) protocol type of described packet after identification grouping, and judge whether described packet needs to resolve, and if do not needed, directly abandons this packet; Otherwise, by the identical packet delivery of protocol type to managing together;
4) according to regular expression rule set, find the protocol-decoding rule corresponding with data protocol type, then the packet that comprises these data is scanned, according to described protocol analysis rule, from packet, extract the resolving information of described data.
Further, following principle is followed in the configuration of described output port grouping:
1) grouping of output port is first according to the business demand of back-end processing system, according to Business Processing type, divide into groups, when a system and a plurality of system have mutually respectively when overlapping in packet rule attribute, same output port there will be in plural grouping;
2) in each packets inner, in order to guarantee data processing task, realize balanced distribution, same output port can repeatedly appear in same group, in grouping, according to the data-handling capacity of back-end system corresponding to each port, determines the allocation proportion of each port data packet flow in this group;
3) in group each output port be distributed in proof load equilibrium time, guarantee that same TCP connects two-way all packets and must be forwarded on same output port, be convenient to rear end gathering and reducing received data.
Further, the setting of described packet rule comprises two kinds of modes:
Be using the IP address of packet and port information as a direct monitored object, based on these IP address informations, rule be arranged in the packet rule list based on address information, this rule tableau format is as follows:
Another kind is the regular set-up mode based on special field information, and this rule tableau format is as follows:
| Rule number | Fields offset amount | Field length | Matching content | Packet behavior |
| ? | ? | ? | ? | ? |
Wherein:
Rule number: be each regular unique identification sequence number;
Source, object IP address: the IP address value of the source of data packet transmission and destination;
Source, object IP mask: the subnet mask of the source of data packet transmission and destination IP field;
Source, destination interface: the source of data packet transmission and the port numbers of destination;
Fields offset amount: the side-play amount of the special field that the needs that start to calculate from IP packet content mate;
Field length: the length that needs the special field of coupling;
Matching content: the matching value of special field, these special field comprise as the receiver in URL address, Email mail and addresser's address field;
Packet behavior: " 0 " represents this data packet discarding, " 1 " represents this package forward to group 1, " 2 " represent this package forward to group 2, " N " represents this Datagram forwarding to group N, " 1 " represents to suspend this rule of use, if all fields in packet rule list except data packet row is field are all 0, represent that this rule is applicable to all packets.
The foundation that the filtration shunting as is after this processed in the setting of packet rule, when the filtration shunting rule match of IP address based on packet of the packet receiving from network and and port information, the filtration that packet will be set according to this rule divide Flow Behavior to be forwarded to corresponding grouping, if comprised in the packet receiving from network while being worth identical data with the special field setting in advance, the IP address information of this packet will be extracted, together with the corresponding filtration setting in advance, divide Flow Behavior, form a dynamic filtration rule of classification based on IP address and port information, write in the packet rule list of IP address based on packet and port information, all subsequent packet that TCP/UDP under this packet and this packet connects are all forwarded to corresponding grouping by the filter packets behavior of setting according to this rule, the packet rule forming based on feature field coupling has aging characteristic, after this connection finishes, this rule corresponding in IP address based on packet and the packet rule list of port information is also by deleted, for the packet newly receiving, to regenerate the new dynamic data rule of classification based on IP address and port information according to the process of above-mentioned special field coupling equally.
Described data filtering shunting is processed, and comprises following link:
(1) initialization: will filter shunting rule and import internal memory, if adopt CAM technology to realize, rule is set in CAM system, the content that system can be set according to user in " the IP address based on packet and the packet rule list of port information " and " the packet rule list based on special field information ", the IP address of generation based on packet and packet rule list and two tables of the packet rule list based on special field information of port information, wherein, the rule that IP address based on packet and the filtration of port information shunting rule list comprise static state setting and regular two parts of dynamic setting, the rule of static state setting is the content that user sets, the rule of dynamic setting is the Rule content that system dynamically generates when the filtration shunting rule match of carrying out based on special field information, because the rule of dynamic setting has ageingly, therefore importing to the IP address based on packet of internal memory and the packet rule list of port information increases " dynamically/static state " and " time-out count " two fields, and form is:
(2) intercepting raw data packets: the packet in intercept network, carry out protocal analysis, according to the Internet protocol data packet format, extract IP packet;
(3) filtration treatment:
First, system is mated the IP packet obtaining according to the filtration shunting rule list of the IP address based on packet setting in advance and port information, if the match is successful, according to " filtering a minute Flow Behavior " of arranging in rule list, packet is divided in each output port grouping, if the rule matching belongs to dynamic setting rule, and packet belongs to connection end packet, remove this dynamic setting rule, otherwise, by the time-out count of this rule clear 0, for situation about also processing over count value, the rule of this dynamic setting is also by deleted,
Packet for rule match failure in the filtration shunting rule list of the IP address with based on packet and port information, shunting with the filtration based on special field information the rule arranging in rule list again mates, if the match is successful, according to the filtration of setting in the filtration shunting rule of this special field information, divide Flow Behavior that packet is divided in corresponding output port grouping, and according to the source IP of this packet, the dynamic filtration shunting rule that the filtration of object IP address and this rule divides Flow Behavior to generate an IP address based on packet and port information joins in the filtration shunting rule list of IP address based on packet and port information, by mating with this new dynamic programming forming, will extract the follow-up data bag connecting under this packet, if packet is failed with the filtration shunting rule match based on special field information, abandon this packet,
(4) shunting is processed: adopt Diffluence Algorithm that high-low-position step-by-step XOR is carried out in the source IP address and the object IP address that are divided into each packet in each grouping, if TCP/UDP packet, again TCP/UDP port numbers and operation result are carried out to XOR again, finally obtain an operation result HASH value, by the output port sum delivery comprising in this operation result HASH value and this grouping, the result obtaining is exactly the output port sequence number of this packet correspondence in affiliated grouping again.
Further, described the identical packet delivery of protocol type is comprised to the concrete steps that manage together:
1) when receiving new data packets, set up new tables of data;
2) when receiving data message, first in described tables of data, search the Bale No. corresponding with it, if can find, then judge whether described data message is last bag, if not, described data message is included in corresponding packet; Otherwise, delete this packet;
3) when described packet occurs that out of order or repeating transmission and described packet are not normally deleted, start timer and in timing, described packet is deleted.
Further, in described step 3, the concrete steps that DFA state transition table is compressed comprise:
Every a line to DFA state transition table, is divided into same group by the identical input character of transition status;
Every a line to DFA state transition table, will divide adjacent or contiguous input character and the transition status thereof at same group to represent by triplet information, and this triplet information is bebinning character, bitmap and transition status;
Every a line to DFA state transition table, when dividing when the input character of same group can not be encoded by single described triplet information, is encoded by a plurality of described tlv triple;
Every a line to DFA state transition table, if the number of its described tlv triple comprising is no more than default threshold values, this statusline leaves in high-speed memory by the mode of above-mentioned tlv triple coding; Otherwise this statusline does not compress, by the mode of Linear array of the one dimension, be stored in the outer DRAM of sheet;
State value is remapped, make to deposit the state value of depositing in the high-speed memory of tlv triple and be all less than the state value that the outer DRAM of sheet deposits; Minimum in the state value that the outer DRAM of sheet is deposited is as cut off value.
Further, described bebinning character is input character minimum in grouping; Described bitmap, with binary representation, calculates this and organizes each input character with respect to the deviant of bebinning character, by bit position corresponding with these deviants on bitmap, is 1, and all the other bit positions are 0.
Further, the concrete grammar of described step 4 comprises:
4a) using the initial state of DFA state transition table and message initial character as initial input;
If 4b) this state is final state, finish coupling; If state value is less than described cut off value, performs step 4c and search the high-speed memory of depositing tlv triple; Otherwise execution step 4d searches the outer DRAM of sheet;
4c) according to the mode of Linear array of the one dimension index, from deposit the high-speed memory of tlv triple, read all tlv triple codings of this statusline, each tlv triple coding Bitmap bit offset value is put to 1 character and mate with this input character; If matched, get the transfering state of this character place tlv triple and the next character of message as input, execution step 4b; If do not match any tlv triple, it fails to match and finish;
4d) according to the mode of two-dimensional linear array indexing, from sheet, in DRAM, read corresponding transfering state; If there is transfering state, get the next character of this transfering state and message as input, execution step 4b; If there is no transfering state, it fails to match and finish.
Compared with prior art, beneficial effect of the present invention is:
The present invention is by carrying out full protocol stack parsing to packet, concrete grammar is to engage by protocol analysis and regular expression rule base, and adopt multithreading to carry out hardware decoding to complicated user service data, greatly improved decoding speed, meet real-time decoding requirement, and reduced the demand to server memory, reduced cost; The present invention also realizes high-speed message by DFA state-transition table mates, and can the threat existing accurately be detected and be defendd, and accurately identifies and remove senior escape technology, will greatly improve the coefficient of safety of network like this.
In addition, the present invention is based on sparse identical with close input character transfering state these two features of DFA state-transition table, adopt the tlv triple coded system of bebinning character, bitmap, transfering state, can effectively compress DFA state-transition table, thereby compression section is put into high-speed memory, effectively reduced the demand of the outer DRAM memory space of sheet.And DFA state-transition table leaves in and in high-speed memory, is also conducive to hardware and realizes high-speed message coupling.
Accompanying drawing explanation
Fig. 1 is the structural representation of protocol analysis system in the present invention;
Fig. 2 is that the present invention compresses DFA state transition table and carries out the hardware configuration schematic diagram of compatible portion with data.
Embodiment
Below in conjunction with accompanying drawing, method of the present invention is further described in detail.
The method of network invasion monitoring of the present invention, it comprises following concrete steps:
The first step, catches the packet in network;
Second step, carries out full protocol stack parsing to caught packet;
The 3rd step, is compiled into DFA state transition table by deterministic finite state machine by regular expression rule set, and DFA state transition table is compressed;
The 4th step, the data after step 2 is resolved write DFA state transition table after compression and do and mate;
The 5th step, output matching result.
Fig. 1 is the structural representation of the protocol analysis system based on stream in this example, and this system comprises router and protocol analysis equipment.Wherein, router is used in communication network repeating business datum, and forwarded data Replica portion is issued to protocol resolver (being DPI protocol analysis equipment); Protocol resolver is parallel on described router, the data that send for receiving router, the data that receive are divided into groups and the data after grouping are carried out to protocol type judgement, according to protocol type, data are sorted out, again data in each agreement are carried out to flow management, to guarantee that same data flow sends in same processing module, decode, finally by processing module, data are carried out to protocol analysis.The mode of this access in parallel is owing to being independently to carry out data protocol parsing, so it is less that mobile system is carried out to the impact of data communication.
The specific implementation process of second step is as follows:
Step 1: protocol resolver carries out initialization, and load the regular expression rule set after compiling.Wherein, DPI chip in protocol resolver could normally be worked after initialization, this step also can comprise before protocol resolver carries out initialization: regular expression rule set is compiled, convert thereof into DPI chip and can identify the data of form, protocol resolver could carry out DPI protocol-decoding to packet like this;
Step 2: read data packet, and it is carried out to packet transaction.This step also comprises that described router stores when packet is divided into groups and by distributed packet; The concrete steps of described packet transaction are:
(1) output port packet configuration rule is set: suppose that High Speed Network filtration shunting access platform has 8 output ports, port-mark number is respectively: 0,1,2,3,4,5,6,7, these ports are each safety monitoring device of opposite rear end respectively.According to the type of service processing demands of each safety monitoring device and data-handling capacity separately, each port is divided into four groupings:
| Packet number | Output port identification number |
| 1 | 3；2；0；2 |
| 2 | 1：4 |
| 3 | 5；7；5；6；7 |
| 4 | 0：4 |
They will process the packet of same type to be divided into the explanation of port in same grouping, and its middle port 2 occurs twice in grouping 1, and the back-end processing equipment that port 2 correspondences are described is by the data volume that receives and process in this output port 1/2nd; Port 4 is divided in the middle of 2 and 4 two groupings, illustrates that port 4 will the responsible data that forward from these two groupings.
(2) filter packets rule is set: filtering the regular setting of shunting is in order to realize the data filtering shunting of coarseness, will the useless data of back-end processing to be filtered out, and the rear end required data of each treatment facility is divided in the port grouping of appointment.The example that packet rule arranges is as follows:
IP address based on packet and the packet rule list of port information:
Packet rule list based on special field information:
| Rule number | Fields offset amount | Field length | Matching content | Filter a minute Flow Behavior |
| 301 | 42 | 15 | xy@yahoo.com.cn | 3 |
Processing procedure according to situation of supposition in (3) to packet b: the rule arranging in the packet rule list of the address information in packet b and IP address based on packet and port information is mated one by one, matching result finds that the match is successful with any rule, by fields offset amount in this packet, be that the value " xy@yahoo.com.cn " of 15 bytes after OX42 byte extracts with the rule in filtration shunting rule list based on special field information and mates one by one again, matching result meets rule 301.According to the filtration arranging in rule 301, divide Flow Behavior, packet b is divided into grouping 3, and extract the IP address information in this packet, together with the filtration arranging in rule 301, divide Flow Behavior, generate in the packet rule list that a new dynamic data filter packets rule joins IP address based on packet and port information, that is:
(5) to being divided into the data of each grouping, according to the Diffluence Algorithm proposing in the present invention, carry out packet transaction.The source IP address 61.125.3.8 of packet a and object IP address 10.10.25.30 high-low-position step-by-step XOR, the end value obtaining is carried out step-by-step XOR with the XOR result of source port number 90 and destination slogan 1290 again, port sum delivery in the result finally obtaining and grouping 4, that is:
{(0X3D7D_0X0A0A)_(0X0308_0X191E)_(0X005A_0X050A)}mod2={0X3777_0X1A16_0X0550}mod2=1
Operation result is 1, shows that this packet a should be from the 2nd the port output of dividing into groups 4, i.e. port 4 outputs.
The source IP address 10.10.19.131 of packet b and the high low byte step-by-step of object IP address 216.136.173.18 XOR, the end value obtaining is carried out step-by-step XOR with the XOR result of source port number 1664 and destination slogan 25 again, port sum delivery in the result finally obtaining and grouping 3, that is:
{(0X0A0A_0XD888)_(0X1383_0XAD12)_(0X0680_0X0019)}mod5={0XD282_0XBE91_0X0699}=4
Operation result is 4, shows that this packet b should be from the 5th the port output of dividing into groups 3, i.e. port 7 outputs.
(6) subsequent packet connecting under packet a, b is received in supposition, because the IP address of affiliated same connection packet is identical with port, therefore the result obtaining through above-mentioned calculating process is also identical, so all subsequent packet that guaranteed identical connection are still from identical port output.
(7) receive the end packet connecting under packet b, remove the rule 7 dynamically arranging.
If the high-rate fitration shunt method that does not rely on connection table that the present invention proposes adopts CAM technology to realize, for 12 road 2.5G POS accesses, possess the data access ability of 30G, meet the linear speed data access demand of high-speed backbone network.
Step 3: the protocol type to the data through grouping is identified, and obtains the protocol type of described data; Judge whether described data need to decode, if do not need, directly abandon described data, otherwise carry out step 4.Wherein, the method for judgement is to judge according to actual requirement whether the data of certain protocol type need to decode;
Step 4: the Check processing module that same data flow distribution is referred in same protocol resolver is processed, and to the data after protocol type identification are carried out to flow management; Described flow management comprises foundation, maintenance, deletion, aging the managing to data flow;
Same data flow is distributed to the concrete steps that the Check processing module in same protocol resolver processes to be comprised:
1) when receiving new data packets, set up new tables of data;
2) when receiving data message, first in described tables of data, search the Bale No. corresponding with it, if can find, then judge whether described data message is last bag, if not, described data message is included in corresponding packet; Otherwise, delete this packet;
3) when described packet occurs that out of order or repeating transmission and described packet are not normally deleted, start timer and in timing, described packet is deleted.
Step 5: described DPI protocol resolver carries out DPI protocol analysis according to regular expression rule base to the data of process flow management module.Described DPI protocol resolver has obtained after the protocol type of described data, according to regular expression rule base, finds the protocol-decoding rule corresponding with described data protocol type; Again the packet that comprises described data is scanned, according to described protocol-decoding rule, from described packet, extract the output information of described data.
It should be noted that, described DPI protocol resolver is supported across bag scanning.For across bag data message, Output rusults in next data message, comprises in even described output information across bag field, need in last bag and current bag, take out corresponding field result, can obtain the described output information across bag.In addition, due to what adopt in the present embodiment, it is the method for regular expression rule base matching and decoding, described output information is the side-play amount of the relatively described packet of matched position, so step 5 also comprises according to described output information, from described packet, extracts decoded information.
Than prior art, the present invention engages by DPI protocol analysis and regular expression rule base, and adopts multithreading to carry out hardware decoding to complicated user service data.Due under the same conditions, DPI hardware decoding speed is the more than 10 times of software decode speed, so the present invention has improved decoding speed greatly, meet real-time decoding requirement, and because DPI hardware supports is across bag scanning, only the last bag in a data flow of need preservation and the information of current bag, so this has just greatly reduced the demand to server memory, reduced cost.
The concrete methods of realizing of the 3rd step and the 4th step is as follows: table 1 shows the DFA state-transition table of existing employing standard two-dimensional linear storage of array form.
As shown in table 1, abscissa 0～255th, input character (8 bit widths, totally 256), ordinate S (O)～S (N) is state.Under this state of each line display, the transfering state of corresponding each input character.If input character does not have transfering state, just represent that it fails to match and finish.According to two-dimensional linear addressable array mode, use (state, input character) from DFA state-transition table, to find transfering state as index.
By the analysis of DFA state-transition table that many regular expressions are converted to, can find two features: a) table is sparse, quite most of (state, input character) do not have transfering state; B), to same state, the transfering state of many adjacent or contiguous input characters is identical.Based on above-mentioned two features, the present invention adopts bitmap coded mode to compress DFA state-transition table.
Table 2 shows the DFA state-transition table that adopts bitmap coded compression storage format of the present invention:
In order to realize from table 1, be compressed to table 2, the concrete compression method of the present invention is as follows:
Step 1), to each statusline, by input character grouping, the input character that transfering state is identical is divided into same group.
Step 2), to each statusline, dividing at identical and the adjacent or contiguous input character of the transfering state of same group and transfering state, by (bebinning character, bitmap, transfering state) tlv triple, represent.
Bebinning character is input character minimum in grouping.Bitmap is with binary representation, and low bit on the right.Calculate the input character of this group with respect to the deviant of bebinning character, by bit position corresponding with these deviants on bitmap 1, all the other bit positions 0.
Step 3), because the width of bitmap is fixed, can be 4～16 bits.Therefore to each statusline, when dividing when the input character of same group can not be encoded by single (bebinning character, bitmap, transfering state) tlv triple, can be encoded by a plurality of tlv triple.The form that final each statusline of formation is encoded by one or more (bebinning character, bitmap, transfering state) tlv triple.
Step 4), to each statusline, if the number of above-mentioned tlv triple is no more than certain default threshold values (such as 8 or 16), this statusline is stored in on-chip SRAM by the mode of above-mentioned tlv triple coding; Otherwise this statusline does not compress, by the mode of Linear array of the one dimension, be stored in the outer DRAM of sheet.
Through above-mentioned compression step, the compressible portion of DFA state-transition table can be left in on-chip SRAM, incompressible part is placed in the outer DRAM of sheet.
Step 5) simultaneously, state value is remapped, make the DFA state value of depositing in on-chip SRAM all be less than the DFA state value that the outer DRAM of sheet deposits.So only need and a cut off value (being the minimum in the state value that outside sheet, DRAM deposits) are big or small is that distinguishable states is to leave in on-chip SRAM, still leaves in the outer DRAM of sheet.
Below by an actual example, compression method of the present invention is specifically described.
Table 3 shows the actual DFA state-transition table fragment of an actual uncompressed.
As shown in table 3, in current state, be 16 o'clock, the transfering state of input character 5,7,8,11 correspondences is 17, and the transfering state of input character 6,9,10,12 correspondences is 23, and other input character does not have transfering state.
First this statusline is divided into groups to input character by transfering state, input character 5,7,8,11 is as a grouping, and input character 6,9,10,12 is as another grouping, and bebinning character is respectively input character 5 and 6 minimum in grouping.
Each input character in grouping being deducted to bebinning character and obtain deviant, is 1 by the bit position of this deviation post of bitmap, and other bit position of bitmap is 0.Such as for input character being 5,7,8,11 grouping, bebinning character is 5, and respectively each character being calculated to deviant is 5-5=0,7-5=2,8-5=3,11-5=6.0,2 of bitmap, 3,6 bit positions are 1, obtain 01001101 (bitmap is with binary representation, and on the right, rightmost is the 0th bit to low bit, and Far Left is the 7th bit).For input character, be 6,9,10,12 grouping, also adopt the formation bitmap that uses the same method.
After this DFA state-transition table process bits compression tlv triple (bebinning character, bitmap, transfering state) compression coding, the row of state 16 correspondences finally can be encoded into following two tlv triple: (5,01001101,17), (6,01011001,23), leave in on-chip SRAM.
Suppose that each state value takies 4 bytes, the byte that each statusline of original DFA state-transition table takies is: 4*256=1024.
Suppose that bitmap field takies a byte, each tlv triple takies 6 bytes, and every row fixedly tlv triple number is 8, uses the byte that each statusline of the DFA state-transition table of bits compression tlv triple of the present invention coding takies to be: 8*6=48.Memory space requirements greatly reduces.
Corresponding with above-mentioned compression, the concrete message matching method of the present invention is as follows:
Step 1) using the initial state of DFA state-transition table and message initial character as initial input.
Step 2), if this state is final state, finish coupling; If state value is less than cut off value, performs step 8 and search on-chip SRAM; Otherwise execution step 9 is searched the outer DRAM of sheet.
Step 3) according to the mode of Linear array of the one dimension index, from on-chip SRAM, read all tlv triple coding of this statusline, each tlv triple coding Bitmap bit offset value is put to 1 character and mate with this message character; If matched, get the transfering state of this character place tlv triple and the next character of message as input, execution step 7; If do not match any tlv triple, it fails to match and finish.
Step 4), according to the mode of two-dimensional linear array indexing, from sheet, in DRAM, read corresponding transfering state; If there is transfering state, get the next character of this transfering state and message as input, execution step 7; If there is no transfering state, it fails to match and finish.
Fig. 2 is the hardware configuration schematic diagram of realizing compression fit part of the present invention.As shown in Figure 2, CPU converts regular expression rule set to DFA state-transition table, and uses the bitmap coded mode of this patent invention to compress, and compression section is write in ASIC/FPGA on-chip SRAM, and compression section does not write in the outer DRAM of ASIC/FPGA sheet.Concrete compression step is described with technique scheme.ASIC/FPGA Memory Controller Hub completes the read-write sequence of SRAM and DRAM and controls.ASIC/FPGA message matching module is responsible for message to go coupling to leave the DFA state-transition table in on-chip SRAM and the outer DRAM of sheet in.Concrete message coupling step is described with technique scheme.
It should be noted that in the present invention that tlv triple after compression not only can leave in sheet in high-speed memory (as SRAM), also can leave in the outer high-speed memory of sheet.And this high-speed memory is except adopting SRAM, also can select the other forms of memory of speed more than existing DRAM, and the high-speed DRAM of new generation of process technological innovation in the future.
Above-described is only the preferred embodiment of the present invention, the invention is not restricted to above embodiment.Be appreciated that the oher improvements and changes that those skilled in the art directly derive or associate without departing from the spirit and concept in the present invention, within all should thinking and being included in protection scope of the present invention.
Claims (10)
1. a network inbreak detection method, is characterized in that, comprises the following steps:
1) catch the packet in network;
2) caught packet is carried out to full protocol stack parsing;
3) by deterministic finite state machine, regular expression rule set is compiled into DFA state transition table, DFA state transition table is compressed;
4) data after step 2 is resolved write DFA state transition table after compression and do and mate;
5) output matching result.
2. the method for claim 1, is characterized in that, the concrete grammar of described step 2 comprises:
1) protocol resolver carries out initialization, and the regularity loading after compiling is expressed collection;
2) read data packet, and it is carried out to packet transaction;
3) protocol type of described packet after identification grouping, and judge whether described packet needs to resolve, and if do not needed, directly abandons this packet; Otherwise, the packet that protocol type is identical is sorted out;
4) according to regular expression rule set, find the protocol-decoding rule corresponding with data protocol type, then the packet that comprises these data is scanned, according to described protocol analysis rule, from packet, extract the resolving information of described data.
3. method according to claim 2, is characterized in that, the described concrete steps that packet is divided into groups comprise:
First output port rule of classification and packet rule are set, the configuration of output port rule of classification is first each output port to be divided into groups according to the Business Processing type of corresponding backend application system, and then according to the disposal ability of back-end system corresponding to each port in group, determine that the allocation proportion of each port processing data packets flow in this group, the configuration of packet rule are according to IP address information or special field, packet to be divided in the middle of each grouping; Carry out again packet processing, first, by protocol analysis, in the raw data packets receiving from network, extract IP packet, according to the packet rule setting, will be divided in each output port grouping with subsequent treatment related data; Then the address and the port information that are divided into the packet in each group are carried out to Hash computing, the port sum delivery that hash value comprises with this grouping again, the result obtaining is exactly this packet corresponding output port sequence number in affiliated grouping.
4. method according to claim 3, is characterized in that, following principle is followed in the configuration of described output port grouping:
1) grouping of output port is first according to the business demand of back-end processing system, according to Business Processing type, divide into groups, when a system and a plurality of system have mutually respectively when overlapping in packet rule attribute, same output port there will be in plural grouping;
2) in each packets inner, in order to guarantee data processing task, realize balanced distribution, same output port can repeatedly appear in same group, in grouping, according to the data-handling capacity of back-end system corresponding to each port, determines the allocation proportion of each port data packet flow in this group;
3) in group each output port be distributed in proof load equilibrium time, guarantee that same TCP connects two-way all packets and must be forwarded on same output port, be convenient to rear end gathering and reducing received data.
5. method according to claim 3, is characterized in that, the setting of described packet rule comprises two kinds of modes:
Be using the IP address of packet and port information as a direct monitored object, based on these IP address informations, rule be arranged in the packet rule list based on address information, this rule tableau format is as follows:
Another kind is the regular set-up mode based on special field information, and this rule tableau format is as follows:
Wherein:
Rule number: be each regular unique identification sequence number;
Source, object IP address: the IP address value of the source of data packet transmission and destination;
Source, object IP mask: the subnet mask of the source of data packet transmission and destination IP field;
Source, destination interface: the source of data packet transmission and the port numbers of destination;
Fields offset amount: the side-play amount of the special field that the needs that start to calculate from IP packet content mate;
Field length: the length that needs the special field of coupling;
Matching content: the matching value of special field, these special field comprise as the receiver in URL address, Email mail and addresser's address field;
Packet behavior: " 0 " represents this data packet discarding, " 1 " represents this package forward to group 1, " 2 " represent this package forward to group 2, " N " represents this Datagram forwarding to group N, " 1 " represents to suspend this rule of use, if all fields in packet rule list except data packet row is field are all 0, represent that this rule is applicable to all packets.
6. method according to claim 5, it is characterized in that, because the rule of dynamic setting has ageingly, therefore importing to the IP address based on packet of internal memory and the packet rule list of port information increases " dynamically/static state " and " time-out count " two fields, and form is:
7. method according to claim 2, is characterized in that, the described concrete steps that the identical packet of protocol type is sorted out comprise:
1) when receiving new data packets, set up new tables of data;
2) when receiving data message, first in described tables of data, search the Bale No. corresponding with it, if can find, then judge whether described data message is last bag, if not, described data message is included in corresponding packet; Otherwise, delete this packet;
3) when described packet occurs that out of order or repeating transmission and described packet are not normally deleted, start timer and in timing, described packet is deleted.
8. the method for claim 1, is characterized in that, in described step 3, the concrete steps that DFA state transition table is compressed comprise:
Every a line to DFA state transition table, is divided into same group by the identical input character of transition status;
Every a line to DFA state transition table, will divide adjacent or contiguous input character and the transition status thereof at same group to represent by triplet information, and this triplet information is bebinning character, bitmap and transition status;
Every a line to DFA state transition table, when dividing when the input character of same group can not be encoded by single described triplet information, is encoded by a plurality of described tlv triple;
Every a line to DFA state transition table, if the number of its described tlv triple comprising is no more than default threshold values, this statusline leaves in high-speed memory by the mode of above-mentioned tlv triple coding; Otherwise this statusline does not compress, by the mode of Linear array of the one dimension, be stored in the outer DRAM of sheet;
State value is remapped, make to deposit the state value of depositing in the high-speed memory of tlv triple and be all less than the state value that the outer DRAM of sheet deposits; Minimum in the state value that the outer DRAM of sheet is deposited is as cut off value.
9. method as claimed in claim 8, is characterized in that,
Described bebinning character is input character minimum in grouping;
Described bitmap, with binary representation, calculates this and organizes each input character with respect to the deviant of bebinning character, by bit position corresponding with these deviants on bitmap, is 1, and all the other bit positions are 0.
10. the method for claim 1, is characterized in that, the concrete grammar of described step 4 comprises:
4a) using the initial state of DFA state transition table and message initial character as initial input;
If 4b) this state is final state, finish coupling; If state value is less than described cut off value, performs step 4c and search the high-speed memory of depositing tlv triple; Otherwise execution step 4d searches the outer DRAM of sheet;
4c) according to the mode of Linear array of the one dimension index, from deposit the high-speed memory of tlv triple, read all tlv triple codings of this statusline, each tlv triple coding Bitmap bit offset value is put to 1 character and mate with this input character; If matched, get the transfering state of this character place tlv triple and the next character of message as input, execution step 4b; If do not match any tlv triple, it fails to match and finish;
4d) according to the mode of two-dimensional linear array indexing, from sheet, in DRAM, read corresponding transfering state; If there is transfering state, get the next character of this transfering state and message as input, execution step 4b; If there is no transfering state, it fails to match and finish.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310398922.8A CN103685221A (en) | 2013-09-05 | 2013-09-05 | A network invasion detection method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310398922.8A CN103685221A (en) | 2013-09-05 | 2013-09-05 | A network invasion detection method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN103685221A true CN103685221A (en) | 2014-03-26 |
Family
ID=50321540
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310398922.8A Pending CN103685221A (en) | 2013-09-05 | 2013-09-05 | A network invasion detection method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103685221A (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104184722A (en) * | 2014-07-25 | 2014-12-03 | 汉柏科技有限公司 | Port group generating method and device of intrusion prevention system |
| CN104283736A (en) * | 2014-08-03 | 2015-01-14 | 成都网安科技发展有限公司 | Network communication quintuple fast matching algorithm based on improved automatic state machine |
| CN105141519A (en) * | 2015-07-24 | 2015-12-09 | 上海红神信息技术有限公司 | Pseudo-network node protection method based on load transformation |
| CN105530243A (en) * | 2015-12-03 | 2016-04-27 | 中国南方电网有限责任公司信息中心 | Realizing method of network attack event quantitative hierarchical algorithm |
| CN105721402A (en) * | 2014-12-04 | 2016-06-29 | 北京航管科技有限公司 | Method and apparatus for analyzing SITA message |
| CN113923002A (en) * | 2021-09-29 | 2022-01-11 | 山石网科通信技术股份有限公司 | Computer network intrusion prevention method and device, storage medium and processor |
| CN115225327A (en) * | 2022-06-17 | 2022-10-21 | 北京启明星辰信息安全技术有限公司 | Intrusion detection method with pre-matching rules based on FPGA network card |
| CN113923002B (en) * | 2021-09-29 | 2024-04-19 | 山石网科通信技术股份有限公司 | Computer network intrusion prevention method, device, storage medium and processor |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1564547A (en) * | 2004-03-25 | 2005-01-12 | 上海复旦光华信息科技股份有限公司 | High speed filtering and stream dividing method for keeping connection features |
| CN101605018A (en) * | 2009-06-17 | 2009-12-16 | 中兴通讯股份有限公司 | A kind of decoding depth message detection protocol method, equipment and system based on stream |
| CN102075430A (en) * | 2011-01-25 | 2011-05-25 | 无锡网芯科技有限公司 | Compression and message matching method for deep message detection deterministic finite automation (DFA) state transfer tables |
-
2013
- 2013-09-05 CN CN201310398922.8A patent/CN103685221A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1564547A (en) * | 2004-03-25 | 2005-01-12 | 上海复旦光华信息科技股份有限公司 | High speed filtering and stream dividing method for keeping connection features |
| CN101605018A (en) * | 2009-06-17 | 2009-12-16 | 中兴通讯股份有限公司 | A kind of decoding depth message detection protocol method, equipment and system based on stream |
| CN102075430A (en) * | 2011-01-25 | 2011-05-25 | 无锡网芯科技有限公司 | Compression and message matching method for deep message detection deterministic finite automation (DFA) state transfer tables |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104184722A (en) * | 2014-07-25 | 2014-12-03 | 汉柏科技有限公司 | Port group generating method and device of intrusion prevention system |
| CN104184722B (en) * | 2014-07-25 | 2017-05-24 | 汉柏科技有限公司 | Port group generating method and device of intrusion prevention system |
| CN104283736B (en) * | 2014-08-03 | 2018-05-22 | 成都网安科技发展有限公司 | A kind of network communication five-tuple Fast Match Algorithm based on improvement automatic state machine |
| CN104283736A (en) * | 2014-08-03 | 2015-01-14 | 成都网安科技发展有限公司 | Network communication quintuple fast matching algorithm based on improved automatic state machine |
| CN105721402A (en) * | 2014-12-04 | 2016-06-29 | 北京航管科技有限公司 | Method and apparatus for analyzing SITA message |
| CN105721402B (en) * | 2014-12-04 | 2019-02-05 | 北京航管科技有限公司 | A kind of method and apparatus parsing SITA message |
| CN105141519A (en) * | 2015-07-24 | 2015-12-09 | 上海红神信息技术有限公司 | Pseudo-network node protection method based on load transformation |
| CN105530243A (en) * | 2015-12-03 | 2016-04-27 | 中国南方电网有限责任公司信息中心 | Realizing method of network attack event quantitative hierarchical algorithm |
| CN105530243B (en) * | 2015-12-03 | 2016-11-16 | 中国南方电网有限责任公司信息中心 | A kind of implementation method of assault quantitative classification algorithm |
| CN113923002A (en) * | 2021-09-29 | 2022-01-11 | 山石网科通信技术股份有限公司 | Computer network intrusion prevention method and device, storage medium and processor |
| CN113923002B (en) * | 2021-09-29 | 2024-04-19 | 山石网科通信技术股份有限公司 | Computer network intrusion prevention method, device, storage medium and processor |
| CN115225327A (en) * | 2022-06-17 | 2022-10-21 | 北京启明星辰信息安全技术有限公司 | Intrusion detection method with pre-matching rules based on FPGA network card |
| CN115225327B (en) * | 2022-06-17 | 2023-10-27 | 北京启明星辰信息安全技术有限公司 | Intrusion detection method with pre-matching rule based on FPGA network card |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103491069A (en) | Filtering method for network data package | |
| CN103685224A (en) | A network invasion detection method | |
| CN103685221A (en) | A network invasion detection method | |
| CN1287570C (en) | High speed filtering and stream dividing method for keeping connection features | |
| CN101267313B (en) | Flooding attack detection method and detection device | |
| CN105429963B (en) | Intrusion detection analysis method based on Modbus/Tcp | |
| CN103841096A (en) | Intrusion detection method with matching algorithm automatically adjusted | |
| CN105337991B (en) | A kind of integrated message flow is searched and update method | |
| CN110650128B (en) | System and method for detecting digital currency stealing attack of Etheng | |
| Dai et al. | Finding persistent items in distributed datasets | |
| CN101018121B (en) | Log convergence processing method and convergence processing device | |
| CN105989061B (en) | Multidimensional data repeats detection fast indexing method under a kind of sliding window | |
| CN102253991B (en) | Uniform resource locator (URL) storage method, web filtering method, device and system | |
| CN103685222A (en) | A data matching detection method based on a determinacy finite state automation | |
| CN101577721A (en) | Method for splitting Broome filter by indexes and inserting, deleting and inquiring methods thereof | |
| CN101465760A (en) | Method and system for detecting abnegation service aggression | |
| CN104468107A (en) | Method and device for verification data processing | |
| CN105491018B (en) | A kind of network data security analysis method based on DPI technology | |
| CN103618733A (en) | Data filtering system and method applied to mobile internet | |
| CN105100023B (en) | Data packet feature extracting method and device | |
| CN107040405A (en) | Passive type various dimensions main frame Fingerprint Model construction method and its device under network environment | |
| CN111523012B (en) | Method, apparatus and computer readable storage medium for detecting abnormal data | |
| CN105407096A (en) | Message data detection method based on stream management | |
| CN104113598A (en) | Three-layer auditing method for database | |
| CN104394180A (en) | Wireless terminal authentication method, wireless router and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20140326 |
|
| RJ01 | Rejection of invention patent application after publication |