CN103475653A - Method for detecting network data package - Google Patents
Method for detecting network data package Download PDFInfo
- Publication number
- CN103475653A CN103475653A CN2013104011602A CN201310401160A CN103475653A CN 103475653 A CN103475653 A CN 103475653A CN 2013104011602 A CN2013104011602 A CN 2013104011602A CN 201310401160 A CN201310401160 A CN 201310401160A CN 103475653 A CN103475653 A CN 103475653A
- Authority
- CN
- China
- Prior art keywords
- algorithm
- module
- matching algorithm
- matching
- pattern matching
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
A method for detecting a network data package comprises the following steps of A) capturing the data package in a network, B) carrying out full protocol stack analysis on the captured data package to obtain protocol variable characteristics, namely, the data of all protocol layers of the original message data package; C) firstly selecting the most suitable multi-mode matching algorithm according to preset mode characteristics and the protocol variable characteristics, and then loading a mode matching algorithm library, and D) dynamically regulating the matching algorithm in the detection process according to a current network state. The full protocol stack analysis is carried out on the data package, decoding speed is greatly improved, real-time decoding requirements are met, and needs for server memory are reduced. In addition, the proper mode matching algorithm is flexibly selected for detected loopholes, matching speed and storage space are optimal, and effective protection on high-level escape invasion is achieved through a high-level reclusion technology.
Description
Technical field
The present invention relates to the network security technology field, relate in particular to a kind of detection method of network packet.
Background technology
Network invasion monitoring is as one of current topmost Active Network Security measure, it connects and is identified and respond by the hostile network on the cyber-net resource, effectively supplement and perfect safety measures such as access control, data encryption, fire compartment wall, virus prevention, improve the integrality of information security foundation structure, become link indispensable in the information system security solution.
Senior reclusion technology (AET, Advanced Evasion Technique), the reclusion stacking network strength of attacking (cyber ?the force) case that is penetrated into the computer attack of various countries' political struggle slightly is shown in not brightly, the Bank of Korea's computer network fault, the New York Times of the U.S. and the attack that Wall Street Journal is subject to that occur recently are enough to illustrate this situation.Obviously hacker's attack means and the variation that matter has occurred ability, according to the report of Garter, from over 2011, the ability of cyber-defence has lagged behind the means of attack far away.And senior reclusion technology (AET) is certainly the technical barrier of headache particularly to IDS/IPS manufacturer, the test (4.15 chapters and sections part) that has increased separately AET the up-to-date IPS testing standard " NSS_Labs_ips group test methodology v6.2 " of announcing from NSS Lab can be found out the attention degree to AET.
Fire compartment wall and IPS are the safety guarantee equipment of core in network, and fire compartment wall carries out the filtration of data usually according to data stream port, address, agreement etc., and IPS further carries out the depth detection of packet.For real understanding and Sampling network packet, the agreement that IPS needs the deep understanding data flow to adopt.If on surface, the protocol format of the saturating data flow of exhaustive analysis is just enough, but fact proved really not so.As far back as 1998, from Tim Newsham and the Thomas Ptacek of Secure Network company, delivered the technical article " insertion, reclusion and Denial of Service attack: avoid network invasion monitoring " about how penetrating IDS/IPS.Nearly 2 years, domestic relevant research, the Xu Jinwei researcher of Headquarters of the General Staff research institute once delivered too much piece of writing article with regard to AET.AET means commonly used have: character string is obscured, four kinds of the violations of encryption and tunneling technique, fragment technology and agreement.
Attack and should consider new interception pattern for senior reclusion, simple feature database match pattern no longer can reach the interception purpose fully, therefore, the present invention will propose a kind of intrusion detection method of automatic adjustment matching algorithm completely newly, and the method will improve the coefficient of safety of network greatly.
Summary of the invention
In order to overcome the defect of prior art, the object of the invention is to propose a kind of detection method that can improve the network packet of network security coefficient.
For achieving the above object, the detection method of network packet of the present invention, it comprises following concrete steps:
A) catch the packet in network;
B) caught packet is carried out to full protocol stack parsing, obtain the agreement variable, be i.e. the protocal layers data of original message packet;
C) select most suitable multi-pattern matching algorithm according to predefined pattern feature and agreement characteristics of variables, loading mode matching algorithm storehouse then, and dynamically adjust matching algorithm according to current network state in testing process.
Further, carry out full protocol stack in described step B and resolve and refer to according to the level of agreement and divide the end of to top and successively resolved, after all protocol datas that comprise application layer protocol are recombinated, more further proceed to resolve.
The concrete steps of further, protocol data being recombinated comprise:
Extract the Expressive Features relevant to Transmission Control Protocol from the raw data packets of catching, analyze TCP message wherein;
TCP message after analyzing is carried out again assembled according to its standard, and be saved in specific internal storage structure
In described step B, the protocal layers data comprise: MAC Address, IP address, protocol type data and other characteristics such as http_url, telnet_user.
Further, described step C comprises statistical analysis module, matching module and assessment handover module; Wherein, described statistical analysis module is used for pattern matching input rule tree is carried out to statistical analysis, and selects optimum multi-pattern matching algorithm according to analysis result; Described matching module is for providing quick multi-mode matching service by the unified call interface for the external call module, and the pattern matching result is described to event feeds back to the assessment handover module; Described assessment handover module is known current network state for the event information according to the matching module feedback, and adjusts accordingly the decision of current multi-pattern matching algorithm.
Further, described statistical analysis module, according to event configuration file set up pattern-matching rule tree, in rule tree, each tree node represents that a certain type-scheme coupling subtask comprises agreement variable or message data load is carried out to pattern matching; The agreement characteristics of variables of each node and associative mode feature in the statistical analysis rule tree; Investigate each alternative multi-pattern matching algorithm feature in algorithms library, select a kind of multi-pattern matching algorithm of the most applicable this rule tree node.
Further, described matching module, utilize the matching algorithm of current selection to provide quick multi-mode matching service for the external call module, network message or the agreement variate-value of external module input are carried out to the multi-key word search operation, and return to the final mode matching result; Generate corresponding pattern matching feedback event according to input and the result of this pattern matching selectively, process for the statistical analysis of assessment handover module simultaneously.
Further, described assessment handover module, according to the event information of matching module feedback, know current network state; Multi-pattern matching algorithm to current selection under current network state is assessed, if assessment result is dissatisfied, investigate each fundamental mode matching algorithm feature in algorithms library, select a kind of multi-pattern matching algorithm of applicable current network state, instruct the algorithmic dispatching engine to complete the dynamic switching to current multi-pattern matching algorithm.
Compared with prior art, beneficial effect of the present invention is:
The present invention, by packet being carried out to full protocol stack parsing, can improve the speed that network intrusion monitoring is analyzed effectively; Can greatly save the match event time, reduce rate of false alarm, improve accuracy rate; When the network event new feature occurring and when paying close attention to the ad hoc network data characteristics, can under the prerequisite of upgrade applications not, promptly these features be increased in the characteristic module that contains the NIDS event base, reach the purpose of warning; Can also pass through user's definable interface flexibly, realize that the renewal of characteristic module and program are irrelevant, guarantee the quick response of nids system to security incident, but and the ability of user's on-site customization characteristic event.
In addition, the present invention take full advantage of various multi-pattern matching algorithms advantage, select flexibly suitable matching algorithm according to the current application scene, reach optimum on matching speed and memory space.
The accompanying drawing explanation
Fig. 1 is the structural representation after full protocol stack is resolved in the present invention;
Fig. 2 is Transmission Control Protocol and a data restoration method flow chart thereof based on parallel processing;
Fig. 3 is the typical applied environment carried out an invention;
Fig. 4 is the output example after the TCP restructuring.
Fig. 5 is the adaptive multi-model matching method technological architecture;
Fig. 6 is static nature statistical flowsheet figure in static adaptation module;
Fig. 7 is state algorithm trade-off decision flow chart in static adaptation module;
Fig. 8 is behavioral characteristics statistical flowsheet figure in the dynamic self-adapting module;
Fig. 9 is dynamic algorithm trade-off decision flow chart in the dynamic self-adapting module.
Embodiment
Below in conjunction with accompanying drawing, method of the present invention is further described in detail.
Network data packet inspection method of the present invention comprises following concrete steps:
The first step, catch the packet in network;
Second step, carry out full protocol stack parsing to caught packet, obtains the agreement variable, i.e. the protocal layers data of original message packet;
The 3rd step, select most suitable multi-pattern matching algorithm according to predefined pattern feature and agreement characteristics of variables, loading mode matching algorithm storehouse then, and dynamically adjust matching algorithm according to current network state in testing process.
The concrete methods of realizing of second step is as follows:
Full protocol stack is resolved the level division referred to according to agreement and is successively resolved the end of to top, after all protocol datas that comprise application layer protocol are recombinated, more further proceeds to resolve.
Its implementation process is to send a series of relevant original message packets to local network, and comprises following initial data.Respective description is as shown in table 1 below:
Table 1
Wherein, agreement variable in upper table is in order to record the characteristic of corresponding data field explanation, and [string.12]=ISS shows that (INTERNET CoNTROL MESSAGEPROTOCOL: the string variable that internet control information agreement) side-play amount of data field is 12 byte places is " ISS " at ICMP;
This resolution system includes:
Protocol analysis module: for this original message packet is carried out to protocol analysis;
Protocol data buffer zone module: for storing each layer protocol data;
Characteristic module: for storing characteristic;
When starting detection method of the present invention, this resolution system is carried out to initialization, read the characteristics such as relevant protocol data, arithmetic type, computing variable name, computing variate-value, characteristic event return value variable from described characteristic module, and be stored in the built-in storage of computer; This characteristic is for " Ping ISS " scanning characteristic event, be specially: agreement variable (data) is that icmp_type and [string.12], arithmetic type are that character string and integer operation type, computing variable name are that to equal operation (=) and comprise operation (^), computing variate-value be that the title of character string ISS and integer value 8, characteristic event return value variable is " length ", and the agreement variable is " icmp_length " accordingly.
The concrete steps of this analytic method are as follows:
Original message packet in as above table 1 is carried out to protocol analysis, obtain each layer protocol data of this original message packet.
The protocol analysis module is carried out protocol analysis to the as above original message packet described in table 1, is the structural representation after protocol analysis of the present invention as shown in Figure 1.Protocol analysis is to divide incremental according to the level of procotol, it successively carries out end of to top, and the protocol data assignment after resolving is to the agreement variable, store the protocol data buffer zone module into, as " 8 " assignment given to " ICMP_type ", will " ISS " assignment to " [String.12] " etc.
The system that all protocol datas that comprise application layer protocol are recombinated comprises:
The network message acquisition equipment, for obtaining the overall network message data of transmission at network;
The network message access device, store specific data structure for the network message by catching into according to specific rule, when implementation agreement analysis and restructuring, can carry out access fast to message;
Transmission Control Protocol is analyzed and reconstruction unit, and the primitive network message after catching for analysis is analyzed Transmission Control Protocol feature wherein, and according to its feature, message is assigned in the network message access device;
The application layer data notifying device, for after the Transmission Control Protocol data recombination, notice application layer program is to data analysis and processing wherein, network enabled management and information security application.
The concrete grammar that all protocol datas that comprise application layer protocol are recombinated comprises the steps:
1. primitive network message capturing platform
For realize at a high speed, realize that the complete of network message catch under large capaciated flow network environment, the present invention has adopted the message capturing technology based on zero-copy in force, the technology that realizes of zero-copy is as follows:
Zero-copy has been realized reducing data copy number of times process that datagram transmits from the network equipment to the user program space, reduces system call, realizes that zero of CPU participates in, and thoroughly eliminates CPU load in this respect.
Realize that the main technology that zero-copy is used is DMA data transmission technology and region of memory mapping techniques.Traditional network data newspaper is processed, and needs to arrive the operating system memory space through the network equipment, and the Installed System Memory space, to twice, user application space copy procedure, also needs to experience the system call that the user sends to system simultaneously.
At first zero duplication technology utilizes the DMA technology that the network data newspaper is directly delivered in the pre-assigned address space of system kernel, avoids the participation of CPU; Simultaneously, the region of memory of storage datagram in system kernel is mapped to the application space of trace routine, also having a kind of mode is to set up buffering area at user's space, and it is mapped to kernel spacing, be similar to the kiobuf technology under the linux system, trace routine directly conducts interviews to this piece internal memory, thereby has reduced the memory copying of system kernel to user's space, has reduced the expense of system call simultaneously.
2. network data message fast access
The present invention adopts the two-dimentional buffering area structure based on the Hash chained list.The Hash watch has advantages of that calculating is simple, locating speed is fast.Hash value utilizes the four-tuple data (source, purpose IP address, source, destination interface) in the TCP message, by designing suitable Hash function, generates.
The thought that the memory management scheme of buffering area adopts the dynamic and stalic state to combine, can guarantee higher access speed, also has higher resource utilization ratio simultaneously.Buffering area adopts two-dimentional logical construction, do not need for from the message capturing system acquisition to the other storage allocation of data space in order to organize buffering area, but directly adopting the relation of internal memory pointer to set up chained list, this strategy has greatly reduced the storage overhead of system, has accelerated access speed.
3.TCP protocol data parallel reassembling
(1) multithreading organizational politics
The network data produced in the large capaciated flow network lower unit interval of environment can be very large, more or less exists relation between network data simultaneously; The thread resources of single system is also limited in addition.Therefore adopt the multi-threaded parallel reorganization scheme of specific policy just can satisfy the demands.
The present invention adopts Thread Pool Technology to organize the Protocol reassembling thread.After application program launching, create immediately the thread (N1) of some, put into idle queues.These threads are all in blocked state, do not consume cpu resource, but take less memory headroom.After task arrives, Buffer Pool is selected an idle thread, and task is imported in this thread and moved.When N1 thread, all after Processing tasks, Buffer Pool creates the new thread of some automatically, for the treatment of more task.After task is finished, thread does not exit yet, but continues to remain in pond the task next time of waiting for.When systematic comparison is idle, most of thread is all always in halted state, thread pool auto-destruct part thread, recovery system resource.This strategy has reduced the overhead that frequent establishment thread brings, and the while has also avoided the number of threads due to massive dataflow too much to cause the defect of system crash.
(2) thread synchronization scheme
When system is given certain link data newspaper a thread and is gone to process, because all datagrams of same link all can be by same thread process, so data master data distribution journey of calling the score of giving the correct time will not be absorbed in wait on processing threads is also finished dealing with.If equally also will be waited for call the score distribution journey notice of master data while in like manner on processing threads, there is no data.Enter busy the grade for fear of thread and take cpu resource, the present invention with two groups of mutexs complete master data call the score distribution journey and packet processing thread synchronously.
In calling the score the distribution journey, master data at first locks corresponding parameter mutex, if now this packet processing thread just reports the distribution journey master data is called the score can wait for release in deal with data, can release after processing threads is finished dealing with this mutex make the master data distribution journey of calling the score continue to carry out, just parameter is carried out at this moment release processing threads again of assignment after having locked the parameter mutex, processing threads just can carry out the processing of data.
4.TCP parallel reassembling state notifying
Selecting suitable opportunity and mode to notify the application layer program is an important problem, uses in the present invention the concept of exchange to be notified as data processing unit.Once exchange is exactly once communicating by letter of client and service end, and client transmission data and service end are to the once exchange of the common formation of the reply of this secondary data.
Exchange each time can be carried out notice three times, is current exchange while just having set up for the first time, and namely server end has just received that first valid data of client give the correct time, and now notify status is TCP_SWAP_CLIENT; Be client while receiving first datagram of server end for the second time, now notify status is TCP_SWAP_SERVER; Notify status is that TCP_SWAP_FINISH can occur when following several situation for the third time:
(1) when the ACK of server end changes.
(2) receive the RST datagram.
(3) receive the FIN datagram.
(4) link overtime.
5. embodiment scene and result are described
In order to verify validity of the present invention, we have built typical applied environment, the typical applied environment that Fig. 3 is embodiment.The hardware environment of Transmission Control Protocol analysis and reorganization server is two Intel (R) Xeon (R) CPU, and dominant frequency is 2G; Memory size is 2G; Software environment is that operating system is Red Hat4.1.1-52, and the kernel version is 2.6.21.
Embodiment, for adopting the tcpdump application program to record the data of the portal of flowing through, forms data file.Select a machine else, operation tcpreplay application program, carry out playback by different speed to operation protocal analysis and the machine of restructuring program, observe the behavior of protocol stack, table 1 and table 2 are respectively protocol stack and process the performance that pure HTTP data on flows and mixed traffic data show under different playback speeds.Fig. 4 is the reduction output that the real network protocol analysis program calls Interface realization of the present invention.
The present embodiment can illustrate no matter be aspect performance or correctness, and the present invention is applicable to utilize (processor quantity is not more than 2) on single node computational resource platform to reduce for implementation agreement under large capaciated flow network environment.
The concrete methods of realizing of the 3rd step is as follows:
Multi-pattern matching algorithm storehouse in adaptive multi-mode matching system of the present invention comprises following 5 basic multi-pattern matching algorithms:
● standard A CBM algorithm: based on finite-state automata Model Establishment pattern matching AC tree, according to the message current character, carry out state transitions, and, in conjunction with the BM algorithm, adjust the character number that match window advances, realize the window quick sliding.This algorithm advantage is that the correlation of matching speed and pattern feature and network the present situation is little; Shortcoming is that AC tree state machine required memory space is large.
● ACBM improves algorithm 1: preserve 256 succeeding states under each state of standard A CBM algorithm, but be the situation of ASCII character or Chinese character for match pattern (keyword), only need 128 possible states.The NextState array size that this algorithm is preserved each state is adjusted into 128, to adapt to the search of ASCII character or Chinese character mode (keyword).
● ACBM improves algorithm 2: the character that occurs of take in set of modes is stored the AC tree as node, and stores the AC tree unlike take state in traditional ACBM algorithm as node.The character quantity that this algorithm is applicable in set of modes is far smaller than 256 situation, can effectively save AC tree memory space.
● standard WM algorithm: show to determine by quick HASH whether the current text substring appears in a certain pattern, if do not occur, utilize wide character BM jump thought to carry out fast moving current text match window; If occur, carry out exact matching.This algorithm advantage is that consumed memory space is little, and under the proper network state, search speed is fast; Shortcoming is that the correlation of search speed and pattern feature and network the present situation is large;
● WM improves algorithm: the algorithm while being 1 byte for the pattern minimum length specially, and the pattern that is 1 byte by length is mated separately, and pattern of surplus is being used the WM algorithmic match.
These basic multi-pattern matching algorithms have all been realized unified calling interface in algorithm United Dispatching module, comprise that algorithm initialization, algorithmic match are called and algorithm unloads processing.
The matching algorithm library module, in storehouse, each fundamental mode matching algorithm object all must be realized the unified call interface, comprises algorithm initialization, pattern matching and algorithm unloading Processing Interface; In storehouse, each basic multi-pattern matching algorithm can add as required new algorithm or delete assignment algorithm.
The static self adaptation stage in adaptive multi-model matching method of the present invention, according to event configuration file set up pattern-matching rule tree, in rule tree, each tree node represents that a certain type-scheme coupling subtask comprises agreement variable or message data load is carried out to pattern matching; The agreement characteristics of variables of each node and associative mode feature in the statistical analysis rule tree; Investigate each alternative multi-pattern matching algorithm feature in algorithms library, select a kind of multi-pattern matching algorithm of the most applicable this rule tree node.
By following four steps, implemented:
At first, set in advance critical parameter value by configuration file.Allow the critical value arranged to comprise:
● MAXNUM_SIMI: the maximum that means the pattern quantity permission that in set of modes, prefix is identical.Be mainly used in standard WM algorithm, if the identical pattern quantity of prefix is greater than MAXNUM_SIMI, in the WM algorithm, HASH subchain search efficiency is lower.
● NUM_OF_SET_AC: mean the quantity of character in set of modes, be mainly used in ACBM and improve algorithm 2, when the pattern character concentrates the number (nChar) of character to be less than this value, use ACBM improvement algorithm 2 to save memory space remarkable.
● WM_LEN: be mainly used in the selection of WM algorithm, when the minimum length of pattern equals 1, and inferior little length is used the WM1 efficiency of algorithm high while being more than or equal to WM_LEN.
● MAXLEN: the maximum length that means pattern.
● MAXCOUNT: the quantity that means pattern.
● LIMIT_OF_MEM: mean the restriction of matching algorithm to internal memory.
● LIMIT_OF_CPU: mean the restriction of matching algorithm to CPU.
Secondly, for selecting various multi-pattern matching algorithms, configure required pattern feature statistics and analysis parameter, while selecting five kinds of algorithms listed above, can be configured to lower 7 parameters, these 7 variablees are described as follows the impact of algorithm:
● minlen: the minimum length of pattern.
● sub_minlen: the inferior little length of pattern.
● nSimil: the number of modes that has mutually same byte prefix.
● nSimi2: the number of modes that has identical two byte prefixes.
● maxlen: the maximum length of pattern.
● count: the total quantity of pattern.
● nChar: the number of character concentrated in the pattern character.
Selective multi-pattern matching algorithm has at present: standard A CBM algorithm, ACBM improve algorithm 1, ACBM improves algorithm 2, standard WM algorithm and WM and improves algorithm.These five kinds of algorithms static self adaptation during the stage scope of application as follows:
● standard A CBM algorithm: be greater than 2 when the set of modes number of elements is greater than 128, nSimi2, and the ACBM memory space is while allowing.
● ACBM improves algorithm 1: when the set of modes number of elements in 128 and NUM_OF_SET_AC between, nSimi2 is greater than MAXNUM_SIMI, and memory space is while allowing.
● ACBM improves algorithm 2: when the set of modes number of elements is less than NUM_OF_SET_AC, nSimi2 is greater than 20, and memory space is while allowing.
● standard WM algorithm: the pattern minimum length at least is greater than 2 bytes, and nSimi2 is less than MAXNUM_SIMI; Perhaps the pattern minimum length is that 1, nSimi1 is less than MAXNUM_SIMI.
● WM improves algorithm: the pattern minimum length is 1, and inferior minimum length is greater than WM_LEN, and nSimi2 is less than MAXNUM_SIMI.
Again, the static nature statistical analysis process computation decision-making foundation parameter f lag shown in 6 with reference to the accompanying drawings.Because the condition of decision-making is many, so quote a decision-making foundation parameter f lag, utilize wherein flag bit to decide called algorithm.Flag be one without symbol 16bit short: unsigned short flag, flag flag bit default value is 0, from the higher bit position to low bit, bit15～bit10: the behavioral characteristics of Record Matching Algorithm, for the dynamic self-adapting stage; Bit7～bit0: logging mode static nature.The flag flag bit is described as follows.
Bit15～bit12: retain;
Bit11=1: represent that matching algorithm incoming message " batter's symbol " probability of occurrence under this agreement variable is large;
Bit10=1: represent that under this agreement variable, the keyword probability that the match is successful is large, corresponding event frequently occurs;
Bit9=1: required memory space is greater than permissible value;
Bit8=1: required CPU value is greater than permissible value;
When bit7 and bit6:minlen=1, bit7=1; Sub_minlen>3, bit6=1; (bit7, bit6) has following combination (0,0) implication minlen>1 and sub_minlen<4; (0,1) implication minlen>3; (1,0) implication minlen=1 and sub_minlen<4; (1,1) implication minlen=1 and sub_minlen>3;
Bit5=1: in all patterns, the pattern quantity that first byte is identical is greater than MAXNUM_SIMI;
Bit4=1: except the pattern that length is 1, the pattern quantity that the first two byte is identical is greater than MAXNUM_SIMI;
Bit3=1: the agreement types of variables is ASCII character or Chinese; Bit3=0: the agreement types of variables is the straight binary character;
bit2＝1：nChar＜NUM_OF_SET_AC；
bit1＝1：maxlen＞MAXLEN；
bit0＝1：count＞MAXCOUNT。
Describe the statistical method of decision parameters in Fig. 6 in detail according to this principle.The input of this module comprises internal memory, CPU situation, and the parameter of statistics.Then module is according to these input values, and to the decision-making foundation parameter f, lag modifies.Decision-making foundation parameter f lag after finally output statistics.
Finally, carry out static multi-pattern matching algorithm trade-off decision process (seeing accompanying drawing 7) based on decision-making foundation parameter f lag.Fig. 8 is input as decision-making foundation parameter and alternative algorithm.When carrying out the algorithm selection, according to the flag bit of flag, to be selected successively, step is as follows:
1) the 7th～4th bit of judgement flag, if (bit7==0& & Bit4==0), mean that the pattern minimum length is greater than 1, and it is few to have the pattern quantity of common two byte prefixes, now is applicable to using the WM algorithm;
2) if (bit7, bit6, bit5, bit4)==(1,0,0,0), mean that the pattern minimum length is 1, inferior little length is less than WM_LEN, and it is few to have the pattern quantity of common two byte prefixes, now selects the WM algorithm;
3) if (bit7, bit6, bit5, bit4)==(1,1,1,0), mean that the pattern minimum length is 1, inferior little length is greater than WM_LEN, and it is few to have the pattern quantity of common two byte prefixes, now is applicable to improving algorithm with WM.
4) judgement bit2, if bit2==1 shows that the concentrated character quantity of pattern character is few, be applicable to improving algorithm 2 with ACBM, greatly reduces memory space, forwards the 7th step to;
5) judgement bit3, if bit3==1, show the pattern character concentrate character quantity in 128 and NUM_OF_SET_AC between, select ACBM to improve algorithm 1, forward the 7th step to;
6) select default algorithm, as ACBM algorithm (if select the ACBM algorithm, forwarding the 7th step to).The ACBM improvement algorithm that now judgement is not suitable for using the WM algorithm and memory space is reduced according to mode feature, therefore utilize the ACBM algorithm to complete search.
7) when selecting to use ACBM algorithm, ACBM to improve algorithm 1 and 2, need judgement shared memory space when realistic model is processed whether to surpass acceptable value, if surpass, select the WM algorithm.
8) export selected algorithm, call the algorithm load-on module, the algorithm load-on module will load initialization function entrance, adaptation function entrance and the unloading function entrance of matching algorithm, and call the initialization function of matching algorithm.
In the dynamic self-adapting stage in adaptive multi-model matching method of the present invention, according to the pattern matching feedback event information that the statistics and analysis pattern matching stage produces, know current network state; Multi-pattern matching algorithm to current selection under current network state is assessed, if assessment result is dissatisfied, investigate each fundamental mode matching algorithm feature in algorithms library, select a kind of multi-pattern matching algorithm of applicable current network state, instruct the algorithmic dispatching engine to complete the dynamic switching to current multi-pattern matching algorithm.
The dynamic self-adapting stage is implemented by four steps:
1) at first, by configuration file, set in advance the algorithm decision parameters, these parameters comprise:
● HOLD_TIME: the locking maximum duration, agreement variable algorithm is adjusted, must be at this agreement variable not within locking time;
● LIMIT_EVENT_DURATION and LIMIT_EVENTN_UM: event-control parameter.Meaning of parameters is that certain event probability of occurrence is high, refers to that the same time exists
The number of times occurred in the LIMIT_EVENT_DURATION time is greater than LIMIT_EVENT_NUM;
● LIMIT_PACKT_DURATION and LIMIT_PACKT_NUM: message characteristic is controlled parameter.Incoming message (text) when meaning of parameters is a certain agreement variable coupling, at LIMIT_PACKT_DURATION in the time, character occurs continuously, such text number is greater than LIMIT_PACKT_NUM;
● LIMIT_FLUX_DURATION and LIMIT_FLUX_NUM: flow control parameter.Meaning of parameters is at LIMIT_FLUX_DURATION in the time, and message flow is greater than LIMIT_FLUX_NUM;
2) consider the impact of network state on various basic multi-pattern matching algorithms according to table 1.
The impact of table 1 network condition on pattern matching algorithm
3) the coupling description event produced in pattern matching process is realized to the behavioral characteristics statistical analysis is to know current network conditions, concrete behavioral characteristics statistical analysis flow process is shown in accompanying drawing 7.
The workflow of Fig. 8 is as follows:
● by configuration, deciding need to be according to which behavioral characteristics for example, to the algorithm adjustment: the feature of the probability of event matches success, matching algorithm incoming message and current network flow etc.;
● these behavioral characteristics are fed back;
● add up these behavioral characteristics, and revise decision-making foundation parameter f lag.For example, when a certain event occurrence number in time LIMIT_EVENT_DURATION is greater than LIMIT_EVENT_NUM, revise the corresponding flag bit of flag;
● locate the associated agreement variable of this behavioral characteristics; And judge whether this agreement variable needs to carry out multi-mode matching, and whether the multi-pattern matching algorithm of this agreement variable is configured to " capable of regulating " etc.;
● output decision-making foundation parameter f lag and agreement variable ID, as the input of dynamic algorithm trade-off decision module.
4), according to the current network situation, in the pattern-matching rule tree, each node (agreement variable) is selected more suitable matching algorithm.Feature according to current alternative five kinds of algorithms, mainly be divided into two classes, and its accommodation is in Table 1.For current this selected two classes algorithm, the WM algorithm is subject to web influence large, so be mainly to adjust to ACBM class algorithm (comprising that it improves algorithm) from the WM algorithm.
The specific works flow process is shown in accompanying drawing 9.
1), according to decision-making foundation parameter f lag, judgement is the dynamic adjustment which kind of reason causes.For example, it is " occurring causing the algorithm adjustment because event is frequent "; Below with this reason, introduce the implementation method that this dynamic algorithm is selected module;
2), if original algorithm is ACBM algorithm (or ACBM1, ACBM2), keep; If algorithm originally is the WM algorithm, need to adjust.According to bit3 in flag and bit4, which kind of ACBM algorithm judgement adjusts.Equally, also need memory space is estimated, if memory space in allowed band, is adjusted into ACBM algorithm (or ACBM1, ACBM2); Otherwise keep original algorithm;
3) export selected results.
The pattern matching stage in adaptive multi-model matching method of the present invention, utilize the matching algorithm of current selection to provide quick multi-mode matching service for the external call module, network message or the agreement variate-value of external module input are carried out to the multi-key word search operation, and return to the final mode matching result; Generate corresponding pattern matching feedback event according to input and the result of this pattern matching selectively, process for the statistical analysis of dynamic self-adapting module simultaneously.
Workflow is as follows:
When the external call module has message to need the execution pattern matching operation, by the pattern matching interface called in the unified call interface module, to realize Fast Pattern Matching, after the pattern matching interface interchange finishes, may produce corresponding coupling and describe event, and be written in the feedback event queue, do statistical analysis for the dynamic self-adapting module, and make where necessary the pattern matching algorithm handover decisions.
In adaptive multi-model matching method of the present invention, behavioral characteristics statistic analysis flow process is described below the dynamic self-adapting stage:
The coupling description event that the pattern matching stage produces is carried out to polymerization, statistical analysis, and statistic analysis result is submitted to " dynamic algorithm trade-off decision module ", for the latter provides the algorithm changeover decision-making foundation.The event that " dynamic algorithm trade-off decision module " is concerned about when decision-making has: a certain event is greater than LIMIT_EVENT_NUM in a period of time (LIMIT_EVENT_DURATION) occurrence number, think that this event probability of occurrence within the unit interval is high, may need corresponding multi-pattern matching algorithm is adjusted; If a certain agreement variable is within a period of time (LIMIT_PACKT_DURATION), the message of inputting during coupling is that " bad character string " (character that similar " AAAAAAAA " repeats like this occurs, and it is high that the prefix of match window and suffix appear at probability in message) number of times be greater than LIMIT_PACKT_NUM, think that the matching algorithm of this agreement variable may need to adjust.
The overall architecture that has meaned this adaptive multi-mode matching system in Fig. 5.Module, this system is divided into to four large modules.When system initialization, the configuration of input rule tree, then call static adaptation module, and the multi-pattern matching algorithm of agreement variable is carried out to initialization; When system works, be input as text to be matched, be output as the result of coupling; And, when system works, call the dynamic self-adapting module multi-pattern matching algorithm of agreement variable is adjusted automatically.Working surfaces, system is divided into three aspects: configuration management aspect, control plane and service aspect.The configuration management aspect is to complete when system initialization, provide service for upper strata when the service aspect is system works, and control plane is in Adaptable System inside, for better services is provided, automatically completes.
The Chinese and English vocabulary annotation table of literary composition
IDS Intrusion Detection System intruding detection system
IPS Intrusion Protection System intrusion prevention system
WM Wu ?Manber algorithm WM algorithm
ACBM Aho ?Corassick Boyer ?Moore algorithm ACBM algorithm
E2XB Exclusion ?base algorithm E2XB algorithm
LKR Long ?Karp ?Rabin algorithm LKR algorithm
Above-described is only the preferred embodiment of the present invention, the invention is not restricted to above embodiment.Be appreciated that other improvement and variation that those skilled in the art directly derive without departing from the spirit and concept in the present invention or associate, within all should thinking and being included in protection scope of the present invention.
Claims (7)
1. the detection method of a network packet, is characterized in that, comprises the following steps:
A) catch the packet in network;
B) caught packet is carried out to full protocol stack parsing, obtain the agreement variable, be i.e. the protocal layers data of original message packet;
C) select most suitable multi-pattern matching algorithm according to predefined pattern feature and agreement characteristics of variables, loading mode matching algorithm storehouse then, and dynamically adjust matching algorithm according to current network state in testing process.
2. the method for claim 1, it is characterized in that, carry out full protocol stack in described step B and resolve and refer to according to the level of agreement and divide the end of to top and successively resolved, after all protocol datas that comprise application layer protocol being recombinated, more further proceed to resolve.
3. method as claimed in claim 2, is characterized in that, the concrete steps that protocol data is recombinated comprise:
Extract the Expressive Features relevant to Transmission Control Protocol from the raw data packets of catching, analyze TCP message wherein;
TCP message after analyzing is carried out again assembled according to its standard, and be saved in specific internal storage structure.
4. the method for claim 1, is characterized in that, described step C comprises statistical analysis module, matching module and assessment handover module; Wherein, described statistical analysis module is used for pattern matching input rule tree is carried out to statistical analysis, and selects optimum multi-pattern matching algorithm according to analysis result; Described matching module is for providing quick multi-mode matching service by the unified call interface for the external call module, and the pattern matching result is described to event feeds back to the assessment handover module; Described assessment handover module is known current network state for the event information according to the matching module feedback, and adjusts accordingly the decision of current multi-pattern matching algorithm.
5. method as claimed in claim 4, it is characterized in that, described statistical analysis module, according to event configuration file set up pattern-matching rule tree, in rule tree, each tree node represents that a certain type-scheme coupling subtask comprises agreement variable or message data load is carried out to pattern matching; The agreement characteristics of variables of each node and associative mode feature in the statistical analysis rule tree; Investigate each alternative multi-pattern matching algorithm feature in algorithms library, select a kind of multi-pattern matching algorithm of the most applicable this rule tree node.
6. method as claimed in claim 4, it is characterized in that, described matching module, utilize the matching algorithm of current selection to provide quick multi-mode matching service for the external call module, network message or the agreement variate-value of external module input are carried out to the multi-key word search operation, and return to the final mode matching result; Generate corresponding pattern matching feedback event according to input and the result of this pattern matching selectively, process for the statistical analysis of assessment handover module simultaneously.
7. method as claimed in claim 4, is characterized in that, described assessment handover module, according to the event information of matching module feedback, is known current network state; Multi-pattern matching algorithm to current selection under current network state is assessed, if assessment result is dissatisfied, investigate each fundamental mode matching algorithm feature in algorithms library, select a kind of multi-pattern matching algorithm of applicable current network state, instruct the algorithmic dispatching engine to complete the dynamic switching to current multi-pattern matching algorithm.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2013104011602A CN103475653A (en) | 2013-09-05 | 2013-09-05 | Method for detecting network data package |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2013104011602A CN103475653A (en) | 2013-09-05 | 2013-09-05 | Method for detecting network data package |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN103475653A true CN103475653A (en) | 2013-12-25 |
Family
ID=49800349
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2013104011602A Pending CN103475653A (en) | 2013-09-05 | 2013-09-05 | Method for detecting network data package |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103475653A (en) |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104184725A (en) * | 2014-07-25 | 2014-12-03 | 汉柏科技有限公司 | Engine detection data updating method and device of intrusion prevention system |
| CN105187224A (en) * | 2014-06-17 | 2015-12-23 | 腾讯科技（深圳）有限公司 | Invasion detection method and device |
| CN106161098A (en) * | 2016-07-21 | 2016-11-23 | 四川无声信息技术有限公司 | A kind of network behavior detection method and device |
| CN107704606A (en) * | 2017-10-17 | 2018-02-16 | 南京茂毓通软件科技有限公司 | The general abstracting method of instant chat protocol element based on customized label language |
| CN107766729A (en) * | 2017-09-11 | 2018-03-06 | 北京天融信网络安全技术有限公司 | A kind of virus characteristic matching process, terminal and computer-readable recording medium |
| CN109743260A (en) * | 2018-12-25 | 2019-05-10 | 南京中新赛克科技有限责任公司 | A kind of device and method that network flow is filtered based on improved ACBM algorithm |
| CN111064631A (en) * | 2019-11-15 | 2020-04-24 | 上海理工大学 | Data capturing and analyzing method for synchronous phasor measuring device of power plant |
| CN111556067A (en) * | 2020-05-09 | 2020-08-18 | 中国航空无线电电子研究所 | Network data protocol description structure and analysis method based on finite state machine |
| CN112751845A (en) * | 2020-12-28 | 2021-05-04 | 北京恒光信息技术股份有限公司 | Network protocol analysis method, system and device |
| CN113765852A (en) * | 2020-06-03 | 2021-12-07 | 深信服科技股份有限公司 | Data packet detection method, system, storage medium and computing device |
| CN114301672A (en) * | 2021-12-28 | 2022-04-08 | 南京中孚信息技术有限公司 | Network risk detection method and device and electronic equipment |
| CN114567688A (en) * | 2022-03-11 | 2022-05-31 | 之江实验室 | FPGA-based collaborative network protocol analysis method and device |
| CN115296878A (en) * | 2022-07-27 | 2022-11-04 | 天翼云科技有限公司 | Message detection method and device, electronic equipment and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1870498A (en) * | 2006-06-26 | 2006-11-29 | 北京启明星辰信息技术有限公司 | Adaptive multi-model matching method and system |
| CN101771697A (en) * | 2010-01-20 | 2010-07-07 | 西安电子科技大学 | Network data stream identification method based on pattern matching method |
-
2013
- 2013-09-05 CN CN2013104011602A patent/CN103475653A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1870498A (en) * | 2006-06-26 | 2006-11-29 | 北京启明星辰信息技术有限公司 | Adaptive multi-model matching method and system |
| CN101771697A (en) * | 2010-01-20 | 2010-07-07 | 西安电子科技大学 | Network data stream identification method based on pattern matching method |
Cited By (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105187224A (en) * | 2014-06-17 | 2015-12-23 | 腾讯科技（深圳）有限公司 | Invasion detection method and device |
| CN104184725A (en) * | 2014-07-25 | 2014-12-03 | 汉柏科技有限公司 | Engine detection data updating method and device of intrusion prevention system |
| CN106161098B (en) * | 2016-07-21 | 2019-04-30 | 四川无声信息技术有限公司 | A kind of network behavior detection method and device |
| CN106161098A (en) * | 2016-07-21 | 2016-11-23 | 四川无声信息技术有限公司 | A kind of network behavior detection method and device |
| CN107766729A (en) * | 2017-09-11 | 2018-03-06 | 北京天融信网络安全技术有限公司 | A kind of virus characteristic matching process, terminal and computer-readable recording medium |
| CN107704606B (en) * | 2017-10-17 | 2021-01-08 | 南京茂毓通软件科技有限公司 | Universal extraction method for instant chat protocol elements based on custom tag language |
| CN107704606A (en) * | 2017-10-17 | 2018-02-16 | 南京茂毓通软件科技有限公司 | The general abstracting method of instant chat protocol element based on customized label language |
| CN109743260A (en) * | 2018-12-25 | 2019-05-10 | 南京中新赛克科技有限责任公司 | A kind of device and method that network flow is filtered based on improved ACBM algorithm |
| CN111064631A (en) * | 2019-11-15 | 2020-04-24 | 上海理工大学 | Data capturing and analyzing method for synchronous phasor measuring device of power plant |
| CN111556067A (en) * | 2020-05-09 | 2020-08-18 | 中国航空无线电电子研究所 | Network data protocol description structure and analysis method based on finite state machine |
| CN111556067B (en) * | 2020-05-09 | 2022-05-20 | 中国航空无线电电子研究所 | Network data protocol description structure based on finite state machine and analytic method |
| CN113765852A (en) * | 2020-06-03 | 2021-12-07 | 深信服科技股份有限公司 | Data packet detection method, system, storage medium and computing device |
| CN113765852B (en) * | 2020-06-03 | 2023-05-12 | 深信服科技股份有限公司 | Data packet detection method, system, storage medium and computing device |
| CN112751845A (en) * | 2020-12-28 | 2021-05-04 | 北京恒光信息技术股份有限公司 | Network protocol analysis method, system and device |
| CN112751845B (en) * | 2020-12-28 | 2022-12-02 | 北京恒光信息技术股份有限公司 | Network protocol analysis method, system and device |
| CN114301672A (en) * | 2021-12-28 | 2022-04-08 | 南京中孚信息技术有限公司 | Network risk detection method and device and electronic equipment |
| CN114301672B (en) * | 2021-12-28 | 2024-01-26 | 南京中孚信息技术有限公司 | Network risk detection method and device and electronic equipment |
| CN114567688A (en) * | 2022-03-11 | 2022-05-31 | 之江实验室 | FPGA-based collaborative network protocol analysis method and device |
| CN114567688B (en) * | 2022-03-11 | 2023-01-24 | 之江实验室 | FPGA-based collaborative network protocol analysis method and device |
| CN115296878A (en) * | 2022-07-27 | 2022-11-04 | 天翼云科技有限公司 | Message detection method and device, electronic equipment and storage medium |
| CN115296878B (en) * | 2022-07-27 | 2023-11-03 | 天翼云科技有限公司 | Message detection method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103475653A (en) | Method for detecting network data package | |
| US10917417B2 (en) | Method, apparatus, server, and storage medium for network security joint defense | |
| Choi et al. | A method of DDoS attack detection using HTTP packet pattern and rule engine in cloud computing environment | |
| CN101018121B (en) | Log convergence processing method and convergence processing device | |
| US20160191558A1 (en) | Accelerated threat mitigation system | |
| US8751787B2 (en) | Method and device for integrating multiple threat security services | |
| US9584533B2 (en) | Performance enhancements for finding top traffic patterns | |
| CN103841096A (en) | Intrusion detection method with matching algorithm automatically adjusted | |
| US20110099631A1 (en) | Distributed Packet Flow Inspection and Processing | |
| US8336098B2 (en) | Method and apparatus for classifying harmful packet | |
| CN103875214A (en) | Intelligent phy with security detection for ethernet networks | |
| CN103746996A (en) | Packet filtering method for firewall | |
| CN101217547A (en) | A flood request attaching filtering method based on the stateless of open source core | |
| CN103248609A (en) | System, device and method for detecting data from end to end | |
| Meng et al. | Adaptive non-critical alarm reduction using hash-based contextual signatures in intrusion detection | |
| CN102497297A (en) | System and method for realizing deep packet inspection technology based on multi-core and multi-thread | |
| Wang et al. | Exploiting Artificial Immune systems to detect unknown DoS attacks in real-time | |
| Khattak et al. | Dofur: Ddos forensics using mapreduce | |
| CN103618720A (en) | Method and system for Trojan network communication detecting and evidence obtaining | |
| Zhang et al. | A firewall rules optimized model based on service-grouping | |
| CA2738690A1 (en) | Distributed packet flow inspection and processing | |
| CN110912887A (en) | Bro-based APT monitoring system and method | |
| CN118041660A (en) | High-speed large-scale concurrent full-volume network flow intrusion detection method and system | |
| Chi | Intrusion detection system based on snort | |
| CN104023000A (en) | Network intrusion detection method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20131225 |
|
| RJ01 | Rejection of invention patent application after publication |