CN1845066A - Automatic protocol recognition method and system - Google Patents
Automatic protocol recognition method and system Download PDFInfo
- Publication number
- CN1845066A CN1845066A CN 200610080453 CN200610080453A CN1845066A CN 1845066 A CN1845066 A CN 1845066A CN 200610080453 CN200610080453 CN 200610080453 CN 200610080453 A CN200610080453 A CN 200610080453A CN 1845066 A CN1845066 A CN 1845066A
- Authority
- CN
- China
- Prior art keywords
- protocol
- agreement
- fingerprint
- type
- sample
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
The invention discloses an automatic protocol identification method and system used in inbreak defense detect (IDS/IPS) product. Wherein, it can automatically identify the protocol type according to the initial report character in the initial period of network protocol communication, and use protocol check rule to check the protocol identified result. Said method comprises two steps as initial protocol sample character extraction and on-line protocol identification, while the first step comprises: the protocol fingerprint extraction of protocol type sample and building relative protocol check rule; the second step comprises: protocol fingerprint quick matching and protocol identification result quick check. The inventive protocol identification system comprises protocol sample character base, protocol fingerprint match engine and protocol check engine, while the protocol fingerprint match engine is based on quick Haxi list method, and the protocol check engine is based on high-efficiency virtual mode.
Description
Technical field
The present invention relates to a kind of automatic protocol recognition method and system that can be used in intrusion detection defence (IDS/IPS) product, it identifies protocol class under it intelligently according to message characteristic in the network data flow, belongs to networking technology area.
Background technology
Intrusion detection/system of defense (Intrusion Detection/Protection System, IDS/IPS) as the important means of network safety prevention, usually be deployed in key network inside/network boundary porch, in the captured in real time network or the message data stream of turnover network and carry out the intelligent comprehensive analysis, find possible intrusion behavior and block in real time.Application layer protocol deep layer analytic technique is widely adopted in current main-stream IDS/IPS product, can be used to realize the intrusion detection based on protocol attack feature and protocol anomaly.Present most IDS/IPS product is all differentiated protocol type under the network message based on port mapping table, such as, as source/eye end mouth in the network message of finding to catch is 80, think that then it is HTTP (Hypertext TransferProtocol) protocol massages, then gives the http protocol analysis engine with this message and carries out protocol-decoding and intrusion detection.Common this port mapping table is definite when the IDS/IPS product export, but allows the keeper to revise to adapt to the needs of actual environment.In recent years, a collection of new-type network agreement has appearred, comprise SIP (SessionInitiation Protocol) and P2P (Peer to peer protocol) agreement etc., they do not adopt the fixed protocol port, but in the agreement operational process dynamic negotiation port; In addition, present various wooden horses, spyware are that some particular processing modes have all been adopted in the intrusion detection of hiding the IDS/IPS product, mainly show as: 1) do not use the fixed communication port to communicate; 2) adopt well-known port to carry out proprietary protocol communication (such as 80 ports); 3) adopt tunneling technique to carry out proprietary protocol communication (such as the HTTP tunneling technique).In this case, the IDS/IPS product can't correctly be discerned protocol type under the message according to port mapping table, and need be according to protocol class under the network message feature intelligent identification message, otherwise the IDS/IPS product will produce a large amount of wrong reports or fail to report.Therefore, have and to develop the intelligent protocol recognition technology that does not depend on protocol port merely, with the wrong report that reduces the IDS/IPS product or fail to report.And this intelligent protocol recognition technology must satisfy following requirement:
Automatically discern protocol type under the message according to message characteristic intelligently, and do not depend on port mapping table merely;
Go out affiliated protocol type according to the early stage message characteristic quick identification of protocol data-flow as far as possible, to carry out protocol-decoding analysis and intrusion detection purpose as early as possible;
Have very high agreement recognition efficiency, algorithm is realized simple as far as possible;
The method highly versatile is supported nearly all agreement, and requires agreement recognition result accuracy rate height.
Summary of the invention
In order to overcome the deficiencies in the prior art, the invention provides a kind of automatic protocol recognition method and system.
The technical solution adopted for the present invention to solve the technical problems is:
A kind of automatic protocol recognition method, comprise two stage steps of protocol sample feature extraction and agreement identification, wherein, described protocol sample feature extraction phases step comprises the agreement fingerprint extraction and the foundation of respective protocol proof rule of protocol type sample, and agreement cognitive phase step comprises that the agreement fingerprint mates fast and the agreement recognition result is verified fast.
The protocol sample feature extraction phases of protocol recognition method, wherein said agreement fingerprint extraction method has the following steps:
For Text Command type protocol sample, with { order+parameter } or { status code+parameter } format description, directly extract order in the protocol sample and status code as the agreement fingerprint;
For fixed header type protocol sample, contained field type in the fixed header is divided into static fields and dynamic field type, seek continuous static type field as much as possible, and its value combination is defined as the type agreement fingerprint.
For other no set form type protocol sample, searching can identify the word of this protocol sample COS as the agreement fingerprint.
The protocol sample feature extraction phases, wherein said protocol verification rule method for building up has following steps:
Extract in the protocol sample except that the agreement fingerprint further feature as the protocol verification rule of the type protocol sample, comprise that the field, word of further identity protocol sample type or other require the satisfied standard of protocol message etc.
Adopt the step of control flow graph (CFG), be described as the protocol verification rule set that certain type protocol sample is set up, in CFG figure: have two special joints of TRUE and FALSE, wherein the FALSE node has only one, and that the TRUE node can have is a plurality of; Except that TRUE and FALSE node, each node is represented a protocol verification rule (Boolean logic) among the CFG, and its execution result is a true or false; Each bar directed edge shows the actual execution route of CFG figure when directed edge source node execution result is true or false among the CFG; This CFG figure begins to carry out from root node, till running into TRUE or FALSE node.
The agreement cognitive phase of protocol recognition method, wherein said agreement fingerprint fast matching method has following steps:
IP message application layer data is imported as text, institute's protocols having fingerprint as set of modes, is adopted multi-mode to join algorithm and finds the affiliated possible assembly of protocols of IP message;
Before carrying out multi-pattern matching algorithm, elder generation classifies to institute's protocols having fingerprint by the skew of agreement fingerprint and sets up Hash table respectively;
To each quasi-protocol fingerprint, begin from IP message application load respective offsets, successively to preceding i=1,2,3.. individual byte is carried out Hash, checks the Hash table subchain simultaneously, travels through subchain under subchain non-NULL situation, promptly call in the subchain related protocol proof rule of each:, then identified protocol type under the IP message if the IP message has passed through the checking of protocol verification rule.
The agreement cognitive phase of protocol recognition method, wherein said protocol verification rule implementation has following steps:
Adopt the virtual machine model to realize the explanation execution of related protocol proof rule;
To translate into the virtual machine program that on virtual machine, directly to carry out with the protocol verification rule set that CFG describes;
Explain the virtual machine program of carrying out to be come by the translation of protocol verification rule set by a virtual machine interpreter, the program execution result promptly is this agreement recognition result.
A kind of automatic protocol recognition system is characterized in that:
Described agreement recognition system comprises agreement fingerprint base, protocol verification rule base, agreement fingerprint matching engine and protocol verification engine, wherein, agreement fingerprint base and protocol verification rule base have been stored agreement fingerprint and protocol verification rule set that the protocol sample feature extraction phases produces respectively, agreement fingerprint matching engine is realized described agreement fingerprint Fast Match Algorithm, and the protocol verification engine has been realized described virtual machine model;
Agreement fingerprint and respective protocol proof rule collection that the protocol sample feature extraction phases is extracted are described with configuration file, during system works, to make up agreement fingerprint Hash table based on this configuration file, and respective protocol proof rule collection be translated into virtual machine program carry out for the protocol verification engine.
The invention has the beneficial effects as follows, the invention solves in the IDS/IPS product self-identifying problem for the non-standard ports procotol.The present invention can communicate by letter the initial stage according to the affiliated protocol type of the automatic identification of message characteristic in early stage in procotol, and adopts the further indentification protocol recognition result of protocol verification rule set correctness.Compare with existing method, the present invention is complete, and protocol communication initial stage message characteristic Intelligent Recognition Network Based goes out affiliated protocol class, and do not depend on fixed port merely, can discern automatically all dynamic port agreements, tunnel protocol, have the fast and high accuracy for examination of agreement recognition speed, all need in the network security product of intelligent protocol self-identifying can be widely used in IDS/IPS etc.
Description of drawings
The present invention is further described below in conjunction with drawings and Examples.
Fig. 1 is two stage of the protocol recognition method workflow diagram that the present invention relates to;
Fig. 2 is that fixed header format protocol fingerprint extraction is given an example;
Left figure: TDS fixed header form;
Right figure: MSRPC fixed header form;
Fig. 3 describes related protocol proof rule collection for example for adopting CFG;
Right figure: http protocol fingerprint HTTP/1.1 proof rule collection;
Left figure: the proof rule collection of http protocol fingerprint Get;
Fig. 4 is based on quick Hash agreement fingerprint matching legend;
Fig. 5 is based on fingerprint matching of quick Hash agreement and protocol verification process flow diagram.
Embodiment
Embodiment 1; Agreement identifying method of the present invention comprises two working stages: early stage protocol sample feature extraction phases and online protocol cognitive phase, the following (see figure 1) of each stage step:
A. protocol sample feature extraction phases;
From the protocol type sample, extract the agreement fingerprint (each quasi-protocol generally has a plurality of agreement fingerprints) of this quasi-protocol according to agreement fingerprint extraction method, and set up the corresponding proof rule collection of this quasi-protocol.
The agreement fingerprint and the respective protocol proof rule collection that extract are stored into respectively in agreement fingerprint base and the protocol verification rule base, use for agreement cognitive phase fingerprint matching engine and protocol verification engine.
B. online protocol cognitive phase;
Catch the unknown protocol categorical data and flow IP message in early stage, extract the possible agreement fingerprint of institute from application layer load, each fingerprint mates in employing multi-pattern matching algorithm and the agreement fingerprint base, identifies possible assembly of protocols;
For each agreement in the above-mentioned possibility assembly of protocols, calling the respective protocol proof rule verifies, if the checking of protocol verification rule is passed through, then identify the affiliated correct protocol type of this IP message, otherwise continue to carry out the relevant proof rule of next candidate's agreement, till identifying correct agreement.
In case identify the affiliated protocol type of IP message, will create one＜source IP, order IP, source port, eye end mouth, agreement ID〉five-tuple, thus make the subsequent packet that belongs to this protocol data-flow need not to carry out this protocol recognition method, with the mitigation system computing cost.
This protocol recognition method is general only need to capture the message of being with application data load in the application protocol initial stage reciprocal process in the 1-2 bout.Therefore, for application layer protocol, need catch the message that TCP three-way handshake finishes back 1-2 band application layer data based on Transmission Control Protocol.This protocol recognition method can identify affiliated protocol class in mutual 1-2 protocol message data of network communication protocol initial stage, therefore have good agreement recognition speed.
This protocol recognition method is set up the set of agreement fingerprint (SigSet) and a respective protocol proof rule collection (RuleSet) for each type protocol sample, wherein, the agreement fingerprint is the successive byte string of a finite length, and it identifies a kind of protocol message of particular type; When the agreement fingerprint is correct set up after, the agreement fingerprint characteristic that carries by the IP message just can be discerned protocol type under it.For example, " GET " can identify the HTTP Request type of message in the http protocol, can be used as the http protocol fingerprint with it; Conversely, carry agreement fingerprint " GET " if find a certain IP message, then protocol type is that the possibility of HTTP is very big under this IP message.But therefore the agreement recognition result possible errors that obtains based on the agreement fingerprint characteristic merely needs additional measures further to verify this agreement recognition result correctness.This protocol recognition method may adopt the protocol verification rule set to come indentification protocol recognition result correctness after the protocol type under identifying the IP message.The protocol verification rule set is relevant with concrete agreement, and it is a set, and each bar rule has defined the Boolean expression that this IP message must satisfy when differentiating an IP message for the type agreement in the set.Therefore, the protocol verification rule set of setting up for a certain protocol type can be regarded as a necessary condition set of the type protocol specification.For the protocol verification rule set that a certain type protocol is set up can or be controlled flow graph (Control Flow Graph with the Boolean expression tree, CFG) describe, this protocol recognition method selects the control flow graph to describe every type of protocol verification rule set, adopts the virtual machine model to realize the explanation execution of protocol verification rule set.
Http protocol fingerprint and proof rule are set up for example;
At first set up the set of agreement fingerprint, and set up respective protocol proof rule collection for http protocol:
The proof rule collection of setting up for http protocol fingerprint " GET " is:
Rule1: must contain " TTP " word string in the text;
Rule2: must comprise in the text " r n ";
Rule3: or the like.
Be the http protocol fingerprint " POST " the proof rule collection set up is:
Rule1: must contain in the text " r n r n " word string;
Rule2: must comprise the Content-Length word string in the text;
Rule3: or the like.
The proof rule collection of setting up for http protocol fingerprint " EAD " is:
Rule1: must contain " TTP " word string in the text;
Rule2: must comprise " f-Modified-Since " word string in the text;
Rule3: or the like.
Then, in the agreement identifying, comprise " GET " fingerprint if find an IP message, protocol class may be HTTP under can discerning it by the agreement fingerprint matching algorithm, at this moment further carry out the pairing proof rule collection of previously defined http protocol agreement fingerprint " GET " and verify this IP message: if this IP message has passed through the checking of all proof rules in the set, then HTTP just is the correct protocol type under this IP message; Otherwise, continue from the IP message, to extract other agreement fingerprint, carry out fingerprint matching find may under protocol type and carry out the related protocol proof rule and verify, till finding correct result.
Embodiment 2; Following steps are contained in automatic protocol recognition method association;
The agreement fingerprint extraction;
Institute's protocols having is divided three classes: 1) Text Command format protocol; 2) fixed header format protocol; 3) no set form agreement.The fingerprint extraction method of this three quasi-protocol is described respectively below.
The Text Command format protocol;
In the Text Command format protocol, all message all may be described with { order+parameter } mode, and the order here not only comprises common protocol command, also comprise the status code of service end response.The example of Text Command format protocol is a lot, comprises POP3, SMTP, FTP, HTTP etc.For this quasi-protocol, only need extraction protocol command and protocol responses status code to get final product as the agreement fingerprint.Such as, for http protocol, the set of the http protocol fingerprint of extraction is { GET, POST, HEAD, HTTP/0.9, HTTP/1.0, HTTP/1.1}.It is 0 that these agreement fingerprints are offset in http protocol message application load, in this case, can omit the sign of agreement fingerprint skew.
The fixed header format protocol;
The fixed header format protocol all is the binary format agreement generally, and in this quasi-protocol, all protocol messages all have the header of a regular length, are variable length data then.Fixed header format protocol example is a lot, comprises DB2, TDS, DNS and MSRPC agreement etc.In the fixed header, generally comprise fields such as Type, version, Length, Flags and ID.According to the difference of value situation of change in procotol reciprocal process, these fields can be divided into two classes: a) static types field, promptly the field value is constant or can only gets a few value, such as Type, Version, fields such as Flags; B) regime type field, promptly the field value condition is unpredictable in advance, such as fields such as Length, ID.Therefore, can only from first type of field, extract the agreement fingerprint.If it is high more that the agreement fingerprint that extracts identifies the degree of accuracy of a certain type protocol, then find the possibility of correct protocol type just high more by the agreement fingerprint matching algorithm for the first time in the agreement identifying, thereby may only need a small amount of protocol verification rule entries just can identify the affiliated protocol class of IP message, and can obviously reduce the computing cost of agreement recognizer.For reaching this purpose, usually continuous static field combination as much as possible is worked the fingerprint that is used as such protocol sample.
The fixed header type protocol fingerprint extraction (see figure 2) of giving an example:
For TDS (Tabular Data System) agreement, two continuous static fields (Type and Status) are arranged in its fixed header, if, can improve the accuracy of agreement fingerprint sign the type agreement therefore with the fingerprint of Type+Status combination as the TDS agreement;
For MSRPC (Microsoft Remote Procedure Call) agreement, there are 5 continuous static fields (Major_Ver, Minor_Ver, PktType, PktFlags and DataRepresentation) in its fixed header, if therefore with above-mentioned 5 field combination as MSRPC agreement fingerprint, can improve the agreement fingerprint and identify the accuracy of the type agreement.
The agreement fingerprint that extracts by this method generally all has 2-4 byte length, and it is very little possibility of collision to occur between each agreement fingerprint, therefore can obviously accelerate the agreement recognition speed.
No set form agreement;
For no set form agreement, can't adopt universal method to extract the agreement fingerprint, and can only extract the possibility of agreement fingerprint at the specific protocol particular analysis.Fortunately, the application layer protocol that belongs to the type is seldom only seen the Auth agreement at present.Therefore, can treat separately for this situation.
Protocol verification rule set (RuleSet) is set up;
The protocol verification rule set is relevant with concrete protocol type, sets up protocol verification rule set purpose for all kinds agreement and mainly contains following 3:
1) can check this agreement recognition result correctness by the protocol verification rule set, promptly identify a certain IP message may under the laggard step of protocol type demonstrate,prove it and whether really meet the type protocol specification (duplication check);
When 2) collision appears in the agreement fingerprint between two agreements, can find correct agreement recognition result (recognition result screening) by protocol verification rule debug protocol type;
3) the protocol verification rule can testing in depth testing protocol data load, finds those tunnel type agreements, such as MSN-over-HTTP and SMB-over-NetBIOS etc.In the agreement identifying, find IP message may under behind the protocol type, must carry out related protocol proof rule collection, and only just find affiliated correct protocol type after the strictly all rules checking in by the related protocol rule set at this IP message.
The protocol verification rule set of setting up for a certain type protocol is a regular collection, adopts control flow graph (CFG) model to describe the protocol verification rule set.As shown in Figure 3, in CFG model representation method, each bar protocol verification rule represents with an oval node, is used to return protocol verification result's the special rules except that TRUE and two of FALSE here, all the other proof rules all are Boolean logics, and its execution result can only be true or false.This protocol verification regular collection begins to carry out from root node, if current protocol verification rule execution result is true, then carry out the proof rule tree in its left side, if be false, then carry out the proof rule tree on its right side, till carrying out TRUE or FALSE node.Fig. 3 is two protocol verification rule set examples: left figure has defined and http protocol fingerprint " GET " relevant protocol verification rule set, the execution of this protocol verification rule set is from root node, a certain IP message has only and has all passed through 1. the 2. 3. checking of bar proof rule, just may return http protocol ID, otherwise return false.Right figure has defined and the http protocol fingerprint " HTTP/1.1 " relevant proof rule collection, it supports HTTP tunnel protocol deep layer to resolve, this rule set begins to carry out from root node: if a certain IP message has passed through the checking of preceding two (1. 2.) protocol verification rules, then discern it and be the http protocol type; Further, if this IP message also satisfies the 3. bar protocol verification rule, then discern it and be the MSN-Over-HTTP agreement; For other situation recognition failures then.
The protocol verification rule set size of setting up for a certain agreement fingerprint directly has influence on the recognition result accuracy and the efficiency of algorithm of this type protocol: when for the protocol verification rule entries of a certain type protocol fingerprint definition many more, then the accuracy of agreement recognition result is just high more, but the agreement recognition efficiency can be lower; When the protocol verification rule entries that is the definition of a certain type protocol fingerprint is few more, then the agreement recognition efficiency can be very high, but may reduce agreement recognition result accuracy, therefore, and reasonable definition protocol verification rule set as required.Suggestion: if the agreement fingerprint is longer, very little with other agreement fingerprint collision possibility, then Ding Yi protocol verification rule entries can seldom even not have; If the agreement fingerprint is shorter, therefore then the possibility with other agreement fingerprint collision is bigger, needs to consider that the more protocol verification rule entries of definition screens the agreement recognition result collection that obtains by agreement fingerprint matching engine to obtain most probable protocol type.
The agreement fingerprint mates fast;
After early stage, protocol sample was extracted stage definitions good all kinds agreement fingerprint and related protocol proof rule, need a kind of fingerprint matching algorithm of agreement efficiently of design, be used for IP message application data being carried out the agreement fingerprint and find and quick coupling, thereby find the affiliated possible protocol type set of this IP message at the agreement cognitive phase.Can adopt multi-pattern matching algorithm to carry out the quick matching process of this agreement fingerprint: with the Text of IP message application layer load data as multi-pattern matching algorithm, the agreement fingerprint set of all definition is as set of modes, use multi-pattern matching algorithm to find all possible protocol type set, then each protocol type in this set is carried out the respective protocol proof rule, the protocol type of debug is till finding suitable protocol type.Realize that based on traditional multi-model matching algorithm agreement fingerprint Fast Match Algorithm has algorithm and realizes advantages such as fast, that versatility is good, but it does not make full use of the agreement fingerprint and is usually located at these characteristics of IP message application load head, therefore, this protocol recognition method has designed an agreement fingerprint Fast Match Algorithm efficiently.
Observe: the skew of the agreement fingerprint of 95% protocol type is 0, is not 0 agreement fingerprint (accounting for 5%) for skew, deviation post all appear at the forward position of IP message application layer load and value all regular, such as in skew 2,4,8,12,16; In addition, institute's protocols having fingerprint length is all shorter, generally is no more than 8 bytes.Therefore, this protocol recognition method is at first classified to the agreement fingerprint by the fingerprint off-set value, adopts quick Hash location algorithm to carry out the agreement fingerprint matching to each quasi-protocol fingerprint then.
The quick Hash location algorithm of agreement fingerprint is described;
With skew is that institute's protocols having fingerprint of 0 is that example is described this quick Hash location algorithm, and being offset for other is not 0 agreement fingerprint classification, can adopt this quick Hash location algorithm equally.
The quick hash algorithm preparatory stage;
With skew be institute's protocols having fingerprint of 0 according to specifying hash algorithm to insert Hash table, the agreement fingerprint of hash value identical (being the agreement fingerprint collision) is contacted, forms agreement fingerprint Hash shown in Figure 4 and shows.
Quick Hash retrieval phase;
This agreement fingerprint quick Hash location and protocol verification algorithm workflow diagram are seen Fig. 5.For the IP message application layer data m that catches, from deviation post 0, successively to i before the m (i=1,2,3 ...) individual successive byte carries out Hash (Hash) computing and (use H (m respectively
1-1), H (m
1-2) ..., H (m
1-maxsig) expression, maxsig is maximum agreement fingerprint length here), and check corresponding Hash subchain situation in the Hash table simultaneously:
If corresponding Hash subchain is empty, then carry out next Hash computing;
Otherwise, carry out concrete protocol verification rule set in the subchain successively:
If the checking of protocol verification rule is passed through, then identified correct protocol type.
Otherwise the protocol verification rule set of next candidate's agreement in the subchain is carried out in continuation, till finding suitable protocol type.
This quick Hash agreement fingerprint matching and protocol verification algorithm false code are described below:
HTab[index]={ NULL} // be offset is 0 agreement fingerprint Hash table
…
// scan maxsig byte before the Payload successively
For(i＝1；i<＝maxsig；i++){
Index=Hash (payload, i); // a preceding i byte is carried out Hash
If (HTab[index]!=NULL) if // the Hash subchain is not empty
// traversal subchain, and call the corresponding protocol proof rule
prec＝HTab[index]；
do{//call?protocol?verify?function?pointer.
ret＝proto_verify(payload，len，prec->rules)；
If // protocol verification passes through, then return agreement ID
if(ret＞0)return?ret；
Prec=prec-〉next; // otherwise continue checking subchain next element
}while(prec！＝NULL)；}}
The protocol verification rule is carried out;
In this protocol recognition method, when the agreement fingerprint that carries by the IP message retrieves affiliated possible protocol type, need call to the protocol verification of the type protocol definition is regular and verify further whether this IP message satisfies the type protocol specification, the problem of implementation of the proto_verify function in the promptly above-mentioned false code.The present invention adopts the CFG model to describe the protocol verification rule, and adopt virtual machine to explain the CFG figure that execution is generated by the protocol verification rule, therefore, early stage the protocol sample feature extraction phases need and will be compiled into the program code that can on virtual machine, directly carry out for the protocol verification rule set (being CFG figure) of all types of protocol definitions; Then, at the agreement cognitive phase, behind the possible protocol type under agreement fingerprint matching engine identifies the IP message, the virtual machine program code that loads the type agreement by the protocol verification engine executions that make an explanation, the program return results is the protocol type ID or 0 (0 represents unknown protocol ID) that identifies.
The virtual machine model class that the protocol verification engine uses in this protocol recognition method is similar to the virtual machine model that uses in the BPF filtrator, and it aims at IP message Treatment Design based on register manipulation, is a very advantages of simplicity and high efficiency virtual machine model.This virtual machine model is by a storer, an indexed registers.An erasable memory, an implication routine counter is formed.It can be carried out and load and storage instruction branch and arithmetic instruction etc.Protocol verification engine based on the virtual machine modelling is virtual machine program input data with IP message application layer load, and the program execution result is the protocol type ID or 0 (expression recognition failures) under this IP message.
By adopting CFG figure to describe the protocol verification rule set of certain type protocol, and by the virtual machine engine proof rule collection that carries on an agreement, make this invent the very complicated protocol verification rule set definition of described protocol recognition method support, protocol verification regular expression ability can reach very thin granularity, can satisfy the protocol verification regular expression demand of all types agreement.In protocol recognition method of the present invention, early stage, the agreement fingerprint and the related protocol proof rule collection of the definition of protocol sample feature extraction phases were provided by configuration file, the keeper can make adjustment between agreement recognition result accuracy and agreement recognition efficiency by protocol verification rule set in the modification configuration file, need not to revise agreement fingerprint matching engine and protocol verification engine virtual machine code, thereby this protocol recognition method have extraordinary framework dirigibility, cross-platform transfer ability.
Claims (7)
1. automatic protocol recognition method, it is characterized in that: described protocol recognition method comprises protocol sample feature extraction and agreement two stages of identification, wherein, described protocol sample feature extraction phases comprises the agreement fingerprint extraction and the foundation of respective protocol proof rule of protocol type sample, and the agreement cognitive phase comprises agreement fingerprint coupling and the quick verification step of agreement recognition result fast.
2. a kind of automatic protocol recognition method as claimed in claim 1 is characterized in that: the protocol sample feature extraction phases of protocol recognition method, and wherein said agreement fingerprint extraction method has following feature:
For Text Command type protocol sample, with { order+parameter } or { status code+parameter } format description, directly extract order in the protocol sample and status code as the agreement fingerprint;
For fixed header type protocol sample, contained field type in the fixed header is divided into static fields and dynamic field type, seek continuous static type field as much as possible, and its value combination is defined as the type agreement fingerprint;
For other no set form type protocol sample, searching can identify the word of this protocol sample COS as the agreement fingerprint.
3. a kind of automatic protocol recognition method as claimed in claim 1 is characterized in that: the protocol sample feature extraction phases of described protocol recognition method, and wherein said protocol verification rule method for building up has following feature:
Extract in the protocol sample except that the agreement fingerprint further feature as the protocol verification rule of the type protocol sample, comprise that the field, word of further identity protocol sample type or other require the satisfied standard of protocol message.
4. a kind of automatic protocol recognition method as claimed in claim 3 is characterized in that: wherein said protocol verification rule method for building up has following feature: adopt the step of CFG, CFG has a FALSE node, and a plurality of TRUE nodes are arranged; Except that TRUE and FALSE node, each node is represented a protocol verification rule Boolean logic among the CFG, and its execution result is a true or false; Each bar directed edge shows the actual execution route of CFG figure when directed edge source node execution result is true or false among the CFG; This CFG figure begins to carry out from root node, till running into TRUE or FALSE node.
5. a kind of automatic protocol recognition method as claimed in claim 1 is characterized in that: the agreement cognitive phase of described protocol recognition method, and wherein said agreement fingerprint fast matching method has following feature:
IP message application layer data is imported as text, institute's protocols having fingerprint as set of modes, is adopted multi-mode to join algorithm and finds the affiliated possible assembly of protocols of IP message;
Before carrying out multi-pattern matching algorithm, elder generation classifies to institute's protocols having fingerprint by the skew of agreement fingerprint and sets up Hash table respectively;
To each quasi-protocol fingerprint, begin from IP message application load respective offsets, successively to preceding i=1,2,3.. byte carried out Hash, checks the Hash table subchain simultaneously, under subchain non-NULL situation, travel through subchain, promptly call in the subchain related protocol proof rule of each.
6. a kind of automatic protocol recognition method as claimed in claim 1 is characterized in that: the agreement cognitive phase of described protocol recognition method, and wherein said protocol verification rule implementation has following feature:
Adopt the virtual machine model to realize the explanation execution of related protocol proof rule;
To translate into the virtual machine program that on virtual machine, directly to carry out with the protocol verification rule set that CFG describes;
Explain the virtual machine program of carrying out to come by the translation of protocol verification rule set by a virtual machine interpreter.
7. automatic protocol recognition system is characterized in that:
Described agreement recognition system comprises agreement fingerprint base, protocol verification rule base, agreement fingerprint matching engine and protocol verification engine, wherein, agreement fingerprint base and protocol verification rule base have been stored agreement fingerprint and protocol verification rule set that the protocol sample feature extraction phases produces respectively, agreement fingerprint matching engine has been realized the agreement fingerprint Fast Match Algorithm described in the claim 4, and the protocol verification engine has been realized the virtual machine model described in the claim 5;
Agreement fingerprint and respective protocol proof rule collection that the protocol sample feature extraction phases is extracted are described with configuration file, during system works, to make up agreement fingerprint Hash table based on this configuration file, and respective protocol proof rule collection be translated into virtual machine program carry out for the protocol verification engine.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB2006100804535A CN100429617C (en) | 2006-05-16 | 2006-05-16 | Automatic protocol recognition method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB2006100804535A CN100429617C (en) | 2006-05-16 | 2006-05-16 | Automatic protocol recognition method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN1845066A true CN1845066A (en) | 2006-10-11 |
| CN100429617C CN100429617C (en) | 2008-10-29 |
Family
ID=37063994
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNB2006100804535A Expired - Fee Related CN100429617C (en) | 2006-05-16 | 2006-05-16 | Automatic protocol recognition method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN100429617C (en) |
Cited By (26)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101577704A (en) * | 2008-05-08 | 2009-11-11 | 北京东华合创数码科技股份有限公司 | Network application-level protocol recognition method and system |
| CN101282251B (en) * | 2008-05-08 | 2011-04-13 | 中国科学院计算技术研究所 | Method for digging recognition characteristic of application layer protocol |
| CN102035698A (en) * | 2011-01-06 | 2011-04-27 | 西北工业大学 | HTTP tunnel detection method based on decision tree classification algorithm |
| CN101335654B (en) * | 2007-06-28 | 2012-04-04 | 中兴通讯股份有限公司 | Interface positioning test method and system |
| CN102413007A (en) * | 2011-10-12 | 2012-04-11 | 上海奇微通讯技术有限公司 | Deep packet inspection method and equipment |
| CN101925895B (en) * | 2007-12-13 | 2012-11-07 | 谷歌公司 | Generic format for efficient transfer data |
| CN103164698A (en) * | 2013-03-29 | 2013-06-19 | 华为技术有限公司 | Method and device of generating fingerprint database and method and device of fingerprint matching of text to be tested |
| CN103914031A (en) * | 2013-12-04 | 2014-07-09 | 哈尔滨安天科技股份有限公司 | RS-485 bus monitor probe circuit automatically adapting to various protocols |
| CN103945287A (en) * | 2013-01-21 | 2014-07-23 | 中兴通讯股份有限公司 | Method for processing message by interface machine and interface machine |
| CN104184726A (en) * | 2014-07-25 | 2014-12-03 | 汉柏科技有限公司 | IPS message omission preventive method and device based on protocol identification |
| CN101425876B (en) * | 2008-12-16 | 2015-04-22 | 北京中创信测科技股份有限公司 | Communication protocol deciphering method and device |
| CN105282133A (en) * | 2014-06-19 | 2016-01-27 | 凯为公司 | A method of forming a Hash input from packet contents and an apparatus thereof |
| CN105530098A (en) * | 2015-12-04 | 2016-04-27 | 北京浩瀚深度信息技术股份有限公司 | Protocol fingerprint automatic extraction method and system |
| CN105939304A (en) * | 2015-06-11 | 2016-09-14 | 杭州迪普科技有限公司 | Tunnel message analysis method and device |
| CN108255675A (en) * | 2018-01-10 | 2018-07-06 | 北京知道创宇信息技术有限公司 | A kind of port diagnostic extracting method, device and computing device |
| CN105678188B (en) * | 2016-01-07 | 2019-01-29 | 杨龙频 | The leakage-preventing protocol recognition method of database and device |
| CN110855576A (en) * | 2015-12-31 | 2020-02-28 | 杭州数梦工场科技有限公司 | Application identification method and device |
| CN111865724A (en) * | 2020-07-28 | 2020-10-30 | 公安部第三研究所 | Information acquisition control implementation method for video monitoring equipment |
| CN112272121A (en) * | 2020-09-21 | 2021-01-26 | 中国科学院信息工程研究所 | Effect verification method and system for flow monitoring |
| CN112367317A (en) * | 2020-11-09 | 2021-02-12 | 浙江大学 | Endogenous safe WAF fingerprint transformation method |
| CN112637223A (en) * | 2020-12-26 | 2021-04-09 | 曙光网络科技有限公司 | Application protocol identification method and device, computer equipment and storage medium |
| CN112714045A (en) * | 2020-12-31 | 2021-04-27 | 浙江远望信息股份有限公司 | Rapid protocol identification method based on equipment fingerprint and port |
| CN112787875A (en) * | 2019-11-06 | 2021-05-11 | 杭州海康威视数字技术股份有限公司 | Equipment identification method, device and equipment, and storage medium |
| CN112995172A (en) * | 2021-02-24 | 2021-06-18 | 合肥优尔电子科技有限公司 | Communication method and communication system for butt joint between Internet of things equipment and Internet of things platform |
| CN113705161A (en) * | 2021-08-10 | 2021-11-26 | 博流智能科技（南京）有限公司 | UVM register model rapid generation method and system, and chip verification method and system |
| CN114124562A (en) * | 2021-12-02 | 2022-03-01 | 湖北天融信网络安全技术有限公司 | Defense method, defense device, electronic equipment and storage medium |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH08214041A (en) * | 1995-02-03 | 1996-08-20 | Max Co Ltd | Automatic protocol recognition device |
| US6895020B2 (en) * | 2001-07-31 | 2005-05-17 | Agilent Technologies, Inc. | Method and apparatus for protocol pattern identification in protocol data units |
| CN1203641C (en) * | 2002-10-11 | 2005-05-25 | 北京启明星辰信息技术有限公司 | Method and system for monitoring network intrusion |
| CN1612135B (en) * | 2003-10-30 | 2012-07-04 | 北京神州绿盟信息安全科技股份有限公司 | Invasion detection (protection) product and firewall product protocol identifying technology |
-
2006
- 2006-05-16 CN CNB2006100804535A patent/CN100429617C/en not_active Expired - Fee Related
Cited By (39)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101335654B (en) * | 2007-06-28 | 2012-04-04 | 中兴通讯股份有限公司 | Interface positioning test method and system |
| CN101925895B (en) * | 2007-12-13 | 2012-11-07 | 谷歌公司 | Generic format for efficient transfer data |
| CN101282251B (en) * | 2008-05-08 | 2011-04-13 | 中国科学院计算技术研究所 | Method for digging recognition characteristic of application layer protocol |
| CN101577704A (en) * | 2008-05-08 | 2009-11-11 | 北京东华合创数码科技股份有限公司 | Network application-level protocol recognition method and system |
| CN101425876B (en) * | 2008-12-16 | 2015-04-22 | 北京中创信测科技股份有限公司 | Communication protocol deciphering method and device |
| CN102035698A (en) * | 2011-01-06 | 2011-04-27 | 西北工业大学 | HTTP tunnel detection method based on decision tree classification algorithm |
| CN102035698B (en) * | 2011-01-06 | 2012-07-25 | 西北工业大学 | HTTP tunnel detection method based on decision tree classification algorithm |
| CN102413007B (en) * | 2011-10-12 | 2014-03-26 | 上海奇微通讯技术有限公司 | Deep packet inspection method and equipment |
| CN102413007A (en) * | 2011-10-12 | 2012-04-11 | 上海奇微通讯技术有限公司 | Deep packet inspection method and equipment |
| CN103945287B (en) * | 2013-01-21 | 2019-05-10 | 中兴通讯股份有限公司 | A kind of method and interface message processor (IMP) of interface message processor (IMP) processing message |
| CN103945287A (en) * | 2013-01-21 | 2014-07-23 | 中兴通讯股份有限公司 | Method for processing message by interface machine and interface machine |
| WO2014110903A1 (en) * | 2013-01-21 | 2014-07-24 | 中兴通讯股份有限公司 | Method used by interface machine to process messages and interface machine |
| CN103164698A (en) * | 2013-03-29 | 2013-06-19 | 华为技术有限公司 | Method and device of generating fingerprint database and method and device of fingerprint matching of text to be tested |
| CN103164698B (en) * | 2013-03-29 | 2016-01-27 | 华为技术有限公司 | Text fingerprints library generating method and device, text fingerprints matching process and device |
| CN103914031A (en) * | 2013-12-04 | 2014-07-09 | 哈尔滨安天科技股份有限公司 | RS-485 bus monitor probe circuit automatically adapting to various protocols |
| CN103914031B (en) * | 2013-12-04 | 2016-08-17 | 哈尔滨安天科技股份有限公司 | A kind of RS-485 bus monitoring probe circuit of self adaptation various protocols |
| CN105282133A (en) * | 2014-06-19 | 2016-01-27 | 凯为公司 | A method of forming a Hash input from packet contents and an apparatus thereof |
| CN105282133B (en) * | 2014-06-19 | 2020-06-23 | 马维尔亚洲私人有限公司 | Method and apparatus for forming hash input from packet content |
| CN104184726A (en) * | 2014-07-25 | 2014-12-03 | 汉柏科技有限公司 | IPS message omission preventive method and device based on protocol identification |
| CN105939304A (en) * | 2015-06-11 | 2016-09-14 | 杭州迪普科技有限公司 | Tunnel message analysis method and device |
| CN105530098A (en) * | 2015-12-04 | 2016-04-27 | 北京浩瀚深度信息技术股份有限公司 | Protocol fingerprint automatic extraction method and system |
| CN105530098B (en) * | 2015-12-04 | 2018-10-09 | 北京浩瀚深度信息技术股份有限公司 | A kind of agreement fingerprint extraction method and system |
| CN110855576A (en) * | 2015-12-31 | 2020-02-28 | 杭州数梦工场科技有限公司 | Application identification method and device |
| CN105678188B (en) * | 2016-01-07 | 2019-01-29 | 杨龙频 | The leakage-preventing protocol recognition method of database and device |
| CN108255675A (en) * | 2018-01-10 | 2018-07-06 | 北京知道创宇信息技术有限公司 | A kind of port diagnostic extracting method, device and computing device |
| CN112787875A (en) * | 2019-11-06 | 2021-05-11 | 杭州海康威视数字技术股份有限公司 | Equipment identification method, device and equipment, and storage medium |
| CN112787875B (en) * | 2019-11-06 | 2022-03-01 | 杭州海康威视数字技术股份有限公司 | Equipment identification method, device and equipment, and storage medium |
| CN111865724A (en) * | 2020-07-28 | 2020-10-30 | 公安部第三研究所 | Information acquisition control implementation method for video monitoring equipment |
| CN111865724B (en) * | 2020-07-28 | 2022-02-08 | 公安部第三研究所 | Information acquisition control implementation method for video monitoring equipment |
| CN112272121A (en) * | 2020-09-21 | 2021-01-26 | 中国科学院信息工程研究所 | Effect verification method and system for flow monitoring |
| CN112367317B (en) * | 2020-11-09 | 2021-09-03 | 浙江大学 | Endogenous safe WAF fingerprint transformation method |
| CN112367317A (en) * | 2020-11-09 | 2021-02-12 | 浙江大学 | Endogenous safe WAF fingerprint transformation method |
| CN112637223A (en) * | 2020-12-26 | 2021-04-09 | 曙光网络科技有限公司 | Application protocol identification method and device, computer equipment and storage medium |
| CN112714045A (en) * | 2020-12-31 | 2021-04-27 | 浙江远望信息股份有限公司 | Rapid protocol identification method based on equipment fingerprint and port |
| CN112995172A (en) * | 2021-02-24 | 2021-06-18 | 合肥优尔电子科技有限公司 | Communication method and communication system for butt joint between Internet of things equipment and Internet of things platform |
| CN112995172B (en) * | 2021-02-24 | 2022-09-09 | 合肥优尔电子科技有限公司 | Communication method and communication system for butt joint between Internet of things equipment and Internet of things platform |
| CN113705161A (en) * | 2021-08-10 | 2021-11-26 | 博流智能科技（南京）有限公司 | UVM register model rapid generation method and system, and chip verification method and system |
| CN113705161B (en) * | 2021-08-10 | 2024-03-22 | 博流智能科技（南京）有限公司 | UVM register model rapid generation method and system, chip verification method and system |
| CN114124562A (en) * | 2021-12-02 | 2022-03-01 | 湖北天融信网络安全技术有限公司 | Defense method, defense device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN100429617C (en) | 2008-10-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN1845066A (en) | Automatic protocol recognition method and system | |
| CN101035111A (en) | Intelligent protocol parsing method and device | |
| US9781139B2 (en) | Identifying malware communications with DGA generated domains by discriminative learning | |
| CN112738126B (en) | Attack tracing method based on threat intelligence and ATT & CK | |
| Bao et al. | {BYTEWEIGHT}: Learning to recognize functions in binary code | |
| US9990583B2 (en) | Match engine for detection of multi-pattern rules | |
| CN106357618B (en) | Web anomaly detection method and device | |
| Bayer et al. | Scalable, behavior-based malware clustering. | |
| US7802009B2 (en) | Automatic reverse engineering of message formats from network traces | |
| US8161550B2 (en) | Network intrusion detection | |
| Shabtai et al. | F-sign: Automatic, function-based signature generation for malware | |
| JP2020530638A (en) | Malware Host NetFlow Analysis System and Method | |
| CN106685803A (en) | Method and system of tracing APT attack event based on phishing mail | |
| CN101034974A (en) | Associative attack analysis and detection method and device based on the time sequence and event sequence | |
| CN105046152B (en) | Malware detection method based on function call graph fingerprint | |
| CN112532642B (en) | Industrial control system network intrusion detection method based on improved Suricata engine | |
| CN112651028B (en) | Vulnerability code clone detection method based on context semantics and patch verification | |
| CN110381009A (en) | A kind of detection method of the rebound shell of Behavior-based control detection | |
| CN105791250B (en) | Application program detection method and device | |
| CN115396138A (en) | Tracing graph reduction method and device | |
| CN101060492A (en) | Talk detection method and talk detection system | |
| CN1571961A (en) | Security hole diagnosis system | |
| CN106650451A (en) | Detection method and device | |
| Mostafa et al. | Netdroid: Summarizing network behavior of android apps for network code maintenance | |
| Boros et al. | Machine Learning and Feature Engineering for Detecting Living off the Land Attacks. |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| C56 | Change in the name or address of the patentee | ||
| CP03 | Change of name, title or address |
Address after: No. 12 South Main Street, Beijing, Haidian District, Zhongguancun Patentee after: Beijing Venus Information Technology Co., Ltd. Address before: No. 12 South Main Street, Beijing, Haidian District, Zhongguancun Patentee before: Beijing Qiming Xingchen Information Technology Co., Ltd. |
|
| C56 | Change in the name or address of the patentee |
Owner name: BEIJING QIMINGXINGCHEN INFORMATION TECHNOLOGY CO., Free format text: FORMER NAME: BEIJING QIMING XINGCHEN INFORMATION TECHNOLOGY CO. LTD. |
|
| ASS | Succession or assignment of patent right |
Owner name: BEIJING QIMINGXINCHEN INFORMATION SECURITY TECHNOL |
|
| C41 | Transfer of patent application or patent right or utility model | ||
| COR | Change of bibliographic data |
Free format text: CORRECT: ADDRESS; FROM: 100081 NO.12 ZHONGGUANCUN SOUTH AVENUE, HAIDIAN DISTRICT, BEIJING TO: 100193 QIMINGXINGCHEN BUILDING, BUILDING 21, ZHONGGUANCUN SOFTWARE PARK, NO.8, DONGBEIWANG WEST ROAD, HAIDIAN DISTRICT, BEIJING CITY |
|
| TR01 | Transfer of patent right |
Effective date of registration: 20100507 Address after: 100193 Beijing city Haidian District Dongbeiwang qimingxingchenmansionproject Building No. 21 West Road No. 8 Zhongguancun Software Park Co-patentee after: Beijing Venusense Information Security Technology Co., Ltd. Patentee after: Beijing Venus Information Technology Co., Ltd. Address before: 100081 Beijing, Zhongguancun, South Street, No. 12, No. Patentee before: Beijing Venus Information Technology Co., Ltd. |
|
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20081029 Termination date: 20150516 |
|
| EXPY | Termination of patent right or utility model |