CN102364897A - Gateway-level on-line network message detection filtering method and apparatus thereof - Google Patents
Gateway-level on-line network message detection filtering method and apparatus thereof Download PDFInfo
- Publication number
- CN102364897A CN102364897A CN2011102943552A CN201110294355A CN102364897A CN 102364897 A CN102364897 A CN 102364897A CN 2011102943552 A CN2011102943552 A CN 2011102943552A CN 201110294355 A CN201110294355 A CN 201110294355A CN 102364897 A CN102364897 A CN 102364897A
- Authority
- CN
- China
- Prior art keywords
- message
- field
- value
- packet filtering
- policy library
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/02—Network architectures or network communication protocols for network security for separating internal from external traffic, e.g. firewalls
- H04L63/0227—Filtering policies
Landscapes
- Engineering & Computer Science (AREA)
- Computer Hardware Design (AREA)
- Computer Security & Cryptography (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a gateway-level on-line network message detection filtering method and an apparatus thereof. The method comprises the following steps: performing online detection of a GET message of hyper text transfer protocol (HTTP); analyzing the GET message so as to acquire a value of a field in the GET message; filtering the detected GET message based on a message filtering strategy database and the value of the field in the GET message, wherein the message filtering strategy database is formed by more than one message filtering strategy and the message filtering strategy is used to determine whether the GET message is an interference message or a non-interference. In the method, through deeply analyzing the value of the field in the GET message of HTTP on a gateway level and based on the message filtering strategy database, filtering is performed to the GET message. On-line real-time cleaning (filtering) can be performed to the interference message generated during HTTP access action of a user. Through the cleaned message, the actual user action can be accurately and real-timely analyzed.
Description
Technical field
The present invention relates to network field, specifically, relate to the online network message of a kind of gateway level and detect filter method and device.
Background technology
The gateway of telecommunications network etc. is provided for can obtaining a large amount of message information that stores in the network in the equipment of message of Network Users'Behaviors Analysis, still, is not that all messages of being stored can both be effective to Network Users'Behaviors Analysis.On the contrary, in the behavior of user capture the Internet, can produce a large amount of noise messages that is unfavorable for Network Users'Behaviors Analysis.For the user behavior analysis in the telecommunications network side, the existence of noise message can produce serious disturbance.But in the prior art, gateway can not discern these noises that are unfavorable for Network Users'Behaviors Analysis well and filter (cleaning) it.
More specifically; Because generally speaking; Have a large amount of contents in the page, for example picture, video, advertisement link, animation link etc. are therefore when the user passes through HTTP (Hyper Text Transfer Protocol) a certain webpage of protocol access through browser; Tend to trigger the GET message (hereinafter referred GET message is also referred to as request message) of a lot of HTTP.
Such as when user capture URL is the webpage of http://sports.sina.com.cn; Is that sports.sina.com.cn, PATH are a large amount of other GET messages the null string with generation except that HOST; But for the analysis user system of behavior; The analysis result of network address http://sports.sina.com.cn of only hoping to have obtained user capture; Whether visited other network address beyond this network address and might not pay close attention to the user, at this moment, except that the interference message of then not expected for the user behavior analysis system most probably for the GET message the visit network address http://sports.sina.com.cn.
But because the GET message of visit http://sports.sina.com.cn and other interference messages have no difference on message structure, so the gateway of prior art can not be well with disturbing packet filtering.Wherein, the GET message is made up of request row (request line), request head (header), null and 4 parts of request msg.To forming, every row is a pair of by field name (also claiming keyword or field) and field value for the request head, field and be worth between separate with English colon ": ".Field comprises User-Agent, Accept, Host etc., in addition, comprises the value of path field in the request row.Fig. 3 shows the example of GET message request.
Under situation about can not filter well to the interference message, not only can cause the waste of network bandwidth resources, Network Users'Behaviors Analysis system server resource etc., also can produce serious the interference to the analysis result of Network Users'Behaviors Analysis.
Summary of the invention
Technical problem to be solved by this invention is to provide the online network message of a kind of gateway level to detect filter method and device.
In order to solve the problems of the technologies described above, to the invention provides the online network message of a kind of gateway level and detect filter method.This method comprises: steps A: the GET message of online detection HTTP; Step B: said GET message is resolved, to obtain the value of the field in the said GET message; And step C: based on the value of the field in packet filtering policy library and the said GET message; Detected GET message is filtered; Wherein, Said packet filtering policy library is made up of an above packet filtering strategy, and said packet filtering strategy is used for confirming that each said GET message is that the interference message also is non-interference message.
Detect filter method according to the online network message of the gateway level of another aspect of the invention; Before steps A; Also carry out the step obtain said packet filtering policy library, wherein, the packet filtering policy library that is obtained is based on the GET message of predetermined quantity and off-line makes up.
Detect filter method according to the online network message of the gateway level of another aspect of the invention, come off-line to make up said packet filtering policy library through following steps: substep A1: the GET message that obtains predetermined quantity; Substep A2: a part of GET message of the GET message that random extraction is obtained; Substep A3: this part the GET message to being extracted is resolved, to obtain the value of the field in the GET message; Substep A4: the field and the value thereof of the GET message that is obtained based on substep A3 make up the packet filtering policy library; Substep A5: according to constructed packet filtering policy library, the GET message of the predetermined quantity that obtains filters in the antithetical phrase steps A 1, according to the constructed packet filtering policy library of filter result adjustment substep A4.
Detect filter method according to the online network message of the gateway level of another aspect of the invention; Among the substep A4; The field and the value thereof of the GET message that is obtained based on substep A3; Receive the field of the GET message that the user obtained according to one or more substep A3 at every turn and the packet filtering strategy that corresponding value is provided with, to make up the packet filtering policy library.
Detect filter method according to the online network message of the gateway level of another aspect of the invention; Among the step C; Mate the value of the field in said packet filtering policy library and the said GET message through even numbers group word lookup tree algorithm, so that detected GET message is filtered.
Detect filter method according to the online network message of the gateway level of another aspect of the invention, also comprise:
Before step B, also to carry out with character '. ' and/or '/' be decollator splits and will split the operation as a plurality of values of this field in the said GET message of value that the back obtained to the value of all or part of field in the said GET message.
Detect filter method according to the online network message of the gateway level of another aspect of the invention, among the step B, only obtain the value of the field of the desired parsing of user in the GET message through hash algorithm.
Detect filter method according to the online network message of the gateway level of another aspect of the invention, confirm the field of the desired parsing of said user through receiving the pre-configured mode of user; Perhaps, confirm the field of the desired parsing of said user through the mode of automatic configuration based on the packet filtering policy library.
Detect filter method according to the online network message of the gateway level of another aspect of the invention; Packet filtering strategy in the said packet filtering policy library has priority attribute; And; In said step C, two different determined results of packet filtering strategy adopt based on the high packet filtering strategy of said packet filtering policy library medium priority and confirm that each said GET message is that the interference message also is non-interference message not simultaneously in according to said packet filtering policy library.
Also provide a kind of online network message of gateway level that is used for to detect the message detection filter that filters according to the present invention.This device comprises: detecting unit, the GET message of its online detection HTTP; Resolution unit, it is resolved said GET message, to obtain the value of the field in the said GET message; Filter element, it filters detected GET message based on the value of the field in the GET message that is obtained.
Message according to another aspect of the invention detects filter, also comprises the policy library construction unit, and said policy library construction unit comprises following subelement: message obtains subelement, and it is responsible for obtaining the GET message of predetermined quantity; Message extracts subelement, a part of GET message of the GET message that the said message acquiring unit of its random extraction is obtained; The packet parsing subelement extracts this part GET message that subelement extracted to said message and resolves, to obtain the value of the field in the GET message; And the structure subelement, the field and the value thereof of the GET message that it is obtained based on said packet parsing subelement make up and are used for the packet filtering policy library that on-line filtration disturbs message; Adjustment unit, it is according to constructed packet filtering policy library, and the GET message that said message is obtained the predetermined quantity that subelement obtains filters, and adjusts the constructed packet filtering policy library of said structure subelement according to filter result.
Message according to another aspect of the invention detects filter; Also comprise: split cells, to be decollator with character '. ' and/or '/' split and will split value that the back the obtained a plurality of values as this field in the said GET message to the value of all or part of field in the said GET message for it.
Compared with prior art, the present invention has the following advantages:
This method is through coming the GET message is filtered based on the value of the field in the GET message of depth analysis HTTP and according to the packet filtering policy library in the gateway level; But online in real time is cleaned the noise message (interference message) that produces in (filtration) user HTTP visit behavior; Thereby, can analyze actual user behavior accurately and real-time through the message after cleaning.
Other features and advantages of the present invention will be set forth in specification subsequently, and, partly from specification, become obvious, perhaps understand through embodiment of the present invention.The object of the invention can be realized through the structure that in specification, claims and accompanying drawing, is particularly pointed out and obtained with other advantages.
Description of drawings
Accompanying drawing is used to provide further understanding of the present invention, and constitutes the part of specification, is used to explain the present invention jointly with embodiments of the invention, is not construed as limiting the invention.In the accompanying drawings:
Fig. 1 is the schematic flow sheet that detects filter method according to the online network message of the gateway level of first embodiment of the invention;
Fig. 2 is the schematic flow sheet that makes up the packet filtering policy library according to first embodiment of the invention;
Fig. 3 illustrates the example of a GET message;
Fig. 4 is the structural representation that detects filter according to the message of second embodiment of the invention;
Fig. 5 is the annexation sketch map that the literary composition of second embodiment of the invention detects filter and communication network.
Embodiment
Below will combine accompanying drawing and embodiment to specify execution mode of the present invention, how the application technology means solve technical problem to the present invention whereby, and the implementation procedure of reaching technique effect can make much of and implement according to this.Need to prove that only otherwise constitute conflict, each embodiment among the present invention and each characteristic among each embodiment can mutually combine, formed technical scheme is all within protection scope of the present invention.
First embodiment
Fig. 1 illustrates present embodiment provides the online network message of gateway level to detect the schematic flow sheet of filter method.Below in conjunction with Fig. 1 present embodiment is elaborated.
Step S110 obtains the packet filtering policy library.The packet filtering policy library is made up of an above packet filtering strategy, and the packet filtering strategy is used for confirming that each message is that the interference message also is non-interference message.
Preferably, the packet filtering policy library that is obtained is based on that the GET message off-line ground of predetermined quantity makes up.Fig. 2 shows each substep that makes up the packet filtering policy library.
Substep S111 obtains the GET message of predetermined quantity.
Particularly, can obtain certain time period of existing network all messages of (such as two hours or 10 hours etc.), and therefrom detect all GET messages.
Substep S112, a part of GET message of the GET message that random extraction S111 obtained.
For example, extract 500 to 5000 in the detected GET message of S111, the example of the content of each field of the message that possibly run into when as far as possible reflecting the user capture website.
Preferably, can carry out random extraction to GET message in the detected GET message of S111 from preset one or more IP address.
Substep S113 resolves the GET message that is extracted, to obtain the value of (all or part of) field in the GET message.
Concrete analytic method is consistent with the step S130 that specifies with hereinafter, does not launch explanation at this.
Substep S114 based on the field and the value thereof of the GET message that is obtained among the substep S113, makes up the packet filtering policy library.
Preferably; Each field from the one or more GET message that obtained to the user and the corresponding value that can show among the substep S113 receive the user at every turn according to each field of the one or more GET message of being showed that obtained and the packet filtering strategy that value is provided with thereof at every turn.The critical field of preferably, being showed includes but not limited to HOST, PATH, UserAgent field.
The user can be provided with such strategy according to the content of being showed: when the value of some critical fielies of a GET message is value whole or certain of this critical field of being shown when a part of, then this message is confirmed as and disturbed message or non-interference message.
For example, when the PATH field in the GET message finishes with character string " .jpg ", then this message is confirmed as the interference message.For ease of explanation, can be with should simply remembering work by strategy:
0001：
Wherein, the sequence number of 0001 this packet filtering strategy of expression, false representes that this packet filtering strategy is the blacklist filter type, that is when the value of the PATH of a GET message comprises character string " .jpg ", this message is confirmed as the interference message.
For another example:
0002：if?HOST＝“sina.”then?true
Wherein, the sequence number of 0002 this packet filtering strategy of expression, true representes that this packet filtering strategy is the white list filter type, that is when the HOST of a GET message is " sina ", this message is confirmed as non-interference message.
Similarly, such strategy can also be set:, then this message is confirmed as interference message or desired message (non-interference message) when the value of certain several critical field of a GET message all satisfies the whole of the corresponding primary key value that shown or certain when a part of.
For example, following strategy can be set:
0003：
Expression is 0003 strategy according to sequence number, when the value of the field HOST of a GET message comprises character string " .jpg " for the value of " sina " and field PATH, this message is confirmed as the interference message.
Preferably, can also priority be set,, adopt the packet filtering strategy determined result higher according to priority when not simultaneously according to two different determined results of packet filtering strategy for each strategy.
For example, be that 0001 and 0002 packet filtering strategy adds priority attribute to sequence number, then can No. 0004 substitute No. 0001 and No. 0002 packet filtering strategy with No. 0005 packet filtering strategy, represent as follows:
0004：
0005：if?HOST＝“sina”then?true?B
At this moment, A and B represent priority A and the priority B lower than priority A respectively.In this case; Even do not add sequence number and be 0001,0002,0003 packet filtering strategy, be that the value of 0004 and 0005 message success filtering policy filtered fields HOST comprises the message of " .jpg " for the value of " sina " and field PATH according to sequence number too.
In brief, in substep S114, the user can be provided with according to the analysis result of substep S113 flexibly, and the user is provided with the strategy of result as the packet filtering policy library.
S115 according to the packet filtering policy library that makes up among the S114, filters (this moment for off-line) to the GET message of the predetermined quantity that obtains among the substep S111, according to filter result adjustment packet filtering policy library.
For example, be the situation of non-interference message for the erroneous judgement that will disturb message, can be with adding new strategy in the packet filtering policy library; And, then can from the packet filtering policy library, delete corresponding strategy for the situation of non-interference message erroneous judgement for the interference message.
Make up (training) packet filtering policy library through above-mentioned S111 to S115 off-line ground based on the data message of a certain period, improved the filtration accuracy of this method greatly.
In addition; Also can carry out once above substep S111 to S115 and come to improve gradually the packet filtering policy library through off-line ground; Reach filter result among predetermined number or the step S114 and meet user's re-set target or reach after the user sets index when carrying out number of times; Again this packet filtering policy library is applied to the subsequent step of this method, the GET message in the network is filtered with online in real time ground.
Combine Fig. 2 that the step S110 of present embodiment has been described above, need to prove that step S110 is optional.Promptly can make up the packet filtering policy library through above-mentioned each step; Also can make up packet filtering policy library and update strategy storehouse through alternate manner; Even can also the packet filtering policy library be set manually in advance, perhaps obtain the packet filtering policy library from third party service provider.
Step S120, online detection GET message.
When carrying out the HTTP access to netwoks, can produce messages such as a large amount of for example GET, POST, TRACE, CONNECT since generally speaking network user's behavior only based on the GET message, the therefore online GET message that detects in this step.Be preferably online and detect in real time in the GET message.As what will mention in following each step, each step of present embodiment can guarantee that with each algorithm this online network message detects the real-time of filter method.
Step S130 resolves the detected GET message of step S120, to obtain the value of the field in the GET message.
Known GET message comprises many fields, for example fields such as HOST, PATH, UserAgent.This step is through resolving to obtain the value of all or part of field each field in the message.
Preferably, can adopt keyword registration Hash (Hash) algorithm, single pass filters out the critical field content.More specifically; In this article; Keyword registration Hash (Hash) algorithm refers to: in the process that detected GET message is resolved to step S120; Can only obtain the value of the critical field of these registered in advance in the GET message, and need not to obtain the value of whole fields according to the field (also being called critical field) of the desired parsing of user through hash algorithm.Like this, through the field that the operational efficiency and the minimizing of raising algorithm will be resolved, can improve the real-time performance of this method.
Need to prove; Keyword registration Hash (Hash) algorithm is a kind of implementation of present embodiment; For example, can also under the situation of registered keyword (being field) not, parse the value of all fields; Or under the situation of registered keyword, do not adopt the hash algorithm but adopt sequential scan mode to filter out critical field and value thereof.
Preferably, confirm the field of the desired parsing of user, can also be based on the packet filtering policy library is confirmed the desired parsing of user through the mode of automatic configuration field through receiving the pre-configured mode of user.For example, can each the field setting that relate in the packet filtering policy library be and be set to critical field automatically.
Step S140 is a decollator with character '. ' and/or '/' to be split the value of (all or part of) field.
In this step, if '. ' and/or '/' arranged in the value of a field, then '. ' and/or '/' split as decollator, a plurality of values of the value that is obtained after splitting as this field.
Generally speaking, in order to submit real-time to, can be only to HOST field and PATH field, split as decollator with character '. ' and/or '/'.
For example, a GET message content comprises " HOST:sports.sina.com.cn ", then through step S130, can obtain the value " sports.sina.com.cn " of the HOST field in the GET message.In this step; Can further split as decollator through the value of step S130 being obtained the field in the GET message with character '. '; The value of the HOST field in this GET message is split as " sports ", " sina ", " com " and " cn " four values, can representes as follows with value1, value2, value3 and value4 respectively:
HOST：
value1＝sports
value2＝sina
value3＝com
value4＝cn
In addition, also can increase other except that '. ' and/or other decollator '/', the form of separator is not limited in '. ' and/or '/'.
Introduce the processing among step S130 and the S140 in the method, improved the flexibility ratio of analyzing efficiency and strategy greatly, thereby improve this method real-time performance.
Step S150; Based on packet filtering policy library and step S130 (when this method comprises step S140; The value of the field in the GET message that is obtained the value of the field in the GET message that is also obtained based on step S140) is filtered detected GET message.
Be that example describes with message shown in Figure 3 below.Can know with reference to figure 3, owing to do not comprise " .jpg " in the value of the PATH field of message shown in Figure 3, and comprise " sina " in the value of HOST field, then, can this message be confirmed as non-interference message according to the policy library that comprises 0001 to No. 0005.
Preferably; The value of the field in the GET message that can also obtain among matching message filtering policy storehouse and the step S130 (and S140) based on even numbers group word lookup (Doub le-Array Trie) tree algorithm algorithm comes detected GET message is filtered, to improve the real-time of this method.
Preferably, can preferentially come detected GET message is filtered according to the high packet filtering strategy of priority.For only comprising No. 0004 and the packet filtering policy library of No. 0005 packet filtering strategy (being called for short strategy) is that example describes.Be that No. 0005 strategy of B filters earlier, message shown in Figure 3 is confirmed as non-interference message, and be that No. 0004 strategy of A filters no longer according to priority according to the priority that is higher than priority A.
Through this method, but online in real time is cleaned the noise message (interference message) that produces in (filtration) user HTTP visit behavior, thereby through the message after cleaning, can accurately analyze actual user behavior.
Second embodiment
Fig. 4 illustrates the structure that detects filter according to the message of present embodiment, below in conjunction with Fig. 4 present embodiment is described.
Message detection filter according to present embodiment comprises detecting unit.The online detection of detecting unit GET message, its performed operation is corresponding with the S120 of first embodiment, no longer launches explanation at this.
Message detection filter according to present embodiment can comprise the policy library construction unit.The policy library construction unit makes up the packet filtering policy library based on the GET message off-line of predetermined quantity, and wherein, the packet filtering policy library is made up of an above packet filtering strategy, and the packet filtering strategy is used for confirming that each message is that the interference message also is non-interference message.The concrete operations of policy library construction unit are corresponding with the step S110 of first embodiment.
Preferably, the policy library construction unit can comprise:
Message obtains subelement, and it is responsible for obtaining the GET message of predetermined quantity, and performed operation is corresponding with the substep S111 of first embodiment;
Message extracts subelement, a part of GET message of the GET message that its random extraction message acquiring unit is obtained, and performed operation is corresponding with the substep S112 of first embodiment;
The packet parsing subelement extracts the GET message that subelement extracted to message and resolves, and to obtain the value of (in all or part of) each field in the GET message, performed operation is corresponding with the substep S113 of first embodiment
Make up subelement, each field and the value thereof of the GET message that it is obtained based on the packet parsing subelement make up and are used for the packet filtering policy library that on-line filtration disturbs message, and performed operation is corresponding with the substep S114 of first embodiment.
In addition, the message detection filter according to present embodiment also comprises resolution unit.Resolution unit is resolved the detected GET message of above-mentioned detecting unit, to obtain the value of the field in the GET message.The operation that resolution unit is carried out is corresponding with the step S130 of first embodiment, no longer launches explanation at this.
Preferably, the message detection filter according to present embodiment can also comprise split cells.To be decollator with character '. ' and/or '/' split and will split a plurality of values that value that the back obtained is made this field in the GET message to the value of all or part of field in the GET message to split cells.The performed concrete operations of split cells be with first embodiment in step S140 corresponding, no longer launch explanation at this.
Message detection filter according to present embodiment also comprises filter element.The value of the field in the GET message that filter element is obtained based on packet filtering policy library and resolution unit (and split cells, when being provided with split cells) is filtered detected GET message.The performed operation of filter element is corresponding with the step S150 among first embodiment, does not launch explanation at this.
Message detection filter according to present embodiment also comprises adjustment unit.Adjustment unit is according to constructed packet filtering policy library, and the GET message that message is obtained predetermined quantity that subelement obtains filters, and makes up the constructed packet filtering policy library of subelement according to the filter result adjustment.
Fig. 5 illustrates the annexation of present embodiment and communication network.As shown in Figure 5, the message of present embodiment detects filter as an independent device, is connected with the carrier class gateway network.
Other embodiment
Method of the present invention promptly can realize in gateway with the mode of software, also can be used as the built in hardware unit of gateway and in gateway, realizes, can also be for through realizing with the online special equipment that is independent of gateway that is connected of one or more gateways.When in gateway, realizing, this gateway also generally has route or nat feature except that the message that comprises second embodiment detects filter.When realizing method of the present invention through the special equipment that is independent of gateway; This special equipment is message of the present invention and detects filter; At this moment, the detecting unit (corresponding to S120) of message detection filter can come online detection GET message through the flow that utilizes operator's mirror image/beam split to obtain.
Those skilled in the art should be understood that; Above-mentioned each module of the present invention or each step can realize that they can concentrate on the single calculation element with the general calculation device, perhaps are distributed on the network that a plurality of calculation element forms; Alternatively; They can realize with the executable program code of calculation element, thereby, can they be stored in the storage device and carry out by calculation element; Perhaps they are made into each integrated circuit modules respectively, perhaps a plurality of modules in them or step are made into the single integrated circuit module and realize.Like this, the present invention is not restricted to any specific hardware and software combination.
Though the execution mode that the present invention disclosed as above, the execution mode that described content just adopts for the ease of understanding the present invention is not in order to limit the present invention.Technical staff under any the present invention in the technical field; Under the prerequisite of spirit that does not break away from the present invention and disclosed and scope; Can do any modification and variation what implement in form and on the details; But scope of patent protection of the present invention still must be as the criterion with the scope that appending claims was defined.
Claims (11)
1. the online network message of gateway level detects filter method, it is characterized in that, comprising:
Steps A: the GET message of online detection HTTP;
Step B: said GET message is resolved, to obtain the value of the field in the said GET message; And
Step C: based on the value of the field in packet filtering policy library and the said GET message, detected GET message is filtered, wherein,
Said packet filtering policy library is made up of the packet filtering strategy more than, and said packet filtering strategy is used for confirming that each said GET message is that the interference message also is non-interference message.
2. method according to claim 1 is characterized in that,
Before said steps A, also carry out the step of obtaining said packet filtering policy library, wherein,
The packet filtering policy library that is obtained is based on that GET message and the off-line of predetermined quantity make up.
3. method according to claim 2 is characterized in that, comes off-line to make up said packet filtering policy library through following steps:
Substep A1: the GET message that obtains predetermined quantity;
Substep A2: a part of GET message of the GET message that random extraction is obtained;
Substep A3: this part the GET message to being extracted is resolved, to obtain the value of the field in the GET message;
Substep A4: the field and the value thereof of the GET message that is obtained based on said substep A3 make up the packet filtering policy library;
Substep A5: according to constructed packet filtering policy library, the GET message of the predetermined quantity that obtains among the said substep A1 is filtered, adjust the constructed packet filtering policy library of said substep A4 according to filter result.
4. method according to claim 3 is characterized in that,
Among the said substep A4; The field and the value thereof of the GET message that is obtained based on said substep A3; Receive the field of the GET message that the user obtained according to one or more said substep A3 at every turn and the packet filtering strategy that corresponding value is provided with, to make up the packet filtering policy library.
5. method according to claim 1 is characterized in that,
Among the said step C, mate the value of the field in said packet filtering policy library and the said GET message through even numbers group word lookup tree algorithm, so that detected GET message is filtered.
6. according to each described method in the claim 1 to 5, it is characterized in that, also comprise:
Before step B, also to carry out with character '. ' and/or '/' be decollator splits the value of all or part of field in the said GET message, with splitting the operation as a plurality of values of this field in the said GET message of value that the back obtained.
7. according to each described method in the claim 1 to 6, it is characterized in that,
Among the step B, only obtain the value of the field of the desired parsing of user in the GET message through hash algorithm.
8. method according to claim 7 is characterized in that,
Confirm the field of the desired parsing of said user through receiving the pre-configured mode of user; Perhaps
Based on the packet filtering policy library, confirm the field of the desired parsing of said user through the mode of automatic configuration.
9. one kind is used for the message detection filter that the online network message of gateway level detects filtration, it is characterized in that, comprising:
Detecting unit, the GET message of its online detection HTTP;
Resolution unit, it is resolved said GET message, to obtain the value of the field in the said GET message;
Filter element, it filters detected GET message based on the value of the field in the GET message that is obtained.
10. message according to claim 9 detects filter, it is characterized in that also comprise the policy library construction unit, said policy library construction unit comprises following subelement:
Message obtains subelement, and it is responsible for obtaining the GET message of predetermined quantity;
Message extracts subelement, a part of GET message of the GET message that the said message acquiring unit of its random extraction is obtained;
The packet parsing subelement extracts this part GET message that subelement extracted to said message and resolves, to obtain the value of the field in the GET message;
Make up subelement, the field and the value thereof of the GET message that it is obtained based on said packet parsing subelement make up and are used for the packet filtering policy library that on-line filtration disturbs message; And
Adjustment unit, it is according to constructed packet filtering policy library, and the GET message that said message is obtained the predetermined quantity that subelement obtains filters, and adjusts the constructed packet filtering policy library of said structure subelement according to filter result.
11. detect filter according to each described message in the claim 9 to 10, it is characterized in that, also comprise:
Split cells, to be decollator with character '. ' and/or '/' split the value of all or part of field in the said GET message for it, with splitting value that the back the obtained a plurality of values as this field in the said GET message.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2011102943552A CN102364897A (en) | 2011-09-30 | 2011-09-30 | Gateway-level on-line network message detection filtering method and apparatus thereof |
| PCT/CN2011/084932 WO2013044565A1 (en) | 2011-09-30 | 2011-12-29 | Method and device for detecting and filtering online gateway-level network packet |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2011102943552A CN102364897A (en) | 2011-09-30 | 2011-09-30 | Gateway-level on-line network message detection filtering method and apparatus thereof |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN102364897A true CN102364897A (en) | 2012-02-29 |
Family
ID=45691449
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2011102943552A Pending CN102364897A (en) | 2011-09-30 | 2011-09-30 | Gateway-level on-line network message detection filtering method and apparatus thereof |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN102364897A (en) |
| WO (1) | WO2013044565A1 (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102684997A (en) * | 2012-04-13 | 2012-09-19 | 亿赞普(北京)科技有限公司 | Classification method, classification device, training method and training device of communication messages |
| CN103078854A (en) * | 2012-12-28 | 2013-05-01 | 北京亿赞普网络技术有限公司 | Message filtering method and device |
| CN103327104A (en) * | 2013-06-25 | 2013-09-25 | 天津汉柏汉安信息技术有限公司 | Method for rendering filtered webpage advertisements to client |
| CN103354546A (en) * | 2013-06-25 | 2013-10-16 | 亿赞普(北京)科技有限公司 | Message filtering method and message filtering apparatus |
| CN103593484A (en) * | 2013-12-03 | 2014-02-19 | 南京安讯科技有限责任公司 | Method for filtering garbage logs during mobile phone internet surfing |
| CN107480190A (en) * | 2017-07-11 | 2017-12-15 | 国家计算机网络与信息安全管理中心 | A kind of filter method and device of non-artificial access log |
| CN108881181A (en) * | 2018-05-30 | 2018-11-23 | 杭州迪普科技股份有限公司 | A kind of filter method and device of message |
| WO2019134277A1 (en) * | 2018-01-02 | 2019-07-11 | 武汉斗鱼网络科技有限公司 | Data filtering method and device, server, and readable storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1350249A (en) * | 2001-12-04 | 2002-05-22 | 上海复旦光华信息科技股份有限公司 | Remote user operation process recording and restoring method |
| CN101266610A (en) * | 2008-04-25 | 2008-09-17 | 浙江大学 | Web active user website accessing mode on-line excavation method |
| CN102098229A (en) * | 2011-03-04 | 2011-06-15 | 北京星网锐捷网络技术有限公司 | Method and device for optimizing and auditing uniform resource locator (URL) as well as network device |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1669899A1 (en) * | 2004-12-08 | 2006-06-14 | France Telecom | Method for processing HTTP and HTML page requests sent or received by a browser towards or from at least one Web server and associated server |
| CN101399843B (en) * | 2007-09-27 | 2012-11-28 | 中兴通讯股份有限公司 | Deepened filtering method for packet |
| US8107502B2 (en) * | 2009-09-11 | 2012-01-31 | Symmetricom, Inc. | Method and apparatus for monitoring packet networks |
| CN101909079B (en) * | 2010-07-15 | 2013-04-24 | 北京迈朗世讯科技有限公司 | User online behavior data acquisition method in backbone link and system |
-
2011
- 2011-09-30 CN CN2011102943552A patent/CN102364897A/en active Pending
- 2011-12-29 WO PCT/CN2011/084932 patent/WO2013044565A1/en active Application Filing
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1350249A (en) * | 2001-12-04 | 2002-05-22 | 上海复旦光华信息科技股份有限公司 | Remote user operation process recording and restoring method |
| CN101266610A (en) * | 2008-04-25 | 2008-09-17 | 浙江大学 | Web active user website accessing mode on-line excavation method |
| CN102098229A (en) * | 2011-03-04 | 2011-06-15 | 北京星网锐捷网络技术有限公司 | Method and device for optimizing and auditing uniform resource locator (URL) as well as network device |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102684997A (en) * | 2012-04-13 | 2012-09-19 | 亿赞普(北京)科技有限公司 | Classification method, classification device, training method and training device of communication messages |
| CN103078854A (en) * | 2012-12-28 | 2013-05-01 | 北京亿赞普网络技术有限公司 | Message filtering method and device |
| CN103078854B (en) * | 2012-12-28 | 2016-04-13 | 北京亿赞普网络技术有限公司 | Message filtering method and device |
| CN103327104A (en) * | 2013-06-25 | 2013-09-25 | 天津汉柏汉安信息技术有限公司 | Method for rendering filtered webpage advertisements to client |
| CN103354546A (en) * | 2013-06-25 | 2013-10-16 | 亿赞普(北京)科技有限公司 | Message filtering method and message filtering apparatus |
| CN103593484A (en) * | 2013-12-03 | 2014-02-19 | 南京安讯科技有限责任公司 | Method for filtering garbage logs during mobile phone internet surfing |
| CN107480190A (en) * | 2017-07-11 | 2017-12-15 | 国家计算机网络与信息安全管理中心 | A kind of filter method and device of non-artificial access log |
| WO2019134277A1 (en) * | 2018-01-02 | 2019-07-11 | 武汉斗鱼网络科技有限公司 | Data filtering method and device, server, and readable storage medium |
| CN108881181A (en) * | 2018-05-30 | 2018-11-23 | 杭州迪普科技股份有限公司 | A kind of filter method and device of message |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2013044565A1 (en) | 2013-04-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102364897A (en) | Gateway-level on-line network message detection filtering method and apparatus thereof | |
| CN107370763B (en) | Asset safety early warning method and device based on external threat information analysis | |
| CN109951500B (en) | Network attack detection method and device | |
| US9912680B2 (en) | Detecting malicious HTTP redirections using user browsing activity trees | |
| EP3142020A1 (en) | Resource downloading method and device | |
| CN110020062B (en) | Customizable web crawler method and system | |
| CN101964025A (en) | XSS (Cross Site Scripting) detection method and device | |
| CN105760379B (en) | Method and device for detecting webshell page based on intra-domain page association relation | |
| US9042863B2 (en) | Service classification of web traffic | |
| CN105376217B (en) | A kind of malice jumps and the automatic judging method of malice nested class objectionable website | |
| CN105635064B (en) | CSRF attack detection method and device | |
| US11763032B2 (en) | Method and system for preserving privacy in an HTTP communication between a client and a server | |
| CN108282441B (en) | Advertisement blocking method and device | |
| CN111818103A (en) | Traffic-based tracing attack path method in network target range | |
| CN114465741B (en) | Abnormality detection method, abnormality detection device, computer equipment and storage medium | |
| CN113810381B (en) | Crawler detection method, web application cloud firewall device and storage medium | |
| CN106713318B (en) | WEB site safety protection method and system | |
| CN111404912A (en) | Domain name detection method and device based on IP white list | |
| WO2016119420A1 (en) | Method, apparatus and communication gateway for detecting malicious access to network resources | |
| CN111818073A (en) | Method, device, equipment and medium for detecting defect host | |
| CN109547294B (en) | Networking equipment model detection method and device based on firmware analysis | |
| CN108737332A (en) | A kind of man-in-the-middle attack prediction technique based on machine learning | |
| CN108259416B (en) | Method for detecting malicious webpage and related equipment | |
| US11159548B2 (en) | Analysis method, analysis device, and analysis program | |
| CN105262720A (en) | Web robot traffic identification method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C12 | Rejection of a patent application after its publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20120229 |