The content of the invention
In order to solve the above-mentioned problems in the prior art, the present invention proposes a kind of network attack
Detecting system and the method for detecting network attack.
According to an aspect of the present invention, it is proposed that a kind of network attack detection system.The system
Including:Network Data Capture unit, it is configured as receiving network request packet from external data source;
Data dispatch unit, the Network Data Capture unit is connected to, is configured as to network request number
According to being distributed;At least one counting unit, the data dispatch unit is connected to, is configured as
Combination based on different network request parameter or network request parameter is within the specified time cycle
The network request packet received from the data dispatch unit is counted, and is generated and closed rule;
And closing management unit, be connected at least one counting unit, be configured as from it is described to
Few counting unit receives the rule of closing generated respectively, and determines whether the envelope to being received
Prohibit rule to be applied, wherein, the data dispatch unit passes through based in network request packet
The combination of source IP address and purpose IP address carries out load balancing, by the network request packet not
The network request packet corresponding with the source IP address closed distributes single at least one counting
Member.
Preferably, the system also includes:Configuration management element, it is configured as from configuration server
Configuration information update is pulled, and resulting configuration information update is accordingly sent to the data
Dispatch unit, at least one counting unit and/or the closing management unit.
Preferably, the system also includes:Counting unit is intercepted, the data is connected to and distributes list
Member, it is configured as counting the network request packet received from the data dispatch unit, its
In, the data dispatch unit is configured as the source IP in the network request packet with being closed
The corresponding network request packet in address distributes the interception counting unit.
Preferably, when at least one counting unit includes the combination based on network request parameter
During the counting unit counted, the system also includes:Secondary counting unit, it is connected to described
The counting unit that combination based on network request parameter is counted, is configured as from the meter connected
Counting unit obtains count results, and the network request ginseng in the combination based on the network request parameter
One of number, secondary counting is carried out using acquired count results, and generate and close rule, it is described
Closing management unit is additionally configured to close rule from what secondary counting unit reception was generated,
And determine whether to be applied to the rule of closing received.
Preferably, the system also includes:Data outputting unit, it is configured as from described at least one
Individual counting unit distinguishes count pick up result, and the count results write into Databasce that will be received.
Preferably, the network request packet is with including request URL, source IP address and/or purpose IP
It is at least one in location, wherein, the request URL includes host machine part, path sections and attribute
Part.
Preferably, the network request parameter includes at least one in the following:Request URL
Host machine part, the path sections of request URL, the attribute section of request URL, source IP address and/
Or purpose IP address.
Preferably, the time cycle specified is the previous unit interval for terminating at current time
Time cycle with the duration specified.
According to another aspect of the present invention, there is provided one kind passes through network attack detection system detectio net
The method of network attack.The network attack detection system includes Network Data Capture unit, data are sent
Bill is first, at least one counting unit and closing management unit.Methods described includes:The network
Data capture unit receives network request packet from external data source;The data dispatch unit passes through
Combination based on the source IP address in network request packet and purpose IP address carries out load balancing, will
The network request packet corresponding with the source IP address closed does not distribute in the network request packet
To at least one counting unit;At least one counting unit is based on different network requests
The combination of parameter or network request parameter within the specified time cycle from the data dispatch unit
The network request packet of reception is counted, and is generated and closed rule;And the closing management list
Member receives the rule of closing generated from least one counting unit respectively, and determines whether pair
The rule of closing received is applied.
Preferably, the system also includes configuration management element, and methods described also includes:Institute
State configuration management element and pull configuration information update from configuration server, and resulting is matched somebody with somebody into confidence
Breath renewal is accordingly sent to the data dispatch unit, at least one counting unit and/or institute
State closing management unit.
Preferably, the system, which also includes the system, also includes intercepting counting unit, and described
Method also includes:The data dispatch unit is by the source IP in the network request packet with being closed
The corresponding network request packet in address distributes the interception counting unit;Described intercept counts list
Member counts to the network request packet received from the data dispatch unit.
Preferably, when at least one counting unit includes the combination based on network request parameter
During the counting unit counted, the system also includes secondary counting unit, and methods described
Also include:The secondary counting unit obtains count results from the counting unit connected, and is based on
One of network request parameter in the combination of the network request parameter, utilize acquired counting knot
Fruit carries out secondary counting, and generates and close rule;The closing management unit is from the secondary counting
What unit reception was generated closes rule, and determines whether to be applied to the rule of closing received.
Preferably, the system also includes data outputting unit, and methods described also includes:Institute
Data outputting unit is stated from least one counting unit difference count pick up result, and will be connect
The count results write into Databasce of receipts.
Preferably, the network request packet is with including request URL, source IP address and/or purpose IP
It is at least one in location, wherein, the request URL includes host machine part, path sections and attribute
Part.
Preferably, the network request parameter includes at least one in the following:Request URL
Host machine part, the path sections of request URL, the attribute section of request URL, source IP address and/
Or purpose IP address.
By using network attack detection system proposed by the invention and the side of detection network attack
Method, when website is brushed by force by such as DDOS attack or malicious user, it can lead to by this algorithm
Cross the attack of various dimensions early warning net to threaten, so as to realize the intercept attack at web portal, after protection
Hold service server normal operation.
Embodiment
It is pointed out that in the description of the detailed description below of the present invention, for convenience
For the sake of, partial content may be directed to specific network frame (for example, Storm frameworks, one kind are increased income
Real-time streaming Computational frame) be illustrated, it is understood that, embodiments of the invention
It is not limited to these specific frameworks.A kind of solution of the term " load balancing " occurred in the present invention
Release the front end forwarding including being carried out to balance backend services server access pressure.It should be appreciated that
, explanation of the above to the term of the present invention, which indicate that, to be easy to enter technical scheme
Row details, and be not used to limit the solution of the present invention.
The present invention is specifically described below with reference to accompanying drawing.
First, Fig. 1 shows network attack detection system 100 according to an embodiment of the invention
Structured flowchart.The network attack defending system 100 includes Network Data Capture unit 110, number
According to dispatch unit 120, at least one counting unit 130 (3 counting units 130 are merely illustrated in Fig. 1,
Here quantity is not limited) and closing management unit 140.
In one embodiment, the Network Data Capture unit 110 is configured as from external data source
Receive network request packet.Preferably, the Network Data Capture unit 110 from data source (such as
Message queue) newest network access data is persistently pulled, and transmit to data dispatch unit.
The network request packet is included in request URL, source IP address and/or purpose IP address extremely
It is few one, wherein, the request URL includes host machine part (Host), path sections (Path)
With attribute section (Param).For example, it is directed to URL:http://search.jd.com
/Book/SearchKeyword=cpu, its host machine part are:Search.jd.com, its path sections
For/Book/Search, its attribute section is keyword=cpu.
In one embodiment, the data dispatch unit 120 is connected to the Network Data Capture list
Member 110, is configured as distributing network request packet.In the network request packet received,
Including corresponding to the data for closing source IP address and corresponding to the data for not closing source IP address, corresponding to
Accordingly close, rear end will not be impacted due to having employed in the data for closing source IP address,
Therefore it need not be taken in again when detecting and attacking, and need to only considers to correspond to not close source
The data of IP address.Specifically, the data dispatch unit 120 is by based in network request packet
Source IP address and purpose IP address combination carry out load balancing, by the network request packet
The network request packet corresponding with the source IP address closed is not distributed at least one counting
Unit.
Certainly, still the data for the source IP address closed can be counted, the counting can
For verifying system interception rate.Specifically, the system may also include:Counting unit is intercepted, even
The data dispatch unit is connected to, being configured as please to the network received from the data dispatch unit
Data are asked to be counted, wherein, the data dispatch unit is configured as the network request number
The network request packet corresponding with the source IP address closed distributes to described intercept and counts list in
Member.
It is by the meaning of data dispatch unit progress load bridging, a topology in system framework
Data processing unit (such as counting unit, closing management unit etc.) and data acquisition in structure
Unit (such as Network Data Capture unit etc.) may be dispersed on different server or JVM and transport
OK, therefore, to can be according to the key of stream data definition when distributing data between data processing unit
Value carries out load bridging, and an extra data dispatch unit is defined in topological structure.Data
The combination that dispatch unit is based on " source IP address+purpose IP address " carries out data distribution as key assignments.
Because the combination key assignments of " source IP address+purpose IP address " is more, so data dispatch unit is based on
The combination of " source IP address+purpose IP address " carries out data distribution as key assignments can realize using more
Thin granularity shunts to data payload, increases efficiency and the flexibility of shunting.
In one embodiment, at least one counting unit 130 is connected to the data dispatch unit
120, the combination based on different network request parameter or network request parameter is configured as to specifying
Time cycle in from the data dispatch unit 120 receive network request packet counted, and
Rule is closed in generation.Specifically, each counting unit 130 is directed to different network request parameter or net
The combination of network required parameter is counted, and the network request parameter includes at least one in the following
It is individual:The host machine part of request URL, the path sections of request URL, request URL attribute section,
Source IP address and/or purpose IP address.For example, each counting unit 130 can be to source IP address, main frame
Part and three dimensions of path sections carry out separate counts respectively, and are produced within each time cycle
Count results and close rule.
Preferably, in one embodiment, when after a time cycle, counting unit 130
Counting reset, counting unit 130 restarts to count to the data in next time cycle.
Preferably, in another embodiment, the time cycle specified is when terminating at current
The time cycle with the duration specified for the previous unit interval carved.Specifically, for this feelings
Condition, counting unit 130 can be counted using time slip-window, and the time window corresponds to eventually
The slip in cycle specified time, over time window of the unit interval terminated in before current time, meter
Counting unit 130 can count to the enumeration data in a nearest time cycle at any time, and base
In this rule is closed to generate.This scheme will hereinafter be carried out detailed with reference to specific embodiment
Thin description.
Preferably, during rule is closed in generation, counting unit 130 will can ask for network
The count results of the particular value of parameter are sought compared with the threshold value specified, and are sealed based on this to generate
Prohibit rule.
For example, when the network request parameter is target ip address, corresponding counting unit
The purpose IP address corresponding to network request packet received in 130 pairs of time cycles specified is carried out
Count, when the counting for a purpose IP address exceedes the threshold value specified, the counting unit
130 pairs of network request numbers corresponding with the purpose IP address being counted within the time cycle
Counted according to corresponding source IP address, and by the one or more (numbers of statistical magnitude highest
Amount can also be preassigned) source IP address add close rule.
For another example when the network request parameter is source IP address, corresponding counting unit 130
Source IP address corresponding to the network request packet that is received in the specified time cycle is counted,
When the counting for source IP address exceedes the threshold value specified, the counting unit is by the source IP
Address adds and closes rule.
Preferably, counting unit 130 can be to add the source IP address appointment penalty coefficient for closing rule,
The penalty coefficient of the higher source IP address of statistical magnitude is also higher.For example, penalty coefficient can be with
Corresponding to closing duration, so as to the higher source IP address of statistical magnitude to close duration longer.Example again
Such as, penalty coefficient may correspond to user and ask reject rate, the higher source IP address of statistical magnitude
Request reject rate it is higher.
In one embodiment, it is single to be connected at least one counting for the closing management unit 140
Member 130, be configured as receiving respectively from least one counting unit 130 generated close rule
Then, and determine whether to be applied to the rule of closing received.
The rule of closing of the generation of at least one counting unit 130 is uniformly sent to closing management list
Member, and decide whether to be issued to such as intercept server by it and applied.The reason for so doing it
One is, each closing rule has a term of validity, before the deadline It is not necessary to one
Source IP address repeats to close, and otherwise can influence the efficiency of intercept server.It is further, it is possible that a certain
Specific source IP address has been identified as that rear end will not be attacked that (that is, the source IP address is in
In white list), then closing management unit 140 also will not be by closing under rule for the source IP address
It is dealt into server.
In one embodiment, the system 100 also includes configuration management element.The configuration management
Unit is configured as pulling configuration information update from configuration server, and by resulting configuration information
Renewal is accordingly sent to the data dispatch unit, at least one counting unit and/or described
Closing management unit.
In one embodiment, when at least one counting unit 130 includes being based on network request
During the counting unit 130 that the combination of parameter is counted, the system also includes secondary counting unit.
The secondary counting unit is connected to the counting that the combination based on network request parameter is counted
Unit 130, and be configured as obtaining count results from the counting unit 130 connected, and be based on
One of network request parameter in the combination of the network request parameter, utilize acquired counting knot
Fruit carries out secondary counting, and generates and close rule.
For example, if host machine part and path sections of a certain counting unit 130 based on URL this
Two network request parameters are counted, i.e. for any one host machine part and path sections
Combination is counted, then the system 100 may also include the secondary counting unit for host machine part,
The secondary counting unit obtains its count results from the counting unit 130, and is directed to the main frame
Partial any particular value, to for the host machine part particular value and either path part particular value
The count results summation of combination, as the count results for the host machine part particular value.When for
After each host machine part particular value is summed respectively, the progress of Intrusion Detection based on host part has just been obtained
The result of counting.
The granularity counted for parameter combination is smaller, in one embodiment, its granularity ng12
N can be expressed asg12=ng1×ng2/ N, wherein ng1It is the granularity counted just for host machine part,
ng2It is the granularity with being counted just for path sections, N is total in the specified time cycle
Data volume (visible, ng12Less than ng1And ng2In any one).
Using this configuration of secondary counting unit, rather than in the combination of network request parameter
Each parameter independently sets a counting unit 130, can reduce the data transfer between node
Amount, so as to improve the level of resources utilization.
Preferably, the closing management unit 140 is additionally configured to receive from the secondary counting unit
What is generated closes rule, and determines whether to be applied to the rule of closing received.
In one embodiment, the system 100 also includes data outputting unit.The data output
Unit is configured as from least one counting unit difference count pick up result, and will be received
Count results write into Databasce.
Next, referring to Fig. 2.Fig. 2, which is shown, according to an embodiment of the invention passes through network
The flow chart of the method 200 of attack detection system detection network attack.The network attack detection system
Including Network Data Capture unit, data dispatch unit, at least one counting unit and closing management
Unit.Methods described 200 starts from providing step S210, and the Network Data Capture unit is from outer
Portion's data sources network request packet.Next, in step S220, the data distribute list
Member is by based on the source IP address in network request packet and the combination of purpose IP address load
Weighing apparatus, by network request number not corresponding with the source IP address closed in the network request packet
According to distributing at least one counting unit.Next, in step S230, described at least one
Combination of the individual counting unit based on different network request parameter or network request parameter is to specified
The network request packet received in time cycle from the data dispatch unit is counted, and is generated
Close rule.Finally, in step S240, the closing management unit is from least one meter
Counting unit receives the rule of closing generated respectively, and determines whether to add the rule of closing received
With application.
Preferably, the system also includes configuration management element, and methods described also includes:Institute
State configuration management element and pull configuration information update from configuration server, and resulting is matched somebody with somebody into confidence
Breath renewal is accordingly sent to the data dispatch unit, at least one counting unit and/or institute
State closing management unit.
Preferably, the system, which also includes the system, also includes intercepting counting unit, and described
Method also includes:The data dispatch unit is by the source IP in the network request packet with being closed
The corresponding network request packet in address distributes the interception counting unit;Described intercept counts list
Member counts to the network request packet received from the data dispatch unit.
Preferably, when at least one counting unit includes the combination based on network request parameter
During the counting unit counted, the system also includes secondary counting unit, and methods described
Also include:The secondary counting unit obtains count results from the counting unit connected, and is based on
One of network request parameter in the combination of the network request parameter, utilize acquired counting knot
Fruit carries out secondary counting, and generates and close rule;The closing management unit is from the secondary counting
What unit reception was generated closes rule, and determines whether to be applied to the rule of closing received.
Preferably, the system also includes data outputting unit, and methods described also includes:Institute
Data outputting unit is stated from least one counting unit difference count pick up result, and will be connect
The count results write into Databasce of receipts.
Preferably, the network request packet is with including request URL, source IP address and/or purpose IP
It is at least one in location, wherein, the request URL includes host machine part, path sections and attribute
Part.
Preferably, the network request parameter includes at least one in the following:Request URL
Host machine part, the path sections of request URL, the attribute section of request URL, source IP address and/
Or purpose IP address.
Preferably, methods described 200 is performed by the system 100 shown in Fig. 1.Above for Fig. 1
In the system 100 various specific descriptions and explanation that carry out, be equally applicable to each step of method 200
Suddenly, will not be repeated here.
Next a specific embodiment is directed to, to the counting unit counted using sliding window
The algorithm and internal storage structure corresponding with used algorithm used in 130 is described in detail.
Specifically, an access counter will be safeguarded in each counting unit 130, is examined for counting
For method of determining and calculating, sampling the data of a certain second merely correctly can not effectively judge to attack IP, therefore
This algorithm uses the data in a nearest period as sampling interval (also referred to as data window).
Simultaneously as the timeline of sampling always in continuous displacement, causes data window also in continuous displacement,
Our data windows of this constantly displacement are referred to as " sliding window ", inside each counting unit 130
Counter be to possess the characteristic of sliding window.Specifically, for Java, can use embedding
The mode for covering Map realizes sliding window.
Fig. 3 shows the schematic diagram of the sliding window according to one particular embodiment of the present invention.Fig. 3
Top show by the lint-long integer time multiple time points of time shaft for indicating.Thereunder,
The progress of time window over time, is constantly slided along timeline, form for it is multiple not
Sliding time window (1-10) in the same time.
Specifically, the counter internal storage structure of counting unit corresponding with the sliding time window can
In a manner of using two-dimensional storage.Specifically, Fig. 4 shows a particular implementation according to the present invention
The schematic diagram of counter internal storage structure in the counting unit of example.Correspond to the master based on URL in Fig. 4
The situation that the combination of machine part (host) and path sections (path) is counted.Wherein, from a left side
Scheme visible, for each specific host and path combination, all store a sliding window counter,
Have in the sliding window counter for (such as lint-long integer time point per second during sliding window
1456475226 and source IP access count list 1456475227) (referring to right figure).Each list
Include each source IP address and its corresponding access times.
By each moment of identical source IP address in the list for sliding time window stored
Access times are added, and have just obtained total access of the source IP address during the sliding time window time
Number.Total access count in for a certain sliding window threshold value of dynamic configuration, then can
Rule is closed for its generation.
Specifically, can be according to following Algorithm Analysis Host/Path access situation, and produce envelope
Prohibit rule:
First, source IP address is corresponded to Host/Path according to access count descending to be ranked up (quickly
Sequence, insertion sort scheduling algorithm are all suitable for);
Then, take visit capacity highest top n IP to be used as according to configuration and close punishment object (white name
Except single IP), specific algorithm is as follows:Calculate top n IP access count summation;Successively to preceding N
Individual IP, IP access counts/top n IP is counted into summation, so as to calculate the penalty coefficient for obtaining the source IP,
The bigger source IP of visit capacity, its penalty coefficient are higher.
It should be appreciated that still, in the counting unit described above counted using sliding window
The algorithm of use and internal storage structure corresponding with used algorithm are exemplary, are not used in
The limitation present invention, the scheme that other are counted using sliding window are equally applicable.
Although combined the preferred embodiments of the present invention show the present invention, this area above
Technical staff will be understood that, without departing from the spirit and scope of the present invention, can be right
The present invention carries out various modifications, replaces and change.Therefore, the present invention should not by above-described embodiment Lai
Limit, and should be limited by appended claims and its equivalent.