CN105072089A - WEB malicious scanning behavior abnormity detection method and system - Google Patents
WEB malicious scanning behavior abnormity detection method and system Download PDFInfo
- Publication number
- CN105072089A CN105072089A CN201510404406.0A CN201510404406A CN105072089A CN 105072089 A CN105072089 A CN 105072089A CN 201510404406 A CN201510404406 A CN 201510404406A CN 105072089 A CN105072089 A CN 105072089A
- Authority
- CN
- China
- Prior art keywords
- keyword
- calling party
- web
- users
- user
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
- H04L63/1425—Traffic logging, e.g. anomaly detection
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
Abstract
The invention discloses a WEB malicious scanning behavior abnormity detection method and a WEB malicious scanning behavior abnormity detection system. The method comprises the following steps: 1) extracting keyword characteristics and statistics characteristics of access users from an access history record, and building keyword vectors and statistic characteristic vectors of the users, 2) traversing the keyword vectors of the users, performing statistics on the user number corresponding to each keyword, and building a global keyword table, 3) calculating the uncommon degree of each keyword according to the global keyword table, calculating original abnormal score values of the access users according to the corresponding uncommon degrees, correcting the original abnormal score values then according to the statistic characteristic vectors of the access users, and obtaining final abnormal score values of the users; 4) for a jump point of a final abnormal score value sequence of all the access users, taking the final abnormal score value corresponding to the jump point as a threshold, and 5) comparing the final abnormal score values of the access values with the threshold, and taking the users as malicious scanning users if the final abnormal score values of the access values are greater than the threshold. An unknown attack behavior can be found, and normal historical data is not relied on.
Description
Technical field
The present invention relates to a kind of method for detecting abnormality towards WEB malice scanning behavior and system, belong to WEB security fields.
Background technology
WEB scanning is that a kind of common WEB accesses behavior, generally refers to the content of web crawlers by certain Rule targeted website.The scanning of WEB malice is with the difference of normal scan, the former target is by information such as scanning discovery website vulnerability, sensitive information, mandate entrances, the target of the latter is then the content information that acquisition website normally provides, as the Html page, picture, CSS file etc.Target due to the two has the difference of internal, and therefore its access behavior also has obvious difference:
First, the access request of WEB malice scanning has obviously different from the access request that normal WEB scans semantically.As, the scanning of WEB malice can send "/robots.txt/1.php " to judge website and whether there is file type parse error leak, send "/web1.rar " and detect whether store backup file existence improperly, send "/servlet/ContentServer? pagename=<script>alert (' Vulnerable') </script> " judge whether website has cross site scripting leak etc.And normal WEB scanning can send request according to web site url, as "/abc/def/201309/201309_3629160.html ", "/images/xian06.gif " etc.
Secondly, WEB maliciously scans in the request sent, and the ratio of the corresponding code of mistake (e.g., 401,404 etc.) generally can be higher, and normal WEB scans because its target is the resource that acquisition website provides, and therefore in its request, the ratio of 200 generally can be higher.But first some WEB malice scanning can be carried out normal webpage and crawl, and after determining website structure, then carries out hostile content scanning, and the ratio of wrong corresponding code in its request can be caused like this to decline.
Again, normal WEB scanning generally all adopts the request of GET method, a small amount of reptile can adopt HEAD method, and the scanning of some WEB malice a large amount of HEAD method can be adopted to judge fast whether website exists file destination, or adopt the methods such as PUT, DELETE to test website whether can revised context.
WEB malice Scanning Detction is generally classified as WEB attack detecting, after normally whether detection WEB request attacks one by one, judge whether the global behavior of assailant belongs to malice scanning again, if general number of times of attack or attack duration exceed certain threshold value, just think that attack belongs to the scanning of WEB malice.The common method of WEB attack detecting can be divided into two classes:
One class is rule detection, by attacking rule match, identifies that the WEB for website vulnerability attacks, as SQL injection attacks, cross-site scripting attack etc.Rule detection method can carry out detecting in real time and interception to WEB malice scanning behavior, but can only detect known query-attack, cannot find unknown query-attack.
Another kind of is abnormality detection based on normal data, by the normal flowing of access study to website, sets up access white list or forward Access Model, only allows the request in white list list or meets the request access of forward Access Model.The accuracy of these class methods depends on normal historical data greatly, if there is no normal historical data, or the normal access behavior kind that normal historical data contains is very few, or has been mixed into attack data in historical data, and the rate of false alarm of these class methods and rate of failing to report all can raise greatly.In existing abnormality detection technology, mostly belong to this class.As disclosed a kind of web intrusion prevention method being applied to application layer in " a kind of web intrusion prevention method and system being applied to application layer " (CN201110117191), the method is given a mark to the behavior of visitor according to the hazardous act preset, and is on the defensive to access behavior by cumulative threat value.Wherein, the hazardous act preset is by obtaining setting up forward model parameter after the study of a large amount of history normal behaviours.Disclose a kind of method of protecting Web and attacking in " method that protection WEB attacks " (CN201410737526), the method carries out attack protection by the method that black and white lists mixes, and wherein white list is also by obtaining after normal action learning.
Summary of the invention
For the technical problem that prior art is deposited, the object of the present invention is to provide method for detecting abnormality and the system of a kind of novel WEB malice scanning behavior, it does not rely on normal visit data, can identify that malice scans user from the website visiting user of magnanimity.
The technical scheme of the method for the invention is by carrying out keyword feature to the calling party in WEB access history record (as WEB access log) and statistical nature extracts, normal users access behavior is utilized to have the feature of similitude and plurality, first the semantically anomalous value of user behavior is calculated according to keyword feature, statistical nature is utilized to revise semantically anomalous value according to heuritic approach again, thus obtain the access exception score value of user, finally calculate the threshold value of user's abnormal score, user abnormal score being exceeded threshold value is identified as malice and scans user.
A kind of WEB malice scanning behavior method for detecting abnormality, its method step comprises:
1) preliminary treatment is carried out to WEB access history record, resolve WEB access history record, mark well-known search engine reptile user, character string is asked to carry out word segmentation processing to the WEB of each user, extraction keyword vector, simultaneously to the access behavior of each user, respectively from visit capacity four aspect statistics of the visit capacity of the visit capacity of total visit capacity, different requesting method, different page type, different answer code, obtain statistical nature vector;
2) travel through the keyword vector of all users, add up the number of users that each keyword is corresponding, build overall antistop list, in table, record the number of users of each keyword and correspondence thereof;
3) the uncommon degree of each keyword is calculated according to overall antistop list, traverse user keyword vector, calculates the original anomaly score value of each user according to the uncommon degree of keyword, more vectorial according to the statistical nature of user, revise original anomaly score value, obtain final abnormal score;
4) exceptional value sequence is formed to the ascending sequence of the final abnormal score of user, calculate the catastrophe point of exceptional value sequence, using final abnormal score corresponding for abnormity point as threshold value, if without catastrophe point, then there is not threshold value;
5) if there is threshold value, then judging whether the final abnormal score of user is greater than this threshold value, if so, then identifying that user is for maliciously scanning user.
Further, described WEB asks character string can all or part of for each primary fields content in WEB request, as request URL, request user agent information USER-AGENT, asks BODY content etc.
Further, described participle processing method is as follows:
Ask character string to be converted into lowercase WEB, use appointment to stop lexicon and ask character string to be decomposed to WEB, record the number of times that each word occurs, build user's keyword vector.
Further, described appointment stop lexicon include but not limited to "/", ". ", "? ", "=", " & ", the character such as ", ".
Further, described user access activity statistical nature vector comprises PUT in the total access times of user, the errored response code number of times received, auxiliary element access times, http protocol and DELETE request method number of times, and wherein auxiliary element refers to the files such as CSS, picture, audio frequency, Office document, PDF.
Further, described user's keyword vector K
u={ (k
i, ck
i) | 0≤i≤m-1}, wherein m is the quantity of user's keyword, k
ibe i-th keyword, c
kifor keyword k
ithe number of times occurred in character string is asked at all WEB of user.
Further, described overall antistop list GK={ (k
i, uc
ki) | 0≤i≤N-1,1≤uc
ki≤ N
u, wherein N is the sum of the different keyword of all users, uc
kifor keyword k
ithe number of users occurred in character string is asked, N at WEB
ufor total number of users.
Further, the computational methods of the uncommon degree of described keyword are as follows:
For user's keyword vector K
u={ (k
i, c
ki) | 0≤i≤m-1} and overall antistop list GK={ (k
i, uc
ki) | 0≤i≤N-1}, k
iuncommon degree P
ki=Log (c
ki) * Log (N
u/ uc
ki* uc
ki), wherein Log (x) is natural logrithm function.
Further, described user's original anomaly divides value calculating method as follows:
If user is marked as well-known search engine reptile, then its original anomaly value is 0, otherwise obtains this user's original anomaly score value by cumulative for the uncommon degree of all keywords in this user's keyword vector.
Further, described user final abnormal score modification method is as follows:
Choose corrected parameter w
1, w
2with w
3, calculate final abnormal score=original anomaly score value * Exp (w
1, the access times of errored response synchronous codes number/total) and * Exp (w
2, the access times of auxiliary element access times/total) and+w
3* the access times of the number of times/total of PUT and DELETE method, wherein Exp (a, b) is exponential function, and a is the truth of a matter, and b is index.
Further, the computational methods of the catastrophe point of described exceptional value sequence are as follows:
Select mutation parameter T, for sequence SA={ α
i| 0≤i≤N
u-1, α
i≤ α
i+1, sequence of calculation SB={ β
j| 0≤j≤N
u-2}, if wherein a
j+1-a
j>T, then β
j=α
j+1-α
j, otherwise β
j=0.For sequence SB, sequence of calculation SC={ γ
k| 0≤k≤N
u-3}, wherein γ
k=β
k+1-β
k.To sequence SC, search k*, make k* for the γ that satisfies condition
k>T or γ
kthe minimum k of <0, if k* exists, the catastrophe point of exceptional value sequence is a
k*+1, otherwise there is not catastrophe point in exceptional value sequence.
A kind of WEB malice scanning behavior detection system, comprises configuration read module, data preprocessing module, abnormality detection module and data memory module.
Described configuration read module is responsible for from data memory module, read configuration parameter information, comprises the access configuration of WEB access history record, well-known search engine reptile IP list, corrected parameter w
1, w
2, w
3and mutation parameter T;
Described data preprocessing module is responsible for reading from data memory module and is resolved original WEB access history record, whether be reptile according to well-known search engine reptile IP list mark user, and calculate the keyword vector of user and statistical nature vector, and by pre-processed results stored in data memory module;
Described abnormality detection module in charge reads pre-processed results from data memory module, carries out abnormality detection according to the keyword of user vector with statistical nature vector, exports malice scanning user's testing result and stored in data memory module;
Described data memory module is responsible for storage system configuration information, original WEB access history record, data prediction result and malice scanning user testing result.
Beneficial effect of the present invention:
Method of the present invention detecting from WEB access history record of automation can identify that WEB malice scans behavior, comprise the WEB attacks such as SQL injection, XSS, WEBSHELL detects access behavior, registration or management interface scanning behavior etc., do not rely on detected rule, do not rely on normal historical data, unknown attack can be found yet.
Accompanying drawing explanation
Fig. 1 is the principle synoptic diagram of a kind of WEB malice of the present invention scanning behavior abnormality detection system.
Fig. 2 is module composition schematic diagram in an embodiment of a kind of WEB malice of the present invention scanning behavior abnormality detection system.
Fig. 3 is the handling process schematic diagram of data preprocessing module in an embodiment of a kind of WEB malice of the present invention scanning behavior abnormality detection system.
Fig. 4 is the handling process schematic diagram of abnormality detection module in an embodiment of a kind of WEB malice scanning behavior abnormality detection system of the present invention.
Embodiment
Fig. 1 is the principle summary of a kind of WEB malice of the present invention scanning behavior abnormality detection system.WEB malice scan abnormalities detection system can carry out data prediction and anomaly analysis to the WEB access history record of input, therefrom finds that malice scans user.
Fig. 2 is module composition schematic diagram in an embodiment of a kind of WEB malice of the present invention scan abnormalities detection system.
In the present embodiment, WEB malice scanning behavior detection system forms by configuring read module, data preprocessing module, abnormality detection module and data memory module.
Data memory module is responsible for storage system configuration information, original WEB access history record, data prediction result and malice scanning user testing result, and data memory module can adopt relational database, non-relational database (as: Elasticsearch system) or the mode of text to realize.The mode of relational database is adopted to realize in the present embodiment.
Configuration read module reads configuration parameter information from data memory module, comprises database access configuration information, the database table name of WEB access history record to be analyzed, well-known search engine reptile IP list, corrected parameter w
1, w
2, w
3and mutation parameter T.
The database access configuration information that data preprocessing module provides according to configuration read module, the database table name of WEB access history record to be analyzed, read from data memory module and resolve original WEB access history record, whether be reptile according to well-known search engine reptile IP list mark user, and calculate the keyword vector of user and statistical nature vector, by pre-processed results stored in data memory module.
The database access configuration information that abnormality detection module provides according to configuration read module, pre-processed results is read from data memory module, carry out abnormality detection according to the keyword of user vector with statistical nature vector, export malice scanning user's testing result and stored in data memory module.
In the present embodiment, WEB access history to be analyzed is recorded as the WEB access log that a website WEB server records, and different user is by accessing IP to distinguish, and WEB asks character string to be the request URL recorded in WEB daily record.
Typical WEB access log record is as follows:
192.168.1.1--[15/May/2015:15:55:29+0800]"GET/abc/font/webfont.php？id＝1HTTP/1.1"20043841"http://www.test.com/abc/""Mozilla/5.0(WindowsNT6.1)AppleWebKit/537.36(KHTML,likeGecko)Chrome/35.0.1916.153Safari/537.36SE2.XMetaSr1.0"。
Wherein 192.168.1.1 is access IP, unique identification user, and GET is requesting method, and 200 is request answer code, "/abc/font/webfont.php? id=1 " be request URL.This request URL is after word segmentation processing, and the keyword obtained is " abc ", " font ", " webfont ", " php ", " id ", " 1 ".
Without loss of generality, the http protocol communication process record that WEB access history record also can record for other network intermediate equipments, Session ID during user also can be combined by the proxy information USER-AGENT in access IP and request or ask is distinguished, and WEB to be analyzed asks character string also can comprise request user agent information USER-AGENT, request BODY content.
The concrete testing process of the present embodiment is as follows:
1, system starts, and configuration read module reads configuration information;
2, data preprocessing module carries out preliminary treatment to WEB daily record, each user is marked whether as search engine reptile and calculate keyword vector and statistical nature vectorial, export user characteristics set { F
u}
u ∈ U;
3, abnormality detection module is according to user characteristics set { F
u}
u ∈ U, build overall antistop list, calculate the abnormal score of user and the threshold value of user's abnormal score, identify that malice scans user, output detections result.
Fig. 3 gives the handling process schematic diagram of data preprocessing module, and its concrete implementation is as follows:
P-1, traversal WEB daily record, obtain access IP list, form user and gather U;
P-2, the user u gathered for user in U, calculate user characteristics F
u=[IR
u, K
u, S
u], export user characteristics set { F
u}
u ∈ U, calculation procedure is as follows:
P-2-1, judge whether it is search engine reptile, if so, then marks IR according to well-known search engine reptile IP list
u=TRUE, otherwise IR
u=FALSE;
P-2-2, from WEB daily record, obtain all access logs of this access IP;
P-2-3, for each daily record extract request URL, use stop lexicon "/", ". ", "? ", "=", " & ", ", " carry out participle to request URL, add up the number of times that each keyword occurs, calculate keyword vector K
u={ (k
i, c
ki) | 0≤i≤m-1}, wherein m is K
unumber of members, k
ibe i-th keyword, c
kifor keyword k
ithe number of times occurred;
P-2-4, all access logs for this access IP, counting statistics characteristic vector S
u=[TH
u, EH
u, AH
u, PH
u], wherein TH
ufor total access times, EH
ufor the number of times of asking answer code to be more than or equal to 400 in daily record, AH
ufor request URL in daily record is the number of times of auxiliary element, PH
ufor requesting method in daily record is the number of times of PUT and DELETE.
Fig. 3 gives the handling process schematic diagram of abnormality detection module, and its idiographic flow is as follows:
D-1, reading user characteristics set { F
u}
u ∈ Uwith parameter w
1, w
2, w
3, T;
D-2, travel through the keyword vector { K of all users
u}
u ∈ U, add up the number of users that each keyword is corresponding, build overall antistop list GK;
D-3, traverse user keyword vector, calculates the uncommon degree of each keyword according to overall antistop list, calculate user's abnormal score { A
u}
u ∈ U;
D-4, ascending for user's abnormal score sequence is formed exceptional value sequence, calculate the catastrophe point of exceptional value sequence, as the threshold value A * of abnormal score, if without catastrophe point, then threshold value is-1;
If D-5 A* is greater than 0, judging whether the final abnormal score of user is greater than A*, if so, then identifying that user is for maliciously scanning user.
Wherein, the detailed process building overall antistop list is as follows:
D-2-1, initialization overall situation antistop list GK=Φ;
D-2-2, get arbitrary user u ∈ U, travel through its keyword vector K
u, for K
uin each keyword k
iif: k
ido not occur in GK, then make uc
ki=1, by (k
i, uc
ki) add in GK; If k
ioccur in GK, then from GK, obtain k
icorresponding uc
ki, make uc
ki=uc
ki+ 1, upgrade GK.
Wherein, user u abnormal score A is calculated
udetailed process as follows:
D-3-1, for user u, if F
uin IR
u=TRUE, then Au=0, otherwise enter D-3-2;
The keyword vector K of D-3-2, traverse user u
u, for K
uin each tuple (k
i, c
ki), from overall antistop list GK, search corresponding uc
ki, calculate the uncommon degree a of ki
ki=Log (c
ki) * Log (N
u/ uc
ki* uc
ki), wherein Log (x) is natural logrithm function, N
ufor user gathers the number of members of U;
The original anomaly score value A of D-3-3, calculating user u
u'=∑ a
ki;
D-3-4, calculate the final abnormal score A of user
u=A
u' * Exp (w
1, EH
u/ TH
u) * Exp (w
2, AH
u/ TH
u)+w
3* PH
u/ TH
u, wherein Exp (a, b) is exponential function, and a is the truth of a matter, and b is index.
Wherein, the detailed process calculating the threshold value A * of user's abnormal score is as follows:
D-4-1, user is gathered the final abnormal score { A of all users in U
u}
u ∈ Uby arranging from small to large, formation sequence SA={ α
i| 0≤i≤N
u-1, α
i≤ α
i+1;
D-4-2, according to mutation parameter T, sequence of calculation SB={ β
j| 0≤j≤N
u-1}, if wherein a
j+1-a
j>T, β
j=α
j+1-α
j, otherwise β
j=0;
D-4-3, according to sequence SB, sequence of calculation SC={ γ
k| 0≤k≤N
u-2}, wherein γ
k=β
k+1-β
k;
D-4-4, to sequence SC, search k*, make k* for the γ that satisfies condition
k>T or γ
kthe minimum k of <0, if k* exists, the catastrophe point of exceptional value sequence is α
k*+1, export A*=α
k*+1, otherwise there is not catastrophe point in exceptional value sequence, exports as A*=-1.
Claims (10)
1. a WEB malice scanning behavior method for detecting abnormality, the steps include:
1) preliminary treatment is carried out to the WEB access history record of calling party, the user wherein belonging to setting search engine reptile user is marked; From this WEB access history record, extract keyword feature and the statistical nature of calling party, build the keyword vector sum statistical nature vector of this user respectively;
2) travel through the keyword vector of all users, add up the number of users that each keyword is corresponding, build overall antistop list;
3) the uncommon degree of each keyword is calculated according to this overall antistop list, travel through the keyword vector of each calling party, the original anomaly score value of this calling party is calculated according to the uncommon degree of keyword, then according to the statistical nature vector of this calling party, revise original anomaly score value, obtain the final abnormal score of this calling party;
4) sequence formation one exceptional value sequence is carried out to the final abnormal score of all calling parties, calculate the catastrophe point of this exceptional value sequence, using the final abnormal score of its correspondence as threshold value; If without catastrophe point, then there is not threshold value;
5) if there is threshold value, then the final abnormal score of calling party and this threshold value are compared, if be greater than this threshold value, then this calling party be identified as malice and scan user.
2. the method for claim 1, it is characterized in that, add up total visit capacity of each calling party, the visit capacity of different requesting method, the visit capacity of different page type, the visit capacity of different answer code, build the described statistical nature vector of this calling party.
3. the method for claim 1, is characterized in that, the keyword vector K of calling party u
u={ (k
i, ck
i) | 0≤i≤m-1}; Wherein, m is the keyword quantity of calling party u, k
ibe i-th keyword, c
kifor keyword k
ithe number of times occurred in character string is asked at all WEB of this calling party u.
4. method as claimed in claim 3, is characterized in that, described overall antistop list GK={ (k
i, uc
ki) | 0≤i≤N-1,1≤uc
ki≤ N
u; Wherein, N is the sum of the different keyword of all calling parties, uc
kifor keyword k
icorresponding number of users, i.e. keyword k
iat uc
kithe WEB of individual calling party asks to occur in character string, N
ufor total number of users.
5. method as claimed in claim 4, it is characterized in that, the method calculating described uncommon degree is: keyword k
iuncommon degree P
ki=Log (c
ki) * Log (N
u/ uc
ki* uc
ki).
6. the method for claim 1, it is characterized in that, the computational methods of described original anomaly score value are: if calling party is step 1) labeled calling party, then its original anomaly score value is 0, otherwise is added up by the uncommon degree of all keywords in the keyword vector of this calling party and obtain the original anomaly score value of this calling party.
7. the method as described in claim 1 or 6, is characterized in that, the method obtaining described final abnormal score is: choose corrected parameter w
1, w
2with w
3, calculate final abnormal score=original anomaly score value * Exp (w
1, the access times of errored response synchronous codes number/total) and * Exp (w
2, the access times of auxiliary element access times/total) and+w
3* the access times of the number of times/total of PUT and DELETE method.
8. the method for claim 1, is characterized in that, the computational methods of described catastrophe point are: select a mutation parameter T, according to exceptional value sequence SA={ α
i| 0≤i≤N
u-1, α
i≤ α
i+1calculate a sequence SB={ β
j| 0≤j≤N
u-2}, meets a
j+1-a
j>T, then β
j=α
j+1-α
j, otherwise β
j=0; Then a sequence SC={ γ is calculated according to sequence SB
k| 0≤k≤N
u-3}, wherein γ
k=β
k+1-β
k; Then from this sequence SC, the γ that satisfies condition is searched
k>T or γ
kthe minimum k of <0, is designated as k*, if k* exists, the catastrophe point of exceptional value sequence SA is a
k*+1, otherwise there is not catastrophe point in exceptional value sequence; Wherein, α
ifor the final abnormal score of i-th calling party in exceptional value sequence SA, N
ufor total calling party number.
9. a WEB malice scanning behavior abnormality detection system, is characterized in that, comprises configuration read module, data preprocessing module, abnormality detection module and data memory module, wherein:
Described configuration read module is responsible for from data memory module, read configuration parameter information, comprises the search engine reptile IP list of the access configuration of WEB access history record, setting;
Described data preprocessing module is responsible for reading from data memory module and is resolved original WEB access history record, whether be reptile according to this search engine reptile IP list mark calling party, and calculate the keyword vector of each calling party and statistical nature vector, and by pre-processed results stored in data memory module;
Described abnormality detection module in charge reads pre-processed results from data memory module, carries out abnormality detection according to the keyword of calling party vector with statistical nature vector, exports malice scanning user's testing result and stored in data memory module;
Described data memory module is responsible for storage system configuration information, original WEB access history record, data prediction result and malice scanning user testing result.
10. system as claimed in claim 9, is characterized in that, the keyword vector of all users of described abnormality detection module walks, adds up the number of users that each keyword is corresponding, build overall antistop list; Then the uncommon degree of each keyword is calculated according to this overall antistop list, travel through the keyword vector of each calling party, the original anomaly score value of this calling party is calculated according to the uncommon degree of keyword, then according to the statistical nature vector of this calling party, revise original anomaly score value, obtain the final abnormal score of this calling party; Then sequence formation one exceptional value sequence is carried out to the final abnormal score of all calling parties, calculate the catastrophe point of this exceptional value sequence, using the final abnormal score of its correspondence as threshold value; If without catastrophe point, then there is not threshold value; If there is threshold value, then the final abnormal score of calling party and this threshold value are compared, if be greater than this threshold value, then this calling party be identified as malice and scan user, export malice scanning user's testing result stored in data memory module.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510404406.0A CN105072089B (en) | 2015-07-10 | 2015-07-10 | A kind of WEB malice scanning behavior method for detecting abnormality and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510404406.0A CN105072089B (en) | 2015-07-10 | 2015-07-10 | A kind of WEB malice scanning behavior method for detecting abnormality and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105072089A true CN105072089A (en) | 2015-11-18 |
| CN105072089B CN105072089B (en) | 2018-09-25 |
Family
ID=54501373
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510404406.0A Active CN105072089B (en) | 2015-07-10 | 2015-07-10 | A kind of WEB malice scanning behavior method for detecting abnormality and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105072089B (en) |
Cited By (33)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105516114A (en) * | 2015-12-01 | 2016-04-20 | 珠海市君天电子科技有限公司 | Method and device for scanning vulnerability based on webpage hash value and electronic equipment |
| CN105554007A (en) * | 2015-12-25 | 2016-05-04 | 北京奇虎科技有限公司 | web anomaly detection method and device |
| CN105915513A (en) * | 2016-04-12 | 2016-08-31 | 内蒙古大学 | Method and device for searching malicious service provider of combined service in cloud system |
| CN106210050A (en) * | 2016-07-12 | 2016-12-07 | 安徽天达网络科技有限公司 | It is a kind of that intelligence is counter shields network crawler system |
| CN106209781A (en) * | 2016-06-27 | 2016-12-07 | 徐汕 | A kind of based on the access recognition methods of statistical exceptional interface |
| CN106254368A (en) * | 2016-08-24 | 2016-12-21 | 杭州迪普科技有限公司 | The detection method of Web vulnerability scanning and device |
| CN106330944A (en) * | 2016-08-31 | 2017-01-11 | 杭州迪普科技有限公司 | Method and device for recognizing malicious system vulnerability scanner |
| CN106453357A (en) * | 2016-11-01 | 2017-02-22 | 北京红马传媒文化发展有限公司 | Network ticket buying abnormal behavior recognition method and system and equipment |
| CN106453355A (en) * | 2016-10-25 | 2017-02-22 | 东软集团股份有限公司 | Data analysis method and apparatus thereof |
| CN106919579A (en) * | 2015-12-24 | 2017-07-04 | 腾讯科技（深圳）有限公司 | A kind of information processing method and device, equipment |
| CN106933905A (en) * | 2015-12-31 | 2017-07-07 | 北京国双科技有限公司 | The monitoring method and device of web page access data |
| CN107302547A (en) * | 2017-08-21 | 2017-10-27 | 深信服科技股份有限公司 | A kind of web service exceptions detection method and device |
| CN107426196A (en) * | 2017-06-30 | 2017-12-01 | 全球能源互联网研究院 | A kind of method and system of identification WEB invasions |
| CN107426141A (en) * | 2016-05-23 | 2017-12-01 | 纬创资通股份有限公司 | Malicious code protection method, system and monitoring device |
| CN107508789A (en) * | 2017-06-29 | 2017-12-22 | 北京北信源软件股份有限公司 | A kind of recognition methods of abnormal data and device |
| CN107547490A (en) * | 2016-06-29 | 2018-01-05 | 阿里巴巴集团控股有限公司 | A kind of scanner recognition method, apparatus and system |
| CN107800690A (en) * | 2017-10-09 | 2018-03-13 | 西安交大捷普网络科技有限公司 | A kind of method for allocating tasks of Distributed Vulnerability Scanning System |
| CN107888571A (en) * | 2017-10-26 | 2018-04-06 | 江苏省互联网行业管理服务中心 | A kind of various dimensions webshell intrusion detection methods and detecting system based on HTTP daily records |
| CN108259473A (en) * | 2017-12-29 | 2018-07-06 | 西安交大捷普网络科技有限公司 | Web server scan protection method |
| CN108768954A (en) * | 2018-05-04 | 2018-11-06 | 中国科学院信息工程研究所 | A kind of DGA Malwares recognition methods |
| CN109040073A (en) * | 2018-08-07 | 2018-12-18 | 北京神州绿盟信息安全科技股份有限公司 | A kind of detection method, device, medium and the equipment of the access of WWW abnormal behaviour |
| CN109145179A (en) * | 2017-07-26 | 2019-01-04 | 北京数安鑫云信息技术有限公司 | A kind of crawler behavioral value method and device |
| CN109582844A (en) * | 2018-11-07 | 2019-04-05 | 北京三快在线科技有限公司 | A kind of method, apparatus and system identifying crawler |
| CN109818954A (en) * | 2019-01-22 | 2019-05-28 | 深信服科技股份有限公司 | Web injection type attack detection method, device, electronic equipment and storage medium |
| CN109871696A (en) * | 2018-12-29 | 2019-06-11 | 重庆城市管理职业学院 | A kind of automatic collection and vulnerability scanning system and method, computer of vulnerability information |
| CN110113228A (en) * | 2019-04-25 | 2019-08-09 | 新华三信息安全技术有限公司 | A kind of network connection detection method and device |
| CN110297854A (en) * | 2019-07-01 | 2019-10-01 | 烟台中科网络技术研究所 | A kind of APP domain name checking method and system |
| CN110912888A (en) * | 2019-11-22 | 2020-03-24 | 上海交通大学 | Malicious HTTP (hyper text transport protocol) traffic detection system and method based on deep learning |
| CN111143654A (en) * | 2019-12-25 | 2020-05-12 | 支付宝(杭州)信息技术有限公司 | Crawler identification method and device for assisting in identifying crawler, and electronic equipment |
| CN111510449A (en) * | 2020-04-10 | 2020-08-07 | 吴萌萌 | Attack behavior mining method based on image big data and big data platform server |
| CN112087414A (en) * | 2019-06-14 | 2020-12-15 | 北京奇虎科技有限公司 | Detection method and device for mining trojans |
| CN113055368A (en) * | 2021-03-08 | 2021-06-29 | 云盾智慧安全科技有限公司 | Web scanning identification method and device and computer storage medium |
| CN117478441A (en) * | 2023-12-28 | 2024-01-30 | 云南建投物流有限公司 | Dynamic access control method and system based on intelligent analysis of user behaviors |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1741458A (en) * | 2004-08-24 | 2006-03-01 | 华为技术有限公司 | Method for detecting user to make malicious IP scanning |
| CN103297435A (en) * | 2013-06-06 | 2013-09-11 | 中国科学院信息工程研究所 | Abnormal access behavior detection method and system on basis of WEB logs |
| US20140108356A1 (en) * | 2012-10-16 | 2014-04-17 | Oki Data Corporation | Information processing apparatus |
| CN104601556A (en) * | 2014-12-30 | 2015-05-06 | 中国科学院信息工程研究所 | Attack detection method and system for WEB |
-
2015
- 2015-07-10 CN CN201510404406.0A patent/CN105072089B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1741458A (en) * | 2004-08-24 | 2006-03-01 | 华为技术有限公司 | Method for detecting user to make malicious IP scanning |
| US20140108356A1 (en) * | 2012-10-16 | 2014-04-17 | Oki Data Corporation | Information processing apparatus |
| CN103297435A (en) * | 2013-06-06 | 2013-09-11 | 中国科学院信息工程研究所 | Abnormal access behavior detection method and system on basis of WEB logs |
| CN104601556A (en) * | 2014-12-30 | 2015-05-06 | 中国科学院信息工程研究所 | Attack detection method and system for WEB |
Cited By (51)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105516114A (en) * | 2015-12-01 | 2016-04-20 | 珠海市君天电子科技有限公司 | Method and device for scanning vulnerability based on webpage hash value and electronic equipment |
| CN105516114B (en) * | 2015-12-01 | 2018-12-14 | 珠海市君天电子科技有限公司 | Method and device for scanning vulnerability based on webpage hash value and electronic equipment |
| CN106919579A (en) * | 2015-12-24 | 2017-07-04 | 腾讯科技（深圳）有限公司 | A kind of information processing method and device, equipment |
| CN105554007A (en) * | 2015-12-25 | 2016-05-04 | 北京奇虎科技有限公司 | web anomaly detection method and device |
| CN105554007B (en) * | 2015-12-25 | 2019-01-04 | 北京奇虎科技有限公司 | A kind of web method for detecting abnormality and device |
| CN106933905A (en) * | 2015-12-31 | 2017-07-07 | 北京国双科技有限公司 | The monitoring method and device of web page access data |
| CN105915513B (en) * | 2016-04-12 | 2019-01-04 | 内蒙古大学 | The lookup method and device of the malicious service supplier of composite services in cloud system |
| CN105915513A (en) * | 2016-04-12 | 2016-08-31 | 内蒙古大学 | Method and device for searching malicious service provider of combined service in cloud system |
| CN107426141B (en) * | 2016-05-23 | 2020-06-09 | 纬创资通股份有限公司 | Malicious code protection method, system and monitoring device |
| CN107426141A (en) * | 2016-05-23 | 2017-12-01 | 纬创资通股份有限公司 | Malicious code protection method, system and monitoring device |
| CN106209781B (en) * | 2016-06-27 | 2019-09-06 | 航天云网科技发展有限责任公司 | One kind accessing recognition methods based on statistical exceptional interface |
| CN106209781A (en) * | 2016-06-27 | 2016-12-07 | 徐汕 | A kind of based on the access recognition methods of statistical exceptional interface |
| CN107547490A (en) * | 2016-06-29 | 2018-01-05 | 阿里巴巴集团控股有限公司 | A kind of scanner recognition method, apparatus and system |
| CN107547490B (en) * | 2016-06-29 | 2020-12-04 | 阿里巴巴集团控股有限公司 | Scanner identification method, device and system |
| CN106210050A (en) * | 2016-07-12 | 2016-12-07 | 安徽天达网络科技有限公司 | It is a kind of that intelligence is counter shields network crawler system |
| CN106254368B (en) * | 2016-08-24 | 2019-09-06 | 杭州迪普科技股份有限公司 | The detection method and device of Web vulnerability scanning |
| CN106254368A (en) * | 2016-08-24 | 2016-12-21 | 杭州迪普科技有限公司 | The detection method of Web vulnerability scanning and device |
| CN106330944A (en) * | 2016-08-31 | 2017-01-11 | 杭州迪普科技有限公司 | Method and device for recognizing malicious system vulnerability scanner |
| CN106453355A (en) * | 2016-10-25 | 2017-02-22 | 东软集团股份有限公司 | Data analysis method and apparatus thereof |
| CN106453357A (en) * | 2016-11-01 | 2017-02-22 | 北京红马传媒文化发展有限公司 | Network ticket buying abnormal behavior recognition method and system and equipment |
| CN107508789B (en) * | 2017-06-29 | 2020-04-07 | 北京北信源软件股份有限公司 | Abnormal data identification method and device |
| CN107508789A (en) * | 2017-06-29 | 2017-12-22 | 北京北信源软件股份有限公司 | A kind of recognition methods of abnormal data and device |
| CN107426196A (en) * | 2017-06-30 | 2017-12-01 | 全球能源互联网研究院 | A kind of method and system of identification WEB invasions |
| CN107426196B (en) * | 2017-06-30 | 2022-06-21 | 全球能源互联网研究院 | Method and system for identifying WEB invasion |
| CN109145179A (en) * | 2017-07-26 | 2019-01-04 | 北京数安鑫云信息技术有限公司 | A kind of crawler behavioral value method and device |
| CN109145179B (en) * | 2017-07-26 | 2019-04-19 | 北京数安鑫云信息技术有限公司 | A kind of crawler behavioral value method and device |
| CN107302547A (en) * | 2017-08-21 | 2017-10-27 | 深信服科技股份有限公司 | A kind of web service exceptions detection method and device |
| CN107800690A (en) * | 2017-10-09 | 2018-03-13 | 西安交大捷普网络科技有限公司 | A kind of method for allocating tasks of Distributed Vulnerability Scanning System |
| CN107800690B (en) * | 2017-10-09 | 2021-07-06 | 西安交大捷普网络科技有限公司 | Task allocation method of distributed vulnerability scanning system |
| CN107888571A (en) * | 2017-10-26 | 2018-04-06 | 江苏省互联网行业管理服务中心 | A kind of various dimensions webshell intrusion detection methods and detecting system based on HTTP daily records |
| CN108259473A (en) * | 2017-12-29 | 2018-07-06 | 西安交大捷普网络科技有限公司 | Web server scan protection method |
| CN108768954B (en) * | 2018-05-04 | 2020-07-10 | 中国科学院信息工程研究所 | DGA malicious software identification method |
| CN108768954A (en) * | 2018-05-04 | 2018-11-06 | 中国科学院信息工程研究所 | A kind of DGA Malwares recognition methods |
| CN109040073A (en) * | 2018-08-07 | 2018-12-18 | 北京神州绿盟信息安全科技股份有限公司 | A kind of detection method, device, medium and the equipment of the access of WWW abnormal behaviour |
| CN109582844A (en) * | 2018-11-07 | 2019-04-05 | 北京三快在线科技有限公司 | A kind of method, apparatus and system identifying crawler |
| CN109871696A (en) * | 2018-12-29 | 2019-06-11 | 重庆城市管理职业学院 | A kind of automatic collection and vulnerability scanning system and method, computer of vulnerability information |
| CN109818954B (en) * | 2019-01-22 | 2021-08-13 | 深信服科技股份有限公司 | Web injection type attack detection method and device, electronic equipment and storage medium |
| CN109818954A (en) * | 2019-01-22 | 2019-05-28 | 深信服科技股份有限公司 | Web injection type attack detection method, device, electronic equipment and storage medium |
| CN110113228A (en) * | 2019-04-25 | 2019-08-09 | 新华三信息安全技术有限公司 | A kind of network connection detection method and device |
| CN110113228B (en) * | 2019-04-25 | 2020-12-18 | 新华三信息安全技术有限公司 | Network connection detection method and device |
| CN112087414A (en) * | 2019-06-14 | 2020-12-15 | 北京奇虎科技有限公司 | Detection method and device for mining trojans |
| CN110297854B (en) * | 2019-07-01 | 2020-07-07 | 烟台中科网络技术研究所 | APP domain name verification method and system |
| CN110297854A (en) * | 2019-07-01 | 2019-10-01 | 烟台中科网络技术研究所 | A kind of APP domain name checking method and system |
| CN110912888B (en) * | 2019-11-22 | 2021-08-10 | 上海交通大学 | Malicious HTTP (hyper text transport protocol) traffic detection system and method based on deep learning |
| CN110912888A (en) * | 2019-11-22 | 2020-03-24 | 上海交通大学 | Malicious HTTP (hyper text transport protocol) traffic detection system and method based on deep learning |
| CN111143654A (en) * | 2019-12-25 | 2020-05-12 | 支付宝(杭州)信息技术有限公司 | Crawler identification method and device for assisting in identifying crawler, and electronic equipment |
| CN111143654B (en) * | 2019-12-25 | 2023-06-16 | 支付宝(杭州)信息技术有限公司 | Crawler identification method and device for assisting in identifying crawler and electronic equipment |
| CN111510449A (en) * | 2020-04-10 | 2020-08-07 | 吴萌萌 | Attack behavior mining method based on image big data and big data platform server |
| CN113055368A (en) * | 2021-03-08 | 2021-06-29 | 云盾智慧安全科技有限公司 | Web scanning identification method and device and computer storage medium |
| CN117478441A (en) * | 2023-12-28 | 2024-01-30 | 云南建投物流有限公司 | Dynamic access control method and system based on intelligent analysis of user behaviors |
| CN117478441B (en) * | 2023-12-28 | 2024-03-12 | 云南建投物流有限公司 | Dynamic access control method and system based on intelligent analysis of user behaviors |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105072089B (en) | 2018-09-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105072089A (en) | WEB malicious scanning behavior abnormity detection method and system | |
| CN107241352B (en) | Network security event classification and prediction method and system | |
| CN107251037B (en) | Blacklist generation device, blacklist generation system, blacklist generation method, and recording medium | |
| CN103297435B (en) | A kind of abnormal access behavioral value method and system based on WEB daily record | |
| CN101370008B (en) | System for real-time intrusion detection of SQL injection WEB attacks | |
| CN103559235B (en) | A kind of online social networks malicious web pages detection recognition methods | |
| CN107579956B (en) | User behavior detection method and device | |
| CN104767757A (en) | Multiple-dimension security monitoring method and system based on WEB services | |
| CN103279710B (en) | Method and system for detecting malicious codes of Internet information system | |
| CN104994091B (en) | Detection method and device, the method and apparatus of defence Web attacks of abnormal flow | |
| CN107992738B (en) | Account login abnormity detection method and device and electronic equipment | |
| CN107437026B (en) | Malicious webpage advertisement detection method based on advertisement network topology | |
| CN103428196A (en) | URL white list-based WEB application intrusion detecting method and apparatus | |
| CN103929440A (en) | Web page tamper prevention device based on web server cache matching and method thereof | |
| US10505986B1 (en) | Sensor based rules for responding to malicious activity | |
| CN114915479B (en) | Web attack stage analysis method and system based on Web log | |
| CN109768992A (en) | Webpage malicious scanning processing method and device, terminal device, readable storage medium storing program for executing | |
| CN110830490A (en) | Malicious domain name detection method and system based on area confrontation training deep network | |
| CN111835777A (en) | Abnormal flow detection method, device, equipment and medium | |
| CN109428857A (en) | A kind of detection method and device of malice detection behavior | |
| CN109756467B (en) | Phishing website identification method and device | |
| CN113076961A (en) | Image feature library updating method, image detection method and device | |
| CN117235600A (en) | User abnormal behavior detection method and system | |
| CN110460620B (en) | Website defense method, device, equipment and storage medium | |
| CN116155519A (en) | Threat alert information processing method, threat alert information processing device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |