CN106027564A - Method and device for detecting security of anti-crawler strategy - Google Patents
Method and device for detecting security of anti-crawler strategy Download PDFInfo
- Publication number
- CN106027564A CN106027564A CN201610537443.3A CN201610537443A CN106027564A CN 106027564 A CN106027564 A CN 106027564A CN 201610537443 A CN201610537443 A CN 201610537443A CN 106027564 A CN106027564 A CN 106027564A
- Authority
- CN
- China
- Prior art keywords
- reptile
- data
- page
- end page
- destination object
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/20—Network architectures or network communication protocols for network security for managing network security; network security policies in general
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
Abstract
The invention discloses a method and a device for detecting the security of an anti-crawler strategy. The method comprises the following steps: embedding an anti-crawler code for implementing the anti-crawler strategy into a first front-end page of a website; detecting whether a user accessing the first front-end page is a crawler or not by using the anti-crawler code, and marking the user detected as the crawler as a target object; verifying whether the target object is the crawler or not, and performing statistics to obtain times that the target object is not the crawler; and calculating the accidental damage rate of the anti-crawler strategy according to the times, wherein the accidental damage rate is used for measuring the security of the anti-crawler strategy. Through adoption of the method and the device, the defect that system damage is caused by improper detection of the security of the anti-crawler strategy in the prior art is overcome; the security of the anti-crawler strategy can be detected accurately; the anti-crawler strategy can be modified or updated conveniently and immediately; the influence of the security of the anti-crawler strategy on the stability of an online system is avoided; and the stability of the system is ensured while the crawler is detected.
Description
Technical field
The present invention relates to a kind of method and device detecting anti-reptile security policy.
Background technology
At present the reptile amount on the Internet increases day by day, and reptile is the most very strange, and time be engraved in evolution,
Anti-reptile mechanism, the most increasingly by stern challenge, needs to issue new anti-reptile strategy continually and solves
The newest reptile.But, the when of carrying out issuing new edition anti-reptile strategy every time, all can be to inline system
Stability impact, if anti-reptile strategy does not has enough safeties, can be while anti-reptile
Inline system is damaged, loses more than gain.
Summary of the invention
The technical problem to be solved in the present invention is to overcome the prior art safety to anti-reptile strategy
Detect the improper defect causing system injury, it is provided that one can detect anti-reptile security policy exactly
Method and device.
The present invention is to solve above-mentioned technical problem by the following technical programs:
The present invention provides a kind of method detecting anti-reptile security policy, is characterized in, described method bag
Include:
S1, embed in the first front end page of website for realizing the anti-reptile code of anti-reptile strategy；
S2, utilize described anti-reptile code detection access described first front end page user whether be reptile,
It is that the user of reptile is designated as destination object by being detected；
S3, verify whether described destination object is reptile, add up the number of times of the non-reptile of described destination object；
S4, according to described number of times calculate described anti-reptile strategy accidental injury rate, described accidental injury rate is used for weighing
The safety of described anti-reptile strategy.
Wherein, anti-reptile strategy is realized by anti-reptile code；The described destination object counted is non-
The number of times of reptile i.e. represents that described anti-reptile code detects into the user of non-reptile the number of times of reptile by mistake, i.e.
The number of times of described anti-reptile code detection mistake.The technical program is by calculating the mistake of described anti-reptile strategy
Rate of hindering is to weigh the safety of anti-reptile strategy, if safety is higher, and can be by anti-reptile policy deployment
In inline system, if safety is relatively low, can be with time update or update anti-reptile strategy, it is to avoid by
The stability of inline system is impacted by the safety in anti-reptile strategy, protects while detection reptile
The stability of card system.
It is preferred that S3Verify whether described destination object is reptile by following steps:
S31, judge whether described destination object accesses the second front end page of described website, the most then institute
State the non-reptile of destination object, if it is not, the most described destination object is reptile.
Wherein, the second front end page and described first front end page have relatedness, have accessed for reptile
The page that generally will not access after first front end page, if destination object accesses the second front end page, then
Represent the non-reptile of described destination object (not being reptile), described anti-reptile code detection mistake, if target
Object does not access the second front end page, then it represents that described destination object is reptile, described anti-reptile code
Detection is correct.The technical program can verify whether described destination object is reptile efficiently and accurately, enters
One step demonstrate,proves the safety of described anti-reptile code.
It is preferred that described anti-reptile code includes that fore-end, described fore-end include for detecting
The code of reptile, S1Including:
S11, in the first front end page of website embed first page；
S12, the code being used for detecting reptile is configured in described first page.
The code that the technical program is configured to detect reptile by described first page embeds the first front end
The page, even if code or first page are wrong, does not interferes with the display of described first front end page yet,
It is easy to the change of code.
It is preferred that described anti-reptile code also includes back partition, described back partition is in client
Write the first data and intercept described in described first data statistics intercept in the second front end page
The sum of the first data；
S1Also include: utilize described back partition to write the first data in described client；
S3Including: utilize described back partition to block in the second front end page described first data of interception statistics
The sum of described first data being truncated to；
Described accidental injury rate is equal to total the number of visiting people of described sum/described second front end page.
The technical program is by judging whether described second front end page is intercepted the first data and judged institute
State whether destination object accesses described second front end page, if described second front end page intercepts the first number
According to, then show that described destination object have accessed described second front end page, the non-reptile of described destination object,
If described second front end page does not intercepts the first data, then show that described destination object does not accesses described
Two front end page, described destination object is reptile.The sum of described first data intercepted is equal to described
The number of times of the non-reptile of destination object.Accidental injury rate in the technical program is the highest, shows described anti-reptile strategy
Safety the lowest, accidental injury rate is the lowest, shows that the safety of described anti-reptile strategy is the highest.
It is preferred that S11Also include: be set in described first front end page and embed the general of described first page
Rate；
Described accidental injury rate is equal to total the number of visiting people of described sum/described probability/described second front end page.
It is preferred that S1Also including: set the effective time of described first data, described first data are super
Lost efficacy when crossing described effective time.
It is preferred that corresponding the first different data of different anti-reptile strategies.
The technical program is particularly suited for detecting the situation of multiple anti-reptile strategy simultaneously, counter climbs for different
Worm strategy (anti-reptile code) sets the first different data, adds up respectively in described second front end page
The sum of the first different data intercepted, judges each anti-reptile strategy by distinguishing the first data
Safety.
It is preferred that described method also includes:
S5, whether relatively described accidental injury rate less than threshold value, the most described anti-reptile security policy；If
No, the most described anti-reptile strategy is dangerous.
The present invention also provides for a kind of device detecting anti-reptile security policy, is characterized in, described device
Including:
Embedded unit, for embedding for realizing the anti-of anti-reptile strategy in the first front end page of website
Reptile code；
Detector unit, for utilizing described anti-reptile code detection to access the user of described first front end page
Whether it is reptile, is that the user of reptile is designated as destination object by being detected；
Authentication unit, is used for verifying whether described destination object is reptile, adds up that described destination object is non-climbs
The number of times of worm；
Computing unit, for calculating the accidental injury rate of described anti-reptile strategy, described accidental injury according to described number of times
Rate is for weighing the safety of described anti-reptile strategy.
It is preferred that whether described authentication unit is reptile by destination object described in following module verification:
Judge module, for judging whether described destination object accesses the second front end page of described website,
The non-reptile of the most described destination object, if it is not, the most described destination object is reptile.
It is preferred that described anti-reptile code includes that fore-end, described fore-end include for detecting
The code of reptile, described embedded unit includes:
Page module, for embedding first page in the first front end page of website；
Configuration module, for being configured to the code being used for detecting reptile in described first page.
It is preferred that described anti-reptile code also includes back partition, described back partition is in client
Write the first data and intercept described in described first data statistics intercept in the second front end page
The sum of the first data；
Described embedded unit also includes:
Data module, is used for utilizing described back partition to write the first data in client；
Described authentication unit includes:
Blocking module, is used for utilizing described back partition to intercept described first data also in the second front end page
The sum of described first data that statistics is intercepted；
Described accidental injury rate is equal to total the number of visiting people of described sum/described second front end page.
It is preferred that described page module is additionally operable to be set in described first front end page embeds described first
The probability of the page；
Described accidental injury rate is equal to total the number of visiting people of described sum/described probability/described second front end page.
It is preferred that described data module is additionally operable to set the effective time of described first data, described first
Data lost efficacy when exceeding described effective time.
It is preferred that corresponding the first different data of different anti-reptile strategies.
It is preferred that described device also includes:
Whether comparing unit, for relatively described accidental injury rate less than threshold value, the most described anti-reptile plan
The safest；If it is not, the most described anti-reptile strategy is dangerous.
On the basis of meeting common sense in the field, above-mentioned each optimum condition, can combination in any, i.e. get Ben Fa
Bright each preferred embodiments.
The most progressive effect of the present invention is: the present invention is by calculating the accidental injury rate of described anti-reptile strategy
Weigh the safety of anti-reptile strategy, the accurate detection to anti-reptile security policy, it is simple to the most right
Anti-reptile strategy is modified or updates, it is to avoid due to steady to inline system of the safety of anti-reptile strategy
Qualitative impact, while detection reptile, ensure the stability of system.
Accompanying drawing explanation
Fig. 1 is the flow chart of the method detecting anti-reptile security policy of present pre-ferred embodiments 1.
Fig. 2 is the schematic diagram of the device detecting anti-reptile security policy of present pre-ferred embodiments 2.
Detailed description of the invention
Further illustrate the present invention below by the mode of embodiment, but the most therefore limit the present invention to
Among described scope of embodiments.
Embodiment 1
A kind of method detecting anti-reptile security policy, as it is shown in figure 1, described method includes:
Step 101, write the anti-reptile code for realizing anti-reptile strategy.Wherein, anti-reptile code
Including fore-end and back partition:
Described fore-end includes the code for detecting reptile；
Back partition is for when client writes the first data and sets described first data effective
Between, described first data lost efficacy when exceeding described effective time, the first data described in when implementing
Cookie can be used (the most also with its plural form cookies, to be stored in user local terminal (i.e. visitor
Family end) on data) realize, the value of cookie and effective time can be self-defined；
Described back partition is additionally operable to intercept in the second front end page described first data of interception statistics
The sum of described first data.
Step 102, in the first front end page of website, embed first page and be set in described first
Front end page embeds the probability of described first page.Wherein, described first front end page is typically
Hotel's details page of the front end page that random access amount is bigger, such as online tourism website；Described first
The page can be by can dynamically configure the iframe of javascript (a kind of literal translation formula script) code
(html label) page realizes, owing to the iframe page is independent sandbox, even if report an error will not be to father for js
The page (the most described first front end page) impacts；Described probability refers in described first front end page
The probability of the described first page of middle embedding, for controlling the occurrence number of described first page, such as, has 100
Individual first front end page, if setting probability as 10%, then just from these 100 first front end page
Select 10 and embed first page, if setting probability as 1, then just end page before these 100 first
Face all embeds first page.
Step 103, issue described anti-reptile code, to realize being configured to the code being used for detecting reptile
The first data are write in described first page and in described client.Described code is utilized automatically to detect
Whether the user going out to access described first front end page is reptile；Utilize described back partition real presently described
Client writes the first data.
The present embodiment is that the user of reptile is designated as destination object, by judging described target by being detected
Object whether access the second front end page of described website with verify described destination object whether as reptile, institute
State the second front end page, with described first front end page, there is relatedness, have accessed the first front end for reptile
The order page of the page that generally will not access after the page, such as online tourism website.If destination object
Access the second front end page, then it represents that the non-reptile of described destination object (not being reptile), described anti-reptile generation
Code detection mistake, described destination object is accidentally injured, if destination object does not access the second front end page,
Then representing that described destination object is reptile, described anti-reptile code detection is correct, and described destination object does not has
Accidentally injured.
Step 104, described first intercepted in the second front end page described first data of interception statistics
The sum of data.Step 104 utilizes described back partition to realize, and wherein intercepts the work of the first data
With being by judging whether described second front end page is intercepted the first data and judged described destination object
Whether access described second front end page, and then verify whether described destination object is reptile.If described
Two front end page intercept the first data, then show that described destination object have accessed described second front end page,
The non-reptile of described destination object, if described second front end page does not intercepts the first data, then shows described
Destination object does not accesses described second front end page, and described destination object is reptile.
Step 105, calculate the accidental injury rate of described anti-reptile strategy, described accidental injury rate be used for weighing described instead
The safety of reptile strategy.
If accidental injury rate is Q, the probability embedding described first page in described first front end page is P,
The sum of described first data intercepted is C, and total the number of visiting people of the second front end page is O:
Q=C/P/O.
As P=1, Q=C/O.
Whether step 106, relatively described accidental injury rate be less than threshold value, and the most described anti-reptile strategy is pacified
Entirely；If it is not, the most described anti-reptile strategy is dangerous.Wherein, described threshold value can be self-defined.If institute
It is safe for stating anti-reptile strategy, then can formally be deployed to produce.
When needs detect multiple anti-reptile strategy simultaneously, different anti-reptile strategy correspondences can be set not
The first same data.The total of the first different data of intercepting is added up respectively in described second front end page
Number, judges the safety of each anti-reptile strategy by distinguishing the first data.
Embodiment 2
The device detecting anti-reptile security policy of the present embodiment, as in figure 2 it is shown, described device includes:
Embedded unit 201, detector unit 202, authentication unit 203, computing unit 204 and comparing unit 205.
Described anti-reptile strategy includes that anti-reptile code, anti-reptile code include fore-end and back partition:
Described fore-end includes the code for detecting reptile；
Back partition is for when client writes the first data and sets described first data effective
Between, described first data lost efficacy when exceeding described effective time, the first data described in when implementing
Cookie can be used (the most also with its plural form cookies, to be stored in user local terminal (i.e. visitor
Family end) on data) realize, the value of cookie and effective time can be self-defined；
Described back partition is additionally operable to intercept in the second front end page described first data of interception statistics
The sum of described first data.
Embedded unit 201, is used for realizing anti-reptile strategy for embedding in the first front end page of website
Anti-reptile code.
Specifically, described embedded unit 201 includes:
Page module 2011, for embedding first page and being set in the first front end page of website
Described first front end page embeds the probability of described first page.Wherein, described first front end page is led to
It can be often hotel's details page of the front end page that random access amount is bigger, such as online tourism website；
Described first page can be by can dynamically configure javascript (a kind of literal translation formula script) code
Iframe (html label) page realizes, owing to the iframe page is independent sandbox, even if js reports an error not
Parent page (the most described first front end page) can be impacted；Described probability refers to before described first
End page face embeds the probability of described first page, for controlling the occurrence number of described first page, example
If any 100 the first front end page, if setting probability as 10%, then just before these 100 first
End page face is selected 10 and embeds first pages, if setting probability as 1, then just at these 100 the
One front end page all embeds first page.
Configuration module 2012, configures the code being used for detecting reptile for the anti-reptile code by issuing
In described first page.
Data module 2013, will write the first data in client for the anti-reptile code by issuing.
The real presently described client of described back partition is utilized to write the first data.
Detector unit 202, for utilizing described code detection to go out the user accessing described first front end page
Whether it is reptile, is that the user of reptile is designated as destination object by being detected.
Authentication unit 203, is used for verifying whether described destination object is reptile, adds up described destination object
The number of times of non-reptile.Specifically, described authentication unit 203 is by destination object described in following module verification
Whether it is reptile:
Judge module 2031, for judge described destination object whether access described website second before end page
Face, the non-reptile of the most described destination object, if it is not, the most described destination object is reptile.Described
Two front end page and described first front end page have relatedness, have accessed the first front end page for reptile
The order page of the page that the most generally will not access, such as online tourism website.If destination object accesses
Second front end page, then it represents that the non-reptile of described destination object (not being reptile), described anti-reptile code is examined
Sniffing by mistake, accidentally injured, if destination object does not access the second front end page, then table by described destination object
Showing that described destination object is reptile, described anti-reptile code detection is correct, and described destination object is not missed
Wound.
Described authentication unit 203 includes:
Blocking module 2032, is used for utilizing described back partition to intercept described first number in the second front end page
According to and add up the sum of described first data intercepted.The effect wherein intercepting the first data is by sentencing
Whether disconnected described second front end page intercepts the first data to judge whether described destination object accesses institute
State the second front end page, and then verify whether described destination object is reptile.If described second front end page
Intercept the first data, then show that described destination object have accessed described second front end page, described target
The non-reptile of object, if described second front end page does not intercepts the first data, then shows described destination object
Not accessing described second front end page, described destination object is reptile.
Computing unit 204, for calculating the accidental injury rate of described anti-reptile strategy, described accidental injury rate is used for weighing
Measure the safety of described anti-reptile strategy.
If accidental injury rate is Q, the probability embedding described first page in described first front end page is P,
The sum of described first data intercepted is C, and total the number of visiting people of the second front end page is O:
Q=C/P/O.
As P=1, Q=C/O.
Whether comparing unit 205, for relatively described accidental injury rate less than threshold value, the most described counter climb
Worm security policy；If it is not, the most described anti-reptile strategy is dangerous.Wherein, described threshold value can be self-defined.
If described anti-reptile strategy is safe, then can formally be deployed to produce.
When needs detect multiple anti-reptile strategy simultaneously, different anti-reptile strategy correspondences can be set not
The first same data.The total of the first different data of intercepting is added up respectively in described second front end page
Number, judges the safety of each anti-reptile strategy by distinguishing the first data.
Although the foregoing describing the detailed description of the invention of the present invention, but those skilled in the art should managing
Solving, these are merely illustrative of, and protection scope of the present invention is defined by the appended claims.This
The technical staff in field, can be to these embodiment party on the premise of without departing substantially from the principle of the present invention and essence
Formula makes various changes or modifications, but these changes and amendment each fall within protection scope of the present invention.
Claims (16)
1. the method detecting anti-reptile security policy, it is characterised in that described method includes:
S1, embed in the first front end page of website for realizing the anti-reptile code of anti-reptile strategy；
S2, utilize described anti-reptile code detection access described first front end page user whether be reptile,
It is that the user of reptile is designated as destination object by being detected；
S3, verify whether described destination object is reptile, add up the number of times of the non-reptile of described destination object；
S4, according to described number of times calculate described anti-reptile strategy accidental injury rate, described accidental injury rate is used for weighing
The safety of described anti-reptile strategy.
2. the method detecting anti-reptile security policy as claimed in claim 1, it is characterised in that
S3Verify whether described destination object is reptile by following steps:
S31, judge whether described destination object accesses the second front end page of described website, the most then institute
State the non-reptile of destination object, if it is not, the most described destination object is reptile.
3. the method detecting anti-reptile security policy as claimed in claim 2, it is characterised in that
Described anti-reptile code includes that fore-end, described fore-end include the code for detecting reptile,
S1Including:
S11, in the first front end page of website embed first page；
S12, the code being used for detecting reptile is configured in described first page.
4. the method detecting anti-reptile security policy as claimed in claim 3, it is characterised in that
Described anti-reptile code also includes back partition, and described back partition is for writing the first data in client
And described first data total that the second front end page intercepts described first data and statistics is intercepted
Number；
S1Also include: utilize described back partition to write the first data in described client；
S3Including: utilize described back partition to block in the second front end page described first data of interception statistics
The sum of described first data being truncated to；
Described accidental injury rate is equal to total the number of visiting people of described sum/described second front end page.
5. the method detecting anti-reptile security policy as claimed in claim 4, it is characterised in that
S11Also include: be set in the probability embedding described first page in described first front end page；
Described accidental injury rate is equal to total the number of visiting people of described sum/described probability/described second front end page.
6. the method detecting anti-reptile security policy as claimed in claim 4, it is characterised in that
Described back partition is additionally operable to set the effective time of described first data, and described first data are exceeding
Lost efficacy when stating effective time.
7. the method detecting anti-reptile security policy as claimed in claim 4, it is characterised in that
Corresponding the first different data of different anti-reptile strategies.
8. the method detecting anti-reptile security policy as claimed in claim 1, it is characterised in that
Described method also includes:
S5, whether relatively described accidental injury rate less than threshold value, the most described anti-reptile security policy；If
No, the most described anti-reptile strategy is dangerous.
9. the device detecting anti-reptile security policy, it is characterised in that described device includes:
Embedded unit, for embedding for realizing the anti-of anti-reptile strategy in the first front end page of website
Reptile code；
Detector unit, for utilizing described anti-reptile code detection to access the user of described first front end page
Whether it is reptile, is that the user of reptile is designated as destination object by being detected；
Authentication unit, is used for verifying whether described destination object is reptile, adds up that described destination object is non-climbs
The number of times of worm；
Computing unit, for calculating the accidental injury rate of described anti-reptile strategy, described accidental injury according to described number of times
Rate is for weighing the safety of described anti-reptile strategy.
10. the device detecting anti-reptile security policy as claimed in claim 9, it is characterised in that
Whether described authentication unit is reptile by destination object described in following module verification:
Judge module, for judging whether described destination object accesses the second front end page of described website,
The non-reptile of the most described destination object, if it is not, the most described destination object is reptile.
11. devices detecting anti-reptile security policy as claimed in claim 10, it is characterised in that
Described anti-reptile code includes that fore-end, described fore-end include the code for detecting reptile,
Described embedded unit includes:
Page module, for embedding first page in the first front end page of website；
Configuration module, for being configured to the code being used for detecting reptile in described first page.
12. devices detecting anti-reptile security policy as claimed in claim 10, it is characterised in that
Described anti-reptile code also includes back partition, and described back partition is for writing the first data in client
And described first data total that the second front end page intercepts described first data and statistics is intercepted
Number；
Described embedded unit also includes:
Data module, is used for utilizing described back partition to write the first data in client；
Described authentication unit includes:
Blocking module, is used for utilizing described back partition to intercept described first data also in the second front end page
The sum of described first data that statistics is intercepted；
Described accidental injury rate is equal to total the number of visiting people of described sum/described second front end page.
13. devices detecting anti-reptile security policy as claimed in claim 12, it is characterised in that
Described page module is additionally operable to be set in the probability embedding described first page in described first front end page；
Described accidental injury rate is equal to total the number of visiting people of described sum/described probability/described second front end page.
14. devices detecting anti-reptile security policy as claimed in claim 12, it is characterised in that
Described back partition is additionally operable to set the effective time of described first data, and described first data are exceeding
Lost efficacy when stating effective time.
15. devices detecting anti-reptile security policy as claimed in claim 12, it is characterised in that
Corresponding the first different data of different anti-reptile strategies.
16. devices detecting anti-reptile security policy as claimed in claim 9, it is characterised in that
Described device also includes:
Whether comparing unit, for relatively described accidental injury rate less than threshold value, the most described anti-reptile plan
The safest；If it is not, the most described anti-reptile strategy is dangerous.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610537443.3A CN106027564B (en) | 2016-07-08 | 2016-07-08 | Detect the method and device of anti-crawler security policy |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610537443.3A CN106027564B (en) | 2016-07-08 | 2016-07-08 | Detect the method and device of anti-crawler security policy |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106027564A true CN106027564A (en) | 2016-10-12 |
| CN106027564B CN106027564B (en) | 2019-05-21 |
Family
ID=57108853
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610537443.3A Active CN106027564B (en) | 2016-07-08 | 2016-07-08 | Detect the method and device of anti-crawler security policy |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106027564B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107147640A (en) * | 2017-05-09 | 2017-09-08 | 网宿科技股份有限公司 | Recognize the method and system of web crawlers |
| CN107943949A (en) * | 2017-11-24 | 2018-04-20 | 厦门集微科技有限公司 | A kind of method and server of definite web crawlers |
| CN109543454A (en) * | 2019-01-25 | 2019-03-29 | 腾讯科技（深圳）有限公司 | A kind of anti-crawler method and relevant device |
| CN111181933A (en) * | 2019-12-19 | 2020-05-19 | 贝壳技术有限公司 | Web crawler detection method and device, storage medium and electronic equipment |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101582887A (en) * | 2009-05-20 | 2009-11-18 | 成都市华为赛门铁克科技有限公司 | Safety protection method, gateway device and safety protection system |
| CN102495861A (en) * | 2011-11-24 | 2012-06-13 | 中国科学院计算技术研究所 | System and method for identifying web crawler |
| CN102790700A (en) * | 2011-05-19 | 2012-11-21 | 北京启明星辰信息技术股份有限公司 | Method and device for recognizing webpage crawler |
| US20130117846A1 (en) * | 2008-10-21 | 2013-05-09 | Lookout, Inc., A California Corporation | System and method for server-coupled application re-analysis to obtain characterization assessment |
| US20160170865A1 (en) * | 2013-09-30 | 2016-06-16 | International Business Machines Corporation | Detecting error states when interacting with web applications |
| CN105743901A (en) * | 2016-03-07 | 2016-07-06 | 携程计算机技术（上海）有限公司 | Server, anti-crawler system and anti-crawler verification method |
-
2016
- 2016-07-08 CN CN201610537443.3A patent/CN106027564B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130117846A1 (en) * | 2008-10-21 | 2013-05-09 | Lookout, Inc., A California Corporation | System and method for server-coupled application re-analysis to obtain characterization assessment |
| CN101582887A (en) * | 2009-05-20 | 2009-11-18 | 成都市华为赛门铁克科技有限公司 | Safety protection method, gateway device and safety protection system |
| CN102790700A (en) * | 2011-05-19 | 2012-11-21 | 北京启明星辰信息技术股份有限公司 | Method and device for recognizing webpage crawler |
| CN102495861A (en) * | 2011-11-24 | 2012-06-13 | 中国科学院计算技术研究所 | System and method for identifying web crawler |
| US20160170865A1 (en) * | 2013-09-30 | 2016-06-16 | International Business Machines Corporation | Detecting error states when interacting with web applications |
| CN105743901A (en) * | 2016-03-07 | 2016-07-06 | 携程计算机技术（上海）有限公司 | Server, anti-crawler system and anti-crawler verification method |
Non-Patent Citations (3)
| Title |
|---|
| 36氪的朋友们: "那些你不知道的爬虫反爬虫套路", 《HTTP://36KR.COM/P/5079327.HTML》 * |
| CTRIPTECH: "关于爬虫，看着一篇就够了", 《HTTPS://SEGMENTFAULT.COM/A/1190000005840672》 * |
| 吴晓晖: "Web爬虫检测技术综述", 《湖北汽车工业学院学报》 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107147640A (en) * | 2017-05-09 | 2017-09-08 | 网宿科技股份有限公司 | Recognize the method and system of web crawlers |
| CN107147640B (en) * | 2017-05-09 | 2019-12-31 | 网宿科技股份有限公司 | Method and system for identifying web crawler |
| CN107943949A (en) * | 2017-11-24 | 2018-04-20 | 厦门集微科技有限公司 | A kind of method and server of definite web crawlers |
| CN107943949B (en) * | 2017-11-24 | 2020-06-26 | 厦门集微科技有限公司 | Method and server for determining web crawler |
| CN109543454A (en) * | 2019-01-25 | 2019-03-29 | 腾讯科技（深圳）有限公司 | A kind of anti-crawler method and relevant device |
| CN111181933A (en) * | 2019-12-19 | 2020-05-19 | 贝壳技术有限公司 | Web crawler detection method and device, storage medium and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106027564B (en) | 2019-05-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104301302B (en) | Go beyond one's commission attack detection method and device | |
| KR102138965B1 (en) | Account theft risk identification method, identification device, prevention and control system | |
| CN106027564A (en) | Method and device for detecting security of anti-crawler strategy | |
| US10057280B2 (en) | Methods and systems of detecting and analyzing correlated operations in a common storage | |
| US9178897B2 (en) | Methods and systems for use in identifying cyber-security threats in an aviation platform | |
| CN101356535B (en) | A method and apparatus for detecting and preventing unsafe behavior of javascript programs | |
| CN109831465A (en) | A kind of invasion detection method based on big data log analysis | |
| CN101950338A (en) | Bug repair method based on hierarchical bug threat assessment | |
| CN103001946B (en) | Website security detection method and equipment | |
| CN109564609A (en) | It mitigates and corrects using the detection of the computer attack of advanced computers decision-making platform | |
| CN109753772A (en) | A kind of account safety verification method and system | |
| CN102970282A (en) | Website security detection system | |
| CN102571870A (en) | Scoring method for web vulnerability scanning | |
| CN105164691A (en) | Optimizing test data payload selection for testing computer software applications using computer networks | |
| CN108965330A (en) | A kind of account number safety guard method and system | |
| CN104580092A (en) | Method and device for conducting security detection on network page | |
| CN105184149B (en) | A kind of method and system for preventing rogue program from frequently obtaining customer position information | |
| KR20160016178A (en) | Program for detecting malignant code distributing network | |
| Madan et al. | Security standards perspective to fortify web database applications from code injection attacks | |
| EP2689373A1 (en) | Detecting attacks on a portable data carrier | |
| CN104253809A (en) | Method and system for detecting network content | |
| CN108270746A (en) | User access request processing method and processing device | |
| KR101229012B1 (en) | Signature detecting device and method | |
| Wirth | Log Jam: Lesson Learned from the Log4Shell Vulnerability | |
| CN105743904B (en) | The leakage detection method and system of the user information of website |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |