CN103414718A - Distributed type Web vulnerability scanning method - Google Patents
Distributed type Web vulnerability scanning method Download PDFInfo
- Publication number
- CN103414718A CN103414718A CN2013103605764A CN201310360576A CN103414718A CN 103414718 A CN103414718 A CN 103414718A CN 2013103605764 A CN2013103605764 A CN 2013103605764A CN 201310360576 A CN201310360576 A CN 201310360576A CN 103414718 A CN103414718 A CN 103414718A
- Authority
- CN
- China
- Prior art keywords
- vulnerability scanning
- url
- node
- main controlled
- controlled node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Computer And Data Communications (AREA)
Abstract
The invention discloses a distributed type Web vulnerability scanning method. The method specifically comprises the steps that urls initially requested by users are sent to a main control node to be dispatched and managed in a unified mode, a vulnerability scanning program uses a crawler system to process the submitted urls, and the probability of repeated url vulnerability scanning is removed through a bloom filter. In terms of the returned result, the remove network is attacked by using a series of attack vectors. In terms of contents returned by the attack, the contents are analyzed and processed according to a corresponding rule, the urls are filtered out of the returned contents to serve as the new urls to be further processed, and all results obtained from the vulnerability scanning process are sent to a statistics center to be processed. By means of the distributed type Web vulnerability scanning method, the speed of the vulnerability scanning process is increased, loads can be scattered to other sub-nodes in a balanced mode when the single point of failure happens in the vulnerability scanning process, and the ability of the system to resist the failure is improved.
Description
Technical field
The present invention relates to the network security technology field, relate in particular to a kind of method of distributed Web vulnerability scanning.
Background technology
Along with various Web application, such as Web bank, ecommerce, personal space, cloud storage etc. constantly enters people's life, if there is hidden danger in these Web application, personal information or even Web station system all can face security risk so.According to statistics, current 75% attack is all undertaken by Web.
Guaranteeing that Web service can move with security and stability, is a vital task of IT department in numerous companies and office.Generally, owing to lacking effective Web security evaluation instrument and checking mechanism, safety problem is difficult to be found at ordinary times, and often when major accident occurs such problem just can be found.
Tradition Web vulnerability scanning software, generally be based on one-of-a-kind system, and the scanner correlation module is installed concentratedly on same computer.Therefore each computer security diagnostic result is mutually isolated, and the assessment of network overall risk and preventive effect are poor, and sweep speed is slow, and single node exists inefficacy can make the danger of whole system collapse.In the Web scanning system, in certain hour, the ability that unit is processed is limited, how to improve the associated treatment of vulnerability scanning multinode, and solves the load balancing that the multinode machine of delaying causes, and is a problem demanding prompt solution.
Summary of the invention
The objective of the invention is in order to overcome the defect of prior art, a kind of method of distributed Web vulnerability scanning is provided, the idiographic flow of the method is: the url of user's initial request, unification is given main controlled node and is carried out dispatching management, the vulnerability scanning process is processed the url of submission with crawler system, and with Bloom filter, get rid of the repeatability of vulnerability scanning url, for the result of returning, with vector of attack, long-range website is attacked, the content of returning for attack, according to respective rule, carry out analyzing and processing, and the url that returned content is filtered out, as new request url, be further processed, the result that all vulnerability scanning processes have been scanned, giving the statistics center processes.
The vulnerability scanning process adopts a main controlled node to be responsible for the overall scheduling of url, to reach the load balancing of distributed treatment, child node is obtained initial url from main controlled node, giving crawler system processes, and by the new url extracted in webpage, after Bloom filter filtered, the capacity of being placed on was in the buffer queue of N, if extract the domain name made new advances, new domain name be passed to main controlled node in real time.
In order to safeguard long the connection, child node regularly sends heartbeat packet to main controlled node, judges whether main controlled node is online.
Server to the server that goes offline or increase is used consistent hashing to carry out the processing of load balancing.
Child node can regularly send to the master control node by cpu information, and main controlled node carries out load balancing according to the scheduling situation of child node to whole system, new domain name is distributed to the child node that the cpu occupation rate is low and process.
The beneficial effect that technical solution of the present invention is brought:
By the present invention, not only improved the speed that vulnerability scanning is processed, and when vulnerability scanning process generation Single Point of Faliure, load meeting equilibrium is distributed to other all child nodes, has improved the ability of system counter fault.
The accompanying drawing explanation
In order to be illustrated more clearly in the embodiment of the present invention or technical scheme of the prior art, below will the accompanying drawing of required use in embodiment or description of the Prior Art be briefly described, apparently, accompanying drawing in the following describes is only some embodiments of the present invention, for those of ordinary skills, under the prerequisite of not paying creative work, can also obtain according to these accompanying drawings other accompanying drawing.
Fig. 1 is method flow diagram of the present invention;
Fig. 2 is the mutual and message processing flow figure of main controlled node and child node in the present invention.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is clearly and completely described, obviously, described embodiment is only the present invention's part embodiment, rather than whole embodiment.Based on the embodiment in the present invention, those of ordinary skills, not making under the creative work prerequisite the every other embodiment obtained, belong to the scope of protection of the invention.
The invention provides a kind of method of distributed Web vulnerability scanning, by the method, solve main controlled node and how to dispatch, and the problem of load balancing of child node program crashing.
As shown in Figure 1, the url of user's initial request, unify to give main controlled node and carry out dispatching management particular flow sheet of the present invention, and the vulnerability scanning process is processed the url of submission with crawler system, and with Bloom filter, gets rid of the repeatability of vulnerability scanning url.For the result of returning, with a series of vector of attacks, long-range website is attacked, the content of returning for attack, according to respective rule, carry out analyzing and processing, and the url that returned content is filtered out, as new request url, be further processed, by the result that all vulnerability scanning processes have scanned, give the statistics center and process.
The vulnerability scanning process adopts a main controlled node to be responsible for the overall scheduling of url, to reach the load balancing of distributed treatment.Child node is obtained some initial url from main controlled node, and give crawler system and process, and by the new url extracted in webpage, after Bloom filter filters, the capacity of being placed on is in the buffer queue of N, if extract the domain name made new advances, new domain name be passed to main controlled node in real time.
During concrete enforcement, the mutual and message processing flow of main controlled node and child node is as shown in Figure 2:
Client sends request, and main controlled node, according to strategy, is distributed to different child nodes by url and processed, and uses consistent hashing for the server of the server gone offline or increase, to carry out the processing of load balancing, the large-scale data migration of avoiding the machine of delaying to produce.In order to safeguard long the connection, child node regularly sends heartbeat packet to main controlled node, judges whether main controlled node is online.The consistent hashing dispatching method makes increase and deletes server very convenient, if n child node arranged, when the machine of delaying occurring or increasing node, only has the object of 1/ (n-1) or 1/ (n+1) to need migration; The consistent hashing algorithm has also increased the method for Virtual Service node, and the buffer memory redistribution while reducing to greatest extent the service node increase and decrease, to reach load balancing.
Child node can regularly send to the master control node by cpu information, and main controlled node can carry out load balancing to whole system according to the scheduling situation of child node, new domain name is distributed to the child node that the cpu occupation rate is low and process.
Child node is initiated web-page requests, and the analyzing responding bag, by the different domain name newly produced, gives main controlled node and dispatches.
After finishing dealing with in a website, each child node is issued main controlled node by the site information result, by main controlled node, result is carried out to last analysis and statistics, and the result that finally will have been analyzed by main controlled node sends to the log collection center.
By Distributed Hole scan procedure of the present invention, improved the speed that vulnerability scanning is processed, and when vulnerability scanning process generation Single Point of Faliure, load meeting equilibrium is distributed to other all child nodes, has improved the ability of system counter fault.
The method of above a kind of distributed Web vulnerability scanning that the embodiment of the present invention is provided is described in detail, applied specific case herein principle of the present invention and execution mode are set forth, the explanation of above embodiment is just be used to helping to understand method of the present invention and core concept thereof; Simultaneously, for one of ordinary skill in the art, according to thought of the present invention, all will change in specific embodiments and applications, in sum, this description should not be construed as limitation of the present invention.
Claims (5)
1. the method for a distributed Web vulnerability scanning, it is characterized in that, the idiographic flow of the method is: the url of user's initial request, unification is given main controlled node and is carried out dispatching management, the vulnerability scanning process is processed the url of submission with crawler system, and with Bloom filter, get rid of the repeatability of vulnerability scanning url, for the result of returning, with vector of attack, long-range website is attacked, the content of returning for attack, according to respective rule, carry out analyzing and processing, and the url that returned content is filtered out, as new request url, be further processed, the result that all vulnerability scanning processes have been scanned, giving the statistics center processes.
2. method according to claim 1, it is characterized in that, the vulnerability scanning process adopts a main controlled node to be responsible for the overall scheduling of url, and to reach the load balancing of distributed treatment, child node is obtained initial url from main controlled node, giving crawler system processes, and by the new url extracted in webpage, after Bloom filter filtered, the capacity of being placed on was in the buffer queue of N, if extract the domain name made new advances, new domain name be passed to main controlled node in real time.
3. method according to claim 1 and 2, is characterized in that, in order to safeguard long the connection, child node regularly sends heartbeat packet to main controlled node, judges whether main controlled node is online.
4. method according to claim 1 and 2, is characterized in that, uses consistent hashing to carry out the processing of load balancing to the server of the server that goes offline or increase.
5. method according to claim 1 and 2, it is characterized in that, child node can regularly send to the master control node by cpu information, and main controlled node carries out load balancing according to the scheduling situation of child node to whole system, new domain name is distributed to the child node that the cpu occupation rate is low and process.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2013103605764A CN103414718A (en) | 2013-08-16 | 2013-08-16 | Distributed type Web vulnerability scanning method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2013103605764A CN103414718A (en) | 2013-08-16 | 2013-08-16 | Distributed type Web vulnerability scanning method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN103414718A true CN103414718A (en) | 2013-11-27 |
Family
ID=49607704
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2013103605764A Pending CN103414718A (en) | 2013-08-16 | 2013-08-16 | Distributed type Web vulnerability scanning method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103414718A (en) |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103699845A (en) * | 2013-12-25 | 2014-04-02 | 北京神州绿盟信息安全科技股份有限公司 | Method and device for displaying scanning progress |
| CN104980309A (en) * | 2014-04-11 | 2015-10-14 | 北京奇虎科技有限公司 | Website security detecting method and device |
| CN105978894A (en) * | 2016-06-27 | 2016-09-28 | 上海柯力士信息安全技术有限公司 | Network security monitoring management system based on security vulnerability scanning cloud platform |
| CN106096008A (en) * | 2016-06-23 | 2016-11-09 | 北京工业大学 | A kind of web crawlers method for finance warehouse receipt wind control |
| CN106161450A (en) * | 2016-07-20 | 2016-11-23 | 上海携程商务有限公司 | Distributed HTTPS monitors method |
| CN106161455A (en) * | 2016-07-25 | 2016-11-23 | 恒安嘉新(北京)科技有限公司 | A kind of multimode and engine distributed cloud management system and detection method |
| CN106790085A (en) * | 2016-12-22 | 2017-05-31 | 国网新疆电力公司信息通信公司 | Vulnerability scanning method, apparatus and system |
| CN107026871A (en) * | 2017-05-15 | 2017-08-08 | 安徽大学 | Web vulnerability scanning method based on cloud computing |
| CN107959662A (en) * | 2016-10-18 | 2018-04-24 | 中国电信股份有限公司 | The method and system of web portal security detection |
| CN108121706A (en) * | 2016-11-28 | 2018-06-05 | 央视国际网络无锡有限公司 | A kind of optimization method of distributed reptile |
| CN110941788A (en) * | 2019-12-17 | 2020-03-31 | 山西云时代技术有限公司 | Cloud environment distributed Web page extraction and analysis system and method for edge computing |
| CN112968917A (en) * | 2021-05-19 | 2021-06-15 | 华东交通大学 | Penetration test method and system for network equipment |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102104601A (en) * | 2011-01-14 | 2011-06-22 | 无锡市同威科技有限公司 | Web vulnerability scanning method and device based on infiltration technology |
| CN103095778A (en) * | 2011-11-07 | 2013-05-08 | 北京知道创宇信息技术有限公司 | Web application firewall and web application safety protection method |
-
2013
- 2013-08-16 CN CN2013103605764A patent/CN103414718A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102104601A (en) * | 2011-01-14 | 2011-06-22 | 无锡市同威科技有限公司 | Web vulnerability scanning method and device based on infiltration technology |
| CN103095778A (en) * | 2011-11-07 | 2013-05-08 | 北京知道创宇信息技术有限公司 | Web application firewall and web application safety protection method |
Cited By (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103699845A (en) * | 2013-12-25 | 2014-04-02 | 北京神州绿盟信息安全科技股份有限公司 | Method and device for displaying scanning progress |
| CN103699845B (en) * | 2013-12-25 | 2017-05-17 | 北京神州绿盟信息安全科技股份有限公司 | Method and device for displaying scanning progress |
| US9954886B2 (en) | 2014-04-11 | 2018-04-24 | Beijing Qihoo Technology Company Limited | Method and apparatus for detecting website security |
| CN104980309A (en) * | 2014-04-11 | 2015-10-14 | 北京奇虎科技有限公司 | Website security detecting method and device |
| WO2015154539A1 (en) * | 2014-04-11 | 2015-10-15 | 北京奇虎科技有限公司 | Website security detection method and apparatus |
| CN106096008A (en) * | 2016-06-23 | 2016-11-09 | 北京工业大学 | A kind of web crawlers method for finance warehouse receipt wind control |
| CN106096008B (en) * | 2016-06-23 | 2021-01-05 | 北京工业大学 | Web crawler method for financial warehouse receipt wind control |
| CN105978894A (en) * | 2016-06-27 | 2016-09-28 | 上海柯力士信息安全技术有限公司 | Network security monitoring management system based on security vulnerability scanning cloud platform |
| CN106161450A (en) * | 2016-07-20 | 2016-11-23 | 上海携程商务有限公司 | Distributed HTTPS monitors method |
| CN106161450B (en) * | 2016-07-20 | 2019-12-20 | 上海携程商务有限公司 | Distributed HTTPS monitoring method |
| CN106161455A (en) * | 2016-07-25 | 2016-11-23 | 恒安嘉新(北京)科技有限公司 | A kind of multimode and engine distributed cloud management system and detection method |
| CN106161455B (en) * | 2016-07-25 | 2019-06-04 | 恒安嘉新（北京）科技股份公司 | A kind of multimode and engine distribution cloud management system and detection method |
| CN107959662A (en) * | 2016-10-18 | 2018-04-24 | 中国电信股份有限公司 | The method and system of web portal security detection |
| CN107959662B (en) * | 2016-10-18 | 2020-12-01 | 中国电信股份有限公司 | Website security detection method and system |
| CN108121706A (en) * | 2016-11-28 | 2018-06-05 | 央视国际网络无锡有限公司 | A kind of optimization method of distributed reptile |
| CN106790085B (en) * | 2016-12-22 | 2020-10-23 | 国网新疆电力公司信息通信公司 | Vulnerability scanning method, device and system |
| CN106790085A (en) * | 2016-12-22 | 2017-05-31 | 国网新疆电力公司信息通信公司 | Vulnerability scanning method, apparatus and system |
| CN107026871B (en) * | 2017-05-15 | 2020-08-25 | 安徽大学 | Web vulnerability scanning method based on cloud computing |
| CN107026871A (en) * | 2017-05-15 | 2017-08-08 | 安徽大学 | Web vulnerability scanning method based on cloud computing |
| CN110941788A (en) * | 2019-12-17 | 2020-03-31 | 山西云时代技术有限公司 | Cloud environment distributed Web page extraction and analysis system and method for edge computing |
| CN112968917A (en) * | 2021-05-19 | 2021-06-15 | 华东交通大学 | Penetration test method and system for network equipment |
| CN112968917B (en) * | 2021-05-19 | 2021-08-06 | 华东交通大学 | Penetration test method and system for network equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103414718A (en) | Distributed type Web vulnerability scanning method | |
| JP2013537344A5 (en) | ||
| CN113489691B (en) | Network access method, network access device, computer readable medium and electronic equipment | |
| CN110493043B (en) | Distributed situation awareness calling method and device | |
| WO2017080161A1 (en) | Alarm information processing method and device in cloud computing | |
| CN106686039B (en) | Resource scheduling method and device in cloud computing system | |
| CN107168844B (en) | Performance monitoring method and device | |
| CN107645483A (en) | Risk Identification Method, risk identification device, cloud risk identification apparatus and system | |
| CN113676563B (en) | Scheduling method, device, equipment and storage medium of content distribution network service | |
| Somani et al. | DDoS victim service containment to minimize the internal collateral damages in cloud computing | |
| CN102833228A (en) | Virus detection system and method for immune network under cloud computing environment | |
| Solaimani et al. | Online anomaly detection for multi‐source VMware using a distributed streaming framework | |
| Mallikarjuna et al. | OLB: a nature inspired approach for load balancing in cloud computing | |
| Zhang et al. | Behavior reconstruction models for large-scale network service systems | |
| CN110493218B (en) | Situation awareness virtualization method and device | |
| CN110471975B (en) | Internet of things situation awareness calling method and device | |
| CN107046546A (en) | A kind of network safety control method and device | |
| Imran et al. | Cloud-niagara: A high availability and low overhead fault tolerance middleware for the cloud | |
| Song et al. | A Hybrid Data Security System of Internet of Things | |
| Lee et al. | Development of an optimal load balancing algorithm based on ANFIS modeling for the clustering web-server | |
| Varghese et al. | Enhancing NameNode fault tolerance in Hadoop over cloud environment | |
| Deshmukh et al. | Intrusion detection system for cloud computing | |
| Pechenkin et al. | Architecture of a scalable system of fuzzing network protocols on a multiprocessor cluster | |
| Qiu et al. | A security controller-based software defined security architecture | |
| Sauber et al. | A novel hadoop security model for addressing malicious collusive workers |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20131127 |