CN114491560A - Vulnerability detection method and device, storage medium and electronic equipment - Google Patents
Vulnerability detection method and device, storage medium and electronic equipment Download PDFInfo
- Publication number
- CN114491560A CN114491560A CN202210100376.4A CN202210100376A CN114491560A CN 114491560 A CN114491560 A CN 114491560A CN 202210100376 A CN202210100376 A CN 202210100376A CN 114491560 A CN114491560 A CN 114491560A
- Authority
- CN
- China
- Prior art keywords
- target
- web application
- url
- page jump
- request
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
- G06F21/57—Certifying or maintaining trusted computer platforms, e.g. secure boots or power-downs, version controls, system software checks, secure updates or assessing vulnerabilities
- G06F21/577—Assessing vulnerabilities and evaluating computer system security
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/951—Indexing; Web crawling techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/955—Retrieval from the web using information identifiers, e.g. uniform resource locators [URL]
- G06F16/9558—Details of hyperlinks; Management of linked annotations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/955—Retrieval from the web using information identifiers, e.g. uniform resource locators [URL]
- G06F16/9566—URL specific, e.g. using aliases, detecting broken or misspelled links
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Computer Hardware Design (AREA)
- Computer Security & Cryptography (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The embodiment of the invention provides a vulnerability detection method, a vulnerability detection device, a storage medium and electronic equipment, wherein the method comprises the following steps: acquiring an initial Uniform Resource Locator (URL) link of a Web application; performing data crawler on the interaction process of the Web application and a server based on a heuristic crawler technology by taking the initial URL link as a crawler inlet to obtain a target request list set corresponding to the Web application; the target request list set comprises at least one piece of request information for triggering the Web application page jump; and detecting the vulnerability of the Web application based on the target request list set. The scheme of the embodiment of the invention not only solves the problem that the traditional crawler cannot load Ajax requests and JavaScript scripts, but also expands the coverage of the Web application and obtains more service request information, thereby being beneficial to greatly reducing the problem of security test report omission of the Web application.
Description
Technical Field
The embodiment of the invention relates to the technical field of network security, in particular to a vulnerability detection method, a vulnerability detection device, a storage medium and electronic equipment.
Background
With the development of science and technology, the requirements on the network security of the Web application are higher and higher, and therefore vulnerability detection needs to be performed on the Web application, the purpose of which is to perform vulnerability detection on the Web application from the perspective of an attacker, obtain potential safety hazards which may exist in the Web application, and perform vulnerability repair by taking effective measures before the attacker utilizes the vulnerability to solve the potential safety hazards and prevent malicious behaviors from occurring.
In the conventional Security test technology, the Application is the most widely at present, and a most simple Web Application Security test method is Dynamic Application Security Testing (DAST), which is a black box test technology and has the technical principle that: the crawler finds the whole Web application structure, obtains information such as a directory structure, request parameters and the like, replaces the request parameters into well-constructed attack loads (called payload for short), sends the attack loads to the server again, and verifies whether security holes exist or not through corresponding message analysis of the server. DAST is able to discover most threats and risks from an attack perspective without paying attention to the implementation details and logical structure of the Web application. But there are limitations because the effect of security vulnerability detection mainly depends on the coverage of the site crawled by the crawler, and the JavaScript script parsing capability is insufficient for Ajax pages.
Another mainstream Web Application Security Testing method is Static Application Security Testing (SAST), which is a white box Testing technique and aims to treat Security threats in the source code development stage, and the technical principle is as follows: and analyzing the semantics, data flow, control flow, configuration, structure and the like of the source code, and matching the vulnerability characteristics of the vulnerability library, thereby accurately positioning the position of the vulnerability in the source code. SAST starts from a code source, has high visibility, can find bugs in time without a user interface, but has obvious limitation, different containers and programming languages need to be distinguished, the scanning time of source codes with large volume is too slow, the final missed scanning result is more in error report, and a large amount of manpower is required to be invested for troubleshooting and misinformation.
Under the background of Web2.0, data interaction of Web application becomes more complex, the coverage of a traditional security testing tool is reduced due to the application of the Ajax dynamic page loading technology, a traditional static crawler cannot interact with a webpage, cannot analyze an Ajax request, and the acquisition of an attack surface is limited.
Disclosure of Invention
The embodiment of the invention provides a vulnerability detection method, a vulnerability detection device, a storage medium and electronic equipment, which can obtain more service request information, thereby being beneficial to greatly reducing the problem of missing report of security test of Web application; meanwhile, the heuristic crawler simulates the operation of a browser, replaces the interaction between manpower and Web application, and greatly reduces the labor cost.
In a first aspect, an embodiment of the present invention provides a vulnerability detection method, including:
acquiring an initial Uniform Resource Locator (URL) link of a Web application;
performing data crawler on the interaction process of the Web application and a server based on a heuristic crawler technology by taking the initial URL link as a crawler inlet to obtain a target request list set corresponding to the Web application; the target request list set comprises at least one piece of request information for triggering the Web application page jump;
and detecting the vulnerability of the Web application based on the target request list set.
In a second aspect, an embodiment of the present invention further provides a vulnerability detection apparatus, including:
the URL link acquisition module is used for acquiring an initial uniform resource locator URL link of the Web application;
a request list set acquisition module, configured to perform data crawling on an interaction process between the Web application and a server based on a heuristic crawling technology with the initial URL link as a crawler entry, and acquire a target request list set corresponding to the Web application; the target request list set comprises at least one piece of request information for triggering the Web application page jump;
and the vulnerability detection module is used for carrying out vulnerability detection on the Web application based on the target request list set.
In a third aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the vulnerability detection method provided in the embodiment of the present invention.
In a fourth aspect, an embodiment of the present invention provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor executes the computer program to implement the vulnerability detection method according to the embodiment of the present invention.
According to the vulnerability detection scheme provided by the embodiment of the invention, an initial Uniform Resource Locator (URL) link of Web application is obtained; performing data crawler on the interaction process of the Web application and a server based on a heuristic crawler technology by taking the initial URL link as a crawler inlet to obtain a target request list set corresponding to the Web application; the target request list set comprises at least one piece of request information for triggering the Web application page jump; and detecting the vulnerability of the Web application based on the target request list set. By the technical scheme provided by the embodiment of the invention, the problem that the traditional crawler cannot load Ajax requests and JavaScript scripts is solved, the coverage of Web application is enlarged, more service request information is obtained, and therefore, the problem of missing report of safety test of the Web application is greatly reduced.
Drawings
Fig. 1 is a flowchart of a vulnerability detection method according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a vulnerability detection apparatus according to another embodiment of the present invention;
fig. 3 is a schematic structural diagram of an electronic device in another embodiment of the present invention.
Detailed Description
Embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present invention are shown in the drawings, it should be understood that the present invention may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, but rather are provided for a more thorough and complete understanding of the present invention. It should be understood that the drawings and the embodiments of the present invention are illustrative only and are not intended to limit the scope of the present invention.
It should be understood that the various steps recited in the method embodiments of the present invention may be performed in a different order and/or performed in parallel. Moreover, method embodiments may include additional steps and/or omit performing the illustrated steps. The scope of the invention is not limited in this respect.
The term "include" and variations thereof as used herein are open-ended, i.e., "including but not limited to". The term "based on" is "based, at least in part, on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions for other terms will be given in the following description.
It should be noted that the terms "first", "second", and the like in the present invention are only used for distinguishing different devices, modules or units, and are not used for limiting the order or interdependence relationship of the functions performed by the devices, modules or units.
It is noted that references to "a", "an", and "the" modifications in the present invention are intended to be illustrative rather than limiting, and that those skilled in the art will recognize that reference to "one or more" unless the context clearly dictates otherwise.
The names of messages or information exchanged between devices in the embodiments of the present invention are for illustrative purposes only, and are not intended to limit the scope of the messages or information.
Fig. 1 is a flowchart of a vulnerability detection method according to an embodiment of the present invention, where the vulnerability detection method is applicable to a Web application, and the vulnerability detection method may be executed by a vulnerability detection apparatus, and the apparatus may be composed of hardware and/or software and may be generally integrated in an electronic device. As shown in fig. 1, the method specifically includes the following steps:
at step 110, an initial uniform resource locator, URL, link for the Web application is obtained.
The initial URL link of the Web application may also be referred to as a Web page address of the Web application, and the home page of the Web application may be opened through the initial URL link of the Web application.
In the embodiment of the invention, the initial URL link of the Web application can be acquired by receiving the URL link of the Web application input by the user, or the initial URL link of the Web application sent by other electronic equipment can be received. It should be noted that, the embodiment of the present invention does not limit the manner of obtaining the initial URL link of the Web application.
The web crawler is also called a web spider, and is an automatic program for simulating a user to browse a network. The purpose of the crawler is to acquire web page resources through page downloading, and extracting hyperlinks in the web page resources can acquire the topology structure of the whole website.
Since the Web application is a dynamic website, in the process of accessing the Web application, a new page jump may be triggered through operations such as page access, button click, scrolling, dragging, form filling and emptying, page switching, page closing and the like. Therefore, in the embodiment of the present invention, an initial URL link of a Web application is used as a crawler entry of a Web crawler, a data crawler is performed on an interaction process between the Web application and a server based on a heuristic crawler technology, and a target request list set corresponding to the Web application is obtained, where the target request list set includes at least one request information capable of triggering a Web application page jump. It can be understood that the target request list set includes various request information capable of triggering page jump of the Web application in the process of dynamically accessing the Web application. The request information may include information such as a request URL, a request method, a request header, a request parameter, and a timestamp. For example, the target request list set Q may be represented as: q [ { "request _ url": http:// abctest.com "," request _ method ": POST", "request _ header": User-Agent ": Mozilla/5.0(Windows NT 10.0; Win 64; x64) AppleWebKit/537.36(KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36" }, "request _ body": a ═ test & b ═ 1 "}.
It can be understood that, based on the heuristic crawler technology, the interaction process between the Web application and the manual mode can be simulated, such as manual operations of page access, button click, dragging, form filling and emptying, page switching, page closing and the like, so that request information in the interaction process between the Web application and the server can be obtained, the problem that the traditional crawler cannot load Ajax requests and JavaScript is solved, the coverage of the Web application is enlarged, more service request information is obtained, the problem of security test report omission is greatly reduced, the heuristic crawler does not need manual participation in the data crawling process, and labor cost is greatly reduced.
And step 130, performing vulnerability detection on the Web application based on the target request list set.
For example, each request message in the target request list set may be sent to a server corresponding to the Web application, and whether vulnerability detection exists in the Web application is determined according to feedback information of the server corresponding to the Web application.
Optionally, performing vulnerability detection on the Web application based on the target request list set includes: and performing security test on each request message in the target request list set through an interactive application program security test IAST tool so as to perform vulnerability detection on the Web application. The method has the advantages that full-automatic interactive vulnerability detection based on the IAST technology is achieved, human resources are liberated, time cost is greatly reduced, and full-automatic operation of vulnerability detection based on the IAST technology is really achieved.
Illustratively, the IAST tool has an agent function and an instrumentation function, wherein each request message in the target request list set can be replayed to a port monitored by the IAST tool through the agent function of the IAST tool, the IAST tool automatically rewrites the request message (namely, a normal service request) in the target request list set into a security test request, sends the security test request to a server corresponding to the Web application, matches data in a rule base of the IAST according to a message returned by the server corresponding to the Web application, and judges whether the Web application has a corresponding vulnerability according to a matching result. Optionally, an Agent may be deployed in a server corresponding to the Web application through an iatt tool insertion function, where the Agent is transparent and visible to a tester, and a probe is driven into a specific position under the condition that the original logic of the target program is complete. Optionally, the vulnerability detection result of the agent function based on the iatt tool and the vulnerability detection result of the instrumentation function based on the iatt tool may be used together as the final vulnerability detection result for the Web application.
According to the vulnerability detection scheme provided by the embodiment of the invention, an initial Uniform Resource Locator (URL) link of Web application is obtained; performing data crawler on the interaction process of the Web application and a server based on a heuristic crawler technology by taking the initial URL link as a crawler inlet to obtain a target request list set corresponding to the Web application; the target request list set comprises at least one piece of request information for triggering the Web application page jump; and detecting the vulnerability of the Web application based on the target request list set. By the technical scheme provided by the embodiment of the invention, the problem that the traditional crawler cannot load Ajax requests and JavaScript scripts is solved, the coverage of Web application is enlarged, more service request information is obtained, and therefore, the problem of missing report of safety test of the Web application is greatly reduced.
In some embodiments, performing data crawling on the interaction process between the Web application and a server based on a heuristic crawling technique with the initial URL link as a crawler entry, and acquiring a target request list set corresponding to the Web application includes: page loading is carried out on the Web application based on the initial URL link, and a Document Object Model (DOM) tree corresponding to the Web application page is obtained; traversing the DOM tree, and determining at least one preset target tag in the DOM tree; the target tag is a tag capable of triggering the Web application page to jump; respectively triggering page jump events bound with the target tags, and acquiring page jump requests corresponding to the page jump events; and taking the set generated by the page jump request as a target request list set corresponding to the Web application. The method has the advantages that the target request list set corresponding to the Web application can be quickly and accurately crawled, the coverage area of the Web application is enlarged, and therefore the missing report rate of the Web application security vulnerability detection can be greatly reduced.
In the embodiment of the invention, the Web application is subjected to page loading based on the initial URL link of the Web application, so that the loaded Web application page is displayed. Specifically, text information such as HTML, CSS, JavaScript script, and the like returned by a server corresponding to the Web application may be acquired through a browser environment provided by a pyppeteeer framework, and the browser forms a Document Object Model (DOM) tree corresponding to a Web application page by parsing an HTML file. In addition, when the CSS file exists in the Web application page, the browser can construct a CSSOM tree by analyzing the CSS style; and then combining the DOM tree and the CSSOM tree, generating a rendering tree through a rendering engine, calculating attributes such as the size of a positioning coordinate and the like to perform layout, drawing a Web page image, thereby completing the loading operation of the Web application page, and displaying the Web application page in a browser. It should be noted that, in the process of loading and rendering the Web application page by the browser, a statement of redirection jump, such as window.
In the embodiment of the invention, the DOM tree comprises tag nodes capable of triggering Web application page skipping and tag nodes incapable of triggering Web application page skipping, the DOM is traversed, and at least one preset target tag in the DOM tree is determined, wherein the target tag is a tag capable of triggering Web application page skipping. For example, the preset target tags may include tags < a >, < button >, < span >, < img >, < input >, < textarea >, < div >, and the like. Where < a > may be understood as a link tag, < button > may be understood as a button tag, < span > may be understood as a text tag, < img > may be understood as a picture tag, < input > may be understood as a text input box tag, < textarea > may be understood as a text box tag defining a text length, and < div > may be understood as a frame tag. And when at least one preset target tag exists in the DOM tree, respectively triggering page jump events bound with the target tags in the DOM tree. In the embodiment of the invention, the page jump request sent by the browser to the server corresponding to the Web application can be acquired, wherein the page jump request can comprise a requested URL (uniform resource locator), a requested parameter and an HTTP (hyper text transport protocol) request method. In the embodiment of the invention, a set generated by triggering the page jump request corresponding to the page jump event bound with each target tag is used as a target request list set corresponding to the Web application.
Optionally, before the page jump event bound to each target tag is triggered respectively and the page jump request corresponding to the page jump event is acquired, the method further includes: adding URL links corresponding to all target tags into the initial URL queue to generate a target URL queue; respectively triggering page jump events bound with the target tags, and acquiring page jump requests corresponding to the page jump events, wherein the page jump requests comprise: and sequentially triggering URL links in the target URL queue based on the first-in first-out order to trigger page jump events bound with each target tag, and acquiring page jump requests corresponding to the page jump events until the target URL queue is empty.
The initial URL queue may be a queue only containing initial URL links of the Web application, or a queue containing initial URL links of the Web application and other determined URL links that can trigger page jumps of the Web application. In the process of performing data crawler on the interaction process of the Web application and the server based on the heuristic crawler technology by taking the initial URL link as a crawler inlet, deleting the crawled URL link from the initial URL queue, adding the URL link corresponding to each target label in the determined DOM tree into the initial URL queue, and generating a target URL queue, wherein the target URL queue is a queue dynamically changing in real time. In the embodiment of the invention, the URL links in the target URL queue are sequentially triggered based on the first-in first-out sequence so as to trigger the page jump event bound with each target label, and the page jump request corresponding to the page jump event is obtained until the target URL queue is empty. It can be understood that when the target URL queue is empty, it indicates that data crawling of an event that can trigger a page jump in an interaction process between the Web application and the server is completed, so that a page jump request corresponding to the page jump event related to the Web page is crawled, and a coverage area of the Web application is enlarged.
In some embodiments, before adding the URL links corresponding to the respective target tags to the initial URL queue, the method further includes: aiming at the URL link corresponding to each target label, judging whether the URL link which is the same as or similar to the URL link corresponding to the current target label exists in the initial URL queue or a preset URL link database; the preset URL link database is a database which stores crawled URL links related to the Web application; adding URL links corresponding to the target tags into an initial URL queue, wherein the URL links comprise: and when no URL link which is the same as or similar to the URL link corresponding to the current target label exists in the initial URL queue or a preset URL link database, adding the URL link corresponding to the current target label into the initial URL queue. The method has the advantages that the phenomenon that the crawled URL is placed in the queue to be grabbed again to cause resource waste and overlarge system load can be effectively avoided, and the efficiency of detecting the Web application vulnerability can be effectively improved.
It can be understood that for a Web application with a complex structure, there may be a result that the URL that has been captured is put into the queue to be captured again, which may cause resource waste and excessive system load, and therefore, the URL link may be subjected to deduplication operation. For example, whether a URL link that is the same as or similar to a URL link corresponding to the current target tag exists in the initial URL queue or in the preset URL link database is determined for the URL link corresponding to each target tag, and if not, it is determined that the URL link corresponding to the current target tag is not crawled, the URL link corresponding to the current target tag is added to the initial URL queue. Specifically, a table may be newly created in the preset URL link database to store URL links that have been crawled through the Web application, and an index may be added to the URL links. Before adding the URL link corresponding to the target label into the initial URL queue, searching whether the URL link corresponding to the target label exists in a preset URL link database or not based on indexes corresponding to all URL links in the preset URL link database, if not, showing that crawling is not carried out on the URL link corresponding to the target label, further judging whether URL links same as or similar to the URL link corresponding to the target label exist in the initial URL queue or not, and if not, adding the URL link corresponding to the target label into the initial URL queue. When similarity detection is performed on the URL links, the request parameter key values in the URL links may be the same, and the URL links with different value values may be determined as similar URL links, for example: http:// demo.com/a ═ 1 and http:// demo.com/a ═ 2 are two similar URL links; the URL links that are determined to be similar only by the URL links with different request orders in the plurality of request parameters may be determined to be similar, for example: http:// demo.com/a ═ 1& b ═ 2 and http:// demo.com/b ═ 3& a ═ 4 are two similar URL links; the URL links with the same request parameter length and the hash value of the random number-letter combination may also be determined as similar URL links, for example: com/a3ds38fac and http:// demo com/d34ef6ae0 are two similar URL links.
In some embodiments, before the set generated by the page jump request is used as the target request list set corresponding to the Web application, the method further includes: sequentially triggering target events bound by other tags in the DOM tree, judging whether the target events are events capable of triggering page skipping or not, and if so, acquiring event triggering requests corresponding to the target events; wherein, other tags are tags in the DOM tree except the target tag; taking the set generated by the page jump request as a target request list set corresponding to the Web application, wherein the set comprises: and taking the set generated by the page jump request and the event trigger request as a target request list set corresponding to the Web application. This has the advantage that the coverage area of the Web application can be further enlarged.
In the embodiment of the invention, the target tag is a tag which is set according to experience and can trigger the webpage jump of the Web application, so that events bound by other tags in the DOM tree can also be events capable of triggering the webpage jump, and in order to enlarge the coverage area of the Web application, the target events bound by other tags in the DOM tree can be sequentially triggered, namely, a user is simulated to trigger the target events bound by other tags in the DOM tree. When a target event bound by other tags in the DOM tree is triggered, judging whether the target event is an event capable of triggering page jump, if so, acquiring an event trigger request corresponding to the target event, and using a set generated by the page jump request and the event trigger request as a target request list set corresponding to the Web application. It should be noted that when triggering target events bound by other tags in the DOM tree, in addition to the situation that the target events bound by other tags in the DOM tree are triggered in a manner of simulating a user click, there is a special situation to trigger the target events, for example, a button can be clicked to trigger the target events when form information has a necessary item. In the embodiment of the present invention, for this situation, a dictionary may need to be set in the heuristic crawler program, the label name of the form is matched in the key value of the dictionary, and if the label name is matched to any value in the value corresponding to the key value, data filling is performed. If the dictionary is { "username": "admin" }, for a user name input box in the login type form, if the label name of the input box is username, the key value of the dictionary can be just matched, and the heuristic crawler can fill admin into the input box, so as to trigger the target event.
Optionally, before sequentially triggering target events of other tag bindings in the DOM tree, the method further includes: determining the time length required for acquiring the page jump request; sequentially triggering target events bound by other tags in the DOM tree, and judging whether the target events are events capable of triggering page jump, wherein the steps of: and when the duration is less than the preset duration, sequentially triggering target events bound by other tags in the DOM tree, and judging whether the target events are events capable of triggering page jump. The method has the advantage that the coverage area of the Web application is further increased on the premise that the acquisition efficiency of the target request list set corresponding to the Web application is effectively guaranteed.
In the embodiment of the invention, the time length required for acquiring all page jump requests is determined, when the time length is less than the preset time length, the time consumption of data crawler on the basis of a heuristic crawler technology in the interaction process of the Web application and the server is less, in order to acquire more comprehensive page jump requests, target events bound by other labels in a DOM tree can be further triggered in sequence, whether the target events are events capable of triggering page jump is judged, and if yes, a set generated by the page jump requests and the event trigger requests is used as a target request list set corresponding to the Web application.
Fig. 2 is a schematic structural diagram of a vulnerability detection apparatus according to another embodiment of the present invention. As shown in fig. 2, the apparatus includes: a URL link acquisition module 210, a request list set acquisition module 220 and a vulnerability detection module 230. Wherein the content of the first and second substances,
a URL link obtaining module 210, configured to obtain an initial uniform resource locator URL link of the Web application;
a request list set obtaining module 220, configured to perform data crawling on an interaction process between the Web application and a server based on a heuristic crawling technique with the initial URL link as a crawler entry, and obtain a target request list set corresponding to the Web application; the target request list set comprises at least one piece of request information for triggering the Web application page jump;
and the vulnerability detection module 230 is configured to perform vulnerability detection on the Web application based on the target request list set.
The vulnerability detection device provided by the embodiment of the invention obtains an initial Uniform Resource Locator (URL) link of Web application; performing data crawler on the interaction process of the Web application and a server based on a heuristic crawler technology by taking the initial URL link as a crawler inlet to obtain a target request list set corresponding to the Web application; the target request list set comprises at least one piece of request information for triggering the Web application page jump; and detecting the vulnerability of the Web application based on the target request list set. By the technical scheme provided by the embodiment of the invention, the problem that the traditional crawler cannot load Ajax requests and JavaScript scripts is solved, the coverage of Web application is enlarged, more service request information is obtained, and therefore, the problem of missing report of safety test of the Web application is greatly reduced.
Optionally, the request list set obtaining module includes:
a DOM tree obtaining unit, configured to perform page loading on the Web application based on the initial URL link, and obtain a DOM tree of a document object model corresponding to a Web application page;
the target tag determining unit is used for traversing the DOM tree and determining at least one preset target tag in the DOM tree; the target tag is a tag capable of triggering the Web application page to jump;
the page jump request acquisition unit is used for respectively triggering page jump events bound with the target tags and acquiring page jump requests corresponding to the page jump events;
and the request list set acquisition unit is used for taking the set generated by the page jump request as a target request list set corresponding to the Web application.
Optionally, the apparatus further comprises:
the URL link adding unit is used for adding the URL link corresponding to each target label into an initial URL queue before respectively triggering the page jump event bound with each target label and acquiring the page jump request corresponding to the page jump event, so as to generate a target URL queue;
the page jump request obtaining unit is configured to:
and sequentially triggering URL links in the target URL queue based on the first-in first-out order to trigger page jump events bound with each target tag, and acquiring page jump requests corresponding to the page jump events until the target URL queue is empty.
Optionally, the apparatus further comprises:
the URL link judging unit is used for judging whether URL links which are the same as or similar to the URL link corresponding to the current target label exist in the initial URL queue or a preset URL link database aiming at the URL link corresponding to each target label before the URL link corresponding to each target label is added into the initial URL queue; the preset URL link database is a database which stores crawled URL links related to the Web application;
the URL link adding unit is used for:
and when no URL link which is the same as or similar to the URL link corresponding to the current target label exists in the initial URL queue or a preset URL link database, adding the URL link corresponding to the current target label into the initial URL queue.
Optionally, the apparatus further comprises:
a target event judging unit, configured to sequentially trigger target events bound to other tags in the DOM tree before taking the set generated by the page jump request as a target request list set corresponding to the Web application,
judging whether the target event is an event capable of triggering page jump, if so, acquiring an event triggering request corresponding to the target event; wherein, other tags are tags in the DOM tree except the target tag;
the request list set obtaining unit is configured to:
and taking the set generated by the page jump request and the event trigger request as a target request list set corresponding to the Web application.
Optionally, the apparatus further comprises:
the time length obtaining unit is used for determining the time length required by obtaining the page jump request before sequentially triggering target events bound by other tags in the DOM tree;
the target event judging unit is used for:
and when the duration is less than the preset duration, sequentially triggering target events bound by other tags in the DOM tree, and judging whether the target events are events capable of triggering page jump.
Optionally, the vulnerability detection module is configured to:
and performing security test on each request message in the target request list set through an interactive application program security test IAST tool so as to perform vulnerability detection on the Web application.
The device can execute the methods provided by all the embodiments of the invention, and has corresponding functional modules and beneficial effects for executing the methods. For technical details which are not described in detail in the embodiments of the present invention, reference may be made to the methods provided in all the aforementioned embodiments of the present invention.
Embodiments of the present invention further provide a storage medium containing computer-executable instructions, which when executed by a computer processor, are configured to perform the vulnerability detection method provided by the embodiments of the present invention.
Storage medium-any of various types of memory devices or storage devices. The term "storage medium" is intended to include: mounting media such as CD-ROM, floppy disk, or tape devices; computer system memory or random access memory such as DRAM, DDRRAM, SRAM, EDORAM, Lanbas (Rambus) RAM, etc.; non-volatile memory such as flash memory, magnetic media (e.g., hard disk or optical storage); registers or other similar types of memory elements, etc. The storage medium may also include other types of memory or combinations thereof. In addition, the storage medium may be located in a first computer system in which the program is executed, or may be located in a different second computer system connected to the first computer system through a network (such as the internet). The second computer system may provide program instructions to the first computer for execution. The term "storage medium" may include two or more storage media that may reside in different locations, such as in different computer systems that are connected by a network. The storage medium may store program instructions (e.g., embodied as a computer program) that are executable by one or more processors.
Of course, the storage medium containing the computer-executable instructions provided in the embodiments of the present invention is not limited to the above-described vulnerability detection operation, and may also perform related operations in the vulnerability detection method provided in any embodiments of the present invention.
The embodiment of the invention provides electronic equipment, wherein the vulnerability detection device provided by the embodiment of the invention can be integrated in the electronic equipment. Fig. 3 is a block diagram of an electronic device according to an embodiment of the present invention. The electronic device 300 may include: the vulnerability detection method comprises a memory 301, a processor 302 and a computer program which is stored on the memory 301 and can be run by the processor, wherein the processor 302 implements the vulnerability detection method according to the embodiment of the invention when executing the computer program.
The electronic equipment provided by the embodiment of the invention acquires an initial Uniform Resource Locator (URL) link of a Web application; performing data crawler on the interaction process of the Web application and a server based on a heuristic crawler technology by taking the initial URL link as a crawler inlet to obtain a target request list set corresponding to the Web application; the target request list set comprises at least one piece of request information for triggering the Web application page jump; and detecting the vulnerability of the Web application based on the target request list set. By the technical scheme provided by the embodiment of the invention, the problem that the traditional crawler cannot load Ajax requests and JavaScript scripts is solved, the coverage of Web application is enlarged, more service request information is obtained, and therefore, the problem of missing report of safety test of the Web application is greatly reduced.
The vulnerability detection device, the storage medium and the electronic device provided in the above embodiments can execute the vulnerability detection method provided in any embodiment of the present invention, and have the corresponding functional modules and beneficial effects for executing the method. For technical details that are not described in detail in the above embodiments, reference may be made to the vulnerability detection method provided in any embodiment of the present invention.
It is to be noted that the foregoing description is only exemplary of the invention and that the principles of the technology may be employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (10)
1. A vulnerability detection method is characterized by comprising the following steps:
acquiring an initial Uniform Resource Locator (URL) link of a Web application;
performing data crawler on the interaction process of the Web application and a server based on a heuristic crawler technology by taking the initial URL link as a crawler inlet to obtain a target request list set corresponding to the Web application; the target request list set comprises at least one piece of request information for triggering the Web application page jump;
and detecting the vulnerability of the Web application based on the target request list set.
2. The method of claim 1, wherein performing data crawling on the interaction process between the Web application and a server based on a heuristic crawling technique by using the initial URL link as a crawler entry to obtain a target request list set corresponding to the Web application comprises:
page loading is carried out on the Web application based on the initial URL link, and a Document Object Model (DOM) tree corresponding to the Web application page is obtained;
traversing the DOM tree, and determining at least one preset target tag in the DOM tree; the target tag is a tag capable of triggering the Web application page to jump;
respectively triggering page jump events bound with the target tags, and acquiring page jump requests corresponding to the page jump events;
and taking the set generated by the page jump request as a target request list set corresponding to the Web application.
3. The method according to claim 2, before triggering the page jump event bound to each target tag and obtaining the page jump request corresponding to the page jump event, further comprising:
adding URL links corresponding to all target tags into the initial URL queue to generate a target URL queue;
respectively triggering page jump events bound with the target tags, and acquiring page jump requests corresponding to the page jump events, wherein the page jump requests comprise:
and sequentially triggering URL links in the target URL queue based on the first-in first-out order to trigger page jump events bound with each target tag, and acquiring page jump requests corresponding to the page jump events until the target URL queue is empty.
4. The method of claim 3, further comprising, prior to adding the URL link corresponding to each target tag to the initial URL queue:
aiming at the URL link corresponding to each target label, judging whether the URL link which is the same as or similar to the URL link corresponding to the current target label exists in the initial URL queue or a preset URL link database; the preset URL link database is a database which stores the crawled URL links related to the Web application;
adding URL links corresponding to the target tags into an initial URL queue, including:
and when no URL link which is the same as or similar to the URL link corresponding to the current target label exists in the initial URL queue or a preset URL link database, adding the URL link corresponding to the current target label into the initial URL queue.
5. The method of claim 2, wherein before the generating the set of page jump requests as a set of target request lists corresponding to the Web application, further comprising:
sequentially triggering target events bound by other tags in the DOM tree, judging whether the target events are events capable of triggering page skipping or not, and if so, acquiring event triggering requests corresponding to the target events; wherein, other tags are tags in the DOM tree except the target tag;
taking the set generated by the page jump request as a target request list set corresponding to the Web application, including:
and taking the set generated by the page jump request and the event trigger request as a target request list set corresponding to the Web application.
6. The method of claim 5, further comprising, before sequentially triggering target events of other tag bindings in the DOM tree:
determining the time length required for acquiring the page jump request;
sequentially triggering target events bound by other tags in the DOM tree, and judging whether the target events are events capable of triggering page jump, wherein the steps of:
and when the duration is less than the preset duration, sequentially triggering target events bound by other tags in the DOM tree, and judging whether the target events are events capable of triggering page jump.
7. The method of claim 1, wherein detecting vulnerabilities of the Web application based on the set of target request lists comprises:
and performing security test on each request message in the target request list set through an interactive application program security test IAST tool so as to perform vulnerability detection on the Web application.
8. A vulnerability detection apparatus, comprising:
the URL link acquisition module is used for acquiring an initial Uniform Resource Locator (URL) link of the Web application;
a request list set acquisition module, configured to perform data crawling on an interaction process between the Web application and a server based on a heuristic crawling technology with the initial URL link as a crawler entry, and acquire a target request list set corresponding to the Web application; the target request list set comprises at least one piece of request information for triggering the Web application page jump;
and the vulnerability detection module is used for carrying out vulnerability detection on the Web application based on the target request list set.
9. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processing device, carries out the vulnerability detection method according to any of claims 1-7.
10. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the vulnerability detection method according to any of claims 1-7 when executing the computer program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210100376.4A CN114491560A (en) | 2022-01-27 | 2022-01-27 | Vulnerability detection method and device, storage medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210100376.4A CN114491560A (en) | 2022-01-27 | 2022-01-27 | Vulnerability detection method and device, storage medium and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114491560A true CN114491560A (en) | 2022-05-13 |
Family
ID=81477067
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210100376.4A Pending CN114491560A (en) | 2022-01-27 | 2022-01-27 | Vulnerability detection method and device, storage medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114491560A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117112435A (en) * | 2023-09-08 | 2023-11-24 | 清科万道(北京)信息技术有限公司 | Vulnerability linkage detection result fusion method, storage medium and electronic equipment |
| JP7464804B1 (en) | 2024-01-10 | 2024-04-09 | 株式会社ユービーセキュア | Security Test System |
-
2022
- 2022-01-27 CN CN202210100376.4A patent/CN114491560A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117112435A (en) * | 2023-09-08 | 2023-11-24 | 清科万道(北京)信息技术有限公司 | Vulnerability linkage detection result fusion method, storage medium and electronic equipment |
| CN117112435B (en) * | 2023-09-08 | 2024-01-26 | 清科万道(北京)信息技术有限公司 | Vulnerability linkage detection result fusion method, storage medium and electronic equipment |
| JP7464804B1 (en) | 2024-01-10 | 2024-04-09 | 株式会社ユービーセキュア | Security Test System |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10567407B2 (en) | Method and system for detecting malicious web addresses | |
| US9614862B2 (en) | System and method for webpage analysis | |
| CN101964025B (en) | XSS detection method and equipment | |
| US8296722B2 (en) | Crawling of object model using transformation graph | |
| CN103279710B (en) | Method and system for detecting malicious codes of Internet information system | |
| US8621613B1 (en) | Detecting malware in content items | |
| CN109347882B (en) | Webpage Trojan horse monitoring method, device, equipment and storage medium | |
| CN111552854A (en) | Webpage data capturing method and device, storage medium and equipment | |
| CN114491560A (en) | Vulnerability detection method and device, storage medium and electronic equipment | |
| CN113114680B (en) | Detection method and detection device for file uploading vulnerability | |
| CN103647678A (en) | Method and device for online verification of website vulnerabilities | |
| CN108846286A (en) | Cross site scripting leak detection method and device | |
| EP3077950A1 (en) | Directed execution of dynamic programs in isolated environments | |
| CN112637361B (en) | Page proxy method, device, electronic equipment and storage medium | |
| CN103177115A (en) | Method and device of extracting page link of webpage | |
| US10129278B2 (en) | Detecting malware in content items | |
| CN112395485A (en) | Policy big data mining method and device, computer equipment and storage medium | |
| Mitropoulos et al. | Time present and time past: analyzing the evolution of JavaScript code in the wild | |
| CN106371987A (en) | Test method and device | |
| Khodayari et al. | It’s (dom) clobbering time: Attack techniques, prevalence, and defenses | |
| Wi et al. | Diffcsp: Finding browser bugs in content security policy enforcement through differential testing | |
| CN107026854B (en) | Vulnerability verification method and device | |
| Qu | Research on password detection technology of iot equipment based on wide area network | |
| CN112287349A (en) | Security vulnerability detection method and server | |
| Noskov | Smart City Webgis Applications: Proof of Work Concept For High-Level Quality-Of-Service Assurance |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |