CN113704760B - Page detection method and related device - Google Patents
Page detection method and related device Download PDFInfo
- Publication number
- CN113704760B CN113704760B CN202111015548.XA CN202111015548A CN113704760B CN 113704760 B CN113704760 B CN 113704760B CN 202111015548 A CN202111015548 A CN 202111015548A CN 113704760 B CN113704760 B CN 113704760B
- Authority
- CN
- China
- Prior art keywords
- page
- detection
- detected
- webpage
- browser
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 215
- 230000008859 change Effects 0.000 claims abstract description 9
- 238000000034 method Methods 0.000 claims description 23
- 238000012545 processing Methods 0.000 claims description 19
- 238000012038 vulnerability analysis Methods 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 9
- 230000006870 function Effects 0.000 claims description 8
- 235000014510 cooky Nutrition 0.000 claims description 7
- 230000001960 triggered effect Effects 0.000 claims description 7
- 230000004044 response Effects 0.000 claims description 6
- 230000003068 static effect Effects 0.000 abstract description 9
- 230000014509 gene expression Effects 0.000 abstract description 4
- 230000009286 beneficial effect Effects 0.000 abstract 1
- 230000008569 process Effects 0.000 description 14
- 238000012360 testing method Methods 0.000 description 7
- 239000000243 solution Substances 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 4
- 238000002347 injection Methods 0.000 description 4
- 239000007924 injection Substances 0.000 description 4
- 238000011161 development Methods 0.000 description 3
- VYZAMTAEIAYCRO-UHFFFAOYSA-N Chromium Chemical compound [Cr] VYZAMTAEIAYCRO-UHFFFAOYSA-N 0.000 description 2
- BUGBHKTXTAQXES-UHFFFAOYSA-N Selenium Chemical compound [Se] BUGBHKTXTAQXES-UHFFFAOYSA-N 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 2
- 230000000903 blocking effect Effects 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 230000009193 crawling Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 229910052711 selenium Inorganic materials 0.000 description 2
- 239000011669 selenium Substances 0.000 description 2
- 230000015556 catabolic process Effects 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000005242 forging Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000013515 script Methods 0.000 description 1
- 230000004083 survival effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
- G06F21/55—Detecting local intrusion or implementing counter-measures
- G06F21/56—Computer malware detection or handling, e.g. anti-virus arrangements
- G06F21/562—Static detection
- G06F21/563—Static detection by source code analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
- G06F21/57—Certifying or maintaining trusted computer platforms, e.g. secure boots or power-downs, version controls, system software checks, secure updates or assessing vulnerabilities
- G06F21/577—Assessing vulnerabilities and evaluating computer system security
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Virology (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Debugging And Monitoring (AREA)
Abstract
The application discloses a page detection method, which comprises the following steps: acquiring a page to be detected; dynamically detecting the webpage to be detected in a virtual running environment to obtain a detection result; the virtual running environment is a browser environment constructed according to a browser interface. The webpage to be detected is dynamically detected in the virtual operation environment, wherein the virtual operation change is a browser environment constructed according to a browser interface, namely, the webpage to be detected is operated in a virtual browser, and is subjected to dynamic detection instead of static detection, so that detection of dynamic expressions such as JS events and the like is realized, coverage rate of the detection of the webpage is improved, detection points are avoided from being missed, and accuracy of the detection of the webpage is improved. The application also provides a page detection device, a server and a computer readable storage medium, which have the beneficial effects.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a page detection method, a page detection device, a server, and a computer readable storage medium.
Background
With the continuous development of internet technology, more and more network vulnerability attacks are presented. Most vulnerability attacks occur on the application layer of the website, i.e., in the page. Thus, page vulnerability detection is an important ring of website site vulnerability detection.
In the related art, the page vulnerability detection is mainly implemented by crawling all url (Uniform Resource Locator ) of the website, and performing vulnerability detection (for example, SQL (Structured Query Language, structured query language) injection, xss (Cross SITE SCRIPTING, cross-site scripting attack) attack, directory traversal, command injection, cross-site request forging, etc.) on url various input points of the website. Moreover, the input points detected by the vulnerability scanner are mainly in static performances such as GET and POST parameters, access resource names, reffer, userAgent, cookie and the like. However, with the continuous development of page technology, more and more dynamic expressions exist in pages, page vulnerability detection cannot be comprehensively detected, and vulnerability detection coverage rate is insufficient, so that some vulnerability cannot be detected, and page detection accuracy is reduced.
Therefore, how to improve the accuracy of page detection is a major concern for those skilled in the art.
Disclosure of Invention
The application aims to provide a page detection method, a page detection device, a server and a computer readable storage medium, so that a page is dynamically detected in a virtual running environment, and the page detection accuracy is improved.
In order to solve the above technical problems, the present application provides a page detection method, including:
acquiring a page to be detected;
dynamically detecting the webpage to be detected in a virtual running environment to obtain a detection result; the virtual running environment is a browser environment constructed according to a browser interface.
Optionally, dynamically detecting the web page to be detected in the virtual running environment to obtain a detection result, including:
Performing dynamic detection operation on the webpage to be detected in the virtual environment to obtain the detection result; the dynamic detection operation at least comprises one or more of JS detection, DOM trigger detection and callback processing.
Optionally, when the dynamic detection operation includes the JS detection, the step of performing a dynamic detection operation on the web page to be detected in the virtual environment to obtain the detection result includes:
Loading the webpage to be detected in the virtual environment, and injecting a target JS code into the webpage to be detected;
and performing vulnerability tracking on the target JS codes to obtain tracking results and taking the tracking results as the detection results.
Optionally, when the dynamic detection operation includes the DOM trigger detection, the step of performing a dynamic detection operation on the web page to be detected in the virtual environment to obtain the detection result includes:
Loading the webpage to be detected in the virtual environment, and triggering each event in the webpage to be detected to obtain a page snapshot under each event;
and performing vulnerability analysis on all the page snapshots to obtain the detection result.
Optionally, when the dynamic detection operation includes the callback processing, the step of performing the dynamic detection operation on the web page to be detected in the virtual environment to obtain the detection result includes:
loading the webpage to be detected in the virtual environment, and adding hooks at the tail of the webpage to be detected;
and when the hook is triggered, performing vulnerability analysis on the log report of the webpage to be detected through a callback function to obtain the detection result.
Optionally, acquiring the page to be detected includes:
adding the scanned page to be detected into a task queue;
And task threads in the task thread cluster acquire the pages to be detected from the task queue according to the sequence.
Optionally, the method further comprises:
After the dynamic detection is finished, the task thread judges whether the browser environment survives;
and if not, restarting the browser environment.
The application also provides a page detection device, which comprises:
the page acquisition module is used for acquiring a page to be detected;
The dynamic detection module is used for dynamically detecting the webpage to be detected in the virtual running environment to obtain a detection result; the virtual running environment is a browser environment constructed according to a browser interface.
The application also provides a server, comprising:
A memory for storing a computer program;
A processor for implementing the steps of the page detection method as described above when executing the computer program.
The present application also provides a computer readable storage medium having stored thereon a computer program which when executed by a processor implements the steps of the page detection method as described above.
The page detection method provided by the application comprises the following steps: acquiring a page to be detected; dynamically detecting the webpage to be detected in a virtual running environment to obtain a detection result; the virtual running environment is a browser environment constructed according to a browser interface.
The webpage to be detected is dynamically detected in the virtual operation environment, wherein the virtual operation change is a browser environment constructed according to a browser interface, namely, the webpage to be detected is operated in a virtual browser, and is subjected to dynamic detection instead of static detection, so that detection of dynamic expressions such as JS events and the like is realized, coverage rate of the detection of the webpage is improved, detection points are avoided from being missed, and accuracy of the detection of the webpage is improved.
The present application also provides a page detection device, a server, and a computer-readable storage medium, which have the above advantages, and are not particularly limited herein.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only embodiments of the present application, and that other drawings can be obtained according to the provided drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flowchart of a page detection method according to an embodiment of the present application;
FIG. 2 is a flowchart of another page detection method according to an embodiment of the present application;
Fig. 3 is a schematic structural diagram of a page detection device according to an embodiment of the present application.
Detailed Description
The core of the application is to provide a page detection method, a page detection device, a server and a computer readable storage medium, so that the page is dynamically detected in a virtual running environment, and the page detection accuracy is improved.
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments of the present application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
In the related art, the page vulnerability detection is mainly implemented by crawling all url of the website and performing vulnerability detection (such as SQL injection, xs attack, directory traversal, command injection, cross-site request forgery, etc.) on all url input points of the website. Moreover, the input points detected by the vulnerability scanner are mainly in static performances such as GET and POST parameters, access resource names, reffer, userAgent, cookie and the like. However, with the continuous development of page technology, more and more dynamic expressions exist in pages, page vulnerability detection cannot be comprehensively detected, and vulnerability detection coverage rate is insufficient, so that some vulnerability cannot be detected, and page detection accuracy is reduced.
Therefore, the application provides a page detection method, which dynamically detects the webpage to be detected in a virtual operation environment, wherein the virtual operation change is a browser environment constructed according to a browser interface, namely, the webpage to be detected is operated in a virtual browser, and the dynamic detection is performed instead of only performing static detection on the webpage, so that the detection of each dynamic representation such as JS events and the like is realized, the coverage rate of the page detection is improved, the missing detection point is avoided, and the accuracy of the page detection is improved.
In order to improve the coverage rate of page detection, rather than just detecting static links in the page, the page is detected in an environment simulating real operation, and detection accuracy is improved. The following describes a page detection method provided by the present application through an embodiment.
Referring to fig. 1, fig. 1 is a flowchart of a page detection method according to an embodiment of the present application.
In this embodiment, the method may include:
S101, acquiring a page to be detected;
This step aims at acquiring the page to be detected. The page to be detected is a page that needs to be dynamically detected in this embodiment. The task thread may acquire a page to be detected.
Further, the scan engine may grab the corresponding page to be detected from the network, and then the step in this embodiment detects the page to be detected.
Also, it is understood that for convenience of description, a page to be detected may be described as a task.
Further, in order to improve efficiency of detecting multiple pages, in the alternative, task thread clusters are used to process multiple pages to be detected. Thus, this step may include:
Step 1, adding a scanned page to be detected into a task queue;
And step 2, task threads in the task thread cluster acquire pages to be detected from the task queue according to the sequence.
It can be seen that in this alternative, how to process a plurality of pages to be detected is mainly described. In the alternative scheme, a plurality of pages to be detected are obtained through a scanning engine, and then the pages to be detected are sent to a plurality of task threads through a task queue, so that each task thread can obtain the corresponding pages to be detected from the task queue and carry out detection processing.
Obviously, by sending the pages to be detected to the task threads through the task queue in the alternative scheme, the concurrent processing of the pages to be detected is realized, and the detection performance is improved. And the page to be detected can be used as a task and run in a non-blocking mode, and when the task is put into a task queue, the task thread can be called to detect the task as long as an available task thread exists.
In addition, in order to manage the execution process of each task thread and keep the page checking process running stably, the embodiment may further include:
Step 1, after the dynamic detection is finished, judging whether the browser environment survives by a task thread;
and step2, if not, restarting the browser environment.
It can be seen that this alternative is mainly described how task threads are managed. In order to avoid the browser environment which cannot be automatically started due to breakdown, whether the browser environment corresponding to each task thread survives or not is judged, and if not, the browser environment is restarted.
In addition, before the browser environment loads the page to be detected, it needs to be ensured that at least more than one window in the browser environment is opened, so as to prevent some JS from closing the access page, so that in actual operation, the browser can also switch to other available windows.
In addition, the cookie of the browser can be cleaned up, and then the browser in the browser environment can accept the preset cookie.
When a task thread is shut down, it needs to wait for it to complete if there is a running task, some errors or bots have been avoided. And stopping consuming the new task, and closing the browser environment.
When the task thread cluster is closed, traversing each task thread instance in the closed task thread cluster, and cleaning a task queue, a disk file, a callback function and a task to be processed.
S102, dynamically detecting a webpage to be detected in a virtual running environment to obtain a detection result; the virtual running environment is a browser environment constructed according to a browser interface.
On the basis of S101, the step aims at dynamically detecting the webpage to be detected in the virtual running environment to obtain a detection result; the virtual running environment is a browser environment constructed according to a browser interface. That is, the page to be detected is run in the constructed browser environment, so that the page is detected in the process of actually loading and running the page.
The process of constructing the browser environment may be to use the selenium call webdriver to merge into the chrome browser. Among them, selenium is a tool for Web application testing. The main functions include: testing compatibility with browsers-testing your application program to see if it can work well on top of different browsers and operating systems. Test system functions-create regression tests verify software functions and user requirements. Supporting automatic recording actions and automatically generating test scripts. Wherein webdriver is a browser interface provided for the test tool. The chrome browser is a browser program adopted in the embodiment.
In the prior art, the detection mode does not detect the dynamic content such as JS. Only static elements such as url in a page are detected, so that the coverage rate of detection is low. Therefore, in this embodiment, the page to be detected is dynamically detected in the virtual running environment, that is, in the detection process, the page to be detected is provided with support of DOM (Document Object Model ), JS (JavaScript, interpreted or just-in-time compiled programming language) and AJAX (Asynchronous Javascript And XML, asynchronous JavaScript and XML (Extensible Markup Language, extensible markup language)) through the browser environment, so that operations such as further triggering or skipping of the contents are realized, and finally detection is realized.
Further, in order to dynamically detect the page to be detected, that is, to trigger an event in each aspect of the page or detect the page by adopting a tracking process, the steps may include:
performing dynamic detection operation on the webpage to be detected in the virtual environment to obtain a detection result; the dynamic detection operation at least comprises one or more of JS detection, DOM trigger detection and callback processing.
It can be seen that, in this alternative solution, it is mainly explained that the dynamic detection operation performed on the web page to be detected at least includes one or more of JS detection, DOM trigger detection, and callback processing.
The JS detection refers to performing JS taint tracking on a page to be detected in a preset function environment, and analyzing a vulnerability of the web page according to a tracking result. It can be seen that the detection mainly detects the JS code in the page. Specifically, a preset JS code is injected into a page to be detected under the same scope as an introduced stain.
The DOM triggering detection refers to exploring a DOM tree of a page to be detected, acquiring page snapshots in each state, and analyzing all page snapshots explored by the page to be detected to obtain a detection result. It can be seen that the detection is mainly to detect events in the page that can be triggered, such as input events including, but not limited to JS, url of multi-element extraction, and ajax request links.
And the callback processing is to analyze according to the obtained report, log or summarized data after the page loading or running is finished, so as to obtain a detection result.
It will be appreciated that the three detection operations in this alternative may be combined arbitrarily into one or more combinations for detection. Specifically, the selection may be made according to the range or angle of dynamic detection. For example, the combination of DOM trigger detection and callback processing may be used, or the combination of JS detection, DOM trigger detection, and callback processing may be used.
Based on the alternative, when the dynamic detection operation includes JS detection, the step of performing the dynamic detection operation on the web page to be detected in the virtual environment to obtain a detection result may include:
step 1, loading a webpage to be detected in a virtual environment, and injecting a target JS code into the webpage to be detected;
and step 2, performing vulnerability tracking on the target JS codes to obtain tracking results and taking the tracking results as detection results.
It can be seen that, in this alternative solution, how to perform JS tracking detection is mainly described. In the tracking process, page snapshots generated by triggering of events and clicking of JS links are needed to be saved, and each url-opened window and an http response object are saved. Wherein the tracking process requires skipped resources such as events that have been triggered and links to clicks, etc. In addition, in the tracking process, cross-domain url needs to be ignored so as to avoid endless scanning and avoid the problems of browser environment blocking and the like.
Based on the alternative, when the dynamic detection operation includes DOM trigger detection, the step of performing the dynamic detection operation on the web page to be detected in the virtual environment to obtain a detection result may include:
step 1, loading a webpage to be detected in a virtual environment, and triggering each event in the webpage to be detected to obtain a page snapshot under each event;
And step 2, performing vulnerability analysis on all page snapshots to obtain a detection result.
It can be seen that in this alternative, it is mainly described how to perform DOM trigger detection, that is, how to perform DOM exploration and detection. Wherein DOM exploration mainly triggers events in the page in a simulated manner.
In DOM exploration, JS-related input events include, but are not limited to, change, blu, focus, select, keyup, keypress, keydown, input, etc. Also, url may be extracted from link, href, src elements. The proxy can be further arranged to intercept requests to extract ajax request links, and the maximum concurrent request number of the browser network can be controlled through a proxy mode, so that performance consumption is reduced.
Wherein the browser's DOM tree is explored and a snapshot of each state change is captured until there are no other states available. And limiting the depth of the DOM tree, traversing all elements with events, and triggering the events to capture. Wherein the real text input can be simulated and related events triggered, and the request response is captured and parsed into page elements. HTTP requests executed by Web pages (AJAX, etc.) are also captured and converted to pages to facilitate vulnerability analysis. The page snapshot dictionary is stored after the event triggers and clicks on the JS link. Event DOMs that are not related to auditable DOM forms or cookies can be deleted while traversing them to save detection time.
Based on the alternative, when the dynamic detection operation includes callback processing, the step of performing the dynamic detection operation on the webpage to be detected in the virtual environment to obtain a detection result may include:
step 1, loading a webpage to be detected in a virtual environment, and adding hooks at the tail of the webpage to be detected;
and 2, when the hook is triggered, performing vulnerability analysis on the log report of the webpage to be detected through a callback function to obtain a detection result.
It can be seen that, in this alternative, how to perform callback processing is mainly described, that is, how to perform callback processing is described. When no other detection is performed, the detection result may be obtained by analyzing the log data after the page is loaded or operated in the alternative. When other detection is performed, the detection result may be obtained by obtaining the result of the other detection and analyzing the result.
In summary, the embodiment dynamically detects the web page to be detected in the virtual running environment, wherein the virtual running change is a browser environment constructed according to a browser interface, that is, the web page to be detected is run in a virtual browser, and the dynamic detection is performed instead of merely performing static detection on the web page, so that detection on each dynamic representation such as a JS event is realized, coverage rate of the web page detection is improved, missing detection points are avoided, and accuracy of the web page detection is improved.
In order to further improve the accuracy of detecting the page in the embodiment, the efficiency of the detection process is improved, and the operation problem is avoided. The page detection method provided by the application is further described below through a specific embodiment.
Referring to fig. 2, fig. 2 is a flowchart of another page detection method according to an embodiment of the application.
In this embodiment, vulnerability detection is performed on scanned pages to be detected by a task thread cluster, and performing operations including JS detection, DOM trigger detection, and callback processing on each page to be detected, where the method may include:
s201, adding the scanned page to be detected into a task queue;
s202, task threads in a task thread cluster acquire pages to be detected from a task queue according to the sequence;
S203, dynamically detecting the webpage to be detected in the virtual running environment to obtain a detection result; the virtual running environment is a browser environment constructed according to a browser interface; the dynamic detection operation comprises JS detection, DOM trigger detection and callback processing.
Therefore, in this embodiment, JS detection, DOM trigger detection and callback processing are performed on the web page to be detected in the virtual running environment, and a corresponding detection result is finally obtained.
The task thread corresponding to each browser has the maximum survival time, so that the browser is prevented from processing a large amount of task memory leakage, and the task thread has the maximum number of retries for some overtime tasks. After the browser loads the page, a timeout task occurs too long for the target element to appear on the page.
And running the task through a browser of the task thread, and finally generating a page generated by triggering an event, clicking a JavaScript link and capturing an AJAX request. Since the browser may sometimes crash, checks ensure that the browser is alive before or after each job task is run. If not, the new browser environment is restarted in time. And WebDriver errors occur in the running process, and the browser also needs to be restarted in time to ensure the stability of the detection process.
Therefore, in this embodiment, the dynamic detection is performed on the web page to be detected in the virtual running environment, where the virtual running change is a browser environment configured according to the browser interface, that is, the web page to be detected is run in the virtual browser, and the dynamic detection is performed, instead of merely performing the static detection on the web page, so that the detection on each dynamic representation such as the JS event is realized, the coverage rate of the web page detection is improved, the detection point is avoided from being omitted, and the accuracy of the web page detection is improved.
The following describes a page detection device provided in an embodiment of the present application, and the page detection device described below and the page detection method described above may be referred to correspondingly.
Referring to fig. 3, fig. 3 is a schematic structural diagram of a page detection device according to an embodiment of the application.
In this embodiment, the apparatus may include:
the page acquisition module 100 is configured to acquire a page to be detected;
the dynamic detection module 200 is configured to dynamically detect a webpage to be detected in a virtual running environment, so as to obtain a detection result; the virtual running environment is a browser environment constructed according to a browser interface.
Optionally, the dynamic detection module 200 is specifically configured to perform a dynamic detection operation on a web page to be detected in a virtual environment, so as to obtain a detection result; the dynamic detection operation at least comprises one or more of JS detection, DOM trigger detection and callback processing.
Optionally, the dynamic detection module 200 is specifically configured to load a webpage to be detected in a virtual environment, and inject a target JS code into the webpage to be detected; and performing vulnerability tracking on the target JS codes to obtain tracking results and taking the tracking results as detection results.
Optionally, the dynamic detection module 200 is specifically configured to load a webpage to be detected in a virtual environment, and trigger each event in the webpage to be detected to obtain a page snapshot under each event; and performing vulnerability analysis on all page snapshots to obtain a detection result.
Optionally, the dynamic detection module 200 is specifically configured to load a webpage to be detected in a virtual environment, and add a hook at the end of the webpage to be detected; and when the hook is triggered, performing vulnerability analysis on the log report of the webpage to be detected through the callback function to obtain a detection result.
Optionally, the page obtaining module 100 is specifically configured to add the scanned page to be detected to a task queue; task threads in the task thread cluster acquire pages to be detected from the task queue according to the sequence.
Optionally, the apparatus may further include:
the maintenance module is used for judging whether the browser environment survives or not by the task thread after the dynamic detection is finished; if not, restarting the browser environment.
The embodiment of the application also provides a server, which comprises:
A memory for storing a computer program;
A processor for implementing the steps of the page detection method as described above when executing the computer program.
Embodiments of the present application also provide a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the page detection method as described above.
In the description, each embodiment is described in a progressive manner, and each embodiment is mainly described by the differences from other embodiments, so that the same similar parts among the embodiments are mutually referred. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative elements and steps are described above generally in terms of functionality in order to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. The software modules may be disposed in Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
The page detection method, the page detection device, the server and the computer readable storage medium provided by the application are described in detail above. The principles and embodiments of the present application have been described herein with reference to specific examples, the description of which is intended only to facilitate an understanding of the method of the present application and its core ideas. It should be noted that it will be apparent to those skilled in the art that various modifications and adaptations of the application can be made without departing from the principles of the application and these modifications and adaptations are intended to be within the scope of the application as defined in the following claims.
Claims (9)
1. A method for detecting a page, comprising:
Acquiring a webpage to be detected;
Performing dynamic detection operation on the webpage to be detected in a virtual environment to obtain a detection result; wherein the virtual environment is a browser environment constructed according to a browser interface;
When the dynamic detection operation comprises DOM trigger detection, loading the webpage to be detected in the virtual environment, and triggering each event in the webpage to be detected to obtain a page snapshot under each event; performing vulnerability analysis on all the page snapshots to obtain the detection result; exploring the DOM tree of the browser and capturing a page snapshot of each state change until no other states are available; limiting the depth of the DOM tree, traversing all elements with events, and triggering the events to capture; simulating real text input and triggering related events, capturing a request response and analyzing the request response into page elements; capturing HTTP requests executed by the Web page and converting the HTTP requests into pages; storing a page snapshot dictionary after triggering an event and clicking a JS link; events not related to auditable DOM forms or cookies are deleted while traversing them.
2. The page detection method of claim 1, wherein the dynamic detection operation further comprises at least a combination of one or more of JS detection and callback processing.
3. The page detection method as recited in claim 2, wherein when the dynamic detection operation includes the JS detection, the step of performing a dynamic detection operation on the web page to be detected in the virtual environment to obtain the detection result includes:
Loading the webpage to be detected in the virtual environment, and injecting a target JS code into the webpage to be detected;
and performing vulnerability tracking on the target JS codes to obtain tracking results and taking the tracking results as the detection results.
4. The page detection method according to claim 2, wherein when the dynamic detection operation includes the callback processing, the step of performing a dynamic detection operation on the web page to be detected in the virtual environment to obtain the detection result includes:
loading the webpage to be detected in the virtual environment, and adding hooks at the tail of the webpage to be detected;
and when the hook is triggered, performing vulnerability analysis on the log report of the webpage to be detected through a callback function to obtain the detection result.
5. The method for detecting a page according to any one of claims 1 to 4, wherein acquiring the web page to be detected includes:
adding the scanned webpage to be detected into a task queue;
And task threads in the task thread cluster acquire the webpage to be detected from the task queue according to the sequence.
6. The page detection method as recited in claim 5, further comprising:
After the dynamic detection is finished, the task thread judges whether the browser environment survives;
and if not, restarting the browser environment.
7.A page detection apparatus, characterized by comprising:
the page acquisition module is used for acquiring a webpage to be detected;
The dynamic detection module is used for executing dynamic detection operation on the webpage to be detected in the virtual environment to obtain a detection result; wherein the virtual environment is a browser environment constructed according to a browser interface;
When the dynamic detection operation comprises DOM trigger detection, loading the webpage to be detected in the virtual environment, and triggering each event in the webpage to be detected to obtain a page snapshot under each event; performing vulnerability analysis on all the page snapshots to obtain the detection result; exploring the DOM tree of the browser and capturing a page snapshot of each state change until no other states are available; limiting the depth of the DOM tree, traversing all elements with events, and triggering the events to capture; simulating real text input and triggering related events, capturing a request response and analyzing the request response into page elements; capturing HTTP requests executed by the Web page and converting the HTTP requests into pages; storing a page snapshot dictionary after triggering an event and clicking a JS link; events not related to auditable DOM forms or cookies are deleted while traversing them.
8. A server, comprising:
A memory for storing a computer program;
A processor for implementing the steps of the page detection method according to any one of claims 1 to 6 when executing the computer program.
9. A computer readable storage medium, characterized in that the computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements the steps of the page detection method according to any of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111015548.XA CN113704760B (en) | 2021-08-31 | 2021-08-31 | Page detection method and related device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111015548.XA CN113704760B (en) | 2021-08-31 | 2021-08-31 | Page detection method and related device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113704760A CN113704760A (en) | 2021-11-26 |
| CN113704760B true CN113704760B (en) | 2024-05-24 |
Family
ID=78658278
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111015548.XA Active CN113704760B (en) | 2021-08-31 | 2021-08-31 | Page detection method and related device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113704760B (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2012166113A1 (en) * | 2011-05-31 | 2012-12-06 | Hewlett-Packard Development Company, L.P. | Automated security testing |
| US8752183B1 (en) * | 2012-07-10 | 2014-06-10 | Hoyt Technologies, Inc. | Systems and methods for client-side vulnerability scanning and detection |
| CN104836779A (en) * | 2014-02-12 | 2015-08-12 | 携程计算机技术（上海）有限公司 | XSS vulnerability detection method, system and Web server |
| CN106022135A (en) * | 2016-02-23 | 2016-10-12 | 北京工业大学 | Automatic detection system capable of dynamically determining XSS vulnerability |
| CN106228067A (en) * | 2016-07-15 | 2016-12-14 | 江苏博智软件科技有限公司 | Malicious code dynamic testing method and device |
| CN109189686A (en) * | 2018-08-30 | 2019-01-11 | 中国平安人寿保险股份有限公司 | Automation regression testing method, apparatus, storage medium and computer equipment |
| CN110716973A (en) * | 2019-09-23 | 2020-01-21 | 杭州安恒信息技术股份有限公司 | Big data based security event reporting platform and method |
| CN111212055A (en) * | 2019-12-30 | 2020-05-29 | 上海安洵信息技术有限公司 | Non-invasive website remote detection system and detection method |
| CN111262839A (en) * | 2020-01-09 | 2020-06-09 | 深信服科技股份有限公司 | Vulnerability scanning method, management equipment, node and storage medium |
| CN111949903A (en) * | 2020-08-28 | 2020-11-17 | 杭州安恒信息技术股份有限公司 | Webpage data acquisition method, device and equipment and readable storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9317694B2 (en) * | 2013-12-03 | 2016-04-19 | Microsoft Technology Licensing, Llc | Directed execution of dynamic programs in isolated environments |
-
2021
- 2021-08-31 CN CN202111015548.XA patent/CN113704760B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2012166113A1 (en) * | 2011-05-31 | 2012-12-06 | Hewlett-Packard Development Company, L.P. | Automated security testing |
| US8752183B1 (en) * | 2012-07-10 | 2014-06-10 | Hoyt Technologies, Inc. | Systems and methods for client-side vulnerability scanning and detection |
| CN104836779A (en) * | 2014-02-12 | 2015-08-12 | 携程计算机技术（上海）有限公司 | XSS vulnerability detection method, system and Web server |
| CN106022135A (en) * | 2016-02-23 | 2016-10-12 | 北京工业大学 | Automatic detection system capable of dynamically determining XSS vulnerability |
| CN106228067A (en) * | 2016-07-15 | 2016-12-14 | 江苏博智软件科技有限公司 | Malicious code dynamic testing method and device |
| CN109189686A (en) * | 2018-08-30 | 2019-01-11 | 中国平安人寿保险股份有限公司 | Automation regression testing method, apparatus, storage medium and computer equipment |
| CN110716973A (en) * | 2019-09-23 | 2020-01-21 | 杭州安恒信息技术股份有限公司 | Big data based security event reporting platform and method |
| CN111212055A (en) * | 2019-12-30 | 2020-05-29 | 上海安洵信息技术有限公司 | Non-invasive website remote detection system and detection method |
| CN111262839A (en) * | 2020-01-09 | 2020-06-09 | 深信服科技股份有限公司 | Vulnerability scanning method, management equipment, node and storage medium |
| CN111949903A (en) * | 2020-08-28 | 2020-11-17 | 杭州安恒信息技术股份有限公司 | Webpage data acquisition method, device and equipment and readable storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 基于DOM树序列值比对的SQL注入漏洞检测;罗明宇等;《计算机工程与设计》;20150228;第36卷(第2期);第350-354页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113704760A (en) | 2021-11-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105553917B (en) | Method and system for detecting webpage bugs | |
| US20060190561A1 (en) | Method and system for obtaining script related information for website crawling | |
| US8752183B1 (en) | Systems and methods for client-side vulnerability scanning and detection | |
| US9235640B2 (en) | Logging browser data | |
| EP2891100B1 (en) | Security scan based on dynamic taint | |
| CN110765464B (en) | Vulnerability detection method, device, equipment and computer storage medium | |
| US10699017B2 (en) | Determining coverage of dynamic security scans using runtime and static code analyses | |
| US20170316202A1 (en) | Rasp for scripting languages | |
| CN104980309A (en) | Website security detecting method and device | |
| CN101490685A (en) | A method for increasing the security level of a user machine browsing web pages | |
| Huang et al. | UChecker: Automatically detecting php-based unrestricted file upload vulnerabilities | |
| US7496636B2 (en) | Method and system for resolving Universal Resource Locators (URLs) from script code | |
| EP3104279B1 (en) | Testing interactive network systems | |
| KR20180075881A (en) | Method and Apparatus for Analyzing Web Vulnerability for Client-side | |
| CN106599270B (en) | Network data capturing method and crawler | |
| Choi et al. | HXD: Hybrid XSS detection by using a headless browser | |
| CN114666104A (en) | Penetration testing method, system, computer equipment and storage medium | |
| US9098704B2 (en) | Method for function capture and maintaining parameter stack | |
| CN114238978A (en) | Vulnerability scanning system, vulnerability scanning method and computer equipment | |
| CN114491560A (en) | Vulnerability detection method and device, storage medium and electronic equipment | |
| CN113704760B (en) | Page detection method and related device | |
| Tatli et al. | WIVET—benchmarking coverage qualities of web crawlers | |
| CN110691005A (en) | Website monitoring system and method | |
| CN112507346A (en) | Vulnerability scanning system | |
| CA2538504C (en) | Method and system for obtaining script related information for website crawling |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |