A kind of evidence obtaining type website vulnerability scanning method and system
Technical field
The invention belongs to network application security fields, relate in particular to a kind of evidence obtaining type website vulnerability scanning method and system.
Background technology
Along with the development of internet, diverse network application emerges in an endless stream, and has met dramatically user's various demands.But, in Network application and development process, often more focus on the realization in function, but leave over lower various safety problem, this is generally because progress pressure, programmer's horizontal constraints, design imperfection or inadvertent cause.Common web portal security problem comprises: the leak that code causes the filtration deficiency of input data, the leak that server configuration aspect causes, sensitive information leakage etc.General very difficult these leaks of stopping completely in application development process.Therefore adopting website hole scanner to carry out safety assessment to website is a well selection.
Website vulnerability scanning system generally captures all pages in website, then the page grabbing is analyzed one by one, finds safety problem as much as possible.Yet several factors cause such scan mode to be very easy to cause wrong report, report some in fact also non-existent leaks, Website development and maintainer are caused to the waste in time and efforts.These factors comprise: the 1) dynamic of a lot of webpages, causes website vulnerability scanning system misjudgment; 2) the decision logic imperfection of website vulnerability scanning system itself; 3) current applied environment is very complicated, and the decision logic of website vulnerability scanning system often cannot cover situation about likely occurring.
In existing known technology, the technology comparatively approaching with this method is a kind of " the Web vulnerability scanning method based on infiltration technology " by name, and its committed step is " comprise scanning and analyze two stages ".This technology has mainly realized the method that website leak is found.But as previously described, such method easily produces wrong report, but the method that does not provide good filtration to report by mistake.Staff can adopt and scan in this way some leaks, but may a part be really to exist, and a part is wrong report.They can only select to tolerate these wrong reports, or manually leak are analyzed again one by one.
Summary of the invention
The object of the invention is to make up the deficiency of website using vulnerability scanning method, a kind of vulnerability scanning method and scanning system are provided, adopt automatic and manual evidence collecting method, improve the accuracy rate of website vulnerability scanning system scanning, submit more accurate believable website vulnerability scanning result to.
A kind of evidence obtaining type website vulnerability scanning method is provided, for carrying out Hole Detection to being scanned website, comprises the following steps:
A) page captures: from being scanned the initial page of website, start to resolve, obtain the page link that is scanned website, then page link is deposited in to system data library module, and guarantee that same page link does not repeat to deposit in, from system data library module, extract again the page link of having preserved and do not process through page crawl step, carry out page access, and extract new page link and be deposited into system data library module, until captured all pages that are scanned website;
B) vulnerability scanning: in the page grabbing at step a, judge whether not yet to carry out in addition the page of Hole Detection, if not yet do not carry out the page of Hole Detection, go to step e, otherwise carry out: select a page that not yet carries out Hole Detection to carry out Hole Detection, for this page, according to different leaks separately corresponding detection logic carry out leak analysis (kind of leak be very many, every kind of leak has different detection methods, it is known that part correlation technique has belonged to industry), if find that leak goes to step c, otherwise repeated execution of steps b;
C) automatic evidence-collecting: to the leak of finding in step b, carry out automatic evidence-collecting according to the evidence obtaining logic of different leaks, and automatic fitration falls wrong report, acquisition can prove the evidence obtaining result that this leak exists;
D) information: the leak of finding according to step b, the manual evidence collecting method that can take for different leaks, automatically collects the needed evidence obtaining information of manual evidence collecting method and preserves, and goes to step b other pages are carried out to Hole Detection after completing;
E) manual evidence obtaining: the evidence obtaining information of using steps d to collect, the corresponding manual evidence collecting method that can take according to different leaks carries out manual evidence obtaining, and obtains for confirming the manual evidence obtaining result of the authenticity of leak.
In the present invention, according to dissimilar website leak, evidence obtaining result in described step c and step e comprises following data: page length, http response code, the page request response time, the message of page request process transmitting-receiving, HTTP head, session and Cookie, content of pages, otherness between a plurality of page request is (as the difference of page response code, the difference of page length, the difference of content etc.), (part leak can be used browser to collect evidence for the actual displayed effect of the page in browser and sectional drawing thereof, it is different that different leaks is inputted in browser, such as " cross site scripting " can be revised the title at interface or eject a dialog box, the display effect of " catalogue browsing " is all sub-directories and the son file showing in browser under a catalogue, different situations also has a lot, will not enumerate).
In the present invention, the evidence obtaining information in described steps d comprises following data: URL, HTTP head, session and Cookie, submission of sheet data, HTTP request data package, the http response packet for further evidence obtaining of automatically constructing.
In the present invention, the automatic evidence-collecting in described step c comprises following methods:
1) directly using the response contents length of the leak place page, answer code, response time, message, content of pages as evidence obtaining result, and judge with this whether leak exists;
2) on the basis of parent page request, modify, automatically construct at least one new http request, and the data (comprising response contents length, answer code, response time, message, content of pages) of these different requests are contrasted, using the otherness of contrast acquisition as evidence obtaining result, and judge with this whether leak exists;
3) in the returned content of the leak place page, mate the text that meets feature, using the content matching as evidence obtaining result, and judge with this whether leak exists;
4) automatic imitation submission of sheet obtain request results, and judge with this whether leak exists;
5) use browser kernel and browser to resolve and typesetting the content of the page, carry out the script in content of pages, the result using the Output rusults of parsing and typesetting as evidence obtaining, and judge with this whether leak exists.
In the present invention, the craft of described step e evidence obtaining is optional, can optionally perform step as required e (manual evidence obtaining, can be more accurate than automatic evidence-collecting in people's presence); Described manual evidence obtaining comprises following methods:
1) use the URL for further evidence obtaining of the automatic structure obtaining in steps d, in browser, access, check that actual URL carries out and shows result, in conjunction with the feature of leak, judge whether leak exists, and can will show the preservation of result sectional drawing;
2) use HTTP to simulate the device of giving out a contract for a project, by the URL for further collecting evidence, and the HTTP head, session and the Cookie that in steps d, obtain, submission of sheet data message, inserting HTTP simulation gives out a contract for a project in device, and send HTTP request, check the content of pages returning, in conjunction with the feature of leak, judge whether leak exists;
3) in browser, by the submission of sheet data of obtaining in steps d, insert in the list of the leak place page, and submission form, check that the page after submission form shows result, in conjunction with the feature of leak, judge whether leak exists;
4) utilize specific purpose tool, described specific purpose tool refers to the instrument of leak evidence obtaining or penetration testing, by HTTP head, session and the Cookie, the submission of sheet data message that obtain in the URL of further evidence obtaining, steps d, insert further evidence obtaining or infiltration in specific purpose tool, and check result, in conjunction with the feature of leak, judge whether leak exists;
5) the HTTP request data package of obtaining in steps d and http response packet are presented in user interface, and use highlighted, font-weight method that important content is highlighted, in conjunction with the feature of leak, whether artificial judgment leak exists.
In the present invention, website leak and evidence obtaining result can be simultaneously displayed in the user interface of program or system, or output in form simultaneously; User judges according to these information whether leak exists, and the character of leak and harmfulness.
In the present invention, described a kind of evidence obtaining type website vulnerability scanning method has learning functionality: user judges whether leak is reported by mistake, system is carried out record to user's judged result, while again there is identical leak in follow-up scanning, according to the user's judged result in record, filter out the leak of wrong report, i.e. wrong report is fallen in automatic fitration in performing step c.
Evidence obtaining type website based on described scan method vulnerability scanning system is provided, comprises page handling module, vulnerability scanning module, automatic evidence-collecting module, information module, manual evidence obtaining module, task issues and administration module, scanning result demonstration and administration module, Reports module, system data library module, item file administration module;
Described page handling module is used for from being scanned the initial page of website or the page link that system data library module is preserved, new page link is obtained in parsing, then the page link obtaining is deposited in system data library module, and guarantee that same page link does not repeat to deposit in (the system data library module in page handling module also can replace with item file);
Described vulnerability scanning module is used for judging whether the page has through Hole Detection, and detects not passing through the page of Hole Detection;
Described automatic evidence-collecting module is carried out automatic evidence-collecting for leak that vulnerability scanning module is found, and automatic fitration falls wrong report, and acquisition can prove the evidence obtaining result that this leak exists;
The leak of described information module for vulnerability scanning module is found, collects and preserves the evidence obtaining information for manual evidence obtaining automatically;
The evidence obtaining information of described manual evidence obtaining module for utilizing information module to collect, carries out manual evidence obtaining, and obtains for confirming the manual evidence obtaining result of the authenticity of leak;
Described task issues and administration module starts the operational means of scans web sites process for offering user, reach the final purpose that leak is found and collected evidence, by supporting that user issues scan task on user interface, and it is managed, comprise time-out, stop, deleting, configure;
Described scanning result shows and administration module is used in scanning process and after having scanned, scan-data is presented on interface, and support user to check scan-data, manual evidence obtaining result and output report, scan-data comprises the leak of discovery, the page link of corresponding website, manual evidence obtaining information;
Described Reports module is for the result of Hole Detection is outputed to unique file, and unique file is as the interchange between staff, file or as the foundation of improving web station system, unique file can be directed in third party's system and further process;
Described system data library module is for the configuration information of storage system, the leak of website, page link, and the configuration information of system comprises account and log information;
Described item file administration module, for item file is managed, comprises establishment, deletion, data writing; Item file refers to the file of the link of energy memory page and leak data, and the task that each task issues and administration module issues, has a corresponding item file.
Principle of the present invention is: after scanning leak, utilize the feature of leak itself and the return data of the page, this leak is carried out to checking again or repeatedly, get rid of the leak of wrong report.
Compared with prior art, the invention has the beneficial effects as follows:
The invention provides a kind of website vulnerability scanning method that accuracy rate is higher, allow staff can produce like a cork the leak report of the more believable website of portion, without the authenticity that re-uses other means and judge leak, can increase work efficiency to a great extent.
Evidence obtaining type scan method can automatically be filtered leak.Adopt in this way, user can just take the more believable report of portion when autoscan finishes.For still there is individually the leak of doubtful point, can adopt " manual evidence obtaining " step described in this method, directly rapidly leak is reaffirmed.
Accompanying drawing explanation
Fig. 1 is the main module structure figure of the evidence obtaining type website vulnerability scanning system in the present invention.
Fig. 2 is the flow process of the evidence obtaining type website vulnerability scanning method in the present invention.
Embodiment
First it should be noted that, the present invention relates to Web and database technology, is that computer technology is in a kind of application of field of information security technology.In implementation procedure of the present invention, can relate to the application of a plurality of software function modules.Applicant thinks, as reading over application documents, accurate understanding is of the present invention realize principle and goal of the invention after, in conjunction with existing known technology in the situation that, those skilled in the art can use the software programming technical ability of its grasp to realize the present invention completely.Aforementioned software functional module comprises but is not limited to: page handling module, vulnerability scanning module, automatic evidence-collecting module, information module, manual evidence obtaining module, task issues and administration module, scanning result demonstration and administration module, Reports module, system data library module, item file administration module etc., this category of all genus that all the present patent application files are mentioned, applicant will not enumerate.
The object of this invention is to provide a kind of website vulnerability scanning method that accuracy rate is higher.By the method for evidence obtaining, filter the leak of wrong report.Evidence obtaining mainly comprises automatic evidence-collecting and two steps of manual evidence obtaining, and wherein automatic evidence-collecting automatically completes in scanning process.
Evidence obtaining type website vulnerability scanning method specifically comprises the following steps:
A) page captures: from being scanned the initial page of website, start to resolve, obtain the page link that is scanned website, then page link is deposited in to system data library module, and guarantee that same page link does not repeat to deposit in, from system data library module, extract again the page link of having preserved and do not process through page crawl step, carry out page access, and extract new page link and be deposited into system data library module, until captured all pages that are scanned website.Can Adoption Network reptile, the various ways such as regular expression, simulation parsing, or various ways combines to carry out, and also can adopt existing web crawlers of increasing income at present, if Labin, Nurch etc. is the web crawlers project of increasing income that popularity is higher.
B) vulnerability scanning: in the page grabbing at step a, judge whether not yet to carry out in addition the page of Hole Detection, if not yet do not carry out the page of Hole Detection, go to step e, otherwise carry out: select a page that not yet carries out Hole Detection to carry out Hole Detection, for this page, according to different leaks separately corresponding detection logic carry out leak analysis (kind of leak be very many, every kind of leak has different detection methods, it is known that part correlation technique has belonged to industry), if find that leak goes to step c, otherwise repeated execution of steps b.Due to the diversity of leak, the present invention adopts policy library to realize writing of Hole Detection logic, and policy library provides unified Hole Detection framework, and every leak is written as respectively to a strategy in policy library.When user issues task, can select to want which strategy in implementation strategy storehouse.In scanning process, for each page, all can carry out user-selected All Policies one time.In the present invention, current step should be found leak as much as possible, and minimizing is failed to report.In the leak method of website, report by mistake and fail to report conflict often, need in scanning strategy, make balance.But in this step, needing top-priority is to reduce rate of failing to report.
C) automatic evidence-collecting: to the leak of finding in step b, carry out automatic evidence-collecting according to the evidence obtaining logic of different leaks, and automatic fitration falls wrong report, acquisition can prove the evidence obtaining result that this leak exists.Automatic evidence-collecting automatically completes in scanning process, after every strategy execution in described policy library completes, if the leak that discovery may exist, just start the process of automatic fitration and evidence obtaining, if be judged as wrong report, this result is dropped automatically, otherwise deposits in database.The foundation of automatic evidence-collecting and the possibility of result include but not limited to following information: message, HTTP head, session and the Cookie of page length, http response code, page request response time, the transmitting-receiving of page request process, content of pages etc., can also comprise the otherness between a plurality of page request in addition, as difference of the difference of the difference of page response code, page length and content etc.Except the content of datumization, evidence obtaining result can also comprise actual displayed effect and the sectional drawing thereof of the page in browser, part leak can be used browser to collect evidence, it is different that different leaks is inputted in browser, such as " cross site scripting " can be revised the title at interface or eject a dialog box, the display effect of " catalogue browsing " is all sub-directories and the son file showing in browser under a catalogue, and different situations also has a lot, will not enumerate.Automatic evidence-collecting includes but not limited to following methods: 1) directly using the response contents length of the leak place page, answer code, response time, message, content of pages as evidence obtaining result, and judge with this whether leak exists; 2) on the basis of parent page request, modify, automatically the one or more new http requests of structure, and the data such as the response contents length of these different requests, answer code, response time, message, content of pages are contrasted, using the otherness of contrast acquisition as evidence obtaining result, and judge with this whether leak exists; 3) in the returned content of the leak place page, mate the text that meets feature, using the content matching as evidence obtaining result, and judge with this whether leak exists; 4) automatic imitation submission of sheet obtain request results, and judge with this whether leak exists.5) use browser kernel and browser to resolve and typesetting the content of the page, carry out the script in content of pages.Result using the Output rusults of parsing and typesetting as evidence obtaining, and judge with this whether leak exists.
D) information: the leak of finding according to step b, the corresponding manual evidence collecting method that can take for this leak, by needed evidence obtaining information preservation, goes to step b other pages are carried out to Hole Detection after completing.Relevant Data Source can come from the page itself, the HTTP packet of transmitting-receiving and the result of program automatic analysis.For the leak that may exist having been found that, after automatic fitration, due to the complicacy of detection technique, still can not guarantee accuracy rate completely, therefore may exist and need human assistance to judge.After described Hole Detection strategy completes, relevant information is collected automatically, and these information will be used as source and the foundation of manual evidence obtaining.User, in needs, can judge according to these information the authenticity of leak, and by judged result storage or output.The information that may be collected includes but not limited to: URL, HTTP head, session and the Cookie for further evidence obtaining automatically constructing, submission of sheet data, HTTP request data package, http response packet etc.
E) manual evidence obtaining: the evidence obtaining information of using steps d to collect, the corresponding manual evidence collecting method that can take according to this leak carries out manual evidence obtaining, and obtains for confirming the manual evidence obtaining result of the authenticity of leak.Manual evidence obtaining is mainly used for manual operations, visual inspection is main, so information except obtaining in described steps d, in system of the present invention, also should comprise some instruments, facilitate user's manual operation, as: browser, simulation based on http protocol give out a contract for a project device, leak verification tool, Web application penetration testing instrument, network packet captures and analytical tool etc.Also relevant information should be presented in user interface, be convenient to user and check.According to dissimilar leak, manual evidence obtaining also should be used diverse ways.Operable method includes but not limited to: 1) use the URL for further collecting evidence of the automatic structure obtaining in steps d, in browser, access, check that actual URL carries out and shows result, in conjunction with the feature of leak, judges whether leak exists.Can the preservation of result sectional drawing will be shown simultaneously; 2) use HTTP to simulate the device of giving out a contract for a project, by the URL for further collecting evidence, and the information such as the HTTP head, session and the Cookie that obtain in steps d, submission of sheet data, inserting HTTP simulation gives out a contract for a project in device, and send HTTP request, check the content of pages returning, in conjunction with the feature of leak, judge whether leak exists; 3) in browser, by the submission of sheet data of obtaining in steps d, insert in the list of the leak place page, and submission form, check that the page after submission form shows result, in conjunction with the feature of leak, judge whether leak exists; 4) utilize other evidence obtaining of special-purpose leak or penetration testing instruments, for the URL further collecting evidence, and the information such as the HTTP head, session and the Cookie that obtain in steps d, submission of sheet data, insert further evidence obtaining or infiltration in specific purpose tool, and check result, in conjunction with the feature of leak, judge whether leak exists; 5) the HTTP request data package of obtaining in steps d and http response packet are presented in user interface, and use the methods such as highlighted, font-weight that important content is highlighted, in conjunction with the feature of leak, whether artificial judgment leak exists.
The craft evidence obtaining of step e is optional, if user is still lack confidence to the leak of reporting in form, can select as required to perform step e.
Evidence obtaining type website vulnerability scanning method, can be simultaneously displayed on website leak and evidence obtaining result in the user interface of program or system, or output in form simultaneously; User judges according to these information whether leak exists, and character and the harmfulness of judgement leak.Evidence obtaining type website vulnerability scanning method, also there is learning functionality: user judges whether leak is reported by mistake, system is carried out record to user's judged result, while again there is identical leak in follow-up scanning, filters out the leak of wrong report according to the user's judged result in record.
Evidence obtaining type website vulnerability scanning system based on described scan method, can include but not limited to lower module: page handling module, vulnerability scanning module, automatic evidence-collecting module, information module, manual evidence obtaining module, task issues and administration module, scanning result demonstration and administration module, Reports module, system data library module, item file administration module etc.
Below each module is further illustrated:
1) page handling module is used for from being scanned the initial page of website or the page link that system data library module is preserved, new page link is obtained in parsing, then the page link obtaining is deposited in system data library module, and guaranteeing that same page link does not repeat to deposit in, database described here also can replace with item file.
2) vulnerability scanning module is used for judging whether the page has through Hole Detection, and detects not passing through the page of Hole Detection, and different leaks has different detection methods.
3) automatic evidence-collecting module is carried out automatic evidence-collecting for leak that vulnerability scanning module is found, and automatic fitration falls wrong report, and acquisition can prove the evidence obtaining result that this leak exists.
4) leak of information module for vulnerability scanning module is found, collects and preserves the evidence obtaining information for manual evidence obtaining automatically.
5) the evidence obtaining information of manual evidence obtaining module for utilizing information module to collect, carries out manual evidence obtaining, and obtains for confirming the manual evidence obtaining result of the authenticity of leak.About the operation of this module, user is optional, can carry out as required.
6) task issues and administration module refers to that user issues scan task on user interface, and it is managed, as suspended, stop, deletion, configuration etc.Its objective is and offer the operational means that user starts scans web sites process, reach the final purpose that leak is found and collected evidence.
7) scanning result demonstration and administration module refer in scanning process and after having scanned, and by contents such as the leak of finding, the page link of corresponding website, manual evidence obtaining information, are presented on interface.User can check related data, manual evidence obtaining and output report.
8) Reports module refers to the result of Hole Detection is outputed to independently in file, for the interchange between staff, file or as the foundation of improving web station system, the file of output also can be directed in third party's system for further processing.
9) system data library module is for the configuration information of storage system, account and log information etc., the information such as leak that also can store website and page link.
10) item file administration module, wherein item file refer to can memory page the file of the data such as link and leak, the task that each task issues and administration module issues, can have a corresponding item file.Described item file administration module refers to the module that item file is managed, as establishment, deletion, data writing etc.
In system, user's scanning process is as follows: user first issue task and etc. to be scanned completing.In scanning process or after the end of scan, user can check scanning result, carries out manual evidence obtaining.Finally as required scanning result and evidence obtaining result are outputed to form.
By evidence obtaining type website vulnerability scanning method, in the vulnerability scanning system of evidence obtaining type website, realized, just can be completed the function of evidence obtaining type website vulnerability scanning.
Finally, it should be noted that above what enumerate is only specific embodiments of the invention.Obviously, the invention is not restricted to above embodiment, can also have a lot of distortion.All distortion that those of ordinary skill in the art can directly derive or associate from content disclosed by the invention, all should think protection scope of the present invention.