CN113849825A - Testing method and device for multi-bug scanner - Google Patents

Testing method and device for multi-bug scanner Download PDF

Info

Publication number
CN113849825A
CN113849825A CN202111268004.4A CN202111268004A CN113849825A CN 113849825 A CN113849825 A CN 113849825A CN 202111268004 A CN202111268004 A CN 202111268004A CN 113849825 A CN113849825 A CN 113849825A
Authority
CN
China
Prior art keywords
application
cve
cves
determining
vulnerability scanner
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111268004.4A
Other languages
Chinese (zh)
Inventor
刘敏
高学玲
刘丽君
赵博
周黎
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nsfocus Technologies Inc
Nsfocus Technologies Group Co Ltd
Original Assignee
Nsfocus Technologies Inc
Nsfocus Technologies Group Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nsfocus Technologies Inc, Nsfocus Technologies Group Co Ltd filed Critical Nsfocus Technologies Inc
Priority to CN202111268004.4A priority Critical patent/CN113849825A/en
Publication of CN113849825A publication Critical patent/CN113849825A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
    • G06F21/57—Certifying or maintaining trusted computer platforms, e.g. secure boots or power-downs, version controls, system software checks, secure updates or assessing vulnerabilities
    • G06F21/577—Assessing vulnerabilities and evaluating computer system security

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Hardware Design (AREA)
  • Computer Security & Cryptography (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Computing Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Stored Programmes (AREA)

Abstract

The application relates to the technical field of network security, in particular to a testing method and a testing device of a multi-vulnerability scanner, which respectively aim at each application which supports scanning of each vulnerability scanner and determine the number of samples corresponding to one application according to the number of CVEs (composite video edge) and the total number of the CVEs of the one application; randomly sampling a preset number of version information from the obtained version information set according to the number of samples corresponding to each application, and determining application samples corresponding to the preset number of version information; respectively calling each vulnerability scanner, and scanning the container environment generated by each application sample to obtain each CVE scanned by each vulnerability scanner; and respectively determining an accuracy detection result of the vulnerability scanner according to the CVE of the vulnerability scanner to be tested and the acquired reference data set aiming at each vulnerability scanner. Therefore, the detection accuracy of various types of vulnerability scanners can be evaluated at the same time.

Description

Testing method and device for multi-bug scanner
Technical Field
The present application relates to the field of network security technologies, and in particular, to a method and an apparatus for testing a multi-vulnerability scanner.
Background
As computer system security issues continue to emerge, different industries have paid more attention to computer system security issues, and as a result, various types of vulnerability scanners have increased. The types and detection accuracy of Common Vulnerabilities & expositions (CVE) that different vulnerability scanners can scan may be different, so how the detection accuracy of each vulnerability scanner can be determined provides an important index for a user to select a vulnerability scanner, which becomes an urgent problem to be solved.
In the related art, when the detection accuracy of a vulnerability scanner is evaluated, generally, only a single type of vulnerability scanner can be evaluated, and if the detection accuracy of multiple types of vulnerability scanners needs to be evaluated, the method in the related art obviously cannot meet the scene needs of the multiple vulnerability scanners due to the lack of evaluation standards. Therefore, how to evaluate the detection accuracy of multiple types of vulnerability scanners at the same time becomes a problem to be solved urgently.
Disclosure of Invention
The embodiment of the application provides a testing method and a testing device for multiple vulnerability scanners, so that the detection accuracy of multiple types of vulnerability scanners can be evaluated at the same time.
The embodiment of the application provides the following specific technical scheme:
a testing method of a multi-vulnerability scanner comprises the following steps:
respectively determining the number of samples corresponding to one application according to the number of CVEs (composite video edge) of the one application and the total number of the CVEs for each application which supports scanning by each vulnerability scanner;
randomly sampling a preset number of version information from the obtained version information set according to the number of samples corresponding to each application, and determining application samples corresponding to the preset number of version information, wherein the version information set at least comprises each version information corresponding to each application supported and scanned by each vulnerability scanner;
respectively calling the vulnerability scanners, scanning the container environment generated by each application sample, and obtaining each CVE scanned by each vulnerability scanner;
and respectively determining the accuracy detection result of the vulnerability scanner according to the CVEs of the vulnerability scanner to be tested and the obtained reference data set aiming at the vulnerability scanners, wherein the reference data set comprises all CVEs corresponding to the applications which are supported and scanned by the vulnerability scanners.
Optionally, determining the number of samples corresponding to one application according to the number of CVEs and the total number of CVEs of the one application includes:
determining the number of CVEs corresponding to each application which supports scanning of each vulnerability scanner and the total number of the CVEs, wherein the total number of the CVEs represents the sum of the number of the CVEs;
calculating the ratio of the number of CVEs for one application to the total number of CVEs;
and determining the number of samples corresponding to the application according to the ratio and the preset total number of samples.
Optionally, determining the number of CVEs corresponding to each application that each vulnerability scanner supports scanning specifically includes:
respectively determining each application which each vulnerability scanner supports scanning, and corresponding type information and application CVE number of the CVE, wherein the application CVE number is the number of the CVE contained in one vulnerability scanner;
and respectively calculating the sum of the CVE number of any application program in each vulnerability scanner aiming at each application, and taking the sum as the CVE number of the application.
Optionally, the obtaining manner of the reference data set is as follows:
querying each application and each version information corresponding to each application from a standard vulnerability database;
respectively determining the CVE contained in the application under each version information according to each version information of one application aiming at each application;
a reference data set containing CVE contained in each application under each version information is generated.
Optionally, the determining, for each vulnerability scanner, an accuracy detection result of the vulnerability scanner according to each CVE of the vulnerability scanner to be tested and the obtained reference data set specifically includes:
respectively aiming at each CVE, if a CVE is determined to be contained in the reference data set, determining that the detection result of the CVE is correct, otherwise, determining that the detection result of the CVE is false alarm;
if the CVE contained in the reference data set is determined not to be contained in each CVE, determining that the detection result of the CVE is a false negative;
and determining the accuracy detection result of the vulnerability scanner according to the detection result of each CVE.
Optionally, determining an accuracy detection result of the vulnerability scanner according to the detection result of each CVE specifically includes:
respectively determining the number of CVEs with correct detection results, the number of CVEs with false detection results and the number of CVEs with false detection results for each application, and determining the total number of the CVEs contained in the reference data set by the application;
respectively aiming at each application, determining the accuracy rate of the application according to the number and the total number of CVEs of which the detection result of the application is correct, determining the false alarm rate of the application according to the number and the total number of CVEs of which the detection result is false alarm, and determining the false alarm rate of the application according to the number and the total number of CVEs of which the detection result is false alarm;
and determining the accuracy detection result of the vulnerability scanner according to the accuracy rate, the false alarm rate and the missing report rate corresponding to each application.
A testing apparatus of a multi-vulnerability scanner, comprising:
the first determining module is used for respectively determining the number of samples corresponding to one application according to the number of CVEs and the total number of the CVEs of the one application for each application which the vulnerability scanner supports scanning;
the sampling module is used for randomly sampling a preset number of version information from the acquired version information set according to the number of samples corresponding to each application, and determining application samples corresponding to the preset number of version information, wherein the version information set at least comprises the version information corresponding to each application which is supported and scanned by each vulnerability scanner;
the scanning module is used for calling each vulnerability scanner respectively, scanning the container environment generated by each application sample and obtaining each CVE scanned by each vulnerability scanner;
and the detection module is used for determining the accuracy detection result of the vulnerability scanner according to each CVE of the vulnerability scanner to be tested and the acquired reference data set aiming at each vulnerability scanner, wherein the reference data set comprises all CVEs corresponding to each application which supports scanning of each vulnerability scanner.
Optionally, when the number of samples corresponding to one application is determined according to the number of CVEs and the total number of CVEs of the one application, the sampling module is specifically configured to:
determining the number of CVEs corresponding to each application which supports scanning of each vulnerability scanner and the total number of the CVEs, wherein the total number of the CVEs represents the sum of the number of the CVEs;
calculating the ratio of the number of CVEs for one application to the total number of CVEs;
and determining the number of samples corresponding to the application according to the ratio and the preset total number of samples.
Optionally, when determining the number of CVEs corresponding to each application that each vulnerability scanner supports scanning, the scanning module is specifically configured to:
respectively determining each application which each vulnerability scanner supports scanning, and corresponding type information and application CVE number of the CVE, wherein the application CVE number is the number of the CVE contained in one vulnerability scanner;
and respectively calculating the sum of the CVE number of any application program in each vulnerability scanner aiming at each application, and taking the sum as the CVE number of the application.
Optionally, when obtaining the reference data set, the method further includes:
the query module is used for querying each application and each version information corresponding to each application from the standard vulnerability database;
a second determining module, configured to determine, for each application, a CVE included in the application under each version information according to each version information of the application;
and the generating module is used for generating a reference data set containing the CVE contained in each application under each version information.
Optionally, the detection module is specifically configured to:
respectively aiming at each CVE, if a CVE is determined to be contained in the reference data set, determining that the detection result of the CVE is correct, otherwise, determining that the detection result of the CVE is false alarm;
if the CVE contained in the reference data set is determined not to be contained in each CVE, determining that the detection result of the CVE is a false negative;
and determining the accuracy detection result of the vulnerability scanner according to the detection result of each CVE.
Optionally, when determining the accuracy detection result of the vulnerability scanner according to the detection result of each CVE, the detection module is specifically configured to:
respectively determining the number of CVEs with correct detection results, the number of CVEs with false detection results and the number of CVEs with false detection results for each application, and determining the total number of the CVEs contained in the reference data set by the application;
respectively aiming at each application, determining the accuracy rate of the application according to the number and the total number of CVEs of which the detection result of the application is correct, determining the false alarm rate of the application according to the number and the total number of CVEs of which the detection result is false alarm, and determining the false alarm rate of the application according to the number and the total number of CVEs of which the detection result is false alarm;
and determining the accuracy detection result of the vulnerability scanner according to the accuracy rate, the false alarm rate and the missing report rate corresponding to each application.
An electronic device comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, wherein the processor executes the program to realize the steps of the testing method of the multi-vulnerability scanner.
A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method for testing a multi-vulnerability scanner described above.
In the embodiment of the application, the number of samples corresponding to an application is determined according to the number of CVEs and the total number of the CVEs for the application, the version information of a preset number is randomly sampled from the obtained version information set according to the number of the samples corresponding to the application, the application samples corresponding to the version information of the preset number are determined, the vulnerability scanners are called respectively, the container environment generated by the application samples is scanned, the CVEs scanned by the vulnerability scanners are obtained, and the accuracy detection result of the vulnerability scanners is determined according to the CVEs of the vulnerability scanners to be tested and the obtained reference data set. In this way, the CVE of each application which supports scanning of each vulnerability scanner is used as a scanning sample, and the reference data set is obtained, so that the scanned CVE and the reference data set are compared, the accuracy of each vulnerability scanner is detected, a unified standard can be provided for detection of various types of vulnerability scanners, detection of various types of vulnerability scanners is realized, and the accuracy of detection is improved.
Drawings
Fig. 1 is a flowchart illustrating a testing method for a multi-vulnerability scanner according to an embodiment of the present disclosure;
FIG. 2 is a schematic diagram illustrating division of version influence intervals according to an embodiment of the present application;
FIG. 3 is a diagram illustrating dividing the number of CVEs based on versions in an embodiment of the present application;
FIG. 4 is another flowchart of a method for testing a multi-vulnerability scanner according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of a testing apparatus of a multi-vulnerability scanner in an embodiment of the present application;
fig. 6 is a schematic structural diagram of an electronic device in an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
As computer system security issues continue to emerge, different industries have also placed increasing emphasis on computer system security issues. Therefore, various types of vulnerability scanners are also gradually increased, and the number of vulnerabilities contained in each product, the accuracy of the vulnerabilities and the like are important indexes for a user to evaluate the quality of the product. The CVE that different vulnerability scanners can scan and the accuracy of detection may be different, so how to determine the accuracy of detection of each vulnerability scanner provides an important index for a user to select a vulnerability scanner, and a problem to be solved urgently is provided.
In the related art, when the detection accuracy of a vulnerability scanner is evaluated, the accuracy of a certain vulnerability scanner can be evaluated only for a single type of scanner, and the evaluation is based on the evaluation of the vulnerability added by the vulnerability scanner. If there are multiple types of scanners to be evaluated, due to the lack of standards, what is used as a standard for comparison of the multiple types of vulnerability scanners to obtain accuracy data, and therefore, the use of the test method in the related art obviously cannot meet the requirements of a scene. If one vulnerability scanner is selected as a standard, adverse factors are inevitably generated on other vulnerability scanners, so when a plurality of vulnerability scanners are compared, how to more objectively evaluate the accuracy of the vulnerability scanner becomes a problem to be solved. In addition, how to verify the report condition of the bugs in different bug scanners needs to set up a test environment for scanning without knowing rules, in the related art, when the test environment is set up, the test environment is basically based on a physical machine or a virtual machine, and when the number of applications and systems is increased, setting up, copying and maintaining a container environment also becomes time-consuming and labor-consuming.
In order to solve the above problem, an embodiment of the present application provides a testing method for multiple vulnerability scanners, where for each application that each vulnerability scanner supports scanning, a sample number corresponding to one application is determined according to a CVE number and a CVE total number of the application, a preset number of version information is randomly sampled from an acquired version information set according to the sample number corresponding to each application, application samples corresponding to the preset number of version information are determined, each vulnerability scanner is called, a container environment generated by each application sample is scanned, each CVE scanned by each vulnerability scanner is obtained, and an accuracy detection result of each vulnerability scanner is determined according to each CVE of a vulnerability scanner to be tested and the acquired reference data set. Therefore, the reference data is used as the comparison standard, different vulnerability scanners use the same standard to perform accuracy calculation, and the output accuracy detection result can be more objective and credible.
Based on the above embodiment, referring to fig. 1, a flowchart of a testing method for a multi-bug scanner in the embodiment of the present application is shown, which specifically includes:
step 100: and respectively determining the number of samples corresponding to one application according to the number of CVEs of the one application and the total number of the CVEs for each application which supports scanning by each vulnerability scanner.
In the embodiment of the application, when hierarchical sampling is performed, the number of samples corresponding to each application which each vulnerability scanner supports scanning is determined.
It should be noted that, because the number of applications and CVEs corresponding to the applications that are supported by each vulnerability scanner is very large, sampling needs to be performed from each application that is supported by each vulnerability scanner according to a preset hierarchical sampling rule and the number of CVEs of each application, and the application obtained by sampling is used as sample data of each vulnerability scanner, so as to construct a container environment containing the sample data. That is, the samples in the embodiment of the present application are obtained by sampling from applications that each vulnerability scanner supports scanning.
In addition, it should be noted that, in the standard vulnerability database, the number of CVEs corresponding to each application is large, which does not mean that the number of CVEs that can be scanned by an actual vulnerability scanner is large. For one application, the total number of CVEs contained in the standard vulnerability database is large, but the total number of CVEs that can be scanned by an actual vulnerability scanner is small. Then, if the total number of CVEs contained in the standard database is used as the basis of the hierarchical sampling, the proportion of samples obtained by the hierarchical sampling is relatively small. The accuracy of the finally determined vulnerability scanner also has a deviation, so the total number of applied CVEs added to each vulnerability scanner to be compared needs to be used as a basis for hierarchical sampling. Therefore, in the embodiment of the present application, the number of CVEs corresponding to each application of each vulnerability scanner and the total number of CVEs that can be scanned by each vulnerability scanner are used as a basis for hierarchical sampling.
A possible implementation manner is provided in the embodiment of the present application for the step of determining the number of samples corresponding to an application, and the following describes in detail the step of determining the number of samples corresponding to an application according to the number of CVEs and the total number of CVEs of an application in the embodiment of the present application, and specifically includes:
s1: and determining the number of CVEs corresponding to each application which supports scanning by each vulnerability scanner and the total number of the CVEs.
Wherein the total number of CVEs represents the sum of the numbers of the CVEs.
Specifically, when step S1 is executed, the method specifically includes:
a1: and respectively determining each application which each vulnerability scanner supports scanning, and the type information and the application CVE number of each corresponding CVE.
The number of applied CVEs is the number of CVEs applied to a vulnerability scanner.
In the embodiment of the present application, because the applications supported by different vulnerability scanners and the version information of the applications may be different, and the types of CVEs that can be scanned by each vulnerability scanner may also be different.
Therefore, in the embodiment of the present application, first, for each vulnerability scanner, each application supported by one vulnerability scanner and the number of applications supported by the vulnerability scanner are obtained, and an application set including each application of the vulnerability scanner is generated.
For example, suppose that the accuracy of 3 vulnerability scanners needs to be detected, the 3 vulnerability scanners are respectively Pa、PbAnd Pc. Vulnerability scanner PaThe method supports scanning of 5 applications, namely app1、app 2、app 3、app4And app5Then bug scanner PaThe set of applications of (a) may be expressed as: pa＝{app 1,app 2,app 3,app4,app5}, vulnerability scanner PaThe number of applications supporting scanning is 5. Vulnerability scanner PbThe method supports scanning of 4 applications, namely app1、app3、app4And app7Then bug scanner PbThe set of applications of (a) may be expressed as: pb＝{app1,app3,app4,app7}, vulnerability scanner PbThe number of applications supporting scanning is 4. Vulnerability scanner PcThe method supports scanning of 4 applications, namely app2、app3、app4And app8Then bug scanner PcThe set of applications of (a) may be expressed as: pc＝{app2,app3,app4,app8}, scanner to be tested PcThe number of applications supporting scanning is 4.
The application may be mysql, for example, which is not limited in the embodiments of the present application.
Further, in the embodiment of the application, after the applications, the number of the applications, and the application set, which are supported and scanned by each vulnerability scanner, are obtained, the application sets corresponding to all vulnerability scanners are subjected to union processing, so that an application set including all applications is obtained.
For example, assume that the 3 vulnerability scanners are P eacha、PbAnd PcThen the application set P can be expressed as: p ═ Pa∪Pb∪Pc。
And then, determining the type information of the CVE and the application CVE number corresponding to each application which supports scanning of each vulnerability scanner, wherein the application CVE number is the number of the CVE contained in one vulnerability scanner.
A2: and respectively calculating the sum of the CVE number of any application program in each vulnerability scanner aiming at each application, and taking the sum as the CVE number of the application.
In the embodiment of the application, after acquiring the application CVE number of any application in each vulnerability scanner, the CVE number of each application can be acquired by calculating the sum of the application CVEs in each vulnerability scanner and using the obtained sum as the CVE number of the application in each vulnerability scanner.
The number of the CVEs represents the total number of the CVEs applied to all vulnerability scanners, and is the sum of the number of the CVEs applied to each vulnerability scanner.
For example, assume that there are 3 vulnerability scanners, namely vulnerability scanners PaLeak scanner PbAnd vulnerability scanner Pc. Vulnerability scanner PaThe method supports scanning of 4 applications, namely app1、app2、app3、app4Then determine app1Corresponding application CVE numberIn an amount of cvea(app1)，app2The corresponding number of applied CVE's is CVEa(app2)，app3The corresponding number of applied CVE's is CVEa(app3)，app4The corresponding number of applied CVE's is CVEa(app4). Vulnerability scanner PbThe method supports scanning of 4 applications, namely app1、app3、app4、app5Then determine app1The corresponding number of applied CVE's is CVEb(app1)，app3The corresponding number of applied CVE's is CVEb(app3)，app4The corresponding number of applied CVE's is CVEb(app4)，app5The corresponding number of applied CVE's is CVEb(app5). Vulnerability scanner PcThe method supports scanning of 4 applications, namely app2、app3、app4、app6Then determine app2The corresponding number of applied CVE's is CVEc(app2)，app3The corresponding number of applied CVE's is CVEc(app3)，app4The corresponding number of applied CVE's is CVEc(app4)，app6The corresponding number of applied CVE's is CVEc(app6)。
Thus, the number of application CVEs for each application in different vulnerability scanners can be expressed as:
cve(Pa)＝{cvea(app1),cvea(app2),cvea(app3),cvea(app4)}；
cve(Pb)＝{cveb(app1),cveb(app3),cveb(app4),cveb(app5)}；
cve(Pc)＝{cvec(app2),cvec(app3),cvec(app4),cvec(app6)}。
then, after determining the application CVE number corresponding to each application in each vulnerability scanner, P is calculatedaMiddle app1Application CVE number CVEa(app1) And PbMiddle app1Application CVE number CVEb(app1) to get the apps1The corresponding number of CVEs is SUM (CVE)a(app1),cveb(app1)). Calculating PaMiddle app2Application CVE number CVEa(app2) And PcMiddle app2Application CVE number CVEc(app2) To get the app2The corresponding number of CVEs is SUM (CVE)a(app2),cvec(app2)). Calculating PaMiddle app3Application CVE number CVEa(app3) And PbMiddle app3Application CVE number CVEb(app3) And PcApp in (1)3Application CVE number CVEc(app3) SUM of (SUM) (cve)a(app3),cveb(app3)，cvec(app3)). Calculating PaMiddle app4Application CVE number CVEa(app4) And PbMiddle app4Application CVE number CVEb(app4) And PcApp in (1)4Application CVE number CVEc(app4) SUM of (SUM) (cve)a(app4),cveb(app4)，cvec(app4)). Computing app5The corresponding number of CVEs is CVEb(app5). Computing app6The corresponding number of CVEs is CVEc(app6)。
And finally, determining the total number of CVEs corresponding to the CVEs which can be scanned by each vulnerability scanner to be tested.
For example, the CVE total number of all apps included in each vulnerability scanner is CVE (app), CVE (app) ═ CVE (app)1),cve(app2),cve(app3),cve(app4),cve(app5),cve(app6)}。
S2: the ratio between the number of CVEs for an application and the total number of CVEs is calculated.
In the embodiment of the application, the ratio of the number of the CVEs of the application program to the total number of the CVEs of all the vulnerability scanners containing the application is calculated.
S3: and determining the number of samples corresponding to one application according to the ratio and the preset total number of samples.
In the embodiment of the present application, the calculated ratio is used as the ratio of the CVE of the applied CVE in all vulnerability scanners. And then, determining the number of samples corresponding to the samples which should be extracted by the application program according to the calculated ratio and the preset total number of samples.
For example, assuming that the application is mysql, the preset total number of samples is 20, and the calculated number of CVEs of mysql accounts for 10% of the total number of CVEs corresponding to the CVEs included in all vulnerability scanners, the product between the preset total number of samples and the ratio is calculated, and the number of extracted mysql samples is determined to be 2.
It should be noted that, in the embodiment of the present application, the number of samples directly affects the error condition of the final accuracy detection result. When the number of samples is larger, the accuracy of the hierarchical sampling can be improved, but the sampling cost is higher, and conversely, when the number of samples is smaller, the accuracy of the hierarchical sampling can be reduced, but the cost is lower, so that the number of samples of each application can be determined according to actual requirements.
In addition, it should be noted that, in the embodiment of the present application, since the hierarchical sampling processing is performed based on all applications of all to-be-bug scanners, the number of samples may affect all bug scanners, and the influence of the part may not be taken into account under the same influence.
Step 110: and randomly sampling a preset number of version information from the acquired version information set according to the number of samples corresponding to each application, and determining application samples corresponding to the preset number of version information.
The version information set at least comprises version information corresponding to each application which is supported by each vulnerability scanner to scan.
In the embodiment of the application, after the number of samples of each application is determined, version information corresponding to the number of samples is extracted from an acquired version information set in a random sampling mode, and application program samples, namely sample data, corresponding to the extracted version information are determined.
It should be noted that, in the embodiment of the present application, a generation process of the version information set is as follows: and respectively aiming at each application supported and scanned by each vulnerability scanner, acquiring version information corresponding to the version of the application containing a certain CVE from the standard vulnerability database, thereby generating a version information set containing the version information.
For example, referring to fig. 2, for a schematic diagram of dividing the version influence interval in the embodiment of the present application, the applications are app1, app2, app3, app4, and app5, the version information of app1 is banner1 and banner2, the version information of app2 is banner1, the version information of app3 is banner1 and banner2, the influence version of app4 is banner1, and the version information of app5 is banner1 and banner2, respectively.
After the sample space is determined, the container environment can be built based on the application samples.
It should be noted that, in the embodiment of the present application, a container environment of each application sample is built by using a docker container technology, and a plurality of container environments are built in one virtual machine, and an environment port is mapped out, so that the container environment can be obtained by calling the environment port.
Further, be a periodic process to the accuracy contrast, the container environment that will have now built is preserved, and later stage direct use can submit efficiency of software testing greatly, selects the Harbor warehouse to carry out the save of sample environment, and it is more convenient to maintain on the one hand, and on the other hand, when sample data used at the later stage, the pull in directly following the Harbor warehouse, convenient migration, duplication.
The development and operation of Docker container application are reliable mirror image management, and the Harbor is an enterprise-level Docker Registry management project, provides a role-based access control mechanism, and controls the organization and access authority of the mirror image through the project. The container technology is a new emerging and popular technology behind virtualization, cloud computing and big data, improves the utilization rate of hardware resources, facilitates quick and transverse capacity expansion of business of enterprises, realizes a self-healing function of business downtime, greatly simplifies creation and maintenance of containers from file systems and network interconnection to process isolation and the like, and enables the Docker technology to be lighter and faster than the virtual machine technology.
Step 120: and respectively calling each vulnerability scanner, scanning the container environment generated by each application sample, and obtaining each CVE scanned by each vulnerability scanner.
In the embodiment of the application, the vulnerability scanners are respectively called, and the determined container environments are respectively scanned through the vulnerability scanners, so that the applications with the CVEs, the CVEs of each application and the number of the CVEs which can be scanned by each vulnerability scanner are determined.
Step 130: and respectively determining an accuracy detection result of the vulnerability scanner according to the CVE of the vulnerability scanner to be tested and the acquired reference data set aiming at each vulnerability scanner.
The reference data set comprises all CVEs corresponding to applications which are supported by the vulnerability scanners to scan.
Firstly, a detailed explanation is given to an obtaining manner of a reference data set in the embodiment of the present application, which specifically includes:
s1: and querying each application and each version information corresponding to each application from the standard vulnerability database.
In the embodiment of the application, each application contained in each vulnerability scanner is inquired from the standard vulnerability database, and the version information corresponding to the version influenced by each application is obtained from the standard vulnerability database.
It should be noted that, for the same application, applications in some versions may be affected by the CVE, but applications in some versions may not be affected, and therefore, version information corresponding to the versions of the affected applications needs to be determined.
The version information represents version information corresponding to a version of the application affected by the CVE, for example, the version information of the application mysql is 8.0.0, which is not limited in this embodiment of the application.
In addition, it should be noted that the standard vulnerability database includes CVE information of all applications, and the standard vulnerability database may be, for example, a security vulnerability sharing platform, which is not limited in this embodiment of the present application.
S2: and respectively determining the CVE contained in the application under each version information according to each version information of one application.
In the embodiment of the application, the CVE included in each version information of the application is determined for each version information of each application.
Step S2 in the embodiment of the present application is described in detail below by taking one of the applications as an example.
First, version information of an application is determined.
Then, the CVE and the application version CVE number contained in the application under any version information are determined respectively aiming at the application under each version information.
It should be noted that the number of CVEs of an application version characterizes the total number of CVEs that an application contains in a certain version. For example, the application is mysql, the version information is 8.0.0, and under version 8.0.0, the number of CVEs contained in mysql is num (CVE (8.0.0)).
Finally, the CVE and the number of the application versions CVE contained in the application under each version information can be obtained through the steps.
For example, the step S2 in the embodiment of the present application is described in detail below by using a specific example, taking an application as mysql as an example, referring to fig. 3, which is a schematic diagram of dividing the CVE number based on versions in the embodiment of the present application, first obtaining the CVE total number of all applications in the standard vulnerability database as CVE (app), then classifying the applications according to the type information of the applications, and obtaining the CVE number of the application version of each application, which is CVE (app) respectively1)、CVE(app2)、CVE(app3)、CVE(app4) And CVE (app)5). Then, determining version information corresponding to the version influenced by each application, namely the app1The version information of (1) is Banner1 and Banner2, app2Version of (1)The information is Banner1, app3The version information of (1) is Banner1 and Banner2, app4Is Banner1, app5The version information of (1) is Banner1 and Banner2, and application apps under version information Banner1 are obtained1Application version CVE number num (CVE (banner1)) and application app under version information banner21CVE number num (CVE (banner2)), and the app under version information banner1 is determined2CVE number num (CVE (banner1)), and the app under version information banner1 is determined3Application version CVE number num (CVE (banner1)) and app under version information banner23CVE number num (CVE (banner2)), and the app under version information banner1 is determined4CVE number num (CVE (banner1)), and the app under version information banner1 is determined5Application version CVE number num (CVE (banner1)) and app under version information banner25Number num of application versions CVE (CVE (banner 2)).
S3: a reference data set containing CVE contained in each application under each version information is generated.
In the embodiment of the application, after the CVEs included in the applications under the version information and the application version CVEs are determined, the reference data set is generated according to the CVEs, and the generated reference data set also corresponds to the application version CVE number of the applications under the version information.
In this way, the accuracy detection result of the vulnerability scanner can be determined according to each CVE of any vulnerability scanner and the acquired reference data set for each vulnerability scanner.
In the embodiment of the present application, when determining the accuracy detection result of the vulnerability scanner to be tested, the method specifically includes:
s1: and respectively aiming at each CVE, if one CVE is determined to be contained in the reference data set, determining that the detection result of the CVE is correct, and otherwise, determining that the detection result of the CVE is false alarm.
S2: and if the CVE contained in the reference data set is determined not to be contained in each CVE, determining that the detection result of the CVE is false negative.
In the embodiment of the application, for each vulnerability scanner, after any vulnerability scanner is determined to scan the container environment, each CVE and the detection result of each CVE are obtained.
The detection result of the CVE may be specifically classified into the following three types, but is not limited to the following three types.
The first detection result: and (4) correct.
And if the CVE obtained by scanning is determined to be contained in the reference data set, determining that the detection result corresponding to the CVE is correct. That is to say, after the vulnerability scanner to be tested scans the container environment, the vulnerability CVE existing in the container environment is determined, and if the vulnerability CVE is correctly reported, the detection result of the CVE is determined to be correct.
The second detection result is as follows: and (4) false alarm.
And if the CVE obtained by scanning is determined not to be contained in the reference data set, determining that the detection result corresponding to the CVE is false alarm. That is, the CVE does not exist in the container environment, and after the vulnerability scanner scans the container environment, the CVE is obtained by scanning, but actually, the CVE does not exist in the application corresponding to the CVE, and therefore, the detection result of the CVE is determined to be a false alarm.
Further, in this embodiment of the application, if each CVE in the reference data set corresponds to one CVE identifier, it may be determined whether the CVE identifier corresponding to the scanned CVE is included in the reference data set, so as to determine whether the CVE identifier obtained by scanning is included in the reference data set.
The third detection result is: and (6) failing to report.
If the CVE existing in the reference data set is determined not to be scanned by the vulnerability scanner, that is, the CVE actually existing in the container environment is not scanned by the vulnerability scanner, the detection result of the CVE is determined to be false alarm.
For example, with vulnerability scanners PaApplication app of1For example, if PaScanned-out app1And if the CVE exists in the reference data set, determining that the detection result of the CVE is correct. If PaScanned-out app1CVE ofAnd if the CVE does not exist in the reference data set, determining that the detection result of the CVE is false alarm. If the CVE existing in the reference data set is determined, the CVE is not detected by the PaAnd if the CVE is scanned out, determining that the detection result of the CVE is a false positive.
S3: and determining the accuracy detection result of the vulnerability scanner according to the detection result of each CVE.
Specifically, when step S3 is executed, the method specifically includes:
a1: respectively aiming at each application, determining the number of CVEs of which the detection result is correct, the number of CVEs of which the detection result is false report and the number of CVEs of which the detection result is false report, and determining the total number of the CVEs contained in a reference data set.
In the embodiment of the application, for each application of the vulnerability scanner, which supports scanning, firstly, CVEs corresponding to any scanned application are determined, detection results corresponding to the CVEs are obtained, then, the number of CVEs with correct detection results is determined, the number of CVEs with false detection results is determined, and the number of CVEs with false detection results is determined. Finally, the total number of CVEs that the application should contain in the reference data set is determined.
For example, assume that the application is an app1Bug scanner PaCVE scanned is a1，a2，a3，a4Let a be1Is correct, a2The detection result of (a) is false alarm3The detection result of (a) is false positive4If the detection result is correct, determining that the number of the CVEs with the correct detection result is 2, the number of the CVEs with the false detection result is 1, and determining that the total number of the CVEs which should be included in the reference data set by the application is 4.
It should be noted that, in step a1, since the reference data set includes the CVEs that each application should include, the total number of the CVEs that each application should include may be determined by counting the CVEs that each application includes, and in the actual application process, the total number of the CVEs that each application should include in the reference data set is the sum of the number of the CVEs whose detection results are correct, the number of the CVEs whose detection results are false positives, and the number of the CVEs whose detection results are false negatives.
A2: respectively aiming at each application, determining the accuracy rate of the application according to the number and the total number of the CVEs of which the detection result of the application is correct, determining the false alarm rate of the application according to the number and the total number of the CVEs of which the detection result is false alarm, and determining the false alarm rate of the application according to the number and the total number of the CVEs of which the detection result is false alarm.
In the embodiment of the application, for each application, a first ratio between the number and the total number of correct CVEs is calculated as a detection result of any application, the calculated first ratio is used as the accuracy of the application, a second ratio between the number and the total number of false-alarm CVEs is calculated as a detection result of the application, the calculated second ratio is used as the false-alarm rate of the application, a third ratio between the number and the total number of false-alarm CVEs is calculated as a detection result of the application, and the calculated third ratio is used as the false-alarm rate of the application.
For example, assume vulnerability scanner PaApplication app of1，app1The number of correct CVEs of (1) is TP (app)1) Total amount is cve (app)1) Then app1The accuracy of (d) may be expressed, for example, as:
Figure BDA0003327606160000171
among them, accuracy (app)1) Is app1Accuracy of (1), TP (app)1) To detect the number of CVEs that result in correctness, CVE (app)1) Is the total number.
app1Has a false alarm rate of FP (app)1) The number of CVEs with false alarms is FP (app1) and the total number is CVE (app1)1) Then app1The false alarm rate of (a) may be expressed as:
FPR(app1)＝(FP(app1))/(cve(app1))
wherein FPR (app)1) Is app1False alarm rate of (FP) (app)1) For the number of CVEs whose detection result is false positive, CVE (app)1) Is the total number.
app1Has a false negative rate of FNR (app)1) And the number of CVEs with the detection result of false negative is FNR (app)1) Total amount is cve (app)1) Then app1The false negative rate of (c) may be expressed as:
Figure BDA0003327606160000181
wherein FPR (app)1) Is app1False alarm rate of (FP) (app)1) For the number of CVEs whose detection result is false negative, CVE (app)1) Is the total number.
A3: and determining the accuracy detection result of the vulnerability scanner according to the accuracy rate, the false alarm rate and the missing report rate corresponding to each application.
In the embodiment of the application, for the accuracy, the false alarm rate and the missing report rate corresponding to each application, when the accuracy of the loophole scanner is high and the false alarm rate and the loophole rate are both low, the operation is more dominant.
In the embodiment of the application, the CVE applied to each standard vulnerability database is used as a comparison standard, and different vulnerability scanners use the same standard to perform accuracy calculation, so that output data can be more objective and credible. And for the selection of the sample of the container environment, the sample space coverage can be diversified by adopting the hierarchical sampling according to the number of the CVEs of the applications supported and scanned by each vulnerability scanner. In addition, the accuracy of the vulnerability scanner is measured from three dimensions of accuracy, missing report rate and false report rate, the comprehensiveness of comparison is increased, a container environment is built by using a docker technology, the resource consumption of building the container environment is reduced, and meanwhile, a Harbor warehouse is used for maintaining container environment mirror images, so that the maintenance and the migration are convenient.
Based on the foregoing embodiment, referring to fig. 4, another flowchart of a method for testing a multi-vulnerability scanner in the embodiment of the present application is shown, which specifically includes:
step 400: and starting.
Step 401: and acquiring the number of each application and the corresponding application CVE which are supported by the vulnerability scanner A to be scanned.
It should be noted that the application in the embodiment of the present application is used to search the corresponding reference data set from the standard vulnerability database, and on the other hand, is used as an input of the hierarchical sampling of the container environment.
Step 402: and acquiring the number of each application and the corresponding application CVE which are supported by the vulnerability scanner B for scanning.
Step 403: and acquiring the number of each application and the corresponding application CVE which are supported by the vulnerability scanner C.
In the embodiment of the present application, each application of the scanning supported by the vulnerability scanner C is:
Pc＝{app2,app3,app4,app8....}
the number of the applications CVE corresponding to each application supported by the vulnerability scanner C is as follows:
cve(Pb)＝{cveb(app1),cveb(app3),cveb(app4),cveb(app5)…}
step 404: and storing the application set and the CVE number corresponding to each application into a test database.
In the embodiment of the present application, an application set including applications that are scanned by vulnerability scans A, B and C is generated, and the number of applications that can be scanned by A, B, C, that is, the sum of the corresponding CVEs in A, B, C, is calculated for each application.
Specifically, it can be expressed as:
cve(app1)＝SUM(cvea(app1),cveb(app1),..)
cve(app2)＝SUM(cvea(app2),cvec(app2),..)
cve(app)＝{cve(app1),cve(app2),…}
and finally, storing the application set and the CVE number corresponding to each application into a test database.
Step 405: and acquiring a reference database from the standard vulnerability database.
The reference database at least comprises a reference data set, a total number of CVEs, version information influenced by the CVEs, and the total number of the CVEs contained in the applications under each version information.
It should be noted that the reference database in the embodiment of the present application is used as a comparison standard for determining the detection result of the CVE.
In addition, it should be noted that, according to the application sets obtained in step 404, version information of the CVE and the affected version of the CVE corresponding to each application set, the number of the CVEs of the applications of different affected versions, and these data are used as criteria for subsequent accuracy judgment, are obtained from the standard vulnerability database.
In addition, it should be noted that, in this step, 3 data are obtained, which are the total number CVE (app) corresponding to each application, the version information CVE (banner) corresponding to the affected version of each CVE, and the number of CVEs included in the application program under the version information.
Step 406: and randomly sampling a preset number of version information from the acquired version information set according to the number of samples corresponding to each application, and determining application samples corresponding to the preset number of version information.
In the embodiment of the application, the application samples corresponding to the version information corresponding to the sample number are randomly sampled from the version information set by a random sampling method according to the determined sample number corresponding to each application.
Step 407: and judging whether a sample mirror image exists in the Harbor warehouse or not, if so, executing a step 408, and if not, executing a step 409.
Step 408: and acquiring the mirror image from the warehouse and building a container environment based on container technology.
Step 409: and acquiring the mirror image from the external network warehouse by using docker and constructing a container environment.
Step 410: and completing the construction of the container environment.
Step 411: the container environment is scanned separately A, B, C.
Step 412: the CVEs after scanning are acquired A, B, C.
Step 413: and determining accuracy of each application corresponding to A, B, C and accuracy detection results of A, B, C according to each CVE of A, B, C.
Step 414: and (6) ending.
Based on the above embodiment, referring to fig. 5, a schematic structural diagram of a testing apparatus for a multi-bug scanner in the embodiment of the present application is shown, which specifically includes:
a first determining module 500, configured to determine, for each application that each vulnerability scanner supports scanning, a sample number corresponding to one application according to the CVE number and the CVE total number of the one application;
a sampling module 510, configured to randomly sample a preset number of version information from an obtained version information set according to the number of samples corresponding to each application, and determine application samples corresponding to the preset number of version information, where the version information set at least includes version information corresponding to each application that each vulnerability scanner supports scanning;
a scanning module 520, configured to respectively invoke the vulnerability scanners, scan the container environment generated by each application sample, and obtain each CVE scanned by each vulnerability scanner;
the detection module 530 is configured to determine, for each vulnerability scanner, an accuracy detection result of the vulnerability scanner according to each CVE of a vulnerability scanner to be tested and an obtained reference data set, where the reference data set includes all CVEs corresponding to applications that each vulnerability scanner supports scanning.
Optionally, when determining the number of samples corresponding to one application according to the number of CVEs and the total number of CVEs of the one application, the sampling module 510 is specifically configured to:
determining the number of CVEs corresponding to each application which supports scanning of each vulnerability scanner and the total number of the CVEs, wherein the total number of the CVEs represents the sum of the number of the CVEs;
calculating the ratio of the number of CVEs for one application to the total number of CVEs;
and determining the number of samples corresponding to the application according to the ratio and the preset total number of samples.
Optionally, when determining the number of CVEs corresponding to each application that each vulnerability scanner supports scanning, the scanning module 520 is specifically configured to:
respectively determining each application which each vulnerability scanner supports scanning, and corresponding type information and application CVE number of the CVE, wherein the application CVE number is the number of the CVE contained in one vulnerability scanner;
and respectively calculating the sum of the CVE number of any application program in each vulnerability scanner aiming at each application, and taking the sum as the CVE number of the application.
Optionally, when obtaining the reference data set, the method further includes:
the query module 540 is configured to query the applications and version information corresponding to the applications from the standard vulnerability database;
a second determining module 550, configured to determine, for each application, a CVE included in each application under each version information according to each version information of the application;
and a generating module 560, configured to generate a reference data set including the CVE included in each application under each version information.
Optionally, the detecting module 530 is specifically configured to:
respectively aiming at each CVE, if a CVE is determined to be contained in the reference data set, determining that the detection result of the CVE is correct, otherwise, determining that the detection result of the CVE is false alarm;
if the CVE contained in the reference data set is determined not to be contained in each CVE, determining that the detection result of the CVE is a false negative;
and determining the accuracy detection result of the vulnerability scanner according to the detection result of each CVE.
Optionally, when determining the accuracy detection result of the vulnerability scanner according to the detection result of each CVE, the detection module 530 is specifically configured to:
respectively determining the number of CVEs with correct detection results, the number of CVEs with false detection results and the number of CVEs with false detection results for each application, and determining the total number of the CVEs contained in the reference data set by the application;
respectively aiming at each application, determining the accuracy rate of the application according to the number and the total number of CVEs of which the detection result of the application is correct, determining the false alarm rate of the application according to the number and the total number of CVEs of which the detection result is false alarm, and determining the false alarm rate of the application according to the number and the total number of CVEs of which the detection result is false alarm;
and determining the accuracy detection result of the vulnerability scanner according to the accuracy rate, the false alarm rate and the missing report rate corresponding to each application.
Based on the above embodiments, referring to fig. 6, a schematic structural diagram of an electronic device in an embodiment of the present application is shown.
An embodiment of the present application provides an electronic device, which may include a processor 610 (CPU), a memory 620, an input device 630, an output device 640, and the like, wherein the input device 630 may include a keyboard, a mouse, a touch screen, and the like, and the output device 640 may include a Display device, such as a Liquid Crystal Display (LCD), a Cathode Ray Tube (CRT), and the like.
Memory 620 may include Read Only Memory (ROM) and Random Access Memory (RAM), and provides processor 610 with program instructions and data stored in memory 620. In this embodiment of the present application, the memory 620 may be used to store a program of a testing method of any multi-bug scanner in this embodiment of the present application.
The processor 610 is configured to execute the testing method of any multi-vulnerability scanner in the embodiment of the present application according to the obtained program instructions by calling the program instructions stored in the memory 620.
Based on the foregoing embodiments, in the embodiments of the present application, a computer-readable storage medium is provided, where a computer program is stored, and when the computer program is executed by a processor, the computer program implements the method for testing the multi-vulnerability scanner in any of the above method embodiments.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.

Claims (10)

1. A testing method of a multi-vulnerability scanner is characterized by comprising the following steps:
respectively determining the number of samples corresponding to one application according to the number of CVEs (composite video edge) of the one application and the total number of the CVEs for each application which supports scanning by each vulnerability scanner;
randomly sampling a preset number of version information from the obtained version information set according to the number of samples corresponding to each application, and determining application samples corresponding to the preset number of version information, wherein the version information set at least comprises each version information corresponding to each application supported and scanned by each vulnerability scanner;
respectively calling the vulnerability scanners, scanning the container environment generated by each application sample, and obtaining each CVE scanned by each vulnerability scanner;
and respectively determining the accuracy detection result of the vulnerability scanner according to the CVEs of the vulnerability scanner to be tested and the obtained reference data set aiming at the vulnerability scanners, wherein the reference data set comprises all CVEs corresponding to the applications which are supported and scanned by the vulnerability scanners.
2. The method of claim 1, wherein determining the number of samples corresponding to one application according to the number of CVEs and the total number of CVEs of the one application comprises:
determining the number of CVEs corresponding to each application which supports scanning of each vulnerability scanner and the total number of the CVEs, wherein the total number of the CVEs represents the sum of the number of the CVEs;
calculating the ratio of the number of CVEs for one application to the total number of CVEs;
and determining the number of samples corresponding to the application according to the ratio and the preset total number of samples.
3. The method of claim 2, wherein determining the number of CVEs corresponding to each application that each vulnerability scanner supports scanning specifically comprises:
respectively determining each application which each vulnerability scanner supports scanning, and corresponding type information and application CVE number of the CVE, wherein the application CVE number is the number of the CVE contained in one vulnerability scanner;
and respectively calculating the sum of the CVE number of any application program in each vulnerability scanner aiming at each application, and taking the sum as the CVE number of the application.
4. The method of claim 1, wherein the reference data set is obtained by:
querying each application and each version information corresponding to each application from a standard vulnerability database;
respectively determining the CVE contained in the application under each version information according to each version information of one application aiming at each application;
a reference data set containing CVE contained in each application under each version information is generated.
5. The method according to claim 4, wherein the determining, for each vulnerability scanner, the accuracy detection result of one vulnerability scanner according to each CVE of the vulnerability scanner to be tested and the acquired reference data set specifically comprises:
respectively aiming at each CVE, if a CVE is determined to be contained in the reference data set, determining that the detection result of the CVE is correct, otherwise, determining that the detection result of the CVE is false alarm;
if the CVE contained in the reference data set is determined not to be contained in each CVE, determining that the detection result of the CVE is a false negative;
and determining the accuracy detection result of the vulnerability scanner according to the detection result of each CVE.
6. The method as claimed in claim 5, wherein determining the accuracy detection result of the vulnerability scanner according to the detection result of each CVE comprises:
respectively determining the number of CVEs with correct detection results, the number of CVEs with false detection results and the number of CVEs with false detection results for each application, and determining the total number of the CVEs contained in the reference data set by the application;
respectively aiming at each application, determining the accuracy rate of the application according to the number and the total number of CVEs of which the detection result of the application is correct, determining the false alarm rate of the application according to the number and the total number of CVEs of which the detection result is false alarm, and determining the false alarm rate of the application according to the number and the total number of CVEs of which the detection result is false alarm;
and determining the accuracy detection result of the vulnerability scanner according to the accuracy rate, the false alarm rate and the missing report rate corresponding to each application.
7. A testing device for a multi-vulnerability scanner is characterized by comprising:
the first determining module is used for respectively determining the number of samples corresponding to one application according to the number of CVEs and the total number of the CVEs of the one application for each application which the vulnerability scanner supports scanning;
the sampling module is used for randomly sampling a preset number of version information from the acquired version information set according to the number of samples corresponding to each application, and determining application samples corresponding to the preset number of version information, wherein the version information set at least comprises the version information corresponding to each application which is supported and scanned by each vulnerability scanner;
the scanning module is used for calling each vulnerability scanner respectively, scanning the container environment generated by each application sample and obtaining each CVE scanned by each vulnerability scanner;
and the detection module is used for determining the accuracy detection result of the vulnerability scanner according to each CVE of the vulnerability scanner to be tested and the acquired reference data set aiming at each vulnerability scanner, wherein the reference data set comprises all CVEs corresponding to each application which supports scanning of each vulnerability scanner.
8. The apparatus of claim 7, wherein when determining the number of samples corresponding to one application according to the number of CVEs and the total number of CVEs of the one application, the sampling module is specifically configured to:
determining the number of CVEs corresponding to each application which supports scanning of each vulnerability scanner and the total number of the CVEs, wherein the total number of the CVEs represents the sum of the number of the CVEs;
calculating the ratio of the number of CVEs for one application to the total number of CVEs;
and determining the number of samples corresponding to the application according to the ratio and the preset total number of samples.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the steps of the method of any of claims 1-6 are implemented when the program is executed by the processor.
10. A computer-readable storage medium having stored thereon a computer program, characterized in that: the computer program when executed by a processor implementing the steps of the method of any one of claims 1 to 6.
CN202111268004.4A 2021-10-29 2021-10-29 Testing method and device for multi-bug scanner Pending CN113849825A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111268004.4A CN113849825A (en) 2021-10-29 2021-10-29 Testing method and device for multi-bug scanner

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111268004.4A CN113849825A (en) 2021-10-29 2021-10-29 Testing method and device for multi-bug scanner

Publications (1)

Publication Number Publication Date
CN113849825A true CN113849825A (en) 2021-12-28

Family

ID=78983378

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111268004.4A Pending CN113849825A (en) 2021-10-29 2021-10-29 Testing method and device for multi-bug scanner

Country Status (1)

Country Link
CN (1) CN113849825A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117395080A (en) * 2023-12-08 2024-01-12 北京升鑫网络科技有限公司 Encryption system scanner detection method, device, electronic equipment and storage medium

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117395080A (en) * 2023-12-08 2024-01-12 北京升鑫网络科技有限公司 Encryption system scanner detection method, device, electronic equipment and storage medium
CN117395080B (en) * 2023-12-08 2024-02-09 北京升鑫网络科技有限公司 Encryption system scanner detection method, device, electronic equipment and storage medium

Similar Documents

Publication Publication Date Title
US20210352099A1 (en) System for automatically discovering, enriching and remediating entities interacting in a computer network
US9378114B2 (en) Code analysis method, code analysis system and computer storage medium
US9864672B2 (en) Module specific tracing in a shared module environment
CN107885660B (en) Fund system automatic test management method, device, equipment and storage medium
US20110296383A1 (en) Mechanism for Performing Dynamic Software Testing Based on Test Result Information Retrieved in Runtime Using Test Result Entity
US8316120B2 (en) Applicability detection using third party target state
US20150066869A1 (en) Module Database with Tracing Options
US20230144818A1 (en) Malicious software detection based on api trust
US10394793B1 (en) Method and system for governed replay for compliance applications
CN113849825A (en) Testing method and device for multi-bug scanner
CN114911706A (en) Use case recommendation method, device, equipment and storage medium
CN113535577B (en) Application testing method and device based on knowledge graph, electronic equipment and medium
CN112882957B (en) Test task validity checking method and device
CN114139160A (en) Method and system for determining software vulnerability influence range
CN109951527B (en) Virtualization system-oriented hypervisor integrity detection method
CN116595523A (en) Multi-engine file detection method, system, equipment and medium based on dynamic arrangement
US11921854B2 (en) Malware detection quality control
US11709993B2 (en) Efficient concurrent invocation of sheet defined functions including dynamic arrays
US8875092B2 (en) Certifying software components using a defect tracking system
CN110297625B (en) Application processing method and device
CN115220859A (en) Data input mode monitoring method and device, electronic equipment and storage medium
CN110955595A (en) Problem repairing method and device, electronic equipment and computer readable storage medium
JP2019144873A (en) Block diagram analyzer
CN111274585B (en) Method, device, equipment and medium for detecting unauthorized vulnerability of Web application
US12001545B2 (en) Detecting stack pivots using stack artifact verification

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination