CN114168464A - Distributed fuzzy test method and system based on dynamic scheduling - Google Patents
Distributed fuzzy test method and system based on dynamic scheduling Download PDFInfo
- Publication number
- CN114168464A CN114168464A CN202111473283.8A CN202111473283A CN114168464A CN 114168464 A CN114168464 A CN 114168464A CN 202111473283 A CN202111473283 A CN 202111473283A CN 114168464 A CN114168464 A CN 114168464A
- Authority
- CN
- China
- Prior art keywords
- test
- fuzzy
- tester
- node
- rate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3676—Test management for coverage analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3684—Test management for test design, e.g. generating new test cases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The application discloses a distributed fuzzy test method and a distributed fuzzy test system based on dynamic scheduling. In the method, a scheduling server configures access addresses and interface authentication information of each test node; configuring a corresponding target test node and establishing a fuzzy test relation with a scheduling server; testing by using the test case after the multi-node seed synchronization to obtain the coverage rate and the coverage rate increase rate; and then calculating to obtain the convergence rate of each fuzzy tester, when the convergence rate is lower than a preset convergence threshold value, replacing the fuzzy tester, continuously executing the test, finally obtaining the test results of all the fuzzy testers, removing the duplicate of the results to determine new defects, and adding the new defects into the new defect set after the duplicate removal. Therefore, the fuzzy testers which are difficult to exert effects are removed in real time, and other fuzzy testers which can still find new defects are added, so that the waste of computing power in the distributed cluster is avoided, and the overall efficiency is improved.
Description
Technical Field
The invention relates to the technical field of distributed fuzzy test, in particular to a distributed fuzzy test method based on dynamic scheduling.
Background
The fuzzy test is an automatic software testing technology, a large amount of random data is generated through a fuzzy tester (Fuzzer), deep defects of software are triggered, problem analysis and positioning are carried out through various detectors (such as Address Santizer and the like), safety and quality problems of various kinds of software are found and repaired, and safety and robustness of a software system are improved.
One type of existing fuzz testing techniques is the traditional single point mode. For example, the architecture of the tools such as Defensics of New thinking technology company, Peach of Gitlab company and the like is designed as traditional single-machine software, and a single-point mode is taken as a main use scene. The method comprises the steps that software runs on a single server, an object to be tested is tested through modes of network communication, files, command line access and the like, and the object to be tested only runs one instance. When the test system is oriented to a complex object to be tested, if a distributed test mode is needed, a user needs to manually start and add test nodes, a parallel test cluster is manually formed to test a single object to be tested, and each node in the cluster works independently and does not have cooperativity.
Another type of existing fuzz testing technology supports distributed testing, such as the Cluster fuzzy platform, marketed by Google, the Onefuzzy platform, marketed by Microsoft, and the like. The platform supports more than 1000 nodes to run simultaneously for parallel testing, and provides functions of cluster management and the like. Specifically, each node of the platform runs a fuzzy tester (Fuzzer), continuously generates test case data, tests the object to be tested, and the object to be tested runs a plurality of instances and is distributed on the platform nodes. The platform can support the operation of various different fuzzy testers, and the different fuzzy testers have different advantages and disadvantages in test performance aiming at different objects to be tested due to different algorithms and architectures, so that the overall test effect can be effectively improved by selecting and optimizing the fuzzy testers. In the test operation process of the platform, the fuzzy tester and the verification detector operated by each node are preset, and the optimization can be carried out only by repeated retesting.
Based on the characteristics, the traditional fuzzy test technology in the single-point mode does not have large-scale parallel test support, so the test capability is limited by the limitation of computing resources. With the increase of the complexity of the current software system, the testing capability and the testing efficiency of the single-point mode cannot meet the requirements of the complex object to be tested, and finally the test coverage rate is low, the coverage rate is slowly increased, and the number of defects is small. However, for the existing distributed fuzzy test technology, although cluster expansion can be effectively performed and a plurality of fuzzy testers are operated in parallel to improve the overall test capability, the existing distributed fuzzy test technology still has limitations. Because the running process is static, the combination of the fuzzy testers is not changed any more in the test, and the dynamic adjustment and optimization can not be carried out according to the test process. Aiming at the problem, manual tuning can only be adopted in the prior art, and the test must be repeatedly restarted based on the analysis of the test process and the result data, so that the test cost is increased and the overall efficiency is reduced.
Disclosure of Invention
Based on this, the embodiment of the application provides a distributed fuzzy test method and system based on dynamic scheduling, which can find out that the fuzzy tester whose current test coverage does not increase any more is replaced by the fuzzy tester which still increases faster, so that the computational power of the distributed cluster is effectively utilized, and the overall test efficiency is improved.
In a first aspect, a distributed fuzz testing method based on dynamic scheduling is provided, and the method includes:
the scheduling server configures the access address and interface authentication information of each test node and establishes connection with the scheduling server; the test nodes comprise a fuzzy tester and test environment dependent items;
configuring a corresponding target test node and establishing a fuzzy test relation with a scheduling server according to the obtained target test item; the fuzzy test relation comprises multi-node integration synchronization of the seed test cases generated by each target test node;
testing all the fuzzy testers to obtain the coverage rate of each fuzzy tester; calculating the coverage rate increase rate of each fuzzy tester according to the time change based on the coverage rate of each fuzzy tester; calculating the convergence rate of each fuzzy tester according to the coverage rate increase rate of each fuzzy tester, and when the convergence rate of each fuzzy tester is lower than a preset convergence threshold value, replacing the fuzzy tester and continuously executing the test;
and acquiring the test results of all the fuzzy testers, removing the results to determine new defects, and adding the new defects into the removed new defect set.
Optionally, the configuring, according to the obtained target test item, a corresponding target test node and a scheduling server to establish a fuzzy test relationship includes:
establishing a mapping table containing target test item global coverage information;
in the whole fuzzy test process, integrating the results of all fuzzy testers in real time, comparing the results with a global mapping table, identifying seed cases capable of covering new paths or new branches, and updating the coverage mapping table;
marking the use case as an interesting seed use case, and adding the use case into a global use case pool;
and the test node synchronously transfers the newly added seed cases from the global case pool to the local case pool and continues to test.
Optionally, the obtaining test results of all the fuzzy testers, performing result deduplication to determine new defects, and adding the new defects into the deduplicated new defect set further includes:
collecting error-reporting stack information output by testing, and forming a set with the error stack information which has finished de-duplication currently;
traversing the error stack information set, comparing error stacks frame by frame, and judging whether the error stacks are repeated;
if the defect is a repeated defect, discarding the defect, otherwise, confirming the new defect and adding the new defect set with the repeated defect.
Optionally, the configuring, by the dispatch server, the access address and the interface authentication information of each test node, and establishing a connection with the dispatch server includes:
and the dispatching server configures an access address and interface identity authentication information, is arranged in the test node, and issues and deploys the test node to the distributed cluster host by using a cluster management tool.
Optionally, when there is a convergence rate of the fuzz tester lower than a preset convergence threshold, replacing the fuzz tester, further includes:
sequencing the fuzzy testers according to the convergence rate;
when a fuzzy tester is found to be converged, a scheduling operation is triggered, and the fuzzy tester type with low convergence rate in the queue is used for replacing the fuzzy tester type with high convergence rate.
Optionally, the method further comprises:
feeding back the test result to a dispatching server;
and stopping the test by the scheduling server, and stopping the fuzzy tester by the corresponding test node.
Optionally, the convergence rate is a ratio of a convergence threshold to a coverage rate increase rate.
In a second aspect, a distributed fuzz testing system based on dynamic scheduling is provided, the system comprising:
the test management module is used for enabling the scheduling server to configure the access address and the interface authentication information of each test node and establish connection with the scheduling server; the test nodes comprise a fuzzy tester and test environment dependent items;
the case synchronization module is used for configuring corresponding target test nodes and establishing a fuzzy test relation with the scheduling server side according to the obtained target test items; the fuzzy test relation comprises the integration and synchronization of the seed test cases generated by each target test node;
the dynamic scheduling module is used for testing all the fuzzy testers to obtain the coverage rate of each fuzzy tester; calculating the coverage rate increase rate of each fuzzy tester according to the time change based on the coverage rate of each fuzzy tester; calculating the convergence rate of each fuzzy tester according to the coverage rate increase rate of each fuzzy tester, and when the convergence rate of each fuzzy tester is lower than a preset convergence threshold value, replacing the fuzzy tester and continuously executing the test;
and the result duplication removing module is used for acquiring the test results of all the fuzzy testers, removing the duplication of the results so as to determine new defects, and adding the new defects into the duplicated new defect set.
Optionally, the use case synchronization module specifically includes:
establishing a mapping table containing target test item global coverage information;
in the whole fuzzy test process, integrating the results of all fuzzy testers in real time, comparing the results with a global mapping table, identifying seed cases capable of covering new paths or new branches, and updating the coverage mapping table;
marking the use case as an interesting seed use case, and adding the use case into a global use case pool;
and the test node synchronously transfers the newly added seed cases from the global case pool to the local case pool and continues to test.
Optionally, the result deduplication module specifically includes:
collecting error-reporting stack information output by testing, and forming a set with the error stack information which has finished de-duplication currently;
traversing the error stack information set, comparing error stacks frame by frame, and judging whether the error stacks are repeated;
if the defect is a repeated defect, discarding the defect, otherwise, confirming the new defect and adding the new defect set with the repeated defect.
In the technical scheme provided by the embodiment of the application, firstly, a scheduling server configures an access address and interface authentication information of each test node and establishes connection with the scheduling server; configuring a corresponding target test node and establishing a fuzzy test relation with a scheduling server; then, obtaining a test case with synchronized multi-node seeds, and testing all the fuzzy testers to obtain coverage rate; and based on obtaining the coverage rate increase rate of each fuzzy tester; and calculating the convergence rate of each fuzzy tester according to the coverage rate increase rate of each fuzzy tester, when the convergence rate of each fuzzy tester is lower than a preset convergence threshold value, replacing the fuzzy tester, continuously executing the test, finally obtaining the test results of all the fuzzy testers, removing the results to determine new defects, and adding the new defects into the removed new defect set. The method has the advantages that the fuzzy testers which are difficult to exert effects can be removed in real time in the parallel testing scene of the heterogeneous fuzzy testers, and other fuzzy testers which can still find new defects are added, so that the waste of computing power in the distributed cluster is avoided, and the overall efficiency is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below. It should be apparent that the drawings in the following description are merely exemplary, and that other embodiments can be derived from the drawings provided by those of ordinary skill in the art without inventive effort.
Fig. 1 is an overall test flowchart of a distributed fuzzy test method based on dynamic scheduling according to an embodiment of the present application;
fig. 2 is a schematic diagram of a use case synchronization mechanism of a distributed fuzzy test method based on dynamic scheduling according to an embodiment of the present application;
fig. 3 is a dynamic tuning flowchart of a distributed fuzzy test method based on dynamic scheduling according to an embodiment of the present application;
FIG. 4 is a flowchart illustrating result deduplication of a distributed fuzzy test method based on dynamic scheduling according to an embodiment of the present application;
fig. 5 is an architecture diagram of a distributed fuzz testing system based on dynamic scheduling according to an embodiment of the present application.
Detailed Description
The present invention is described in terms of particular embodiments, other advantages and features of the invention will become apparent to those skilled in the art from the following disclosure, and it is to be understood that the described embodiments are merely exemplary of the invention and that it is not intended to limit the invention to the particular embodiments disclosed. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The application provides a distributed fuzzy test method based on dynamic scheduling. On one hand, the cloud-end architecture is based on, the control scheduling of the global level is carried out by the coordination service on the server, and the support of large-scale distributed testing is realized. On the other hand, the dynamic scheduling method based on the statistical data is realized, and after a certain fuzzy tester (Fuzzer) is started, the running state of the fuzzy tester is continuously collected and monitored. When the operation effect of a certain tester is found to be reduced, the operation of the tester can be automatically stopped, and another tester of other types can be selected for testing, so that the reduction of the whole testing effect caused by the convergence of a single tester is effectively avoided.
Referring to fig. 1, a flowchart of a distributed fuzz testing method based on dynamic scheduling provided by an embodiment of the present application is shown, where the method may include the following steps:
step 101, the scheduling server configures the access address and the interface authentication information of each test node, and establishes connection with the scheduling server.
The test nodes comprise a fuzzy tester and test environment dependent items.
In the embodiments of the present application, this step is the basic capability of the present application. The core functions of the module are test node management, test cluster establishment and test process management. The core process is the process management of the building and testing of the test cluster, and the step 101 specifically includes:
(1) and the scheduling service configures an access address and interface identity authentication information and is arranged in the Agent test node. And using a cluster management tool to issue and deploy the Agent test node to the distributed cluster host.
(2) And initializing the test node, establishing connection with the scheduling service, and finishing cluster initialization. The test node automatically installs each fuzz tester (Fuzzer) and test environment dependencies.
(3) And (4) newly building a test project by the dispatching service, downloading the program of the object to be tested from the dispatching service by the corresponding test node according to parameters such as the number of nodes set by the test project, and the like, and completing the compiling and detector integration of the object to be tested.
(4) And dispatching the service to start the test, and starting the fuzzy tester by the corresponding test node according to the parameters such as the parallel number and the like set by the test to start the test. And feeding back the test result to the dispatching service.
(5) The dispatch service aborts the test and the corresponding test node stops the fuzzy tester.
And 102, configuring a corresponding target test node and establishing a fuzzy test relation with a scheduling server according to the obtained target test item.
The fuzzy test relation comprises multi-node integration synchronization of the seed test cases generated by the target test nodes.
The case synchronization is an important characteristic of the application different from the traditional parallel test of a plurality of single machine systems, so that the test nodes in the cluster can be effectively cooperated, and the advantage of the distributed cluster computing capability is exerted. The case synchronization module can integrate and synchronize the seed test cases generated by the fuzzy testers of the test nodes, increase the coverage and lifting speed of the fuzzy testers, and reduce unnecessary repeated work of the fuzzy testers, so that the overall test effect is improved. In the embodiment of the present application, as shown in fig. 2, step 102 specifically includes:
(1) and establishing a mapping table containing global coverage information of the object to be measured.
(2) In the whole fuzzy test process, the results of all fuzzy testers are integrated in real time and compared with the global mapping table, the seed cases capable of covering new paths or new branches are identified, and the coverage mapping table is updated.
(3) And marking the use case as an interesting seed use case, and adding the use case into the global use case pool.
The testing node continues to test from the global case pool to the local case pool, and in the embodiment of the application, the efficiency of the test can be effectively improved by synchronously obtaining the multi-node seeds to perform the test.
And 103, testing all the fuzzy testers to obtain the coverage rate increase rate of each fuzzy tester and calculating to obtain the convergence rate of each fuzzy tester, and when the convergence rate of each fuzzy tester is lower than a preset convergence threshold, replacing the fuzzy tester and continuously executing the test.
Wherein, testing all the fuzzy testers to obtain the coverage rate increase rate of each fuzzy tester specifically comprises: testing all the fuzzy testers to obtain the coverage rate of each fuzzy tester; and calculating the coverage rate increase rate of each fuzzy tester according to the time change based on the coverage rate of each fuzzy tester.
In the step, the convergence threshold of the fuzzy tester in each target test node is firstly set, the convergence rate of each fuzzy tester is calculated according to the coverage rate increase rate of each fuzzy tester, and when the convergence rate of the fuzzy tester is lower than the preset convergence threshold, the fuzzy tester is replaced, and the test is continuously executed.
In the process, the coverage rate is defined as the ratio of the number of the paths touched by the test case to the total number of the paths of the program to be tested. The step of testing all the fuzz testers to obtain the coverage rate of each fuzz tester comprises the following steps: collecting the total path number of the programs in each fuzzy tester and the number of the paths touched by the test cases; and calculating the coverage rate of each fuzz tester according to the ratio of the two.
Convergence is defined as the rate of coverage increase remaining below a target threshold for a specified length of time, and is determined to be convergence. The convergence rate is defined as the ratio of the target threshold to the coverage rate increase rate. The target threshold is set according to the type of the object to be tested or the test requirement.
As shown in fig. 3, the dynamic scheduling process specifically includes:
(1) for the initial test, all types of fuzzy testers (Fuzzer) are selected for testing, and the number of each type of tester is evenly distributed.
(2) And when the test is started, the coverage rate monitoring module collects the coverage rate information of each fuzzy tester.
(3) And calculating the convergence rate of each fuzzy tester according to the coverage rate increase condition.
(4) And sequencing the fuzzy testers according to the convergence rate.
(5) When a fuzzy tester is found to be converged, a scheduling operation is triggered, and the fuzzy tester type with low convergence rate in the queue is used for replacing the fuzzy tester type with high convergence rate.
(6) The test continues to be performed.
Dynamic scheduling is a key feature that distinguishes the present application from existing distributed fuzz testing techniques. By collecting test coverage indexes of the fuzzy tester during operation, the test coverage convergence conditions of different fuzzy testers are monitored, and real-time optimization is performed based on a scheduling strategy, so that the distributed cluster computing resource utilization rate is further improved, and the test effect is improved.
And 104, acquiring the test results of all the fuzzy testers, removing the results to determine new defects, and adding the new defects into the removed new defect set.
Result deduplication is an important characteristic for improving the usability of distributed fuzz testing. In the embodiment of the application, the new defect test case (namely, the result deduplication) with the repeated result of the multiple testers removed can be obtained, so that the situation that the multiple fuzzy testers output too much repeated defect information to bring about false alarm interference can be avoided. In the testing process, each new found potential defect, the scheduling server will execute the result deduplication operation, as shown in fig. 4, and the specific process of the result deduplication includes:
(1) and executing the new defect test case by using the object program to be tested added with the detector.
(2) Collecting the output error stack information and forming a set with the error stack information which has finished the duplication elimination.
(3) And traversing the error stack information set, comparing error stacks frame by frame, and judging whether the error stacks are repeated.
(4) If the defect is a repeated defect, discarding the defect, otherwise, the result duplication removing module confirms the new defect and adds the new defect into the duplication removed set.
In summary, the distributed fuzzy test method provided by the application adopts a cloud-end architecture and processes of test cluster management, case synchronization, result deduplication and the like, and has many improvements compared with the prior art. In the single-machine mode, the parallel computing capability cannot be utilized, and the testing efficiency is limited. The parallel operation of the single machine testing tool leads to the increase of the operation cost because of the lack of a cluster management mechanism; the lack of a case synchronization mechanism can cause each node to be tested repeatedly; lack of a result deduplication mechanism will result in reduced usability. The distributed fuzzy test method can improve the coverage speed, reduce repeated calculation, effectively utilize cluster calculation capacity and ensure the usability of test results.
The dynamic scheduling method provided by the application is based on test coverage rate statistics and convergence rate calculation, and can be used for applying calculation power to a fuzzy tester type with higher real-time test capability. The existing distributed fuzzy test method can only statically use various types of fuzzy testers based on early-stage setting, and when a certain type of fuzzy tester converges in advance, the computational power is wasted, and manual optimization can only be performed through repeated retesting. The dynamic scheduling method can automatically find the fuzzy tester with the current test coverage not increasing any more, and replace the fuzzy tester with the fuzzy tester which still increases rapidly, so that the calculation power of the distributed cluster is effectively utilized, and the overall test efficiency is improved.
As shown in fig. 5, an embodiment of the present application further provides a distributed fuzz testing system based on dynamic scheduling. The system generally comprises cloud scheduling service and distributed test nodes, wherein the scheduling service comprises 4 main functional modules of test management, case synchronization, result duplication removal and dynamic scheduling:
the test management module is used for enabling the scheduling server to configure the access address and the interface authentication information of each test node and establish connection with the scheduling server; the test nodes comprise a fuzzy tester and test environment dependent items;
the case synchronization module is used for configuring corresponding target test nodes and establishing a fuzzy test relation with the scheduling server side according to the obtained target test items; the fuzzy test relation comprises the integration and synchronization of the seed test cases generated by each target test node;
the dynamic scheduling module is used for testing all the fuzzy testers to obtain the coverage rate of each fuzzy tester; calculating the coverage rate increase rate of each fuzzy tester according to the time change based on the coverage rate of each fuzzy tester; calculating the convergence rate of each fuzzy tester according to the coverage rate increase rate of each fuzzy tester, and when the convergence rate of each fuzzy tester is lower than a preset convergence threshold value, replacing the fuzzy tester and continuously executing the test;
and the result duplication removing module is used for acquiring the test results of all the fuzzy testers, removing the duplication of the results so as to determine new defects, and adding the new defects into the duplicated new defect set.
In an optional embodiment of the present application, the use case synchronization module specifically includes:
establishing a mapping table containing target test item global coverage information; in the whole fuzzy test process, integrating the results of all fuzzy testers in real time, comparing the results with a global mapping table, identifying seed cases capable of covering new paths or new branches, and updating the coverage mapping table; marking the use case as an interesting seed use case, and adding the use case into a global use case pool; and the test node synchronously transfers the newly added seed cases from the global case pool to the local case pool and continues to test.
In an optional embodiment of the present application, the result deduplication module specifically includes:
collecting error-reporting stack information output by testing, and forming a set with the error stack information which has finished de-duplication currently;
traversing the error stack information set, comparing error stacks frame by frame, and judging whether the error stacks are repeated;
if the defect is a repeated defect, discarding the defect, otherwise, confirming the new defect, and adding the de-duplicated new defect test case set.
For specific limitations of a distributed fuzzy test system based on dynamic scheduling, reference may be made to the above limitations of a distributed fuzzy test method based on dynamic scheduling, and details of the limitations are not repeated here. The various parts of the distributed fuzzy test system based on dynamic scheduling can be wholly or partially realized by software, hardware and the combination thereof. The modules can be embedded in a hardware form or independent from a processor in the device, and can also be stored in a memory in the device in a software form, so that the processor can call and execute operations corresponding to the modules.
The technical features of the embodiments described above may be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the embodiments described above are not described, but should be considered as being within the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the claims. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. A distributed fuzzy test method based on dynamic scheduling is characterized by comprising the following steps:
the scheduling server configures the access address and interface authentication information of each test node and establishes connection with the scheduling server; the test nodes comprise a fuzzy tester and test environment dependent items;
configuring a corresponding target test node and establishing a fuzzy test relation with a scheduling server according to the obtained target test item; the fuzzy test relation comprises multi-node integration synchronization of the seed test cases generated by each target test node;
testing all the fuzzy testers to obtain the coverage rate of each fuzzy tester; calculating the coverage rate increase rate of each fuzzy tester according to the time change based on the coverage rate of each fuzzy tester; calculating the convergence rate of each fuzzy tester according to the coverage rate increase rate of each fuzzy tester, and when the convergence rate of each fuzzy tester is lower than a preset convergence threshold value, replacing the fuzzy tester and continuously executing the test;
and acquiring the test results of all the fuzzy testers, removing the results to determine new defects, and adding the new defects into the removed new defect set.
2. The method of claim 1, wherein the configuring, according to the obtained target test item, a corresponding target test node and a scheduling server to establish a fuzzy test relationship comprises:
establishing a mapping table containing target test item global coverage information;
in the whole fuzzy test process, integrating the results of all fuzzy testers in real time, comparing the results with a global mapping table, identifying seed cases capable of covering new paths or new branches, and updating the coverage mapping table;
marking the use case as an interesting seed use case, and adding the use case into a global use case pool;
and the test node synchronously transfers the newly added seed cases from the global case pool to the local case pool and continues to test.
3. The method of claim 1, wherein the obtaining test results of all fuzzy testers, performing result deduplication to determine new defects, and adding new defects to the deduplicated new defect set comprises:
collecting error-reporting stack information output by testing, and forming a set with the error stack information which has finished de-duplication currently;
traversing the error stack information set, comparing error stacks frame by frame, and judging whether the error stacks are repeated;
if the defect is a repeated defect, discarding the defect, otherwise, confirming the new defect and adding the new defect set with the repeated defect.
4. The method of claim 1, wherein the scheduling server configures an access address and interface authentication information of each test node, and establishes a connection with the scheduling server, including:
and the dispatching server configures an access address and interface identity authentication information, is arranged in the test node, and issues and deploys the test node to the distributed cluster host by using a cluster management tool.
5. The method of claim 1, wherein when there is a ambiguity tester whose convergence rate is lower than a preset convergence threshold, replacing the ambiguity tester, further comprises:
sequencing the fuzzy testers according to the convergence rate;
when a fuzzy tester is found to be converged, a scheduling operation is triggered, and the fuzzy tester type with low convergence rate in the queue is used for replacing the fuzzy tester type with high convergence rate.
6. The method of claim 1, further comprising:
feeding back the test result to a dispatching server;
and stopping the test by the scheduling server, and stopping the fuzzy tester by the corresponding test node.
7. The method of claim 1, wherein the convergence rate is a ratio of a convergence threshold to a coverage rate increase rate.
8. A distributed fuzz testing system based on dynamic scheduling, the system comprising:
the test management module is used for enabling the scheduling server to configure the access address and the interface authentication information of each test node and establish connection with the scheduling server; the test nodes comprise a fuzzy tester and test environment dependent items;
the case synchronization module is used for configuring corresponding target test nodes and establishing a fuzzy test relation with the scheduling server side according to the obtained target test items; the fuzzy test relation comprises multi-node integration synchronization of the seed test cases generated by each target test node;
the dynamic scheduling module is used for testing all the fuzzy testers to obtain the coverage rate of each fuzzy tester; calculating the coverage rate increase rate of each fuzzy tester according to the time change based on the coverage rate of each fuzzy tester; calculating the convergence rate of each fuzzy tester according to the coverage rate increase rate of each fuzzy tester, and when the convergence rate of each fuzzy tester is lower than a preset convergence threshold value, replacing the fuzzy tester and continuously executing the test;
and the result duplication removing module is used for acquiring the test results of all the fuzzy testers, removing the duplication of the results so as to determine new defects, and adding the new defects into the duplicated new defect set.
9. The system according to claim 8, wherein the use case synchronization module specifically includes:
establishing a mapping table containing target test item global coverage information;
in the whole fuzzy test process, integrating the results of all fuzzy testers in real time, comparing the results with a global mapping table, identifying seed cases capable of covering new paths or new branches, and updating the coverage mapping table;
marking the use case as an interesting seed use case, and adding the use case into a global use case pool;
and the test node synchronously transfers the newly added seed cases from the global case pool to the local case pool and continues to test.
10. The system of claim 8, wherein the result deduplication module specifically comprises:
collecting error-reporting stack information output by testing, and forming a set with the error stack information which has finished de-duplication currently;
traversing the error stack information set, comparing error stacks frame by frame, and judging whether the error stacks are repeated;
if the defect is a repeated defect, discarding the defect, otherwise, confirming the new defect and adding the new defect set with the repeated defect.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111473283.8A CN114168464A (en) | 2021-12-02 | 2021-12-02 | Distributed fuzzy test method and system based on dynamic scheduling |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111473283.8A CN114168464A (en) | 2021-12-02 | 2021-12-02 | Distributed fuzzy test method and system based on dynamic scheduling |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114168464A true CN114168464A (en) | 2022-03-11 |
Family
ID=80483117
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111473283.8A Withdrawn CN114168464A (en) | 2021-12-02 | 2021-12-02 | Distributed fuzzy test method and system based on dynamic scheduling |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114168464A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115658534A (en) * | 2022-11-04 | 2023-01-31 | 上海安般信息科技有限公司 | Distributed fuzzy test method and device based on multi-engine fusion |
| CN115858366A (en) * | 2022-12-13 | 2023-03-28 | 北京水木羽林科技有限公司 | Grammar generation-based database automatic test method and system |
| CN115964275A (en) * | 2022-12-13 | 2023-04-14 | 北京水木羽林科技有限公司 | Distributed fuzzy test acceleration method and system |
-
2021
- 2021-12-02 CN CN202111473283.8A patent/CN114168464A/en not_active Withdrawn
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115658534A (en) * | 2022-11-04 | 2023-01-31 | 上海安般信息科技有限公司 | Distributed fuzzy test method and device based on multi-engine fusion |
| CN115658534B (en) * | 2022-11-04 | 2024-01-19 | 上海安般信息科技有限公司 | Distributed fuzzy test method and device based on multi-engine fusion |
| CN115858366A (en) * | 2022-12-13 | 2023-03-28 | 北京水木羽林科技有限公司 | Grammar generation-based database automatic test method and system |
| CN115964275A (en) * | 2022-12-13 | 2023-04-14 | 北京水木羽林科技有限公司 | Distributed fuzzy test acceleration method and system |
| CN115964275B (en) * | 2022-12-13 | 2023-08-29 | 北京水木羽林科技有限公司 | Distributed fuzzy test acceleration method and system |
| CN115858366B (en) * | 2022-12-13 | 2024-03-29 | 北京水木羽林科技有限公司 | Automatic database testing method and system based on grammar generation |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114168464A (en) | Distributed fuzzy test method and system based on dynamic scheduling | |
| US10635473B2 (en) | Setting support program, setting support method, and setting support device | |
| CN106708740B (en) | Script testing method and device | |
| CN106201502B (en) | Cross-host application program deployment method and device | |
| JP2010503074A5 (en) | ||
| CN107656705B (en) | Computer storage medium and data migration method, device and system | |
| WO2019214010A1 (en) | Method and device for monitoring for equipment failure | |
| US10910081B2 (en) | Management of test resources to perform reliability testing of memory components | |
| CN109063005B (en) | Data migration method and system, storage medium and electronic device | |
| CN111324599B (en) | Block chain experiment system and management method | |
| CN108920369A (en) | A kind of server stress test method, device, equipment and storage medium | |
| US20150019722A1 (en) | Determining, managing and deploying an application topology in a virtual environment | |
| CN108170507B (en) | Virtual application management method/system, computer readable storage medium and server | |
| CN109597764A (en) | A kind of test method and relevant apparatus of catalogue quota | |
| CN104410698B (en) | Initiation node abnormality eliminating method and device under a kind of share nothing cluster | |
| Genkin et al. | Automatic, on-line tuning of YARN container memory and CPU parameters | |
| WO2020131809A1 (en) | Multi-dimensional usage space testing of memory components | |
| CN110196751B (en) | Method and device for isolating mutual interference service, electronic equipment and storage medium | |
| CN114281508A (en) | Data batch-flow fusion offline calculation method | |
| CN116346826B (en) | Database node deployment method and device, electronic equipment and storage medium | |
| CN106612213B (en) | Equipment testing method and device | |
| CN109995571B (en) | Method and device for matching server configuration and VNF application | |
| CN105511952A (en) | Resource self-migration method and system based on cloud computing platform | |
| CN111857767B (en) | Virtualization system upgrading method and device based on automatic migration of virtual machine | |
| CN110007934B (en) | Distributed database deployment method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20220311 |