CN112306853A - Fuzzy test method, device, equipment and medium - Google Patents
Fuzzy test method, device, equipment and medium Download PDFInfo
- Publication number
- CN112306853A CN112306853A CN201910707526.6A CN201910707526A CN112306853A CN 112306853 A CN112306853 A CN 112306853A CN 201910707526 A CN201910707526 A CN 201910707526A CN 112306853 A CN112306853 A CN 112306853A
- Authority
- CN
- China
- Prior art keywords
- test
- statement
- generating
- resource
- testing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000010998 test method Methods 0.000 title abstract description 10
- 238000012360 testing method Methods 0.000 claims abstract description 595
- 238000013515 script Methods 0.000 claims abstract description 54
- 238000000034 method Methods 0.000 claims abstract description 36
- 230000006870 function Effects 0.000 claims description 47
- 238000013507 mapping Methods 0.000 claims description 3
- 230000008569 process Effects 0.000 description 10
- 239000012634 fragment Substances 0.000 description 9
- 238000012545 processing Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 8
- 238000004891 communication Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 5
- 230000005540 biological transmission Effects 0.000 description 4
- 230000035772 mutation Effects 0.000 description 4
- 230000007547 defect Effects 0.000 description 3
- 230000007246 mechanism Effects 0.000 description 3
- 230000009182 swimming Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 235000000332 black box Nutrition 0.000 description 2
- 230000007123 defense Effects 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000013522 software testing Methods 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 101000767160 Saccharomyces cerevisiae (strain ATCC 204508 / S288c) Intracellular protein transport protein USO1 Proteins 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 101150047356 dec-1 gene Proteins 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000013467 fragmentation Methods 0.000 description 1
- 238000006062 fragmentation reaction Methods 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000012372 quality testing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3676—Test management for coverage analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3684—Test management for test design, e.g. generating new test cases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a fuzzy test method, a device, equipment and a medium, wherein the method comprises the steps of obtaining test seeds and generating a test random parameter sequence; acquiring a test object set; extracting the test resources pointed by each test object to form a test resource set corresponding to the test object set; generating a variable pool based on the testing random parameter sequence and the testing resource set, wherein all variables required for constructing the testing case of each testing object in the testing object set are stored in the variable pool; generating a test statement based on the test random parameter sequence and the test resource set to form a test statement set corresponding to the test resource set; generating a test resource script based on the variable pool and the test statement set, and outputting all statements in the test resource script out of order to obtain a fuzzy test script recorded with a test case; and testing according to the fuzzy test script to obtain a test result. The invention can greatly improve the testing efficiency.
Description
Technical Field
The invention relates to the field of testing, in particular to a fuzzy testing method, a fuzzy testing device, equipment and a medium.
Background
Fuzz testing is an effective strategy for discovering defects in software under test. It shows great potential for vulnerability detection automation. Meanwhile, because the automation degree of the fuzzy test is high, once the environment is configured, the fuzzy test method can be applied to continuous vulnerability discovery, and is remarkable in human optimization for security researchers or enterprises. The fuzzy test has various forms, such as a switch-case use test mode, which can be connected in series to form an approximately randomized statement stream, but has the problems that the functions to be tested and the contents transferred to the test functions are approximately fixed, and although a large number of random values are generated, the actual code coverage is not high, and further, like the existing use case test mode based on the grammar rule, the automation degree is improved, but the test cases of a multilayer nested mode are difficult to generate, and the test effect is correspondingly influenced.
Disclosure of Invention
In order to solve the technical problem that a high-quality and high-code-coverage fuzz testing method cannot be provided in the prior art, embodiments of the present invention provide a fuzz testing method, apparatus, device, and medium.
In one aspect, the present invention provides a fuzz testing method, including:
acquiring test seeds, and generating a test random parameter sequence based on the test seeds;
acquiring a test object set, wherein the test object set comprises at least one test object;
extracting the test resources pointed by each test object to form a test resource set corresponding to the test object set;
generating a variable pool based on the testing random parameter sequence and the testing resource set, wherein all variables required for constructing the testing case of each testing object in the testing object set are stored in the variable pool;
generating a test statement based on the test random parameter sequence and the test resource set to form a test statement set corresponding to the test resource set;
generating a test resource script based on the variable pool and the test statement set, and outputting all statements in the test resource script out of order to obtain a fuzzy test script recorded with a test case;
and testing according to the fuzzy test script to obtain a test result.
In another aspect, the present invention provides a fuzz testing apparatus, the apparatus comprising:
the test seed module is used for acquiring test seeds and generating a test random parameter sequence based on the test seeds;
the device comprises a test object set acquisition module, a test object set acquisition module and a test object analysis module, wherein the test object set acquisition module is used for acquiring a test object set, and the test object set comprises at least one test object;
the test resource set acquisition module is used for extracting the test resources pointed by each test object to form a test resource set corresponding to the test object set;
a variable pool generating module, configured to generate a variable pool based on the random test parameter sequence and the test resource set, where all variables required for constructing test cases of the test objects in the test object set are stored in the variable pool;
the test statement set generation module is used for generating a test statement based on the test random parameter sequence and the test resource set so as to form a test statement set corresponding to the test resource set;
the fuzzy test script generation module is used for generating a test resource script based on the variable pool and the test statement set, and outputting all statements in the test resource script out of order to obtain a fuzzy test script recorded with a test case;
and the fuzzy test module is used for testing according to the fuzzy test script to obtain a test result.
In another aspect, the present invention provides a fuzz testing apparatus, characterized in that the apparatus comprises a processor and a memory, wherein at least one instruction, at least one program, a set of codes, or a set of instructions is stored in the memory, and the at least one instruction, the at least one program, the set of codes, or the set of instructions is loaded and executed by the processor to implement a fuzz testing method.
In another aspect, the present invention provides a computer storage medium having stored therein at least one instruction, at least one program, set of codes, or set of instructions that is loaded by a processor and that performs a fuzz testing method.
The invention provides a fuzzy test method, a fuzzy test device, equipment and a medium. The invention realizes the automatic generation of the test statement by pertinently designing the function definition language support, the called level nesting and the type matching mechanism so as to obtain the effective or ineffective test script, and the script is used for carrying out the fuzzy test on the test object so as to find the stability problem and the safety problem in the target software. Through relevant verification, the expected value of code coverage is higher after the embodiment of the invention is implemented for a long time. Meanwhile, the black box can be used according to the test scheme, the problem that the black box is difficult to insert piles and does not have a good mode to improve the code coverage rate is solved, and the test efficiency can be greatly improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions and advantages of the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
FIG. 1 is a diagram of an implementation environment of a fuzzy testing method provided by the present invention;
FIG. 2 is a flow chart of a fuzzy testing method provided by the present invention;
FIG. 3 is a flow chart of generating a variable pool based on the test random parameter sequence and the test resource set according to the present invention;
FIG. 4 is a flow chart of a test statement set according to the present invention, wherein the test statement set is generated based on the test random parameter sequence and the test resource set to form a test statement set corresponding to the test resource set;
FIG. 5 is a flow chart of the present invention for invoking simple statement generating functions in the test resources and generating test statements based on the test random argument sequence;
FIG. 6 is a schematic diagram of the logic for generating the test statement set provided by the present invention;
FIG. 7 is a logical framework diagram of a fuzzy testing method in a scenario according to the present invention;
FIG. 8 is a block diagram of a fuzz test provided by the present invention;
fig. 9 is a hardware structural diagram of an apparatus for implementing the method provided by the embodiment of the invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or server that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
In order to make the objects, technical solutions and advantages disclosed in the embodiments of the present invention more clearly apparent, the embodiments of the present invention are described in further detail below with reference to the accompanying drawings and the embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the embodiments of the invention and are not intended to limit the embodiments of the invention. To facilitate the detailed description of the embodiments of the present invention, the related concepts and the related technical status of the embodiments of the present invention will be described first.
JavaScript: a high-level programming language is a dynamic type, object-oriented (prototype-based) scripting language.
Mutation test: for software testing and assessing the quality of existing software. Mutation testing is to observe whether the output is normal or not by slightly modifying the program input so as to test the stability of the program and discover security holes.
And (3) fuzzy testing: is a software testing technology. The core idea is to generate random data input into a program and monitor program exceptions to discover possible program errors. Fuzz testing is often used to detect security breaches of a software or computer system.
Modularization: segmentation, organization, and packaging systems. Each module performs a specific sub-function, and all modules are assembled together in a certain way to form a whole to perform the functions required by the whole system.
Tree: a data set having the nature of a tree structure is simulated. It is a set with hierarchy relationship composed of n (n >0) finite nodes. It is called a "tree" because it looks like an inverted tree, i.e., it is root up and leaf down.
Specific syntax tree, parse tree: is an ordered tree structure with a root node that reflects the grammatical relations of a certain form of language string.
Black box testing: testing performed with complete or substantial ignorance of the source code or implementation of the target program.
White box testing: tests performed with full or substantial knowledge of the source code or implementation of the target program.
And (3) target software: the software to be fuzz tested may be any software that accepts external input and processes the input.
Test case: the set of data or steps used for testing is called a test case.
And dom: a document object model.
Before describing the method for testing the fuzziness in the embodiment of the present invention in detail, an overview of the related art of the fuzziness testing is first provided.
The fuzz test is a 1988 course program originated from professor Barton Miller, university of wisconsin. This project is aimed at testing the reliability of Unix (a multi-user, multi-tasking, time-sharing operating system, supporting multiple processor architectures) by executing large numbers of random inputs in rapid succession until they crash. It also provides an early debugging tool to determine the cause and category of each detected fault. Now, the term fuzz testing has been limited to command line tools only, but a broad concept that applies to all testable things. In 4 months 2012, google announced ClusterFuzz, which is a cloud-based fuzzy test system used for performing security test on functions of a webpage browser which is developed by google mainly. If ClusterFuzz finds some security issues, security researchers can upload their own fuzz testing programs and obtain rewards. In the year 2015, the period of 4 months,
how the fuzzy AFL finds 2014 heartbled holes is shown. (the vulnerability of Heartbed was disclosed in month 4 2014, which is a serious vulnerability that allows attackers to decipher otherwise encrypted communications, such as data from internet banking communications.23 million machines were reported to be vulnerable in month 4 2016). In 2016, 8 months, the first Cyber Grand Challenge general event was initiated by the United states department of Defense Advanced Research Program (DARPA), an automated game that lasted 11 hours. The goal is to develop an automatic defense system that can discover, exploit, and correct software defects in real time.
The test cases are distinguished according to the method for generating the test cases by the fuzz test, and can be divided into the following categories:
completely inputting at random: the applicable mode of both black box and white box tests belongs to the simplest fuzzy test mode, and random data is generated by a random number generator and input to target software.
Boundary/special value input: the black box and white box tests can be applied to test whether the software works normally by inputting boundary values or special values into the target software, wherein the boundary values or the special values are as follows: 0. 1, -1, MAX _ INT +1, MAX _ INT-1, etc., that fall on some mathematically defined or software engineering defined boundary. This test mode can significantly reduce the number of test cases and time, but at the same time is less likely to discover other types of security issues.
And (3) mode input: both black-box and white-box testing may be applied by entering into the target software a fixed pattern specified by the fuzzifier writer, such as the introduction of a string of ten thousand in length, the introduction of a random file, and the like.
switch-case test: both black and white box tests are applicable. I.e. all the functions to be tested are written according to their syntax specifications. Then a random number is used to control which one of them is executed at all, and the random number selection step is repeated to concatenate a nearly randomized statement stream. The problem is that the function tested and the content passed to the test function are both approximately fixed, and although a large number of random values are produced, the actual code coverage is not high. To improve the code coverage, the code amount of the fuzzy test program will be greatly increased, and the test time will be correspondingly greatly increased, resulting in a decrease in efficiency.
Pile inserting branch testing: white box test mode of application. The fuzz test program captures branch boundary coverage and runtime branch execution counts by instrumentation of some instructions during compilation. When the input can lead out a new branch, the input is recorded and the mutation is continued.
Compiling branch test: white box test mode of application. Compiling options such as cross-border detection and the like by using a compiler provided by a fuzz testing tool, linking partial codes of fuzz testing, compiling source codes, and then running the compiled program. The compiled program can carry the test code of the fuzzy test program, and can test branches quickly and accurately and carry out input mutation. The method has the advantages that the coverage rate of the test branches is high, and the defects that only one function can be tested in a fuzzy mode at a time, and more codes need to be manually written to carry out complete test on each module.
Grammar-based generation test: both black and white box tests are applicable. And informing a grammar rule of the generating system in advance, generating statements conforming to the grammar by the generating system according to the grammar rule, and finally combining the statements to generate a complete test program.
In summary, the complete random input, the boundary/special value input, the patterned input, the switch-case use-case test, and the grammar-based generation test can be simultaneously applied to the black-box test and the white-box test, but the complete random input, the boundary/special value input, and the patterned input test have a single test means and limited test effect, and the coverage of the switch-case use-case test code is not high, so that the grammar-based generation test becomes an important research direction in the fuzzy test method.
Technical solutions for fuzz testing by way of grammar-based generation testing are not common in the prior art, and are most representative of jsfunfuzz programs of Mozilla. The jsfunfuzz can not realize interactive access between functions and variables of different types in the test case generation process, and can not temporarily replace the function types of the generated test statements, so that a test case with more threat can not be generated, and the test effect is limited.
In view of this, the embodiments of the present invention provide a fuzz testing method, which aims to generate a test case with a stronger testing effect.
Referring to fig. 1, the implementation scenario includes: the client 01, the test server 03 and the configuration server 05 are in communication connection with each other, and the configuration server 05 may also be disposed inside the test server 03. The configuration server 05 may be configured to select a test object for the fuzz test, and the test server 03 may be configured to obtain a test resource pointed by the test object, generate a test case according to the test resource, and execute the test case in the test object to obtain a test result.
The Client 01 may communicate with the test Server 03 based on a Browser/Server (B/S) mode or a Client/Server (C/S) mode, and obtain a test result by communicating with the test Server 03.
The test server 03 and the configuration server 05 may each include a server operating independently, or a distributed server, or a server cluster composed of a plurality of servers.
An embodiment of the present invention provides a fuzzy testing method, as shown in fig. 2, where the method uses a test server in the implementation environment as an execution subject, and the method specifically includes:
s101, obtaining a test seed, and generating a test random parameter sequence based on the test seed.
The occurrence probability of each parameter in the test random parameter sequence is random probability, but the test random parameter sequence and the test seed have a mapping relation as follows: the same test seed is used regardless of which machine the same length of test random parametric sequence is generated on. Specifically, the test random parameter sequence with the length of N is obtained by sequentially inputting the test seeds into a preset test random parameter generation function N times. If the machine A inputs the test seed into the test random parameter generating function for 5 times to obtain a test random parameter sequence C with the length of 5, and the machine B inputs the test seed into the test random parameter generating function for 5 times to obtain a test random parameter sequence D with the length of 5, the test random parameter sequence C and the test random parameter sequence D are the same sequence.
Obviously, the mapping relationship is beneficial to avoiding test errors caused by tests performed by different machines, and the relative fixity of the test random parameter sequence is also beneficial to reproducing problems generated by the tests.
Specifically, the test random parameter sequence may continuously grow by continuously inputting the test seed into the test random parameter generating function, so as to meet the actual requirement of the fuzzy test. In the actual process of the fuzzy test, in any scene that needs to use the random number, the random parameter generating function may be freely called to obtain the random number based on the test seed and use the random number, and the obtained random number may also be one of the elements in the random parameter sequence.
S103, obtaining a test object set, wherein the test object set comprises at least one test object.
Specifically, each test object may independently perform a function, each test object may be packaged as a module, and different test objects may have different paths and be located in different folders. Thus, a tester may generate a set of test objects by configuring different paths.
In a possible embodiment, a set of test objects may also be obtained based on the test random parametric sequence. For example, there are N objects that can be tested, and a number of test objects are selected from the N objects that can be tested based on the random numbers provided by the test random parameter sequence to form a test object set.
In one possible embodiment, the test object may be loaded by loading a folder corresponding to the test object. For example, the test object may include arrayy, class, deletey, domy, enumy, functional. The domy is a test object in the form of the dom, and different from the prior art, the embodiment of the invention also supports the fuzzy test on the test object in the form of the dom.
And S105, extracting the test resources pointed by each test object to form a test resource set corresponding to the test object set.
Each test object corresponds to its own test resource, and the test resource can be used for automatically constructing a test case for testing the test object related to the test resource. The test resources include at least one initialization script and at least one definition script. For example, the test resources corresponding to the arrayy test object at least include script arraryfuzz.js and script init.js.
In one possible embodiment, all modules that may be tested may be loaded such that individual test objects and their corresponding test resources are automatically loaded at the beginning of the fuzz test. The tester can configure the test object to be tested through the configuration server or randomly obtain the test object based on the test random parameter sequence, so as to obtain the test object set and the corresponding test resource set.
And S107, generating a variable pool based on the testing random parameter sequence and the testing resource set, wherein all variables required for constructing the test cases of all the testing objects in the testing object set are stored in the variable pool.
Specifically, the generating a variable pool based on the test random parameter sequence and the test resource set, as shown in fig. 3, includes:
s1071, obtaining each test resource in the test resource set.
S1073, loading an initialization script in the test resources, wherein an initialization function and a simple statement generating function are defined in the initialization script.
The initialization function may add and initialize the name of a variable for testing to a Global space (Global Context), and call a simple statement generation function to assign an initial value to the variable for testing.
In one possible embodiment, the actions performed by the initialization function each generate an equivalent JavaScript script in the form of a character string. The simple sentence generation function may be used to generate at least one equivalent JavaScript sentence to facilitate initialization of various variables.
S1075, executing the initialization scripts in each test resource, and summarizing variables generated by the initialization scripts to a variable pool.
Specifically, the variable pool as a global resource may be stored in the global space in the form of a global variable, for example, it may be named as g _ tankVariant, which is a JavaScript array that stores all variables required by a test case, so as to facilitate reproduction or tracing of the test case and a problem generated by the test case at a later stage.
In one possible embodiment, an example of code generated for testing variables of a resource includes:
and S109, generating a test statement based on the test random parameter sequence and the test resource set to form a test statement set corresponding to the test resource set.
Specifically, the simple statement generating function in each test resource may also be configured to generate a test statement, and generate a plurality of test statements based on the test random parameter sequence and the test resource set to form a test statement set corresponding to the test resource set, as shown in fig. 4, including:
s1091, for each test resource in the test resource set, calling a simple statement generating function in the test resource and generating a test statement based on the test random parameter sequence.
In one possible embodiment, the test resource further includes a test rule and a test statement generation engine, and the test rule and the test statement generation engine are also recorded in the folder in which the test resource is located. The calling a simple statement generation function in the test resource and generating a test statement based on the test random parameter sequence, as shown in fig. 5, includes:
s10911, determining the operation cycle number N according to the test random parameter sequence.
S10913, calling N times of simple statement generating functions by the test statement generating engine to obtain a first statement set.
S10915, determining a test rule according to the test random parameter sequence.
In particular, the test rule may be used to generate nested test functions or non-nested test functions. Nesting parameters in the nested test function are not limited. The nested test function can be anonymous multi-level nested, so that the method is more suitable for the writing habit of JavaScript codes.
In one possible embodiment, the test rule may be a child array defined in a global variable array. Each sub-array r is called a rule.
The rule is defined as: [ Return value type, number of parameters, parameter type, function definition, statement type ]
Return value type: the designations ARRAY, STRING, NUMBER, OBJECT, UNDEFINED … … are consistent with the basic type of JavaScript. When no value is returned, it is UNDEFINED.
The number of parameters: refers to the number of variables involved in the generation.
The parameter types are as follows: it may be specified whether the parameter must be constant. It can be specified that: CAN BE _ INSTANT _ VAL (which may BE constant), CAN BE _ INSTANT _ VAL (which cannot BE constant), COMMA _ multiple (COMMA separated data set), or base type. When the type is specified as the first three, the type will be used randomly when generating the result. A Bitwise OR may be used to enforce the declaration type.
Function definition: and (3) using a [ number ] form, and specifying a generated result according to the number of parameters and the JavaScript function definition. For example: [1] concat ([2]), when generating the result, appropriate values are filled into this rule according to the parameter type.
Statement type: whether the generated statement is a FRAGMENT or STATEMENT (statement). The fragmented statements may be used to put into other fragmented statements. The combination of N (N >0) fragmentation statements are nested into one declaration. A statement is already a complete piece of statement and cannot be put into other fragmented statements or statements.
Examples of the definition of a rule are as follows:
Var g_ruleList_array_Operations＝new Array(
//retvalue,argcount,argtype,argdecq.,func dec1.,fragment type
[ARRAY,2,ARRAY,CAN_BE_INSTANT_VAL,COMMA_MULTILIST,CAN_BE_INSTANT_VAL,“[1].concat([2])”,FRAGMENT],
[STRING,2,ARRAY,CAN_BE_INSTANT_VAL,STRING,CAN_BE_INSTANT_VAL,“[1].join([2])”,FRAGMENT],
[STRING,1,ARRAY,CAN_BE_INSTANT_VAL,“[1].join([2])”,FRAGMENT],
[UNDEFINED,1,ARRAY,CAN_BE_INSTANT_VAL,“[1].pop()”,FRAGMENT],
[NUMBER,2,ARRAY,CAN_BE_INSTANT_VAL,COMMA_MULTILIST,CAN_BE_INSTAN T_VAL,“[1].push([2])”,FRAGMENT],
[ARRAY,1,ARRAY,CAN_BE_INSTANT_VAL,“[1].reverse()”,FRAGMENT],
[OBJECT,1,ARRAY,CAN_BE_INSTANT_VAL,“[1].shift()”,FRAGMENT],
s10917, generating a test statement according with the test rule according to the test rule and the first statement set.
Specifically, if the test rule is used for generating a nesting test function, nesting statements in the first statement set according to the rule to obtain a test statement; and if the test rule is used for generating a non-nested test statement, directly performing global declaration on the statements in the first statement set to obtain the test statement.
S1093, summarizing the test statements corresponding to the test resources to obtain a test statement set meeting the requirement of the preset quantity.
Specifically, the test statement set includes test statements generated by each test resource, the number of the test statements generated by each test resource is greater than 1, and the number ratio of the test statements generated by each test resource is not limited in the embodiment of the present invention.
FIG. 6 is a logic diagram illustrating the generation of the test statement set.
Firstly, a plurality of test objects are selected from each test object which can be tested through a test random parameter sequence to form a test object set.
Secondly, loading the test resources pointed by each test object in the test object set, wherein the test resources comprise test rules and a test statement generation engine.
Third, for each test resource, a test statement is generated using its test statement engine based on its test rules in conjunction with a test random parameter sequence.
Fourthly, summarizing the test sentences to obtain a test sentence set, wherein the test sentence set should meet the preset quantity requirement. In other preferred embodiments, not only the test statement set should meet the preset number requirement, but also the test statements generated by the respective test resources meet the single preset number requirement, and the single preset number requirement may also be determined according to the test random parameter sequence.
For convenience of understanding, the embodiment of the present invention is described by taking the exercise-related test rule as an example:
sports { { running, "fast running or slow running", "how many meters to run" } { swimming, "which swimming posture", "how long to swim" } … … }
For example, the rule that the parameters in the random parameter sequence are tested to point to "swim", and then the numerical values of the "swim posture" and the "swim duration" are respectively determined according to the parameters in the random parameter sequence, for example, a test sentence determined according to the random parameter sequence at a certain time can be running (jogging, 1000 meters), then swimming (butterfly stroke, 600 meters), and then basketball (30 minutes) … ….
S1011, generating a test resource script based on the variable pool and the test statement set, and outputting all statements in the test resource script out of order to obtain a fuzzy test script recorded with a test case.
And S1013, testing according to the fuzzy test script to obtain a test result.
And taking the fuzzy test script as an input to run a test object to obtain a test result.
In a possible embodiment, a test result capture command for capturing the test result output by the test object during the running process of the fuzz test script can be further added at the end of the fuzz test script. Different test result capture commands can be added according to different test purposes, and the test result capture commands can be JavaScript statements supported by test objects such as document. wr ite (), container. log (), and the like.
In a preferred embodiment, each step of the fuzz test can be automatically and repeatedly executed in a timing driving manner, and the method specifically includes: a timer is set so that the driver repeatedly performs step S103 and the next test round has been performed.
Referring to fig. 7, a logic framework diagram of a fuzz testing method in a scenario according to an embodiment of the present invention is shown. Firstly, all modules which can be tested are loaded at one time, a test object set in the modules is obtained, and a variable pool is generated. The test statement set can be obtained by testing the random parameter sequence through the test resource set pointed by the test object set, and specifically, in the generation process of the test statement set, the test statements can be generated circularly until reaching the statement upper limit of the test statement set. And obtaining a fuzzy test script according to the test statement set, and obtaining a fuzzy test result by executing the fuzzy test script. Of course, the whole fuzz testing process can be executed in a circulating manner again to achieve the effect of repeated testing. During the re-cycling process, the test object and its associated resources do not need to be loaded a second time.
The fuzzy test method realizes the automatic generation of the test statement by pertinently designing the function definition language support, the called level nesting and the type matching mechanism so as to obtain an effective or invalid test script, and the script is used for carrying out fuzzy test on a test object so as to find the stability problem and the safety problem in target software. Through relevant verification, the expected value of code coverage is higher after the embodiment of the invention is implemented for a long time. Meanwhile, the black box can be used according to the test scheme, the problem that the black box is difficult to insert piles and does not have a good mode to improve the code coverage rate is solved, and the test efficiency can be greatly improved.
According to the fuzzy test method, all variables in the variable pool and the generated test statement set can be uniformly output in a same test script in a disordered mode, and the test statements in the test statement set have reference to all the variables in the variable pool, so that the fuzzy test method can establish a correlation relation for various types of data or statements, and the fuzzy test method is different from the prior art that statements in a fuzzy test statement generation mechanism only can call data of the same type. Furthermore, the embodiment of the invention adopts a disorder output method, so that the function type has the possibility of being temporarily replaced, and more threatening test cases can be generated, thereby improving the efficiency of the security test.
Furthermore, in the embodiment of the invention, the test resources are set for each test object, and uniform test resources are not adopted, so that the test accuracy is improved as much as possible. For higher quality testing, in one possible embodiment, a small JavaScript virtual machine is also implemented, so that interpretation function nesting can be type sensitive or not type based on the small JavaScript virtual machine. Type sensitive testing can significantly improve the effective input rate. Testers may also be configured to use non-type based to increase the breadth of the fuzz testing.
Another embodiment of the present invention provides a fuzz testing apparatus, as shown in fig. 8, the apparatus including:
a test seed module 201, configured to obtain a test seed and generate a test random parameter sequence based on the test seed;
a test object set obtaining module 203, configured to obtain a test object set, where the test object set includes at least one test object;
a test resource set obtaining module 205, configured to extract a test resource pointed by each test object to form a test resource set corresponding to the test object set;
a variable pool generating module 207, configured to generate a variable pool based on the random test parameter sequence and the test resource set, where all variables required for constructing test cases of each test object in the test object set are stored in the variable pool;
a test statement set generating module 209, configured to generate a test statement based on the test random parameter sequence and the test resource set, so as to form a test statement set corresponding to the test resource set;
a fuzzy test script generating module 2011, configured to generate a test resource script based on the variable pool and the test statement set, and output each statement in the test resource script out of order to obtain a fuzzy test script in which a test case is recorded;
and the fuzz testing module 2013 is used for testing according to the fuzz testing script to obtain a testing result.
Specifically, the embodiment of the fuzzy test device and the embodiment of the method are all based on the same inventive concept.
The embodiment of the present invention further provides a computer storage medium, where the computer storage medium may store a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the fuzz testing method according to the embodiment of the present invention, and details refer to the method embodiment and are not described herein again.
Further, fig. 9 shows a hardware structure diagram of an apparatus for implementing the method provided by the embodiment of the present invention, and the apparatus may participate in forming or containing the device or system provided by the embodiment of the present invention. As shown in fig. 9, the device 10 may include one or more (shown as 102a, 102b, … …, 102 n) processors 102 (the processors 102 may include, but are not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission device 106 for communication functions. Besides, the method can also comprise the following steps: a display, an input/output interface (I/O interface), a Universal Serial Bus (USB) port (which may be included as one of the ports of the I/O interface), a network interface, a power source, and/or a camera. It will be understood by those skilled in the art that the structure shown in fig. 9 is only an illustration and is not intended to limit the structure of the electronic device. For example, device 10 may also include more or fewer components than shown in FIG. 9, or have a different configuration than shown in FIG. 9.
It should be noted that the one or more processors 102 and/or other data processing circuitry described above may be referred to generally herein as "data processing circuitry". The data processing circuitry may be embodied in whole or in part in software, hardware, firmware, or any combination thereof. Further, the data processing circuitry may be a single, stand-alone processing module, or incorporated in whole or in part into any of the other elements in the device 10 (or mobile device). As referred to in the embodiments of the application, the data processing circuit acts as a processor control (e.g. selection of a variable resistance termination path connected to the interface).
The memory 104 may be used for storing software programs and modules of application software, such as program instructions/data storage devices corresponding to the method described in the embodiment of the present invention, and the processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, so as to implement one of the fuzz testing methods described above. The memory 104 may include high speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, memory 104 may further include memory located remotely from processor 102, which may be connected to device 10 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The transmission device 106 is used for receiving or transmitting data via a network. Specific examples of such networks may include wireless networks provided by the communication provider of the device 10. In one example, the transmission device 106 includes a network adapter (NIC) that can be connected to other network devices through a base station so as to communicate with the internet. In one example, the transmission device 106 can be a Radio Frequency (RF) module, which is used for communicating with the internet in a wireless manner.
The display may be, for example, a touch screen type Liquid Crystal Display (LCD) that may enable a user to interact with a user interface of the device 10 (or mobile device).
It should be noted that: the precedence order of the above embodiments of the present invention is only for description, and does not represent the merits of the embodiments. And specific embodiments thereof have been described above. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, as for the device and server embodiments, since they are substantially similar to the method embodiments, the description is simple, and the relevant points can be referred to the partial description of the method embodiments.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (10)
1. A fuzz testing method, the method comprising:
acquiring test seeds, and generating a test random parameter sequence based on the test seeds;
acquiring a test object set, wherein the test object set comprises at least one test object;
extracting the test resources pointed by each test object to form a test resource set corresponding to the test object set;
generating a variable pool based on the testing random parameter sequence and the testing resource set, wherein all variables required for constructing the testing case of each testing object in the testing object set are stored in the variable pool;
generating a test statement based on the test random parameter sequence and the test resource set to form a test statement set corresponding to the test resource set;
generating a test resource script based on the variable pool and the test statement set, and outputting all statements in the test resource script out of order to obtain a fuzzy test script recorded with a test case;
and testing according to the fuzzy test script to obtain a test result.
2. The method of claim 1, wherein generating a pool of variables based on the sequence of test random parameters and the set of test resources comprises:
acquiring each test resource in the test resource set;
loading an initialization script in the test resource, wherein an initialization function and a simple statement generating function are defined in the initialization script;
and executing the initialization script in each test resource, and summarizing the variables generated by the initialization script to a variable pool.
3. The method of claim 1, wherein: generating a plurality of test statements based on the test random parameter sequence and the test resource set to form a test statement set corresponding to the test resource set, wherein the simple statement generating function in each test resource can be used for generating the test statements;
for each test resource in the test resource set, calling a simple statement generating function in the test resource and generating a test statement based on the test random parameter sequence;
and summarizing the test statements corresponding to the test resources to obtain a test statement set meeting the requirement of the preset quantity.
4. The method of claim 3, wherein said invoking a simple statement generation function in said test resource and generating a test statement based on said test random sequence of arguments comprises:
determining the number N of operation cycles according to the test random parameter sequence;
calling N times of simple statement generating functions by a test statement generating engine to obtain a first statement set;
determining a test rule according to the test random parameter sequence;
and generating a test statement according with the test rule according to the test rule and the first statement set.
5. The method of claim 4, wherein generating a test statement that conforms to the test rule based on the test rule and the first set of statements comprises:
if the test rule is used for generating a nesting test function, nesting the statements in the first statement set according to the rule to obtain a test statement; and if the test rule is used for generating a non-nested test statement, directly performing global declaration on the statements in the first statement set to obtain the test statement.
6. The method of claim 3, wherein:
the test statement set meets the preset quantity requirement;
and the test statement generated by each test resource also meets the quantity requirement of single items, and the quantity requirement of the single items is determined according to the test random parameter sequence.
7. The method of claim 1, wherein:
the occurrence probability of each parameter in the test random parameter sequence is a random probability, and the test random parameter sequence and the test seed have a mapping relation as follows: the test random parametric sequences of the same length generated on different machines using the same test seed are identical.
8. A fuzz testing apparatus, the apparatus comprising:
the test seed module is used for acquiring test seeds and generating a test random parameter sequence based on the test seeds;
the device comprises a test object set acquisition module, a test object set acquisition module and a test object analysis module, wherein the test object set acquisition module is used for acquiring a test object set, and the test object set comprises at least one test object;
the test resource set acquisition module is used for extracting the test resources pointed by each test object to form a test resource set corresponding to the test object set;
a variable pool generating module, configured to generate a variable pool based on the random test parameter sequence and the test resource set, where all variables required for constructing test cases of the test objects in the test object set are stored in the variable pool;
the test statement set generation module is used for generating a test statement based on the test random parameter sequence and the test resource set so as to form a test statement set corresponding to the test resource set;
the fuzzy test script generation module is used for generating a test resource script based on the variable pool and the test statement set, and outputting all statements in the test resource script out of order to obtain a fuzzy test script recorded with a test case;
and the fuzzy test module is used for testing according to the fuzzy test script to obtain a test result.
9. A fuzz testing apparatus, the apparatus comprising a processor and a memory, the memory having stored therein at least one instruction, at least one program, set of codes, or set of instructions, the at least one instruction, the at least one program, set of codes, or set of instructions, loaded and executed by the processor to implement a fuzz testing method according to any of claims 1-7.
10. A computer storage medium having stored therein at least one instruction, at least one program, set of codes, or set of instructions that is loaded by a processor and that performs a fuzz testing method according to any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910707526.6A CN112306853B (en) | 2019-08-01 | 2019-08-01 | Fuzzy test method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910707526.6A CN112306853B (en) | 2019-08-01 | 2019-08-01 | Fuzzy test method, device, equipment and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112306853A true CN112306853A (en) | 2021-02-02 |
| CN112306853B CN112306853B (en) | 2023-12-12 |
Family
ID=74486269
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910707526.6A Active CN112306853B (en) | 2019-08-01 | 2019-08-01 | Fuzzy test method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112306853B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115134278A (en) * | 2021-03-24 | 2022-09-30 | 奇安信科技集团股份有限公司 | Fuzzy test method and device, electronic equipment and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090320002A1 (en) * | 2008-06-20 | 2009-12-24 | Cadence Design Systems, Inc. | Method and system for testing and analyzing user interfaces |
| US20180365139A1 (en) * | 2017-06-15 | 2018-12-20 | Microsoft Technology Licensing, Llc | Machine learning for constrained mutation-based fuzz testing |
| US10164848B1 (en) * | 2014-06-09 | 2018-12-25 | Amazon Technologies, Inc. | Web service fuzzy tester |
| CN109241746A (en) * | 2018-08-29 | 2019-01-18 | 腾讯科技（深圳）有限公司 | Code process method, apparatus calculates equipment and storage medium |
| CN109739755A (en) * | 2018-12-27 | 2019-05-10 | 北京理工大学 | A kind of fuzz testing system executed based on program trace and mixing |
| CN109885479A (en) * | 2019-01-07 | 2019-06-14 | 中国人民解放军战略支援部队信息工程大学 | Software obfuscation test method and device based on path record truncation |
-
2019
- 2019-08-01 CN CN201910707526.6A patent/CN112306853B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090320002A1 (en) * | 2008-06-20 | 2009-12-24 | Cadence Design Systems, Inc. | Method and system for testing and analyzing user interfaces |
| US10164848B1 (en) * | 2014-06-09 | 2018-12-25 | Amazon Technologies, Inc. | Web service fuzzy tester |
| US20180365139A1 (en) * | 2017-06-15 | 2018-12-20 | Microsoft Technology Licensing, Llc | Machine learning for constrained mutation-based fuzz testing |
| CN109241746A (en) * | 2018-08-29 | 2019-01-18 | 腾讯科技（深圳）有限公司 | Code process method, apparatus calculates equipment and storage medium |
| CN109739755A (en) * | 2018-12-27 | 2019-05-10 | 北京理工大学 | A kind of fuzz testing system executed based on program trace and mixing |
| CN109885479A (en) * | 2019-01-07 | 2019-06-14 | 中国人民解放军战略支援部队信息工程大学 | Software obfuscation test method and device based on path record truncation |
Non-Patent Citations (1)
| Title |
|---|
| 蔡军 等: "基于改进轮盘赌策略的反馈式模糊测试方法", 四川大学学报(工程科学版), no. 02, pages 132 - 138 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115134278A (en) * | 2021-03-24 | 2022-09-30 | 奇安信科技集团股份有限公司 | Fuzzy test method and device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112306853B (en) | 2023-12-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Amalfitano et al. | A gui crawling-based technique for android mobile application testing | |
| US11416377B2 (en) | Automated application testing system | |
| CN106201862B (en) | Web services method for testing pressure and device | |
| US20190340512A1 (en) | Analytics for an automated application testing platform | |
| US9697109B2 (en) | Dynamically configurable test doubles for software testing and validation | |
| CN107783873B (en) | Method for realizing automatic testing platform of burner | |
| EP2839375A1 (en) | Testing system for an integrated software system | |
| US20170153969A1 (en) | System and method for executing integration tests in multiuser environment | |
| Wen et al. | Pats: A parallel gui testing framework for android applications | |
| CN106909498A (en) | A kind of java applet injects the method and system of failure | |
| Li et al. | ADAutomation: An activity diagram based automated GUI testing framework for smartphone applications | |
| JP2014021982A (en) | Iterative generation of symbolic test drivers for object-oriented languages | |
| CN112506785A (en) | Automatic testing method, device, equipment and medium for login of Html5 game page | |
| CN114116510A (en) | Interface parameter checking method and device | |
| CN106529304B (en) | A kind of Android applies concurrent leakage location | |
| Menegassi et al. | Automated tests for cross‐platform mobile apps in multiple configurations | |
| CN106557412B (en) | A kind of method and device of fuzz testing | |
| CN112306853B (en) | Fuzzy test method, device, equipment and medium | |
| US10310962B2 (en) | Infrastructure rule generation | |
| Cseppentő et al. | Evaluating code‐based test input generator tools | |
| US20130283238A1 (en) | Testing system for an integrated software system | |
| CN109120473B (en) | Interface request frame implementation method, interface test method and corresponding device | |
| CN111274144A (en) | Unit testing method and system based on network file system operation word interface | |
| CN114281709A (en) | Unit testing method, system, electronic equipment and storage medium | |
| CN113947047B (en) | Interface connection method for verifying design to be tested and related equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |