CN112306853B - Fuzzy test method, device, equipment and medium - Google Patents
Fuzzy test method, device, equipment and medium Download PDFInfo
- Publication number
- CN112306853B CN112306853B CN201910707526.6A CN201910707526A CN112306853B CN 112306853 B CN112306853 B CN 112306853B CN 201910707526 A CN201910707526 A CN 201910707526A CN 112306853 B CN112306853 B CN 112306853B
- Authority
- CN
- China
- Prior art keywords
- test
- generating
- resource
- statement
- script
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000010998 test method Methods 0.000 title claims abstract description 26
- 238000012360 testing method Methods 0.000 claims abstract description 575
- 238000013515 script Methods 0.000 claims abstract description 54
- 238000000034 method Methods 0.000 claims abstract description 35
- 230000006870 function Effects 0.000 claims description 50
- 238000013507 mapping Methods 0.000 claims description 3
- 239000012634 fragment Substances 0.000 description 11
- 230000008569 process Effects 0.000 description 10
- 238000012545 processing Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 7
- 238000004891 communication Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 5
- 230000009182 swimming Effects 0.000 description 5
- 230000005540 biological transmission Effects 0.000 description 4
- 230000035772 mutation Effects 0.000 description 4
- 230000007547 defect Effects 0.000 description 3
- 230000007246 mechanism Effects 0.000 description 3
- 230000007123 defense Effects 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000007774 longterm Effects 0.000 description 2
- 238000013522 software testing Methods 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- VYZAMTAEIAYCRO-UHFFFAOYSA-N Chromium Chemical compound [Cr] VYZAMTAEIAYCRO-UHFFFAOYSA-N 0.000 description 1
- 241000408659 Darpa Species 0.000 description 1
- 229910052804 chromium Inorganic materials 0.000 description 1
- 239000011651 chromium Substances 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 101150047356 dec-1 gene Proteins 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 230000002045 lasting effect Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000012372 quality testing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 238000011076 safety test Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3676—Test management for coverage analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3684—Test management for test design, e.g. generating new test cases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
Abstract
The invention discloses a fuzzy test method, a device, equipment and a medium, wherein the method comprises the steps of obtaining test seeds and generating a test random parameter sequence; acquiring a test object set; extracting the test resources pointed by each test object to form a test resource set corresponding to the test object set; generating a variable pool based on the test random parameter sequence and the test resource set, wherein all variables required by test cases for constructing each test object in the test object set are stored in the variable pool; generating test sentences based on the test random parametric sequence and the test resource set to form a test sentence set corresponding to the test resource set; generating a test resource script based on the variable pool and the test statement set, and outputting each statement in the test resource script in disorder to obtain a fuzzy test script recorded with test cases; and testing according to the fuzzy test script to obtain a test result. The invention can greatly improve the test efficiency.
Description
Technical Field
The present invention relates to the field of testing, and in particular, to a fuzzy testing method, apparatus, device, and medium.
Background
Fuzzing is an effective strategy for discovering defects in software under test. It shows great potential for automation of vulnerability detection. Meanwhile, because the automation degree of the fuzzy test is high, once the environment is configured, the fuzzy test can be applied to continuous vulnerability discovery, and is a remarkable manpower optimization for security researchers or enterprises. The fuzzy test has various forms, such as a switch-case test mode, which can be connected in series to form an approximately randomized statement stream, but has the problems that the function of the test and the content transferred to the test function are approximately fixed, a large amount of random values are generated, but the actual code coverage is not high, and the test effect is correspondingly influenced as compared with the existing case test mode based on grammar rules, which has improved automation degree but is difficult to generate a test case of a multi-layer nested mode.
Disclosure of Invention
In order to solve the technical problem that a fuzzy test method with high quality and high code coverage rate cannot be provided in the prior art, the embodiment of the invention provides a fuzzy test method, a device, equipment and a medium.
In one aspect, the present invention provides a fuzzy test method, the method comprising:
Obtaining a test seed, and generating a test random parameter sequence based on the test seed;
acquiring a test object set, wherein the test object set comprises at least one test object;
extracting the test resources pointed by each test object to form a test resource set corresponding to the test object set;
generating a variable pool based on the test random parameter sequence and the test resource set, wherein all variables required by test cases for constructing each test object in the test object set are stored in the variable pool;
generating test sentences based on the test random parametric sequence and the test resource set to form a test sentence set corresponding to the test resource set;
generating a test resource script based on the variable pool and the test statement set, and outputting each statement in the test resource script in disorder to obtain a fuzzy test script recorded with test cases;
and testing according to the fuzzy test script to obtain a test result.
In another aspect, the present invention provides a ambiguity test apparatus, the apparatus comprising:
the test seed module is used for obtaining test seeds and generating a test random parameter sequence based on the test seeds;
The test object set acquisition module is used for acquiring a test object set, wherein the test object set comprises at least one test object;
the test resource set acquisition module is used for extracting the test resources pointed by each test object to form a test resource set corresponding to the test object set;
the variable pool generation module is used for generating a variable pool based on the test random parameter sequence and the test resource set, wherein all variables required by the test cases for constructing each test object in the test object set are stored in the variable pool;
the test statement set generation module is used for generating test statements based on the test random parameter sequence and the test resource set to form a test statement set corresponding to the test resource set;
the fuzzy test script generation module is used for generating a test resource script based on the variable pool and the test statement set, and outputting each statement in the test resource script in disorder to obtain a fuzzy test script recorded with test cases;
and the fuzzy test module is used for testing according to the fuzzy test script to obtain a test result.
In another aspect, the present invention provides a ambiguity test apparatus, wherein the apparatus includes a processor and a memory, the memory storing at least one instruction, at least one program, a set of codes, or a set of instructions, the at least one instruction, the at least one program, the set of codes, or the set of instructions being loaded and executed by the processor to implement an ambiguity test method.
In another aspect, the present invention provides a computer storage medium, wherein at least one instruction, at least one program, a code set, or a set of instructions is stored in the storage medium, and the at least one instruction, the at least one program, the code set, or the set of instructions is loaded by a processor and performs a ambiguity test method.
The invention provides a fuzzy test method, a fuzzy test device, fuzzy test equipment and a fuzzy test medium. According to the invention, by purposefully designing a hierarchical nesting and type matching mechanism for function definition language support and call, automatic generation of test sentences is realized, so that an effective or invalid test script is obtained, and the script is used for carrying out fuzzy test on a test object, so that stability problems and security problems in target software are found. Through correlation verification, the expected value of code coverage is higher after long-term implementation of the embodiment of the invention. Meanwhile, the black box of the testing scheme can be used, so that the problem that the black box is difficult to insert piles and has no good mode for improving the code coverage rate is solved, and the testing efficiency can be greatly improved.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions and advantages of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are only some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is an environmental diagram of an implementation of a fuzzy test method provided by the present invention;
FIG. 2 is a flow chart of a fuzzy test method provided by the invention;
FIG. 3 is a flow chart for generating a variable pool based on the test random parametric sequence and the test resource set provided by the present invention;
FIG. 4 is a flow chart of a set of test sentences corresponding to the set of test resources, which is provided by the invention, based on the test random parametric sequence and the set of test resources;
FIG. 5 is a flow chart of a simple statement generating function in the test resource and generating test statements based on the test random argument sequence provided by the invention;
FIG. 6 is a schematic diagram of the generation logic of the test statement set provided by the present invention;
FIG. 7 is a diagram of a logic framework of a fuzzy test method in a scenario according to the present invention;
FIG. 8 is a block diagram of a fuzzy test provided by the present invention;
fig. 9 is a schematic hardware structure of an apparatus for implementing the method provided by the embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
It should be noted that the terms "first," "second," and the like in the description and the claims of the present invention and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or server that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed or inherent to such process, method, article, or apparatus, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
In order to make the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the detailed description and specific examples, while indicating the embodiment of the invention, are intended for purposes of illustration only and are not intended to limit the scope of the invention. For the purpose of facilitating a detailed description of embodiments of the present invention, related concepts of embodiments of the present invention and related state of the art will be first described.
JavaScript: a high-level programming language is a dynamic type, object-oriented (prototype-based) scripting language.
Mutation test: for software testing and evaluating the quality of existing software. The mutation test is to observe whether the output is normal or not by slightly modifying the program input so as to test the program stability and find the security hole.
Fuzzy test: is a software testing technology. The core idea is to generate random data for input into a program and monitor program anomalies to find possible program errors. Fuzzy testing is often used to detect security vulnerabilities of software or computer systems.
And (3) modularization: segmentation, organization, and packaging systems. Each module completes a specific sub-function, and all modules are assembled according to a certain method to form a whole to complete the functions required by the whole system.
Tree: a data set having tree-structured properties is simulated. It is a set of n (n > 0) finite nodes that have a hierarchical relationship. It is called a "tree" because it looks like an inverted tree, i.e. it is root up and leaf down.
Specific syntax tree, parse tree: is an ordered tree structure with a root node reflecting the grammatical relations of a string of a certain formal language.
Black box test: tests performed without complete or substantial knowledge of the source code or implementation of the target program.
White box test: the tests performed with full or substantial knowledge of the source code or implementation of the target program.
Target software: the software to be fuzzed may be any software that accepts external inputs and processes the inputs.
Test cases: the collection of data or steps used for testing is referred to as a test case.
doms: document object model.
The embodiment of the invention provides a fuzzy test method, and before the fuzzy test method of the embodiment of the invention is described in detail, the related technology of the fuzzy test is summarized first.
The fuzziness test was a 1988 curriculum project originating from the university of wisconsin Barton Miller. The project aims to test the reliability of Unix (a multi-user, multi-tasking operating system supporting multiple processor architectures) by performing a large number of random inputs in rapid succession until they crash. It also provides early debugging tools to determine the cause and category of each detected fault. The term ambiguity test is now not limited to command line tools, but is a broad concept that applies to everything that can be tested. 20 The google declaration pushes out ClusterFuzz, a cloud-based fuzzy test system for security testing of the functionality of the web browser Chromium browser that it predominatively developed, for month 4 of 12. If clusteriuzz finds some security issues, security researchers can upload their own fuzzy test programs and get rewards. 4 months of 2015, the time of day,it is shown how fuzzy AFL finds 2014 shared holes. (Heartfeed's vulnerability was revealed in month 4 of 2014. This is a serious vulnerability that allows an attacker to decipher otherwise encrypted communications, such as data for an Internet banking communication. Month 4 of 2016 reports that 23 ten thousand machines have vulnerabilities). 8 months 2016, the advanced research program agency (DARPA) of the united states department of defense held the first Cyber Grand Challenge total resolution, an automated game lasting 11 hours. The aim is to develop an automatic defense system capable of discovering, utilizing and correcting software defects in real time.
The test cases are distinguished according to the method of generating the test cases by the fuzzy test and can be divided into the following categories:
completely random input: the black box and white box test are applicable modes, belong to the simplest fuzzy test mode, and generate random data through a random number generator and input the random data to target software.
Boundary/special value input: the black box and white box tests are applicable to test whether the software works normally or not by inputting a boundary value or a special value to the target software, wherein the boundary value or the special value is as follows: 0. values falling on a certain mathematical or software engineering defined boundary, such as 1, -1, max_int+1, max_int-1. This test mode can significantly reduce the number and time of test cases, but at the same time it is unlikely that other types of security issues will be found.
And (3) modeling input: both black box and white box tests are applicable by entering fixed patterns specified by the obfuscating tool writer into the target software, such as entering tens of thousands of strings, entering a random file, and the like.
switch-case test: both black box and white box tests are applicable. I.e. all functions that want to be tested are written according to their grammatical specifications. Then control which of these is performed by a random number and repeat this step of random number selection to concatenate a stream of approximately randomized sentences. However, the problem is that the function of the test and the content passed to the test function are approximately fixed, and the actual code coverage is not high, although a large number of random values are generated. To improve the code coverage, the code amount of the fuzzy test program is greatly increased, and the test time is correspondingly greatly increased, so that the efficiency is reduced.
Pile insertion branch test: the manner in which the white box tests the application. The fuzziness test program captures branch boundary coverage and run-time branch execution count by instrumentation of some instructions during compilation. When an input can bring up a new branch, the input is recorded and mutation is continued.
Compiling branch test: the manner in which the white box tests the application. And compiling source codes by using a compiler provided by the fuzzy test tool, adding compiling options such as out-of-range detection and the like and linking part codes of the fuzzy test, and then running the compiled program. The compiled program can carry the test code of the fuzzy test program, can test branches more quickly and accurately, and can carry out input mutation. The method has the advantages of high test branch coverage rate, and the defect that only one type of function can be tested in a fuzzy way at a time, and each module can be completely tested only by manually writing more codes.
Grammar-based generation testing: both black box and white box tests are applicable. The grammar rule of the generation system is informed in advance, then the generation system generates sentences conforming to the grammar according to the grammar rule, and finally the sentences are combined to generate the complete test program.
In summary, the completely random input, the boundary/special value input, the patterned input, the switch-case test and the grammar-based generation test can be simultaneously applied to the black box test and the white box test, but the completely random input, the boundary/special value input and the patterned input have single test means and limited test effects, and the switch-case test codes have low coverage, so that the grammar-based generation test becomes an important research direction in the fuzzy test method.
The prior art solutions for fuzzing by means of grammar-based generation testing are not very versatile, most typically with the jsfunfuzz program of Mozilla corporation. The jsfunfuzz cannot realize interactive access between different types of functions and variables in the test case generation process, and the function types of the generated test sentences cannot be replaced temporarily, so that more threatening test cases cannot be generated, and therefore, the test effect is limited.
In view of the above, the embodiment of the present invention provides a fuzzy test method, which aims to generate a test case with a stronger test effect, and before detailing the fuzzy test method, the embodiment of the present invention first provides an implementation environment of the fuzzy test method.
Referring to fig. 1, the implementation scenario includes: the client 01, the test server 03 and the configuration server 05 are in communication connection with the test server 03 and the configuration server 05, and the configuration server 05 can also be arranged inside the test server 03. The configuration server 05 may be configured to select a test object for fuzzy testing, and the test server 03 may be configured to obtain a test resource pointed by the test object, generate a test case according to the test resource, and execute the test case in the test object to obtain a test result.
The Client 01 may communicate with the test Server 03 based on Browser/Server (B/S) or Client/Server (C/S) mode, and obtain a test result by communicating with the test Server 03.
The test server 03 and the configuration server 05 may each comprise a server that operates independently, or a distributed server, or a server cluster composed of a plurality of servers.
The embodiment of the invention provides a fuzzy test method, as shown in fig. 2, wherein the method takes a test server in the implementation environment as an execution subject, and the method specifically comprises the following steps:
s101, acquiring test seeds, and generating a test random parameter sequence based on the test seeds.
The probability of occurrence of each parameter in the test random parameter sequence is random, but the test random parameter sequence and the test seed have the following mapping relation: the test random parametric sequences of the same length generated on which machine are identical using the same test seed. Specifically, the test random parameter sequence with the length of N is obtained by sequentially inputting test seeds into a preset test random parameter generating function for N times. If the machine A inputs the test seed into the test random parameter generating function 5 times to obtain a test random parameter sequence C with the length of 5, and the machine B inputs the test seed into the test random parameter generating function 5 times to obtain a test random parameter sequence D with the length of 5, the test random parameter sequence C and the test random parameter sequence D are the same.
Obviously, this mapping is advantageous for avoiding test errors caused by testing by different machines, and the relative stationarity of the test random parameter sequences is also advantageous for reproducing the problems generated by the test.
Specifically, the test random parameter sequence can be continuously grown by continuously inputting the test seeds into a test random parameter generating function, so as to meet the actual fuzzy test requirement. In the actual process of the fuzzy test, any scene requiring the random number can be freely called to obtain the random number based on the test seed and use the random number, and the obtained random number can become one element in the random parameter sequence, so that the later-described execution of certain operation based on the test random parameter sequence can also comprise the situation that the random number is generated at any time and taken at any time, and then certain operation is executed.
S103, acquiring a test object set, wherein the test object set comprises at least one test object.
Specifically, each test object may perform a function independently, each test object may be packaged as a module, and different test objects may have different paths and be located in different folders. Thus, a tester may generate a test object set by configuring different paths.
In a possible embodiment, the test object set may also be obtained based on the test random parametric sequence. For example, there are N objects that can be tested, and a number of test objects are selected from the N objects that can be tested based on a random number provided by a test random parameter sequence to form a test object set.
In one possible embodiment, the test object may be loaded by loading a folder to which the test object corresponds. For example, the test object may include arrayy, classy, delete, domy, enumy, funcly. The domy is a test object in the form of dom, and unlike the prior art, the embodiment of the invention also supports fuzzy test on the test object in the form of dom.
S105, extracting the test resources pointed by each test object to form a test resource set corresponding to the test object set.
Each test object corresponds to its own test resource, which can be used to automatically build test cases for testing the test object to which the test resource relates. The test resource includes at least one initialization script and at least one definition script. For example, the test resources corresponding to the arrayy test object at least comprise script arraryfuzz. Js and script init. Js.
In one possible embodiment, all modules that can be tested may be loaded such that each test object and its corresponding test resources are automatically loaded at the beginning of the fuzzy test. The tester can configure the test object to be tested through the configuration server or obtain the test object randomly based on the test random parameter sequence, so as to obtain a test object set and a corresponding test resource set.
S107, generating a variable pool based on the test random parameter sequence and the test resource set, wherein all variables required by test cases for constructing all test objects in the test object set are stored in the variable pool.
Specifically, the generating a variable pool based on the test random parametric sequence and the test resource set, as shown in fig. 3, includes:
s1071, acquiring each test resource in the test resource set.
S1073, loading an initialization script in the test resource, wherein an initialization function and a simple statement generation function are defined in the initialization script.
The initialization function may add, initialize, and call a simple statement generation function to an initial value for a variable for testing to a Global space (Global Context).
In one possible embodiment, the actions performed by the initialization function all generate equivalent JavaScript scripts in the form of strings. A simple statement generation function may be used to generate at least one equivalent JavaScript statement to facilitate initializing various variables.
S1075, executing the initialization scripts in each test resource, and summarizing variables generated by the initialization scripts into a variable pool.
Specifically, the variable pool as a global resource may be stored in global space in the form of global variables, for example, it may be named g_tankVariant, and is a JavaScript array, which stores all variables required by the test case, so as to facilitate reproduction or tracing of the test case and the problem generated by the test case in a later period.
In one possible embodiment, an example of the code for generating a variable for a test resource includes:
s109, generating test sentences based on the test random parameter sequences and the test resource sets to form test sentence sets corresponding to the test resource sets.
Specifically, the simple statement generating function in each test resource can also be used for generating test statements, and the generating a plurality of test statements based on the test random argument sequence and the test resource set to form a test statement set corresponding to the test resource set, as shown in fig. 4, includes:
S1091, for each test resource in the test resource set, a simple statement generating function in the test resource is called, and test statements are generated based on the test random parameter sequence.
In a possible embodiment, the test resource further comprises a test rule and a test statement generation engine, which are also recorded in a folder in which the test resource resides. The calling the simple sentence generating function in the test resource and generating a test sentence based on the test random argument sequence, as shown in fig. 5, includes:
s10911, determining the operation cycle times N according to the test random parameter sequence.
S10913, calling N times of simple statement generating functions by the test statement generating engine to obtain a first statement set.
S10915, determining a test rule according to the test random parameter sequence.
In particular, the test rules may be used to generate nested test functions or non-nested test functions. The nesting parameters in the nesting test function are not limited. The nested test function can be anonymously multi-stage nested, so that the habit of writing JavaScript codes is more met.
In one possible embodiment, the test rule may be a child array defined in a global variable array. Each child array r is called a rule.
The rules are defined as: [ return value type, parameter quantity, parameter type, function definition, statement type ]
Return value type: the designation ARRAY, STRING, NUMBER, OBJECT, UNDEFINED … … is consistent with the basic type of JavaScript. When no value is returned, it is UNDEFINED.
Parameter number: refers to the number of variable factors involved in the generation.
Parameter type: it may be specified whether the parameter must be constant. It may be specified that: can_be_INSTANT_VAL (which may BE constant), can_BE_INSTANT_VAL (which may not BE constant), COMMA_MULTILIST (COMMA separated data set), or base type. When the designated types are the first three, the types are randomly used when the result is generated. A Bitwise OR may be used to force declaration types.
Function definition: using the form of [ number ], a generated result is specified according to the parameter number and JavaScript function definition. For example: [1] concat ([ 2 ]), when a result is generated, this rule is entered by filling in the appropriate value according to the parameter type.
Statement type: whether the generated statement is a FRAGMENT or STATEMENT statement. The fragment statement may be used to put into other fragment statements. The combination of N (N > 0) shard statements is nested into one declaration. The statement is already a complete one and cannot be put into other fragmented statements or statements.
Examples of definitions of rules are as follows:
Var g_ruleList_array_Operations＝new Array(
//retvalue,argcount,argtype,argdecq.,func dec1.,fragment type
[ARRAY,2,ARRAY,CAN_BE_INSTANT_VAL,COMMA_MULTILIST,CAN_BE_INSTANT_VAL,“[1].concat([2])”,FRAGMENT],
[STRING,2,ARRAY,CAN_BE_INSTANT_VAL,STRING,CAN_BE_INSTANT_VAL,“[1].join([2])”,FRAGMENT],
[STRING,1,ARRAY,CAN_BE_INSTANT_VAL,“[1].join([2])”,FRAGMENT],
[UNDEFINED,1,ARRAY,CAN_BE_INSTANT_VAL,“[1].pop()”,FRAGMENT],
[NUMBER,2,ARRAY,CAN_BE_INSTANT_VAL,COMMA_MULTILIST,CAN_BE_INSTAN T_VAL,“[1].push([2])”,FRAGMENT],
[ARRAY,1,ARRAY,CAN_BE_INSTANT_VAL,“[1].reverse()”,FRAGMENT],
[OBJECT,1,ARRAY,CAN_BE_INSTANT_VAL,“[1].shift()”,FRAGMENT],
s10917, generating test sentences conforming to the test rules according to the test rules and the first sentence sets.
Specifically, if the test rule is used for generating a nested test function, nesting the sentences in the first sentence set according to the rule to obtain test sentences; if the test rule is used for generating non-nested test sentences, directly carrying out global statement on the sentences in the first sentence set to obtain test sentences.
S1093, summarizing test sentences corresponding to the test resources to obtain a test sentence set meeting the preset quantity requirements.
Specifically, the test statement set includes test statements generated by each test resource, the number of test statements generated by each test resource is greater than 1, and the number and the ratio of the test statements generated by each test resource are not limited in the embodiment of the present invention.
As shown in FIG. 6, a schematic diagram of the generation logic of the test statement set is shown.
Firstly, selecting a plurality of test objects from all the test objects which can be tested through testing a random parameter sequence to form a test object set.
Secondly, loading test resources pointed by each test object in the test object set, wherein the test resources comprise test rules and test statement generation engines.
Third, for each test resource, a test statement is generated using its test statement engine based on its test rules in combination with a test random parameter sequence.
Fourth, summarizing the test sentences to obtain a test sentence set, wherein the test sentence set should meet a preset quantity requirement. In other preferred embodiments, not only should the test statement set meet a preset number requirement, but also the test statement generated by each test resource meets a single preset number requirement, which may also be determined according to the test random parameter sequence.
For ease of understanding, the embodiments of the present invention will be described by taking the motion-related test rule as an example:
exercise = { { { running, "fast running or jogging", "how many meters to run" } { swim, "which swimming stroke", "how long to swim" } … … }
For example, the parameter in the test random parameter sequence points to the rule of "swimming", and then the values of "what kind of swimming gesture" and "how long to swim" are respectively determined according to the parameter in the test random parameter sequence, for example, a test sentence determined according to the random parameter sequence can be running (jogging, 1000 meters), then swimming (butterfly swimming, 600 meters), and then basketball (30 minutes) … ….
S1011, generating a test resource script based on the variable pool and the test statement set, and outputting each statement in the test resource script in disorder to obtain a fuzzy test script recorded with test cases.
S1013, testing according to the fuzzy test script to obtain a test result.
And taking the fuzzy test script as an input operation test object to obtain a test result.
In a possible embodiment, a test result capturing command may be further added at the end of the fuzzy test script, where the test result capturing command is used to capture the test result output by the test object during the running process of the fuzzy test script. Different test result capturing commands can be added according to different test purposes, and the test result capturing commands can be JavaScript statements supported by test objects such as document. Wr ite (), control. Log (), and the like.
In a preferred embodiment, the steps of the fuzzy test may also be automatically and repeatedly performed by a timing driving manner, and specifically include: a timer is set so that the drive repeatedly performs step S103, and the next round of testing has been performed.
Referring to fig. 7, a logic frame diagram of a fuzzy test method in a scenario according to an embodiment of the present invention is shown. Firstly, loading all modules which can be tested at one time, acquiring a test object set in the modules, and generating a variable pool. The random parameter sequence is tested through the test resource set pointed by the test object set to obtain the test statement set, and specifically, in the generation process of the test statement set, test statements can be circularly generated until the statement upper limit of the test statement set is reached. And obtaining a fuzzy test script according to the test statement set, and obtaining a fuzzy test result by executing the fuzzy test script. Of course, the whole fuzzy test process can be performed again and again, so as to achieve the effect of repeated test. During the recirculation, the test object and its associated resources do not need to be loaded twice.
According to the fuzzy test method, the automatic generation of test sentences is realized by purposefully designing a hierarchical nesting and type matching mechanism for supporting and calling a function definition language, so that an effective or invalid test script is obtained, and the script is used for carrying out fuzzy test on a test object, so that stability problems and safety problems in target software are found. Through correlation verification, the expected value of code coverage is higher after long-term implementation of the embodiment of the invention. Meanwhile, the black box of the testing scheme can be used, so that the problem that the black box is difficult to insert piles and has no good mode for improving the code coverage rate is solved, and the testing efficiency can be greatly improved.
According to the embodiment of the invention, all variables in the variable pool and the generated test statement set can be output in an out-of-order manner in the same test script, and the test statements in the test statement set have opportunity references to all variables in the variable pool, so that the fuzzy test method in the embodiment of the invention can establish a correlation relationship for various types of data or statements, and is unlike the case that statements in a fuzzy test statement generation mechanism provided by the prior art can only call the data of the same type. Furthermore, the embodiment of the invention adopts an out-of-order output method, so that the probability of temporarily replacing the function type can be realized, more threatening test cases can be generated, and the efficiency of safety test can be improved.
Further, in the embodiment of the invention, the test resources are set for each test object, and unified test resources are not adopted, so that the test accuracy is improved as much as possible. For higher quality testing, in one possible embodiment, a small JavaScript virtual machine is also implemented, and therefore, interpretation function nesting based on the small virtual machine may be type sensitive or non-type based. Type sensitive testing can significantly improve the effective input rate. Testers may also configure use to be non-type based to improve the breadth of the fuzzy test.
Another embodiment of the present invention provides a ambiguity test apparatus, as shown in fig. 8, including:
a test seed module 201, configured to obtain a test seed, and generate a test random parameter sequence based on the test seed;
a test object set obtaining module 203, configured to obtain a test object set, where the test object set includes at least one test object;
a test resource set acquisition module 205, configured to extract test resources pointed by each test object to form a test resource set corresponding to the test object set;
a variable pool generating module 207, configured to generate a variable pool based on the test random parameter sequence and the test resource set, where all variables required for constructing test cases of each test object in the test object set are stored in the variable pool;
A test statement set generating module 209, configured to generate a test statement based on the test random argument sequence and the test resource set, so as to form a test statement set corresponding to the test resource set;
the fuzzy test script generation module 2011 is configured to generate a test resource script based on the variable pool and the test statement set, and perform out-of-order output on each statement in the test resource script to obtain a fuzzy test script recorded with a test case;
and the fuzzy test module 2013 is used for testing according to the fuzzy test script to obtain a test result.
Specifically, the embodiments of the present invention relate to a fuzzy testing device and a fuzzy testing method, which are all based on the same inventive concept.
The embodiment of the invention also provides a computer storage medium, which can store a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing the fuzzy test method according to the embodiment of the invention, and details refer to the method embodiment and are not repeated herein.
Further, fig. 9 shows a schematic diagram of a hardware structure of an apparatus for implementing the method provided by the embodiment of the present invention, where the apparatus may participate in forming or including the device or the system provided by the embodiment of the present invention. As shown in fig. 9, the apparatus 10 may include one or more processors 102 (shown as 102a, 102b, … …,102 n) that may include, but are not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA, a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, the method may further include: a display, an input/output interface (I/O interface), a Universal Serial Bus (USB) port (which may be included as one of the ports of the I/O interface), a network interface, a power supply, and/or a camera. It will be appreciated by those skilled in the art that the configuration shown in fig. 9 is merely illustrative and is not intended to limit the configuration of the electronic device. For example, the device 10 may also include more or fewer components than shown in fig. 9, or have a different configuration than shown in fig. 9.
It should be noted that the one or more processors 102 and/or other data processing circuits described above may be referred to generally herein as "data processing circuits. The data processing circuit may be embodied in whole or in part in software, hardware, firmware, or any other combination. Further, the data processing circuitry may be a single stand-alone processing module, or incorporated in whole or in part into any of the other elements in the device 10 (or mobile device). As referred to in embodiments of the application, the data processing circuit acts as a processor control (e.g., selection of the path of the variable resistor termination connected to the interface).
The memory 104 may be used to store software programs and modules of application software, and the processor 102 executes the software programs and modules stored in the memory 104 to perform various functional applications and data processing, i.e., to implement a fuzzy test method as described above, according to the program instructions/data storage device corresponding to the method according to the embodiment of the present application. Memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, memory 104 may further include memory located remotely from processor 102, which may be connected to device 10 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The transmission means 106 is arranged to receive or transmit data via a network. Specific examples of the network described above may include a wireless network provided by a communications provider of device 10. In one example, the transmission device 106 includes a network adapter (NetworkInterfaceController, NIC) that can connect to other network devices through a base station to communicate with the internet. In one example, the transmission device 106 may be a radio frequency (RadioFrequency, RF) module for communicating wirelessly with the internet.
The display may be, for example, a touch screen type Liquid Crystal Display (LCD) that may enable a user to interact with a user interface of the device 10 (or mobile device).
It should be noted that: the sequence of the embodiments of the present invention is only for description, and does not represent the advantages and disadvantages of the embodiments. And the foregoing description has been directed to specific embodiments of this specification. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for the device and server embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and references to the parts of the description of the method embodiments are only required.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program for instructing relevant hardware, where the program may be stored in a computer readable storage medium, and the storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
The foregoing description of the preferred embodiments of the invention is not intended to limit the invention to the precise form disclosed, and any such modifications, equivalents, and alternatives falling within the spirit and scope of the invention are intended to be included within the scope of the invention.
Claims (7)
1. A method of ambiguity testing, the method comprising:
obtaining a test seed, and generating a test random parameter sequence based on the test seed;
Acquiring a test object set, wherein the test object set comprises at least one test object;
extracting the test resources pointed by each test object to form a test resource set corresponding to the test object set;
generating a variable pool based on the test random parameter sequence and the test resource set, wherein all variables required by test cases for constructing each test object in the test object set are stored in the variable pool;
the simple sentence generating function in each test resource can be used for generating test sentences, and for each test resource in the test resource set, the simple sentence generating function in the test resource is called and test sentences are generated based on the test random parametric sequence; summarizing test sentences corresponding to each test resource to obtain a test sentence set meeting the preset quantity requirements; the method for generating the test statement by calling the simple statement generating function in the test resource and generating the test statement based on the test random parametric sequence comprises the following steps: determining the operation cycle times N according to the test random parameter sequence; invoking N times of simple sentence generating functions by the test sentence generating engine to obtain a first sentence set; determining a test rule according to the test random parameter sequence; if the test rule is used for generating a nested test function, nesting the sentences in the first sentence set according to the rule to obtain a test sentence; if the test rule is used for generating non-nested test sentences, directly carrying out global declaration on the sentences in the first sentence set to obtain test sentences;
Generating a test resource script based on the variable pool and the test statement set, and outputting each statement in the test resource script in disorder to obtain a fuzzy test script recorded with test cases;
and testing according to the fuzzy test script to obtain a test result.
2. The method of claim 1, wherein the generating a variable pool based on the test random parametric sequence and the test resource set comprises:
acquiring each test resource in the test resource set;
loading an initialization script in the test resource, wherein an initialization function and a simple statement generating function are defined in the initialization script;
executing the initialization script in each test resource, and summarizing the variable generated by the initialization script into a variable pool.
3. The method according to claim 1, characterized in that:
the test statement set meets the preset quantity requirement;
and the test sentences generated by each test resource also meet the number requirement of single items, and the number requirement of the single items is determined according to the test random parameter sequence.
4. The method according to claim 1, characterized in that:
the occurrence probability of each parameter in the test random parameter sequence is random probability, and the test random parameter sequence and the test seed have such a mapping relation: the test random parametric sequences of the same length generated on different machines using the same test seed are identical.
5. A ambiguity test apparatus, said apparatus comprising:
the test seed module is used for obtaining test seeds and generating a test random parameter sequence based on the test seeds;
the test object set acquisition module is used for acquiring a test object set, wherein the test object set comprises at least one test object;
the test resource set acquisition module is used for extracting the test resources pointed by each test object to form a test resource set corresponding to the test object set;
the variable pool generation module is used for generating a variable pool based on the test random parameter sequence and the test resource set, wherein all variables required by the test cases for constructing each test object in the test object set are stored in the variable pool;
the test statement set generation module is used for generating test statements by using the simple statement generation function in each test resource, and for each test resource in the test resource set, the simple statement generation function in the test resource is called and test statements are generated based on the test random parameter sequence; summarizing test sentences corresponding to each test resource to obtain a test sentence set meeting the preset quantity requirements; the method for generating the test statement by calling the simple statement generating function in the test resource and generating the test statement based on the test random parametric sequence comprises the following steps: determining the operation cycle times N according to the test random parameter sequence; invoking N times of simple sentence generating functions by the test sentence generating engine to obtain a first sentence set; determining a test rule according to the test random parameter sequence; if the test rule is used for generating a nested test function, nesting the sentences in the first sentence set according to the rule to obtain a test sentence; if the test rule is used for generating non-nested test sentences, directly carrying out global declaration on the sentences in the first sentence set to obtain test sentences;
The fuzzy test script generation module is used for generating a test resource script based on the variable pool and the test statement set, and outputting each statement in the test resource script in disorder to obtain a fuzzy test script recorded with test cases;
and the fuzzy test module is used for testing according to the fuzzy test script to obtain a test result.
6. A ambiguity test apparatus, comprising a processor and a memory, wherein the memory has stored therein at least one instruction, at least one program, code set or instruction set, which is loaded and executed by the processor to implement a ambiguity test method according to any one of claims 1-4.
7. A computer storage medium having stored therein at least one instruction, at least one program, code set or instruction set, the at least one instruction, at least one program, code set or instruction set being loaded by a processor and performing a ambiguity test method according to any one of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910707526.6A CN112306853B (en) | 2019-08-01 | 2019-08-01 | Fuzzy test method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910707526.6A CN112306853B (en) | 2019-08-01 | 2019-08-01 | Fuzzy test method, device, equipment and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112306853A CN112306853A (en) | 2021-02-02 |
| CN112306853B true CN112306853B (en) | 2023-12-12 |
Family
ID=74486269
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910707526.6A Active CN112306853B (en) | 2019-08-01 | 2019-08-01 | Fuzzy test method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112306853B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115134278A (en) * | 2021-03-24 | 2022-09-30 | 奇安信科技集团股份有限公司 | Fuzzy test method and device, electronic equipment and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10164848B1 (en) * | 2014-06-09 | 2018-12-25 | Amazon Technologies, Inc. | Web service fuzzy tester |
| CN109241746A (en) * | 2018-08-29 | 2019-01-18 | 腾讯科技（深圳）有限公司 | Code process method, apparatus calculates equipment and storage medium |
| CN109739755A (en) * | 2018-12-27 | 2019-05-10 | 北京理工大学 | A kind of fuzz testing system executed based on program trace and mixing |
| CN109885479A (en) * | 2019-01-07 | 2019-06-14 | 中国人民解放军战略支援部队信息工程大学 | Software obfuscation test method and device based on path record truncation |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9098635B2 (en) * | 2008-06-20 | 2015-08-04 | Cadence Design Systems, Inc. | Method and system for testing and analyzing user interfaces |
| US10452526B2 (en) * | 2017-06-15 | 2019-10-22 | Microsoft Technology Licensing, Llc | Machine learning for constrained mutation-based fuzz testing |
-
2019
- 2019-08-01 CN CN201910707526.6A patent/CN112306853B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10164848B1 (en) * | 2014-06-09 | 2018-12-25 | Amazon Technologies, Inc. | Web service fuzzy tester |
| CN109241746A (en) * | 2018-08-29 | 2019-01-18 | 腾讯科技（深圳）有限公司 | Code process method, apparatus calculates equipment and storage medium |
| CN109739755A (en) * | 2018-12-27 | 2019-05-10 | 北京理工大学 | A kind of fuzz testing system executed based on program trace and mixing |
| CN109885479A (en) * | 2019-01-07 | 2019-06-14 | 中国人民解放军战略支援部队信息工程大学 | Software obfuscation test method and device based on path record truncation |
Non-Patent Citations (1)
| Title |
|---|
| 基于改进轮盘赌策略的反馈式模糊测试方法;蔡军 等;四川大学学报(工程科学版)(第02期);第132-138页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112306853A (en) | 2021-02-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Varga | A practical introduction to the OMNeT++ simulation framework | |
| CN108885545B (en) | Tools and methods for real-time dataflow programming languages | |
| US9697109B2 (en) | Dynamically configurable test doubles for software testing and validation | |
| US20130339930A1 (en) | Model-based test code generation for software testing | |
| CN107783873B (en) | Method for realizing automatic testing platform of burner | |
| CN103324568A (en) | Method and apparatus for testing programs | |
| Chowdhury et al. | CyFuzz: A differential testing framework for cyber-physical systems development environments | |
| Sottile et al. | Semi-automatic extraction of software skeletons for benchmarking large-scale parallel applications | |
| Santelices et al. | DUA-Forensics: a fine-grained dependence analysis and instrumentation framework based on Soot | |
| CN112306853B (en) | Fuzzy test method, device, equipment and medium | |
| Zeller | Search-based testing and system testing: a marriage in heaven | |
| CN106557412B (en) | A kind of method and device of fuzz testing | |
| TWI626538B (en) | Infrastructure rule generation | |
| US10606732B2 (en) | Hybrid genetic concolic co-verification of hardware and software | |
| Wiederseiner et al. | An open-source tool for automated generation of black-box xunit test code and its industrial evaluation | |
| Cseppentő et al. | Evaluating code‐based test input generator tools | |
| Paydar et al. | An experimental study on flakiness and fragility of randoop regression test suites | |
| US10579761B1 (en) | Method and system for reconstructing a graph presentation of a previously executed verification test | |
| Di Nardo et al. | Model based test validation and oracles for data acquisition systems | |
| Cassez et al. | Wuppaal: Computation of worst-case execution-time for binary programs with uppaal | |
| CN114281709A (en) | Unit testing method, system, electronic equipment and storage medium | |
| Xiao et al. | Advances on improving automation in developer testing | |
| Bree et al. | Removing decorator to improve energy efficiency | |
| Ahmed et al. | An Adaptation Model for Android Application Testing with Refactoring | |
| JP2011154568A (en) | Information processing apparatus, program verification method and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |