CN112256572A

CN112256572A - Random test case generation method and device, electronic equipment and storage medium

Info

Publication number: CN112256572A
Application number: CN202011133389.9A
Authority: CN
Inventors: 杨仍才
Original assignee: Haiguang Information Technology Co Ltd
Current assignee: Haiguang Information Technology Co Ltd
Priority date: 2020-10-21
Filing date: 2020-10-21
Publication date: 2021-01-22
Anticipated expiration: 2040-10-21
Also published as: CN112256572B

Abstract

The present disclosure provides a random test case generation method and apparatus, an electronic device and a storage medium, wherein the method includes: constructing a candidate pool of functions to be tested comprising a plurality of functions; constructing a parameter pool by grouping input and/or output parameters for testing, the parameter pool comprising a plurality of input and/or output parameters; selecting a single function to be tested from a plurality of functions of a candidate pool of functions to be tested or selecting a function combination to be tested to generate a random test case, wherein the function combination to be tested comprises one or more functions to be tested; and generating a test case for a single function to be tested, randomly selecting input parameters required by the single function to be tested from the parameter pool once or for multiple times to obtain a random test case, or generating a test case for a combination of functions to be tested, randomly selecting input parameters required by one or multiple functions to be tested from the parameter pool once or for multiple times to obtain a random test case. The random test case generation method can simply and efficiently generate the random test case.

Description

Random test case generation method and device, electronic equipment and storage medium

Technical Field

The embodiment of the disclosure relates to a random test case generation method and device, an electronic device and a storage medium.

Background

After the functional design of hardware or software is completed, the correctness of the hardware or software is verified. Typical functions are hardware instructions, software functions, etc. Functional verification is the most critical part of design verification, and only designs which can complete preset functions are meaningful designs. Generally, after determining the functional design requirements, it is possible to determine what output it can form at what input. After the design is complete, only the output under the given input needs to be verified to match the expected value.

However, since the input values of some functions are too many or the input forms are complicated and variable, if all the conditions are to be covered, a large number of test cases are required, which requires a lot of manpower, and the testing time is easily too long. However, reducing the test cases may leave some inputs uncovered, which may result in the inability to expose potential design issues and defeat the purpose of testing. This conflict becomes more acute when multiple functions are tested in an integrated manner.

Therefore, how to scientifically generate a Test Case (Test Case, which refers to a specific set of inputs prepared for a function and is provided with corresponding expected outputs) is an important matter for ensuring the validity and reliability of functional verification.

Disclosure of Invention

Embodiments of the present disclosure provide a random test case generation method and apparatus, an electronic device, and a storage medium, so as to generate a random test case simply and efficiently.

At least one embodiment of the present disclosure provides a random test case generation method, including:

constructing a candidate pool of functions to be tested, wherein the candidate pool of functions to be tested comprises a plurality of functions;

building a parameter pool by grouping input parameters and/or output parameters for testing, such that the parameter pool comprises one or more parameter groups; wherein the parameter pool comprises a plurality of input parameters and/or output parameters;

selecting a single function to be tested from a plurality of functions of the candidate pool of functions to be tested or selecting a function combination to be tested to generate the random test case, wherein the function combination to be tested comprises one or more functions to be tested; and is

And generating a test case for the single function to be tested, randomly selecting input parameters required by the single function to be tested from the parameter pool for one or more times to obtain the random test case, or generating a test case for the function combination to be tested, randomly selecting input parameters required by the one or more functions to be tested from the parameter pool for one or more times to obtain the random test case.

For example, in a random test case generation method provided in at least one embodiment of the present disclosure, by grouping input parameters and/or output parameters for testing, the method includes: and grouping the input parameters and/or the output parameters for the single function to be tested or the combination of the functions to be tested according to one or more of the input parameter data type, the output parameter data type, the input parameter value range and the output parameter value range.

For example, in a random test case generation method provided in at least one embodiment of the present disclosure, the plurality of parameter packets include an input parameter packet and an output parameter packet; the input parameter group is used for providing input parameters required by one or more functions to be tested in the single function to be tested or the function combination to be tested; the output parameter group is used for receiving output parameters output by one or more functions to be tested of the single function to be tested or the combination of the functions to be tested, and the output parameter group is also used for providing input parameters required by one or more functions to be tested in the single function to be tested or the combination of the functions to be tested.

For example, in a random test case generation method provided in at least one embodiment of the present disclosure, selecting a combination of functions to be tested from a plurality of functions in the candidate pool of functions to be tested to generate the random test case includes: forming the function combination to be tested including the one or more functions to be tested by randomly selecting at least one function to be tested from a plurality of functions of the function candidate pool to be tested one or more times.

For example, in a random test case generation method provided in at least one embodiment of the present disclosure, when the random test case is generated for the function combination to be tested, constructing the parameter pool includes: constructing an initial parameter pool for the function combination to be tested, wherein the initial parameter pool at least comprises a plurality of input parameters for one or more functions to be tested of the function combination to be tested; sequentially performing a plurality of functional operations to iteratively update the initial parameter pool to obtain the parameter pool, wherein sequentially performing a plurality of functional operations to iteratively update the initial parameter pool comprises:

for each of the plurality of function runs, randomly selecting a currently selected function in the function combination to be tested, randomly selecting input parameters required by the currently selected function from a plurality of parameters currently included in the initial parameter pool, running the currently selected function to obtain current output parameters, and adding the current output parameters to the initial parameter pool for subsequent selection, thereby iteratively updating the initial parameter pool;

and performing the multiple functional operations until the test termination condition is met.

For example, in a random test case generation method provided in at least one embodiment of the present disclosure, constructing an initial parameter pool for the function combination to be tested includes: grouping the initial pool of parameters such that the initial pool of parameters includes at least:

an initial input parameter group including a plurality of initial input parameters used as a plurality of input parameters of one or more functions to be tested of the combination of functions to be tested;

an initial output parameter group for adding a current output parameter of a currently selected function for each random selection for subsequent selection, thereby iteratively updating the initial output parameter group so that the initial parameter pool is iteratively updated; wherein the initial input parameter packet and the initial output parameter packet are non-overlapping with each other.

For example, in a random test case generation method provided in at least one embodiment of the present disclosure, randomly selecting an input parameter required by the currently selected function from a plurality of parameters currently included in the initial parameter pool includes: randomly selecting parameters for input to the currently selected function from among initial input parameters included in the initial input parameter group and output parameters added in the iterative update included in the initial output parameter group according to a predetermined probability.

For example, in a random test case generation method provided in at least one embodiment of the present disclosure, the predetermined probability is such that a probability of selecting a parameter for input to the currently selected function from the initial input parameter group is greater than or less than a probability of selecting a parameter for input to the currently selected function from the initial output parameter group.

For example, in a random test case generation method provided in at least one embodiment of the present disclosure, the test termination condition includes: all functions to be tested in one or more functions to be tested of the function combination to be tested are selected at least once; and/or, the random test case generation method is suitable for generating a random test program, and the total code length of the random test program reaches a preset value.

For example, in a random test case generation method provided in at least one embodiment of the present disclosure, at least one of the parameters in the parameter pool is a constant or a variable, and the variable is used for being assigned when the random test case is actually tested.

For example, in a random test case generation method provided in at least one embodiment of the present disclosure, the function combination to be tested includes a plurality of functions to be tested; generating a test case for the function combination to be tested, randomly selecting input parameters required by the one or more functions to be tested from the parameter pool one or more times to obtain the random test case, wherein the method comprises the following steps:

and constraining a plurality of functions to be tested in the function combination to be tested so as to limit the input parameters required by randomly selecting the functions to be tested from the parameter pool once or for a plurality of times, thereby obtaining the random test case.

For example, in a random test case generation method provided in at least one embodiment of the present disclosure, a plurality of functions of the function combination to be tested belong to N function groups, and the method for randomly selecting input parameters required by the plurality of functions to be tested from the parameter pool multiple times includes:

randomly selecting one or M function groups from the N function groups, randomly selecting one or more tested functions from the one or M function groups respectively, and randomly selecting input parameters required by the one or more tested functions from the parameter pool once or for multiple times to obtain the random test case; wherein N is more than 1, and M is more than 1 and less than or equal to N.

For example, in a random test case generation method provided in at least one embodiment of the present disclosure, a plurality of functions in at least one of the N function groups have similarity therebetween, where the similarity includes: the content of the constraint applied to the input parameters and/or the output parameters of the plurality of functions is the same; wherein the constraints imposed on the input parameters and/or output parameters of the plurality of functions include one or more of: input parameter data type, output parameter data type, input parameter value range and output parameter value range.

For example, in a random test case generation method provided in at least one embodiment of the present disclosure, at least one of the functions to be tested is a software function or an instruction supported by hardware; the input parameters and/or the output parameters respectively comprise one or more of the following: explicit incoming parameter values, implicit software or hardware state.

At least one embodiment of the present disclosure provides a random test case generation apparatus, including:

the system comprises a first construction module, a second construction module and a third construction module, wherein the first construction module is configured to construct a candidate pool of functions to be tested, which comprises a plurality of functions;

a second building module configured to build a parameter pool by grouping input parameters and/or output parameters for testing such that the parameter pool comprises one or more parameter groups; wherein the parameter pool comprises a plurality of input parameters and/or output parameters;

the function selection module is configured to select a single function to be tested or a function combination to be tested from a plurality of functions of the candidate pool of functions to be tested, wherein the function combination to be tested comprises one or more functions to be tested;

and the parameter random selection module is configured to randomly select the input parameters required by the single function to be tested or the one or more functions to be tested in the function combination to be tested one or more times from the parameter pool to obtain the random test case.

At least one embodiment of the present disclosure provides an electronic device, including: a processor and a memory, wherein the memory has stored thereon a computer program that, when executed by the processor, implements the random test case generation method as in any of the examples above.

At least one embodiment of the present disclosure provides a computer-readable storage medium, where a computer program is stored in the storage medium, and when the computer program is executed by a processor, the method for generating a random test case is implemented as in any one of the above examples.

Drawings

In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without creative efforts.

Fig. 1 is a schematic flow chart of a random test case generation method according to some embodiments of the present disclosure;

FIG. 2 is a flow chart of a random unit test case generation method according to some embodiments of the present disclosure;

FIG. 3 is a flow chart of a method for generating a random combinational test case according to some embodiments of the present disclosure;

FIG. 4 is a schematic block diagram of a random test case generation apparatus according to some embodiments of the present disclosure;

FIG. 5 is a schematic block diagram of a random test case generation apparatus according to some embodiments of the present disclosure;

FIG. 6 is a schematic block diagram of another random test case generation apparatus provided in some embodiments of the present disclosure; and

fig. 7 is a schematic diagram of a storage medium according to some embodiments of the present disclosure.

Detailed Description

The technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are only a part of the embodiments of the present disclosure, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments disclosed herein without making any creative effort, shall fall within the protection scope of the present disclosure.

Unless otherwise defined, all terms (including technical and scientific terms) used in the embodiments of the present disclosure have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

The use of "first," "second," and similar terms in the embodiments of the disclosure is not intended to indicate any order, quantity, or importance, but rather to distinguish one element from another. The use of the terms "a" and "an" or "the" and similar referents do not denote a limitation of quantity, but rather denote the presence of at least one. Likewise, the word "comprising" or "comprises", and the like, means that the element or item listed before the word covers the element or item listed after the word and its equivalents, but does not exclude other elements or items. The terms "connected" or "coupled" and the like are not restricted to physical or mechanical connections, but may include electrical connections, whether direct or indirect. Flow charts are used in the disclosed embodiments to illustrate the steps of a method according to an embodiment of the disclosure. It should be understood that the preceding and following steps are not necessarily performed in the exact order in which they are performed. Rather, various steps may be processed in reverse order or simultaneously. Meanwhile, other operations may be added to the processes, or a certain step or steps may be removed from the processes.

The most common solution for generating test cases today is to solidify the test cases. Each function manually selects one or more typical inputs and provides corresponding expected outputs, forms a simple Test Set (Test Set, a plurality of Test cases can form a Test Set), and then fixedly runs the Test Set in each Test. The solidification test case is generally created manually, the method is simple and easy to implement, when the input values are few, although the input values often have good effects, the flexibility is poor, only a part of the input values can be selected for testing, and the increase, decrease, modification and the like of the test case are completed manually. Meanwhile, if a plurality of functions are subjected to combined test, the curing test case is often lost on the coverage surface of the functions, which means that many input combinations are not in the test range, thereby causing some potential problems to be overlooked.

With the higher and higher integration level of the system, more and more functions are needed to be tested, the diversity is larger and larger, and the solidified test case can hardly meet the test requirement. Therefore, it is becoming more common to randomly generate test cases.

Generally, random testing can be used for both unit testing of a single function and combined testing of multiple functions. The random unit test is to select the input value of the function randomly and then check whether it can operate normally or the result is expected. The random combinatorial testing is not only input randomly, but the set of functions it tests may also be random. That is, the random combination test not only tests the correctness of each function when the functions are randomly input individually, but also effectively tests the correctness of the combined action of the functions after the functions are randomly combined. Because the degree of freedom of the random test is large, how to balance the test coverage and the test time, and accordingly, selecting and generating a high-quality random test case is a main problem to be solved in the random test.

The inventor finds that: firstly, establishing a set of functions to be verified, and randomly selecting one function from the set of functions to be verified in each cycle until a termination condition is met; the method is combined into a complete program, and a random test case is formed by carrying out validity verification (such as operation validity judgment, dead cycle judgment and the like) on the program. This approach provides better test coverage since all functions are better accessible. However, the greatest problem of this solution is that the completeness of the test is not enough, although the test can cover many functions, the input form of each function is not necessarily covered effectively, and especially when some functions may generate interaction or linkage under combined conditions, this generation solution cannot be handled well; another problem with this solution is that although the coverage of its functions is high, the test efficiency is often low; because the generation process of the test case is to completely randomly select and combine functions, a great deal of invalid redundancy exists for the test of each function, and some marginal conditions still need to manually construct the test case for supplement. Another problem with this solution is that the randomly selected function, although embodied in the test code, does not necessarily have a substantial effect on the test result, and thus the purpose of verifying the correctness of the test result is not necessarily achieved.

The inventor also found that: and the mode of randomly selecting the functions is limited through certain constraint conditions, so that the functions are more effectively and more purposefully tested in a combined manner. The constraint condition can obviously reduce redundant tests existing in the randomly selected function, and simultaneously, the real effect of each function in the test case can be effectively tested. Therefore, the method has high testing efficiency. However, the main problem of this scheme is that the constraint conditions have relatively high complexity in many cases, and some may also need to make relatively complex combined constraint solutions, so that it is relatively troublesome to implement, and there are many technical difficulties in implementing specific programs, and the universality is limited. In addition, even if the constraints can describe the problem of the required test well, the randomly generated test cases have some points which are difficult to cover, so the random test always needs some solidified test cases to supplement. Another problem is that the complexity of the different functions often varies widely. Some function formats and constraints are very simple, and more similar functions have similar formats and constraints, and the type of function is usually simple in constraint, very high in flexibility and unlikely to be very high in cost due to a large number of tests. However, some functions are very complex, difficult to describe by constraints, and troublesome to test, which easily leads to a severe increase in the complexity of the whole system. Therefore, it is not ideal to make a complete and complex constraint network to realize the benefit of the random testing scheme, and it is not convenient to implement in practical application.

selecting a single function to be tested from a plurality of functions of a candidate pool of functions to be tested or selecting a function combination to be tested to generate a random test case, wherein the function combination to be tested comprises one or more functions to be tested; and for a single function to be tested, generating a test case, randomly selecting input parameters required by the single function to be tested from the parameter pool for one or more times to obtain a random test case, or for a function combination to be tested, randomly selecting input parameters required by one or more functions to be tested from the parameter pool for one or more times to obtain a random test case.

The random test case generation method of the embodiment can simply and efficiently generate the random test case, and in at least one specific example, the parameter pool can be introduced as the input and output area of the function, so that the input and output constraints of the function can be described, and a plurality of independent functions can be directly and effectively connected, for example, the random test case generation method can help to more completely cover all possible input conditions, and the parameter pool can realize the coupling of input and output.

At least one embodiment of the present disclosure further provides a random test case generation apparatus.

At least one embodiment of the present disclosure provides a random test case generation method, for example, the random test case generation method may be applied to generate a random test program, and the following examples are described by taking the purpose of generating a random test program as an example. Of course, under certain conditions, the random test case generation method may also be used to exhaustively generate complete test cases, which is equivalent to that each possible combination can be selected once in the selection process. The embodiment of the present disclosure does not limit a specific application scenario generated by the random test case, and the embodiment of the present disclosure is not exhaustive and repeated herein.

Fig. 1 is a flowchart of a random test case generation method according to some embodiments of the present disclosure. As shown in fig. 1, the random test case generation method includes steps S110 to S130.

Step S110, a candidate pool of functions to be tested is constructed, and the candidate pool of functions to be tested comprises a plurality of functions.

For step S110, for example, in some examples, the candidate pool of functions to be tested is a set of functions to be tested that is constructed according to the functions that need to be tested. For example, the set of functions to be tested includes, but is not limited to, the following four functions to be tested: integer addition IADD, floating point addition FADD, integer to floating point I2F and floating point to integer F2I, where for example the four functions under test involve two variables in total: integer int and floating point float. The four operations to be performed are in the form of:

int IADD(int,int)；

float FADD(float,float)；

float I2F(int)；

int F2I(float)；

the above example describes each function by using a function form of C language, but the function to be tested in the embodiment of the present disclosure may be a software function or an instruction supported by hardware, and the embodiment of the present disclosure does not limit the embodiment of the specific function to be tested.

Step S120, constructing a parameter pool, wherein the parameter pool comprises a plurality of parameters for testing; wherein the pool of parameters is constructed by imposing constraints on input parameters and/or output parameters for the test.

For step S120, for example, in some examples, the parameter pool refers to a set of input parameters and/or output parameters (i.e., output parameters output by the function under test running). For example, the parameter in the parameter pool may be a fixed constant or may be a variable, and the variable or another random test case is generated and then assigned when the actual test is performed, and the variable may be defined as various appropriate types, such as an integer type, a floating point type, and the like, and thus assigned with a corresponding integer, floating point number, and the like.

For example, in some examples, different types of parameters are categorized into different sub-parameter pools, which may also be referred to as parameter groupings, which is equivalent to grouping different types of parameters in a parameter pool. The types of parameters referred to by these examples may be all supported data types, even including plain binary sequences. For example, for the four functions to be tested (i.e., the integer addition IADD, the floating point addition FADD, the conversion from the integer to the floating point I2F, and the conversion from the floating point to the integer F2I), the involved parameter types include an integer type (int type) and a floating point type (float type), and the integer and the floating point are respectively categorized into different sub-parameter pools. Of course, the embodiments of the present disclosure are not limited to the specific form of the type of the parameter, and are not exhaustive here. In some more complex functional tests, more sub-parameter pools can be added, for example, grouping can be performed not only according to parameter types, but also according to the value ranges of parameter values, because some functions may only accept input parameters in a specified range, and the like; for example, positive numbers and negative numbers are grouped into a group, i.e., each is categorized into a different sub-parameter pool.

For step S120, for example, in some examples, a required sub-parameter pool is constructed by applying constraints to the input parameters and/or output parameters of the function to be tested. For example, parameters are grouped into sub-parameter pools (i.e., multiple parameter groupings) to implement constraints on the input and/or output of functions. For example, when a function can only accept a specific type of input, it can only randomly select input parameters from the corresponding sub-parameter pool that conforms to the specific type. For example, in some examples, input parameters and/or output parameters of a function are grouped according to one or more of an input parameter data type, an output parameter data type, an input parameter value range, and an output parameter value range, such that the constructed parameter pool includes one or more parameter groups. Of course, the embodiments of the present disclosure do not limit the specific content of the constraints of the input parameters and the output parameters, and can be freely adjusted according to actual needs.

For example, the following examples illustrate that the four functions to be tested (IADD, FADD, I2F, and F2I) included in the set of functions to be tested support input and output parameters of two types (integer type and floating point type). Of course, the embodiments of the present disclosure do not limit this. Thus, in these examples, a parameter pool including at least two groupings of parameters, an integer pool and a floating point pool, needs to be constructed to enable grouping of parameters.

Step S130, selecting a single function to be tested from a plurality of functions in a candidate pool of functions to be tested (e.g., a function set to be tested), or selecting a function combination to be tested to generate a random test case, where the function combination to be tested includes one or more functions to be tested.

Unit test (UnitTest) refers to the correctness testing of a single function under test. The combined Test (Combinatorial Test) refers to a Test for simultaneously performing correctness on two or more functions to be tested, and is also called an integrated Test (integration Test), i.e., a Test for performing a combination Test on a plurality of functions to ensure that the functions can interact correctly, so that the functions can work normally when being used together. Therefore, according to the type of the generated test case, the test case can be divided into a random unit test case (i.e. a random test case is generated corresponding to the above-mentioned selection of a single function to be tested) and a random combination test case (i.e. a random test case is generated corresponding to the above-mentioned selection of a function to be tested). The random unit test case of the embodiment of the present disclosure may be, but is not limited to, a unit test, and the random combination test case of the embodiment of the present disclosure may be, but is not limited to, a combination test.

The execution subject of each step above may be any device that generates a test case, and the embodiment of the present disclosure is not limited thereto.

Fig. 2 is a flowchart of a random unit test case generation method according to some embodiments of the present disclosure. The following description mainly takes the above function set to be tested including four functions to be tested (IADD, FADD, I2F, and F2I) as an example. As shown in fig. 2, in some examples, the random unit test case generation method includes steps S1100 to S1300 described below.

Step S1100, a function set to be tested is constructed, and the function set to be tested comprises a plurality of functions.

Step S1200, a parameter pool is constructed, and the parameter pool comprises a plurality of input parameters for testing.

Step 1300, selecting a single function to be tested from the function set to be tested, and randomly selecting input parameters required by the single function to be tested from the parameter pool constructed in the step 1200 according to the selected input form of the single function to be tested to obtain a random unit test case.

For step S1300, for example, in some examples, based on the selected input parameters, an expected output value of the single function under test at the corresponding input is obtained, so as to obtain a random unit test case.

For generating random unit test cases, for example, in some examples, input parameters required by a single function to be tested may be randomly selected from a parameter pool one or more times, and one or more test cases may be correspondingly generated, and the multiple test cases may constitute a test set. For simplicity and convenience of expression, the test case described in the embodiment of the present disclosure may represent one test case, or may represent a test set including a plurality of test cases, and details of relevant contents may not be repeated below.

For example, in some examples, when a function is tested for correctness, a simulator is needed to calculate the expected output value of the function at the corresponding input, and only test cases with corresponding inputs and expected outputs can be tested for correctness completely. The simulator simulates a working process of a function in a real environment through a function of a non-native design environment, so as to obtain a correct or reference result of the real function, for example, a CPU software may be used to simulate and execute a hardware instruction, or a CPU function may be used to simulate a GPU function, and the like. The CPU (Central Processing Unit) is a central Processing unit, and the GPU (graphics Processing Unit) is a graphics processor.

For example, in some examples, the integer addition IADD requires two integer input parameters, and the parameter pool includes an input parameter pool (i.e., an integer pool) at this time, and the test case can be generated by randomly selecting two integers from the input parameter pool as inputs. It should be noted that, whether for the random unit test case described above or the random combination test case below, the parameters in any parameter pool of the embodiments of the present disclosure may be reused, for example, two integer input parameters of the integer addition IADD may be from the same integer in the integer pool.

It should be noted that, for generating a random unit test case, at least an input parameter pool for providing input parameters required by a single function to be tested needs to be constructed, and as for whether output parameters of the function need to be constructed for classifying and placing, the embodiment of the present disclosure is not limited, that is, an output parameter pool may be constructed to receive output parameters output by the function to be tested after running, or the output parameter pool may not be constructed, which is not described in detail in the embodiment of the present disclosure.

As can be seen from the above, compared with a general method for generating a random unit test case, the method for generating a random unit test case according to the embodiment of the present disclosure is helpful to cover all possible input situations more completely due to the introduction of a parameter pool; for example, a single initial input to be tested of the embodiments of the present disclosure can cover all necessary types so as to satisfy the input requirement of the function, and a single initial input to be tested of the embodiments of the present disclosure can cover a specific type of special value (such as positive infinity INF of floating point number, non-number NaN, etc.), so as to ensure the correctness of the test covering some special inputs. In addition, for generating the random unit test case, the number of the constructed input parameter pools is not limited, and for example, when necessary requirements such as input types and ranges are met, the random unit test case can be freely selected according to needs. For example, when the function can support any type of input, one input parameter pool can be realized, and for example, when a plurality of input parameter pools are constructed, the input parameter pools can be correspondingly divided for grouping according to different parameter types and even different parameter ranges, so that the function can select random input more conveniently and efficiently.

For example, the random combination test case generated in step S130 has randomness in function selection and randomness in function input. For the combined test, the combined test is meaningful only if there is a correlation between the inputs or outputs of the functions to be tested, or if there is some correlation on some underlying operating resources. If the functions to be tested are not related, the test is equivalent to a plurality of independent unit tests, and the mutual operation does not affect the correctness of the other side, so that the purpose of combination test cannot be achieved.

For example, in some examples, for the combination of functions to be tested, a test case is generated, and input parameters required by one or more functions to be tested in the combination of functions to be tested are randomly selected from the parameter pool one or more times to obtain a random test case. Similarly, for simplicity and convenience of expression, the random test case is one test case or multiple test cases (test sets) corresponding to the one or multiple random selection inputs, that is, the test case related to the embodiment of the present disclosure may represent one test case, or may represent a test set including multiple test cases, and details of related contents may not be repeated below.

Wherein, for the randomness of the function selection of the random combination test, it can be understood as: for example, at least one tested function is randomly selected from a plurality of functions of the candidate pool of tested functions one or more times to form a combination of tested functions, for example, the randomness includes the randomness of the selection of the function types and the randomness of the sequence of the function combinations, so the combination of tested functions is actually a state formed by randomly selecting the functions one or more times. In addition, for randomness on the functional inputs of the random combination test, it can be understood that: for example, the input parameters required to select a function from a plurality of parameters of the parameter pool are random.

The random unit test case generation method described in the above embodiment is helpful to more completely cover all possible related effects of input due to the introduction of the parameter pool, and is also applicable to the generation of the random combination test case in the embodiment of the present disclosure, and details are not described herein. In addition, for the purpose of implementing combined testing, at least one embodiment of the present disclosure can also implement mutual coupling of multiple functions (such as implementing coupling of input and output through the use of a parameter pool), for example, for any function, output parameters of other functions can be selected as input of the function in the parameter pool, which means that, for generating random combined test cases, the parameter pool can be divided into multiple parameter packets including input parameter packets (also referred to as input sub-parameter pools) and output parameter packets (also referred to as output sub-parameter pools) based on the parameter packets. The input parameter group is used for inputting parameters required by one or more functions to be tested in the function combination to be tested; the output parameter packet is used for receiving output parameters output by one or more functions to be tested of the function combination to be tested, and the output parameter packet is also used for providing input parameters required by one or more functions to be tested in the function combination to be tested.

For the generation of random combinational test cases of step S130, for example, in some examples, to ensure test coverage and functional coupling efficiency, the input parameter packets and the output parameter packets are separated (i.e., do not overlap with each other) in the constructed parameter pool. For example, the input parameters of each function may be from an input parameter group, or from an output parameter group, or a part of the input parameters of the function may be from the input parameter group and another part from the output parameter group, and the probabilities of the two selections may be freely adjusted as needed, which is not limited in this disclosure.

As can be seen from the above, the embodiments of the present disclosure may use the grouping parameter pool as the input/output area of the function, so that not only the input/output constraints of the function can be described, but also a plurality of independent functions can be directly and effectively connected.

Fig. 3 is a flowchart illustrating a method for generating a random combinational test case according to some embodiments of the present disclosure. As shown in fig. 3, in some examples, the random combination test case generation method includes steps S1101 to S1305 described below.

Step S1101, constructing a function set to be tested, where the function set to be tested includes a plurality of functions.

Step S1201, an initial parameter pool is constructed, wherein the initial parameter pool at least comprises a plurality of input parameters of one or more functions to be tested for the function combination to be tested.

And step S1301, randomly selecting a current function to be tested from the function set to be tested.

Step S1302, randomly selecting an input parameter required by the currently selected function to be tested from a plurality of input parameters currently included in the initial parameter pool.

Step S1303, the currently selected function to be tested is operated to obtain the corresponding current output parameter, and the current output parameter is sent to the initial parameter pool for subsequent selection, so as to update the parameters in the initial parameter pool.

Step S1304, determining whether a test termination condition is satisfied: if yes, go to the following step S1305; if not, the process goes to step S1301, and the process loops from step S1301 to step S1304 until the test termination condition is satisfied, and then goes to step S1305 described below.

Step S1305, the generation of the random combination test case is finished.

First, for step S1101, for example, in some examples, the set of functions under test includes, but is not limited to, the following four functions under test: the method comprises the following steps of integer addition IADD, floating point number addition FADD, integer conversion to floating point number I2F and floating point conversion to integer F2I, wherein the four functions to be tested relate to two variables: integer int and floating point float.

Next, for step S1201, for example, in some examples, the initial parameter pool is grouped such that the initial parameter pool includes at least an initial input parameter group (also referred to as an initial input sub-parameter pool) and an initial output parameter group (also referred to as an initial output sub-parameter pool), wherein the initial input parameter group includes a plurality of initial input parameters, and the initial input parameters are used as a plurality of input parameters of one or more functions to be tested of the function combination to be tested. The initial output parameter group is used for adding the current output parameters of the current selected function to be tested, which are randomly selected each time, for subsequent selection, so that the initial output parameter group is updated iteratively, and the initial parameter pool is updated iteratively on the whole. Likewise, to improve test coverage and functional coupling efficiency, the initial input parameter packet and the initial output parameter packet are non-overlapping with each other.

Next, for step S1301, for example, in some examples, each time a current function to be tested is selected from the set of functions to be tested is actually selected randomly, for example, the currently selected function to be tested may be an integer addition IADD, a floating point addition FADD, or an integer conversion to a floating point I2F or a floating point conversion to an integer F2I, that is, the function to be tested is completely random when selected.

Next, for step S1302, for example, in some examples, the input parameters of the currently selected function to be tested may be from the initial input parameter group, or from the initial output parameter group, or a part of the input parameters may be from the initial input parameter group and another part of the output parameters may be from the initial output parameter group, and the probabilities of the two selections may be freely adjusted as needed. For example, the probability is configured such that the probability of selecting a parameter for input to the currently selected function to be tested from the initial input parameter group is greater than the probability of selecting a parameter for input to the currently selected function to be tested from the initial output parameter group, which enables the cascade coupling degree of the combination of the functions to be tested to be high, that is, a plurality of functions are strung into a long chain, but the diversity of the function combination and the parameter value is weak at this time. Of course, the probability may also be configured such that the probability of selecting the parameter for input to the currently selected function to be tested from the initial input parameter group is smaller than the probability of selecting the parameter for input to the currently selected function to be tested from the initial output parameter group, which may make the value of the function combination and the parameter more similar, but the degree of cascade coupling of the function combination to be tested is lower.

Next, for step S1303, for example, in some examples, each time the currently selected function to be tested is subjected to a correctness test, the function is generally operated and calculated by a simulator to obtain an expected output value under a corresponding input, and only test cases with corresponding inputs and expected outputs can be subjected to a complete correctness test.

Next, for step S1304, for example, in some examples, the test termination condition may be that all functions to be tested in one or more functions to be tested of the combination of functions to be tested have been selected at least once, or that the total code length of the random test program reaches a predetermined value (for example, when the random test case generation method is applied to generate the random test program). The test termination condition is not limited in the embodiments of the present disclosure, and may be adjusted according to specific applications and settings, which is not described herein again.

For example, in some examples, when the number of times of performing steps S1301 to S1304 in a loop is one, that is, the selected function to be tested is one function, and when the number of times of performing steps S1301 to S1304 in a loop is multiple, that is, the selected function to be tested is multiple functions.

Finally, for example, in some examples, the above-mentioned complete steps S1101 to S1305 may be performed randomly one or more times, and then one test case or multiple test cases (test sets) are obtained correspondingly.

When the embodiment of the disclosure is applied to generating a random test program, for the formed function combination to be tested, all variables in the input parameters are the inputs of the whole combination test program, and the inputs can be assigned as required during testing. All final output parameters of the entire combinatorial test procedure are the results that need to be checked. The final output parameters refer to those output parameters in the output parameter packet that are not used as inputs for other functions; the output parameter of the output parameter packet that is used as input for the other function is referred to as an intermediate output parameter, which may or may not be checked, because the other function uses the intermediate output parameter as input, and checking the output of the other function can confirm the correctness of the intermediate result to some extent.

It should be noted that, in the embodiment of the present disclosure, the flow of the test case generation method provided by each example above may include more or less operations, and these operations may be executed sequentially or in parallel. Although the flow of the test case generation method described above includes a plurality of operations occurring in a particular order, it should be clearly understood that the order of the plurality of operations is not limited. The test case generation method described above may be executed once or multiple times according to a predetermined condition.

The following description will be made of the generation of the random combination test case by taking an example in which the function set to be tested includes four functions to be tested (IADD, FADD, I2F, and F2I).

For example, in some examples, the constructed initial pool of input subparameters is again divided into two groups: an integer input packet a1 and a floating point input packet a 2. For example, the integer input packets have integers i0 and i1, and the floating point input packet has a floating point f 0. In addition, the constructed initial output sub-parameter pool is divided into two groups: integer output packet B1 and floating point output packet B2. Firstly, a function is randomly selected from a function set to be tested, for example, the function is converted into a floating point number I2F from an integer, an integer I1 is randomly selected from an integer input packet A1 to serve as an input of the function I2F, a floating point number f1 is correspondingly output, and then the output floating point number f1 is put into a floating point number output packet B2, so that the floating point number output packet B2 has the floating point number f1, namely the floating point number packets (including a floating point number input packet A2 and a floating point number output packet B2) have the floating point numbers f0 and f 1. And then randomly selecting a next function from the function set to be tested, such as a floating point number addition FADD, respectively selecting a floating point number f0 and a floating point number f1 from a floating point number input packet A2 and a floating point number output packet B2 as the input of the function FADD, correspondingly outputting a floating point number f2, and then putting the output floating point number f2 into a floating point number output packet B2. Therefore, it can be found that the two functions of integer conversion to floating point I2F and floating point addition FADD form a relationship through the floating point output parameter value "f 1", that is, the floating point addition FADD can select the output parameter value f1 of one other function (for example, integer conversion to floating point I2F) as an input in the sub parameter pool (for example, floating point grouping), so as to achieve the purpose of combination testing. It should be noted that, in the test program generation process, the values i1, f1 and f2 of specific variables are irrelevant to program generation, and the randomly selected parameters are all based on variable names rather than variable values. The test program generated here was:

f1＝I2F(i1)；

f2＝FADD(f0,f1)；

the input values required by this procedure are i1 and f0, and f2 is output (or f1 can be output at the same time). In the test, only different I1 and f0 are required to be input, and the value of f2 is checked to detect whether the I2F and FADD functions can work normally at the same time.

It should be noted that the number and the labels of the parameters included in each parameter group listed in the above example are only some examples, so as to facilitate description and understanding of the technical solution of the embodiment of the present disclosure, of course, other multiple parameters may also be not completely listed, and the above randomly selected specific parameter is also only an example as an input, and the input parameter that does not affect the function described in the embodiment of the present disclosure is random, and the point of attention here also applies to the following examples, and the embodiment of the present disclosure is not described again in detail in related contents.

For example, in other examples, the constructed initial input sub-parameter pool is divided into two groups: an integer input packet a1 and a floating point input packet a 2. For example, the integer input packet A1 has integers i0 and i1, and the floating point input packet A2 has a floating point f 0. In addition, the constructed initial output sub-parameter pool is divided into two groups: integer output packet B1 and floating point output packet B2. Firstly, a function is randomly selected from a function set to be tested, for example, a floating point number is converted into an integer F2I, a floating point number F0 is randomly selected from a floating point number input packet A2 to serve as the input of the function F2I, an integer i2 is correspondingly output, and then the output integer i2 is put into an integer output packet B1. Thus, integer output packet B1 has integer i2, i.e., integer packet (including integer input packet a1 and integer output packet B1) has integers i0, i1, and i 2. And then randomly selecting the next function from the function set to be tested, such as integer addition IADD, respectively selecting an integer i1 and an integer i2 from an integer input group A1 and an integer output group B1 as the inputs of the function IADD, correspondingly outputting an integer i3, and then putting the output integer i3 back into an integer output group B1. Therefore, it can be found that the floating point number is converted into the integer F2I and the integer addition IADD forms a relationship through the integer output parameter value "i 2", that is, the functional integer addition IADD can select the output parameter value i2 of one other function (for example, the functional floating point number is converted into the integer F2I) as an input in the sub-parameter pool (for example, the integer grouping), so as to achieve the purpose of the combination test. It should be noted that, in the test program generation process, the values f0, i1 and i2 of specific variables are irrelevant to program generation, and the randomly selected parameters are all based on variable names rather than variable values. The test program generated here was:

i2＝F2I(f0)；

i3＝IADD(i1,i2)；

the input values required by the program are f0 and i1, and i3 is output (i 2 can also be output at the same time). In the test, only different i1 and F0 are needed to be input, and the value of i3 is checked to detect whether the F2I and IADD functions can work normally at the same time.

In addition, for the function set to be tested including four functions to be tested (IADD, FADD, I2F, and F2I), besides the above-mentioned example function combination, there are other possible combinations, for example, in some examples, the function to be tested is formed by sequentially and randomly selecting an integer addition IADD and an integer conversion to a floating point number I2F, and these two functions may also be associated by an intermediate output parameter (i.e., an intermediate output parameter of an integer type), so as to achieve the purpose of the combination test. For another example, in some examples, the function to be tested formed by sequentially randomly selecting floating point number addition FADD and converting the floating point number into the integer F2I is combined, and the two functions may also form an association through an intermediate output parameter (i.e., an intermediate output parameter of a floating point type), so as to achieve the purpose of the combined test. This means that all reasonable combinations of functions to be tested are potential test cases, which are not exhaustive in the embodiments of the present disclosure, and reference may be made to the above embodiments specifically, and based on the above described embodiments, all other reasonable test cases and methods obtained by those skilled in the art without creative labor fall within the scope of the present disclosure.

However, in addition to the above possible function combinations to generate the random combination test case, there is also a function combination manner that cannot implement the combination test for the function set to be tested including four functions to be tested (IADD, FADD, I2F, and F2I); for example, the function combination to be tested is formed by an integer addition IADD and a floating point addition FADD which are randomly selected in sequence, for example, the function combination to be tested is formed by a floating point addition FADD and an integer addition IADD which are randomly selected in sequence, and for example, the function combination to be tested is formed by a floating point addition FADD and an integer which are randomly selected in sequence and converted into a floating point I2F; in these cases, no correlation is formed between functions, and therefore, the purpose of combination test cannot be achieved.

As mentioned above, most of the descriptions in the embodiments are about the mutual coupling between any two functions to be tested in any combination of functions to be tested during the random combination test, which is mainly for explaining the functional coupling, and other parts of the random combination test case are not illustrated one by one, because the combination of functions to be tested not only includes one or two functions to be tested, but also includes a plurality of functions to be tested in any number greater than two, that is, the random combination test can perform the correctness test on two or more functions to be tested at the same time, and the specific manner needs to be determined according to the actual situation, which is not limited in this disclosure and will not be described again.

In addition, the inventor also finds that in some application scenarios, one difficulty of the random combination test is how to balance the test coverage and the test time, so that the test efficiency needs to be improved as much as possible, and various combinations need to be covered as much as possible but cannot have too much redundancy. Therefore, in some examples of the present disclosure, for generating a test case for a function combination to be tested, a manner of randomly selecting a function may be defined by constraining a plurality of functions to be tested in the function combination to be tested, and further input parameters required for randomly selecting a plurality of functions to be tested from a parameter pool one or more times may be defined, so as to obtain a random test case.

For example, in some examples of the present disclosure, functions to be tested are grouped accordingly, so as to constrain a plurality of functions to be tested in a function combination to be tested. Wherein, the functions in the same group have higher similarity and have approximately equivalent functions in the combined test. The interaction of functions between two function groups, generally the internal mechanism is common, therefore, only the corresponding combination between the groups is needed and at least one function to be tested is randomly selected in each group, and all possible combinations are not needed to be carried out on each instruction of each group, thereby greatly reducing the number of required test cases.

For example, in some examples, similarities between the functions of the same group obtained by grouping include: the constraints imposed on the input parameters and/or output parameters of the plurality of functions are the same, for example: input parameter data type, output parameter data type, input parameter value range and output parameter value range. Of course, the content of the similarity is not limited in this embodiment of the disclosure, that is, the embodiment of the disclosure does not limit how to group the functions to be tested, and a person skilled in the art may determine the method according to the actual situation, and the embodiment of the disclosure is not described herein again.

As mentioned above, the above function grouping is an extension of the input and/or output constraint description of functions by parameter pool. The simple grouping mode is simple and easy to implement, is convenient to realize, and can meet most test requirements.

For example, in some examples, the set of functions under test includes, but is not limited to, the following four functions under test: the addition of integers IADD, subtraction of integers ISUB, addition of floating point FADD and subtraction of floating point FSUB all of which may produce an overflow, i.e. the result exceeds the representation range of the current integer or floating point. The overflow status may sometimes be an input or an output, such as an integer carry flag bit in a CPU, but the floating-point overflow generally does not have such a flag bit. Therefore, both the integer addition IADD and the integer subtraction ISUB may operate on the integer carry flag bit, i.e. there is an additional implicit input and/or output parameter, and it must be put into the parameter pool. While the addition of a floating-point number FADD and the subtraction of a floating-point number FSUB do not have such an operation of putting flag bits as input and/or output parameters into the parameter pool. Therefore, the integer addition IADD and the integer subtraction ISUB can be grouped into one group, the floating point addition FADD and the floating point subtraction FSUB can be grouped into one group, and the carry flag bit correlation test only needs to select at least one of the integer addition IADD and the integer subtraction ISUB.

Based on the above, the functions of the embodiments of the present disclosure may be functions of a software layer or instructions of a hardware layer, and correspondingly, the inputs and/or outputs of the functions may be explicit incoming parameter values or implicit software or hardware states (such as carry flags, program counters, etc.), which is not limited by the embodiments of the present disclosure and may be determined according to the actual situation.

For generating a test case for a function combination to be tested containing a plurality of functions to be tested, for example, in some examples, the functions of the function combination to be tested belong to N function groups (N > 1), and the input parameters required by the functions to be tested are randomly selected from a parameter pool for a plurality of times, including: randomly selecting one or M function groups (N and M are integers, and M is more than 1 and less than or equal to N) from the N function groups, randomly selecting one or more tested functions from the one or M function groups respectively, and randomly selecting input parameters required by the one or more tested functions from a parameter pool once or for many times to obtain a random test case.

For example, in other examples, the following includes six functions under test in the set of functions under test: the integer addition IADD, the integer subtraction ISUB, the floating point addition FADD, the floating point subtraction FSUB, the function CC2I, and the function CC2F will be described as examples. The input of function CC2I is a carry flag, the output of function CC2I is an integer, the input of function CC2F is a carry flag, and the output of function CC2F is a floating point. The specific operation of the function CC2I is: when the carry flag bit is 0, 0 of the integer is output, and when the carry flag bit is 1, 1 of the integer is output. The specific operation of the function CC2F is: and outputting 0 of the floating point number when the carry flag bit is 0, and outputting 1 of the floating point number when the carry flag bit is 1. Thus, the two functions of CC2I and CC2F may each be grouped separately.

For example, if the related functions of the carry flag bit need to be tested, the two functions CC2I and CC2F are respectively and separately grouped into one group, and IADD/ISUB is grouped into the same functional group, and FADD/FSUB is grouped into the same functional group, that is, when N is 3, there may be two testing routes:

the first route is (IADD/ISUB) - > CC2I- > (IADD/ISUB), namely the route has 4 reasonable functional combinations, and specifically comprises the following steps: IADD- > CC2I- > IADD, ISUB- > CC2I- > IADD, IADD- > CC2I- > ISUB, ISUB- > CC2I- > ISUB;

the second route is (IADD/ISUB) - > CC2F- > (FADD/FSUB), i.e. this route is also 4 reasonable combinations, specifically including: IADD- > CC2F- > FADD, ISUB- > CC2F- > FADD, IADD- > CC2F- > FSUB, ISUB- > CC2F- > FSUB.

When the work can be grouped without doing work, the two test routes need to be completely covered, namely 8 test cases are needed, and the 8 test cases correspond to the 8 possible combinations. After the functions are grouped, because the IADD/ISUB and the FADD/FSUB are grouped into the same function group, and have the same status during the function combination test, the purpose of performing the combination test between the function groups can be achieved only by selecting one of the IADD/ISUB function groups and one of the FADD/FSUB function groups, so that only two test cases are needed to cover the two test routes (for example, IADD- > CC2I- > IADD and IADD- > CC2F- > FADD, where M is 3, but the embodiment of the present disclosure is not limited thereto, and this is only an example). Therefore, the embodiment of the disclosure can more effectively achieve the same test coverage rate with fewer test cases through function grouping. It should be noted that, the example of obtaining the test case through the functional grouping mainly describes how to perform the processes of functional grouping and acceleration and function random selection, and as for how to implement the construction of the parameter pool and how to specifically generate the content of the random test case, reference may be made to the content of the test case generated by the other combinations of functions to be tested, which is not described herein again in the embodiments of the present disclosure.

The random test program generated in the embodiment of the present disclosure is not necessarily limited to the correctness of the function of the verification program itself. Through random input herein, consistency between multiple different processes based on the input can be verified. For example, after a random test case is generated, one is compiled by inputting a compiler without optimization, and the other is compiled by inputting a compiler with optimization. The two operations theoretically generate code that, although different, generally need to be consistent in terms of execution results with the same input. Therefore, the embodiments of the present disclosure may also be used for the generation of random test cases in such scenarios.

The algorithm and the preparation work of the random test case generation method disclosed by the embodiment are simple and easy to implement, the practicability is good, the test coverage rate of the generated test case is high enough, the redundancy of the test case can be obviously reduced, the test efficiency is improved, a good compromise can be formed between the test coverage rate and the test cost, and most of actual test requirements can be met.

Fig. 4 is a schematic block diagram of a random test case generation apparatus according to some embodiments of the present disclosure. In the example shown in fig. 4, the random test case generation apparatus 100 includes a first building module 110, a second building module 120, a function selection module 130, and a parameter random selection module 140. For example, these modules may be implemented by software, hardware, or a combination of the two, etc., and the embodiments of the present disclosure do not limit this. The following examples are the same and will not be described in detail.

The first constructing module 110 is configured to construct a candidate pool of functions to be tested, which includes a plurality of functions, for example, the first constructing module 110 may implement step S110, and a specific implementation method thereof may refer to a related description of step S110, which is not described herein again.

The second building module 120 is configured to build a parameter pool by imposing constraints on input parameters and/or output parameters for testing, wherein the parameter pool comprises a plurality of input parameters and/or output parameters for testing. For example, the second building module 120 may implement step S120, and the specific implementation method may refer to the related description of step S120, which is not described herein again.

The function selection module 130 is configured to select a single function under test or a combination of functions under test from a plurality of functions of the candidate pool of functions under test, wherein the combination of functions under test includes one or more functions under test. The parameter random selection module 140 is configured to randomly select input parameters required by a single function to be tested or one or more functions to be tested in a function combination to be tested from the parameter pool one or more times to obtain a random test case. For example, the step S130 may be implemented by the interaction of the function selection module 130 and the parameter random selection module 140, and the specific implementation method may refer to the related description of the step S130, which is not described herein again.

It should be noted that, in the embodiment of the present disclosure, the random test case generation apparatus 100 may include more or fewer modules, and the connection relationship between the modules is not limited and may be determined according to actual requirements. The specific configuration of each module is not limited. For technical effects of the random test case generation apparatus 100, reference may be made to the technical effects of the random test case generation method provided in the embodiments of the present disclosure, and details are not described here.

Fig. 5 is a schematic block diagram of a random test case generation device according to some embodiments of the present disclosure. As shown in fig. 5, the random test case generation apparatus 200 includes a processor 210 and a memory 220. Memory 220 is used to store non-transitory computer readable instructions (e.g., one or more computer program modules). The processor 210 is configured to execute non-transitory computer readable instructions that, when executed by the processor 210, may perform one or more of the steps of the random test case generation method above. The memory 220 and the processor 210 may be interconnected by a bus system and/or other form of connection mechanism (not shown).

For example, the processor 210 may be a Central Processing Unit (CPU), a Digital Signal Processor (DSP), or other form of processing unit having data processing capabilities and/or program execution capabilities, such as a Field Programmable Gate Array (FPGA), or the like; for example, the Central Processing Unit (CPU) may be an X86 or ARM architecture or the like. The processor 210 may be a general-purpose processor or a special-purpose processor that may control other components in the random test case generation apparatus 200 to perform desired functions.

For example, memory 220 may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. Volatile memory can include, for example, Random Access Memory (RAM), cache memory (or the like). The non-volatile memory may include, for example, Read Only Memory (ROM), a hard disk, an Erasable Programmable Read Only Memory (EPROM), a portable compact disc read only memory (CD-ROM), USB memory, flash memory, and the like. One or more computer program modules may be stored on the computer-readable storage medium and executed by the processor 210 to implement the various functions of the random test case generation apparatus 200. Various applications and various data, as well as various data used and/or generated by the applications, and the like, may also be stored in the computer-readable storage medium.

It should be noted that, in the embodiment of the present disclosure, reference may be made to the description about the random test case generation method above for specific functions and technical effects of the random test case generation device 200, and details are not described here.

Fig. 6 is a schematic block diagram of another random test case generation apparatus 300 according to some embodiments of the present disclosure. The random test case generation device 300 is, for example, suitable for implementing the random test case generation method provided by the embodiment of the present disclosure. The random test case generating apparatus 300 may be a user terminal or the like. It should be noted that the random test case generation apparatus 300 shown in fig. 6 is only an example, and does not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.

As shown in fig. 6, the random test case generation apparatus 300 may include a processing device (e.g., a central processing unit, a graphic processor, etc.) 310, which may perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)320 or a program loaded from a storage device 380 into a Random Access Memory (RAM) 330. In the RAM330, various programs and data necessary for the operation of the random test case generating apparatus 300 are also stored. The processing device 310, the ROM 320, and the RAM330 are connected to each other by a bus 340. An input/output (I/O) interface 350 is also connected to bus 340.

Generally, the following devices may be connected to I/O interface 350: input devices 360 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 370 including, for example, a Liquid Crystal Display (LCD), speakers, vibrators, or the like; storage 380 including, for example, magnetic tape, hard disk, etc.; and a communication device 390. The communication device 390 may allow the random test case generating apparatus 300 to communicate with other electronic devices wirelessly or by wire to exchange data. While FIG. 6 illustrates the random test case generation apparatus 300 having various means, it is to be understood that not all of the illustrated means are required to be implemented or provided, and that the random test case generation apparatus 300 may alternatively be implemented or provided with more or fewer means.

For example, the random test case generation method provided by the embodiment of the present disclosure may be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a non-transitory computer readable medium, the computer program comprising program code for performing the random test case generation method described above. In such embodiments, the computer program may be downloaded and installed from a network through communication device 390, or installed from storage device 380, or installed from ROM 320. When the computer program is executed by the processing device 310, the random test case generation method provided by the embodiment of the disclosure may be executed.

At least one embodiment of the present disclosure also provides a storage medium for storing non-transitory computer-readable instructions, which when executed by a computer may implement the random test case generation method of any embodiment of the present disclosure. By using the storage medium, the test case can be generated through the algorithm of the random test case generation method, the algorithm is simple in structure and good in practicability, the test coverage rate of the generated test case is high enough, and the test efficiency is high. At least one embodiment provides a storage medium that also provides a good compromise between test coverage and test cost to meet most practical test requirements.

Fig. 7 is a schematic diagram of a storage medium according to some embodiments of the present disclosure. As shown in fig. 7, the storage medium 400 is used to store non-transitory computer readable instructions 410. For example, the non-transitory computer readable instructions 410, when executed by a computer, may perform one or more steps in a random test case generation method according to the above.

For example, the storage medium 400 may be applied to the random test case generating apparatus 200 described above. For example, the storage medium 400 may be the memory 220 in the random test case generating apparatus 200 shown in fig. 5. For example, the relevant description about the storage medium 400 may refer to the corresponding description of the memory 220 in the random test case generating device 200 shown in fig. 5, and will not be described herein again.

The following points need to be explained:

(1) the drawings of the embodiments of the disclosure only relate to the structures related to the embodiments of the disclosure, and other structures can refer to common designs.

(2) Without conflict, embodiments of the present disclosure and features of the embodiments may be combined with each other to arrive at new embodiments.

The above description is only a specific embodiment of the present disclosure, but the scope of the present disclosure is not limited thereto, and the scope of the present disclosure should be subject to the scope of the claims.

Claims

1. A random test case generation method comprises the following steps:

2. The random test case generation method of claim 1,

by grouping input parameters and/or output parameters for testing, comprising:

and grouping the input parameters and/or the output parameters for the single function to be tested or the combination of the functions to be tested according to one or more of the input parameter data type, the output parameter data type, the input parameter value range and the output parameter value range.

3. The random test case generation method of claim 1,

the plurality of parameter packets includes an input parameter packet and an output parameter packet;

the input parameter group is used for providing input parameters required by one or more functions to be tested in the single function to be tested or the function combination to be tested;

the output parameter group is used for receiving output parameters output by one or more functions to be tested of the single function to be tested or the combination of the functions to be tested, and the output parameter group is also used for providing input parameters required by one or more functions to be tested in the single function to be tested or the combination of the functions to be tested.

4. The random test case generation method of claim 1,

selecting a function combination to be tested from a plurality of functions of the candidate pool of functions to be tested to generate the random test case, wherein the method comprises the following steps:

forming the function combination to be tested including the one or more functions to be tested by randomly selecting at least one function to be tested from a plurality of functions of the function candidate pool to be tested one or more times.

5. The random test case generation method of any one of claims 1 to 4,

when the random test case is generated for the function combination to be tested, the parameter pool is constructed, and the method comprises the following steps:

constructing an initial parameter pool for the function combination to be tested, wherein the initial parameter pool at least comprises a plurality of input parameters for one or more functions to be tested of the function combination to be tested;

sequentially performing a plurality of functional operations to iteratively update the initial parameter pool to obtain the parameter pool, wherein sequentially performing a plurality of functional operations to iteratively update the initial parameter pool comprises:

6. The random test case generation method of claim 5,

constructing an initial parameter pool for the function combination to be tested, comprising:

grouping the initial pool of parameters such that the initial pool of parameters includes at least:

7. The random test case generation method of claim 6,

randomly selecting input parameters required for the currently selected function from a plurality of parameters currently included in the initial parameter pool, including:

randomly selecting parameters for input to the currently selected function from among initial input parameters included in the initial input parameter group and output parameters added in the iterative update included in the initial output parameter group according to a predetermined probability.

8. The random test case generation method of claim 7,

the predetermined probability is such that the probability of selecting a parameter from the initial input parameter grouping for input to the currently selected function is greater than or less than the probability of selecting a parameter from the initial output parameter grouping for input to the currently selected function.

9. The random test case generation method of claim 5,

the test termination condition includes:

all functions to be tested in one or more functions to be tested of the function combination to be tested are selected at least once; and/or the presence of a gas in the gas,

the random test case generation method is suitable for generating a random test program, and the total code length of the random test program reaches a preset value.

10. The random test case generation method of claim 1,

at least one of the parameters in the parameter pool is a constant or a variable, and the variable is used for being assigned when the random test case is actually tested.

11. The random test case generation method of any one of claims 1-4, 6-10,

the function combination to be tested comprises a plurality of functions to be tested;

generating a test case for the function combination to be tested, randomly selecting input parameters required by the one or more functions to be tested from the parameter pool one or more times to obtain the random test case, wherein the method comprises the following steps:

12. The random test case generation method of claim 11,

the functions of the function combination to be tested belong to N function groups,

randomly selecting input parameters required by the functions to be tested from the parameter pool for multiple times, wherein the input parameters comprise:

13. The random test case generation method of claim 12,

a plurality of functions in at least one of the N functional groups have similarities therebetween, the similarities including: the content of the constraint applied to the input parameters and/or the output parameters of the plurality of functions is the same;

wherein the constraints imposed on the input parameters and/or output parameters of the plurality of functions include one or more of: input parameter data type, output parameter data type, input parameter value range and output parameter value range.

14. The random test case generation method of any one of claims 1-5, 6-10, 12-13,

at least one of the functions to be tested is an instruction supported by a software function or hardware;

the input parameters and/or the output parameters respectively comprise one or more of the following: explicit incoming parameter values, implicit software or hardware state.

15. A random test case generation apparatus includes:

16. An electronic device, comprising:

a processor and a memory, wherein the processor is capable of processing a plurality of data,

wherein the memory has stored thereon a computer program that, when executed by the processor, implements the random test case generation method of any of claims 1 to 14.

17. A computer-readable storage medium, wherein the storage medium has stored therein a computer program which, when executed by a processor, implements the random test case generation method of any one of claims 1 to 14.