CN100351811C - Method and device for testing software product robustness - Google Patents
Method and device for testing software product robustness Download PDFInfo
- Publication number
- CN100351811C CN100351811C CNB2004100886733A CN200410088673A CN100351811C CN 100351811 C CN100351811 C CN 100351811C CN B2004100886733 A CNB2004100886733 A CN B2004100886733A CN 200410088673 A CN200410088673 A CN 200410088673A CN 100351811 C CN100351811 C CN 100351811C
- Authority
- CN
- China
- Prior art keywords
- test
- detecting information
- data
- information
- testing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Abstract
The present invention discloses a testing method for the robustness of a software product, which comprises the steps that a tested object is analyzed, and a testing message is extracted from the tested object; the testing message which is extracted is displayed, and the testing message input of users is received; according to the testing message which is extracted and the testing message input, a test case is formed; the test case is combined with the tested object to form a testing target code; the testing target code is operated so as to test the robustness of a software product. The present invention also relates to a testing device for the robustness of a software product. The present invention can realize the automatic generation and the automatic extraction of the testing message of the tested object.
Description
Technical field
The present invention relates to software product reliability demonstration technology, particularly relate to a kind of method of testing and device of software product robustness.
Background technology
Development along with infotech, the application of software product is increasingly extensive, affect numerous areas such as people's work, studying and living, and software product is when design or operation, need test its application reliability, so that improve targetedly, for example, to the test of the robustness of software product.The test of software product is generally included test to software systems or component software, and so-called component software is meant to have stand-alone interface, can finish the software module of certain function.
Software robustness is the degree of software systems or assembly normal operation under unusual input or pressure environment condition.Robustness testing is exactly the robustness that unusual input of manufacturing, simulation or environment come verifying software system or assembly operating.
The software robustness test is along with the software test conception development, and from white box, the test of grey box the earliest, (white-box testing is meant a kind of method of testing according to the inner structure design test case of tested program to develop into present Black-box Testing; The test of ash box is paid close attention to the realization of the test case of function or intermodule between white box and Black-box Testing; Black-box Testing then goes to select test case based on the analysis of description, and does not consider its inner realization); Tested object expands to software systems, assembly also from original function, module; But the thought of test, not too big variation of notion.Along with the tested software reliability requirement constantly improves, impel measuring technology to need development constantly, effect, the efficient of robustness testing are also had higher requirement.
Seeing also Fig. 1, is the signal of a kind of artificial realization component software robustness Black-box Testing method in the prior art.Tested module comprises several A PI (Application Progress Interface, application programming interfaces) function, and wherein the legal span of the n value of function API _ n (int n) is n＜0 or n＞10.Before testing, construct test case test_API_n () by the tester, according to analysis, get illegal random value between the 0-10 as input to tested function, tested function is carried out unusual value inject; Subsequently, in test environment, call this test case and carry out the software robustness test; And monitoring system under test (SUT) running status, constantly corrigendum, validation test design.
The defective of this method is: it is higher that personnel participate in design, the structure degree of test case, and peopleware, workload etc. is also had higher requirements, whole test mid-early stage design, structure cost plenty of time and human cost.In relating to test case change, dislocation more problem can appear especially, can produce deviation to the understanding of test case as different personnel's design experiences, therefore on the design of test case, change, dislocation, can have problems such as test case is complete, availability, maintainability; The test case size of code is big, can cause the difficulty of design, change, dislocation to increase, and is not easy to transfer and surveys; The design of test case, change, dislocation also can bring influence to positioning problems.
In a word, adopt this method to carry out the design of test case, structure, need artificial mode to write test data and test case, will inevitably cause that test data can not be shared, test code repeats to write, testing efficiency reduction, code transposability variation, the difficult problem of case maintenance management, finally causes the repeatability of above-mentioned work.
The prior art of another kind of software robustness test can be consulted the paper of CMU (Carnegie Mellon University, CARNEGIE MELLON UNIVERSITY) a piece " HIGH PERFORMANCEROBUST COMPUTER SYSTEMS " by name.This paper has disclosed a kind of robustness testing instrument at the unusual input of component software api interface: the Ballista testing tool.
With reference to Fig. 2, this instrument provides data type to describe and touches plate, the parameter in the tested function is classified, as files classes, internal memory class, value class etc., and in every kind with its value of different symbologies, each symbol all is a kind of definition (being similar to function or grand) of value mode.
In test process, tested object API (write (...) function) is touched plate according to the data type description be converted to parameter type, during tool configuration test case (code), the value touch each parameter of symbol construction that plate takes is described from data type according to the user.Three kinds of symbols as arrow indication in the example of Fig. 2 are respectively: FD_OPEN_RD, BUFF_NULL, SIZE_16, expression respectively: parameter 1 is taken a read-write handle that opens file; Parameter 2 is taken a null pointer; Parameter 3 is taken numerical value 16.Last test case is write (FD_OPEN_RD, BUFF_NULL, SIZE_16).That is to say that instrument can be that tested construction of function goes out test environment and value according to selected symbol when structure test case (code).
With respect to traditional Black-box Testing method shown in Figure 1, this Ballista testing tool defines with data type description template, data type and describes tested object and data, and then generation test case, avoided the repetitive construct of robustness testing data to a certain extent, realized testing sharing of data between different functions, reached the automatic generation of test case.Specifically, come the management data type, make data type have inheritability, extensibility, be convenient to maintenance management the data type with the notion of class; Come the management testing data with data type, test case does not rely on the function of tested function, but the combination of different types of data value in instrument that function uses; This instrument is with data type and structure of transvers plate tested object and value, and promptly by the different identification symbol, the structure test environment is finished the test value, thereby realizes the robustness testing data sharing, reaches the automatic generation of test case.
But this Ballista testing tool does not still break away from the complex process of traditional artificial participation test case design, need by the template that instrument provides describe relevant data type and value (as, the data type description template that needs the user to provide with instrument, come newtype obtaining value method that does not have in the defined declaration instrument or the value of replenishing existing type, so that test is used), therefore still can't avoid some problems that run in the design of traditional test use-case.And, brought some new problems:
At first, data type and template are described, and have limited user's use.Though instrument provides some fundamental types, can't solve a large amount of new data structure test needs.For such data, the user need describe the building method and the value of new data type with template, and workload is big, requires high.
Secondly, test data is limited in scope.Come the management testing data with data type, brought convenience for the instrument robotization generates test case, also limited the usable range of data itself simultaneously, make test ossify, test data is unreasonable etc., causes the insufficient of test.
Once more, the test data reusability is limited.The measurand interface parameters, complicated and changeable, various informative, describe certain type with the type template, often can not express the meaning of tested object fully, even if be described, also can cause its reusability to descend because of factors such as environments for use.
The 4th, the MTD deficiency.Complexity, the diversity of the inside definition of data structure, make template to the description in inside configuration member territory become complicated and changeable, tool data type template is become unlikely to the structure of test data, can't satisfy the test in a large amount of inner member of labyrinth territory.
The 5th, under the situation of mass data type, be not easy to safeguard.In use data type and data can constantly increase, and data type and data describe out (being similar to coding) by template, therefore can cause the problem in the later maintenance.
At last, test data lacks dirigibility.The test value that dtd goes out will provide explanation in tool palette, just can not arbitrarily change or produce the data that need like this concerning user and instrument, causes on the value dumbly, increases operation easier.
Summary of the invention
The technical matters that the present invention solves provides a kind of method of testing and device of software product robustness, can realize the automatic generation of measurand detecting information and extraction automatically, avoid detecting information is manually described the process of structure, and then realize the automatic generation of test case (test code).
For this reason, the technical scheme of technical solution problem of the present invention is: a kind of method of testing of software product robustness is provided, comprises step:
1) resolves the measurand application programming interfaces, from described measurand application programming interfaces, extract first detecting information;
2) first detecting information of the described extraction of demonstration receives second detecting information that the user imports;
3) described first detecting information and described second detecting information are carried out the information pre-service, generate the interface function of detecting information list item and described detecting information list item;
4) detecting information with the complicated type data of storing in the described detecting information list item generates constructed fuction;
5) first detecting information stored in the described detecting information list item and second detecting information and described constructed fuction are carried out the value structure, generate test case;
6) described test case is compiled, is connected with measurand, generate the test target code;
7) the described test target code of operation.
Preferably, resolve the measurand application programming interfaces in the described step 1), from described measurand application programming interfaces, extract first detecting information specifically: the measurand application programming interfaces are compiled and grammatical analysis by the syntax parsing device; The grammatical analysis result who obtains is converted to first detecting information; Described first detecting information comprises the layer of structure relation in member territory in tested function name, number of parameters, parameter type, the parameter.
The data type storehouse provided second test data of basic universal test data as user's input when preferably, the test-types in second detecting information of described user's input was for the acquiescence mode.
Preferably, described step 2), on the graphical user interface interface, show first detecting information of described extraction, and on described graphical user interface interface, receive second detecting information of user's input; Described second detecting information comprises whether testing, test-types, test mode and test data.
Preferably, in the described step 4), the data type storehouse generates identification and the value that constructed fuction provides the fundamental type test data for the detecting information of described complicated type data.
Preferably, described step) the described test target code of described operation comprises:
71) finish the initialization of test macro and the externally registration of test control interface, the establishment of test control task, test execution task starts;
72) by pre-provisioning request registered test case is called the behavior of control and measuring process, the control information of user's input in data that produce in the record test process and the test process.
Preferably, the described test target code of the described operation of described step 7) also comprises: the detecting information of storing in the described detecting information list item of dynamic queries.
The present invention also provides a kind of proving installation of software product robustness, comprising:
Detecting information is resolved extraction unit, is used to resolve the measurand application programming interfaces, extracts first detecting information from described measurand application programming interfaces;
The user test information input unit is used to show the detecting information of described extraction, receives second detecting information of user's input;
The test case generation unit comprises detecting information pretreatment unit, labyrinth generation unit and test code generation unit;
The detecting information pretreatment unit is used for described first detecting information and described second detecting information are carried out the information pre-service, generates the interface function of detecting information list item and described detecting information list item;
The labyrinth generation unit, the detecting information that is used for complicated type data that described detecting information list item is stored generates constructed fuction;
The test code generation unit is used for first detecting information that described detecting information list item is stored and second detecting information and described constructed fuction and carries out the value structure, generates test case;
The object code performance element is used for described test case is compiled, is connected with measurand, generates the test target code, and moves described test target code;
The data type storehouse is used to store basic universal test data, for the generation and the operation of test case provides basic universal test data and the information service of fundamental type data identification.
Preferably, described data type storehouse comprises universal class storehouse, expansion type storehouse and private type storehouse.
Preferably, described object code performance element comprises:
Unit of testing and controlling is used to finish the registration of the external test control interface of the initialization of tested software system and described tested software system, and the establishment of test control task, test execution task starts;
The test execution unit is used for by pre-provisioning request registered test case being called, the behavior of control and measuring process;
The test record unit is used for writing down the data that produce in the control information of test process user input and the test process.
With respect to prior art, the invention has the beneficial effects as follows: at first, because the present invention analyzes the parameter information that obtains tested api interface automatically, parameter decomposition such as labyrinth, pointer are arrived base class, need not describe by the type template, the defined parameters type, thoroughly solved the process that detecting information is manually described structure.And this technical scheme does not rely on tested object language form, and is simply clear and definite to the requirement of tested object, and it realizes principle advanced person, reasonable, and stronger versatility is arranged.Secondly, press detecting information, test data fully, automatically generate test case (test code) and data, do not need to intervene artificially.This technical scheme realizes the automatic conversion of detecting information to test code, do not need manually to participate in the generation of code, make artificial design code load reduce to zero, user's change Test Design of can following one's bent, and needn't worry the Code Design workload for this reason born out, greatly boost productivity.Once more, owing to need not detecting information is manually described structure, thereby the change degree certainly of raising instrument reduces the manually-operated difficulty, has improved work efficiency.
In addition, in preferred version of the present invention, because detecting information is gone up very clear showing at GUI (graphic user interface), the user can be like a cork according to interface parameters information, layering on demand, branch territory are provided with, select, and have avoided describing interface parameters and member's sampling process by the operation data type template.No matter this instrument does not have too many requirement from operability, availability, applicability, as long as operating personnel are had basic WINDOWS graphical tool operating experience and even can the operation of basic programming knowledge personnel.
Increased test option, type, the input of value information such as (scopes), the use-case (test code) that makes generation satisfies actual test needs to test data and more dirigibility of test call.(test code) output from the instrument start-up to the test case, omnidistance user can finish as long as by the instrument requirement tested object, typing detecting information are set.The personnel that need not under default situations intervene the generation that can realize use-case.The user that is embodied as of this technical scheme provides flexile test setting function, thereby rich and varied positioning problems means, method also is provided indirectly, thereby provides good approach for the solution as early as possible of test problem.
Can preserve, call, revise detecting information, data, improve the maintainability of detecting information, data, ward off and exempted to repeat to generate, the inconvenience that brings is set, reduce maintenance in the future.Management to test case no longer need be carried out at code or written literal, and only the information that needs the preservation user to be provided with on view gets final product.Thereby change over management to substantive test code or file, reuse historical information and become free and relaxed, and needn't worry again, and need design is understood or change, spend workload again because personnel change or long-time the placement.This technical scheme can realize abilities such as use-case test dispatching, problem reproduction flexibly, reaches good process record, information printing mechanism, guarantees the test case of each execution, can finish test by user's intention.
This technical scheme can further expand and realize other multiple test function as required, as unit testing, the test of message bag or the like, has big application space; This technical scheme has realized use-case design new model, makes the written use-case design of loaded down with trivial details complexity, changes into the view form, and means such as the user selects, makes up, input make design, change, increase more easy, have improved design efficiency.Design concept of the present invention is reasonable, flow process is clear, the job stability height; Energy produces test case (test code) according to the information that the user designs within a short period of time, has stronger response speed.
Description of drawings
Fig. 1 is a kind of synoptic diagram of software robustness method of testing of prior art;
Fig. 2 is the synoptic diagram of the software robustness method of testing of another kind of prior art;
Fig. 3 is a system schematic of implementing robustness testing method of the present invention;
Fig. 4 is the process flow diagram of software product robustness method of testing of the present invention;
Fig. 5 is the detecting information synoptic diagram that extracts among the present invention;
Fig. 6 is the synoptic diagram that receives user's detecting information input among the present invention;
Fig. 7 is the data type library facility structural representation that the present invention adopts;
Fig. 8 is the process flow diagram that generates test case among the present invention;
Fig. 9 is the synoptic diagram that generates test code in the embodiment of the invention;
Figure 10 is the block diagram of software product robustness proving installation of the present invention;
Figure 11 is the block diagram of object code operating part among the present invention.
Embodiment
See also Fig. 3, for implementing the system schematic of robustness testing method of the present invention.The inventive method comprises automatic generating portion of test case and test target code operating part.
At first, on computing machine 910, generate test case automatically, promptly finish the automatic generation of test case (test code) automatically; Pass through means such as compiling, connection again, generate with measurand and can carry out the target test code; Subsequently, by computing machine 910 this can be carried out the target test code and be loaded on the measurand operation hardware platform 930, and start this and can carry out the operation of target test code by connecting lines such as netting twine, Serial Port Line 920.The target test code is in service by connecting lines such as netting twine, Serial Port Line 920 and computing machine 910 exchange messages, comprise: monitor message, operation and control information, test data dynamic queries, control etc., 910 in computing machine provides information analysis, control, data, services etc.
Among the present invention, described measurand operation hardware platform 930 includes but not limited to the hardware platform for running software such as computer, veneer.Described test data is meant the detecting information tables of data that generates in the method implementation process, can be stored in as required on the computing machine 910 or after the test target code compiles connection to be loaded on the hardware platform 930.
Seeing also Fig. 4, is the process flow diagram of software product robustness method of testing of the present invention.
Step S1, measurand API detecting information resolve and extract.
For robustness testing, when generating test case (test code) automatically, need know how to construct measurand and test environment.As the important source of robustness testing instrument detecting information, this step is analyzed measurand exactly, therefrom extracts relevant information, as: the layer of structure relation in tested function name, number of parameters, parameter type, the interior member territory of parameter etc.
The detailed process of this step is: input includes the measurand file of aforementioned detecting information, is that example is exactly * .h, * .obj etc. with the C language.By the syntax parsing device, this class file is compiled, obtain the grammatical analysis result, and then therefrom extract the information of each tested API, and convert to and can be kept in internal memory with certain frame mode by the data message of computer Recognition, use for subsequent step.
Wherein, the effect of syntax parsing device is exactly that grammer detects, and tested file is carried out grammatical analysis, decomposes the purpose of complex information to elementary cell (basic data type) thereby reach, so that extract these detecting informations.The syntax parsing device can be a module in the code compile instrument (GCC), generative grammar inforamtion tree after the validity checking of process grammer, pre-service.Grammatical analysis is exactly to extract the information that the present invention needs from the syntactic information tree that generates, as various data types and hierarchical relationship etc.
Be that example describes with the C language below, suppose that measurand is the testfile.h file, the text formatting of measurand api function TestApi_n (...) in testfile.h is described below:
……
struct?b{
struct?c{
int?c1；
int?c2；
}
char?*d；
}
……
TestApi_n(int?a，struct?b?m)；
……
Testfile.h is carried out grammatical analysis as input object, obtain TestApi_n information after finishing information extraction.As shown in Figure 5, the information of square frame content representation function, parameter, nodes domains concerns under reaching before and after dotted line and arrow are represented, thereby finishes decomposition and extraction to tested api function TestApi_n (int a, struct b m) essential information.
Step S2, the input of user test information.
As another source of detecting information, this step is finished the detecting information input of user to each territory.That is to say, on the basis that measurand API detecting information is extracted, by GUI (graphical user interface, graphics user interface) interface display, whether the detecting information in each territory of measurand of user input is carried out record, and detecting information comprises: test, test-types, test mode, test data.By the detecting information setting of user, finish the input of user test information to the measurand territory.
As shown in Figure 6, for a certain function, parameter or node member territory, the user operates by gui interface, and this territory Member Users's detecting information is set, and by the interface realize to node Member Users detecting information preservation, inquire about, call.Empty arrow is represented the front and back in this territory and other member territory, is concerned under up and down.
See also Fig. 7,, need to adopt the data type storehouse in order to finish purpose of the present invention.
Template Information in the related data typelib is safeguarded in the data type storehouse, and externally provide relevant information and interfaces such as fundamental type identification, test data, if the user is necessary the data in a certain typelib are replenished, can replenish test data and information according to template description document that provides or the correlation type template of opening original definition description document.
The data type storehouse is succession and the development to the template description technique of prior art, different is that its status in scheme changes to some extent with effect, because the present invention introduces the tested object parameter information is resolved, so template is described the complex data type value mode and is only used in special circumstances.
The data type storehouse is used to provide base class detecting information (fundamental type identification, value), particularly when the user selects the test-types in a certain territory to be the acquiescence mode, it provides test data for the final goal run time version: because to some basic data types (as char int etc.) when generating test case automatically, discern, finish the identification of these information by the data type storehouse; Data type stock is being put some basic universal test data, if the user selects for use is exactly these data wherein, will provide interface function so, so that test case can be obtained these test datas when carrying out.
In addition,, can be classified in the data type storehouse,, be respectively applied for and deposit not same-action, other template description document of level as universal class storehouse, expansion type storehouse, private type storehouse etc. for ease of the management of template file.
The universal class storehouse: the user can not at will be provided by the data type value that provides, and can only be operated by the professional, is applicable to universal test.
What data type stock was put is some independently data type files, and what deposit in these type files is the various value construction process of the type that the form with code provides.As file: Rchar.TPL/Rint.TPL etc., wherein describe the general value structure of the type and analyse the structure process.
As: char construct_char (int i)
{
……
switct(i)
case?1：
return?0；
case2：
return?255；
……
}
The expansion type storehouse: the user can be as required voluntarily the data of additional demand in relevant data type, as replenishing of universal test data.
Private type storehouse: can generate special value and environment description code according to the description of user for specific data type value mode, the detecting information deficiency of on gui interface, importing with replenish user.
That is to say that the effect in data type storehouse is the information that provides according to template, for support (test data is provided) and the effect of serving (the fundamental type identifying information is provided) are played in generation, the operation of test data, test code (test case).Can pass through some interface functions, as query type information, the value of fundamental type waits and realizes.
Step S3 generates test code, test data.
It is test data, test code that this step is used for converting tested API detecting information to test case, unites after compiling generates executable file with measured target, is loaded on the object run equipment, and measurand is implemented test.
Because the diversity of combinations such as the randomness that the user is provided with each territory of measurand, data, type, option, it is in service how these information to be embodied in test code timely and accurately, finishes the Test Design of user expectation, is the key of this step.Be better understanding, with the test case separated into two parts, promptly test data and test code are described.Certainly, test data is not must be to separate or merge in realization with code, just is defined as two for convenience of description here, so that the present invention is had better expression and understanding.
See also Fig. 8, this step can be divided into three sub-steps, is respectively: detecting information pre-service, labyrinth generate, test code generates.
Step S31 carries out the detecting information pre-service.
Integrating step S1 forms a complete detecting information data structure to the extraction description of tested object information and the input of step S2 user test information, is used for the generation of follow-up test use-case.The such information table structure of each part all can independently realize information preservation, call, handle again, thereby improve the maintainability of follow-up use.
Detailed process is: after further classification of detecting information (information of the API information of extraction, user's input), arrangement, processing, information translation is become certain data table items, interface output, use for the test case (test code) of next substep or generation.
Described certain data table items comprises detecting information conversion table and list of values etc., and the interface function of interface input is finished the visit of user to detecting information table and list of values.
What deposit in the detecting information tables of data that generates is the detecting information of user to each parameter field or member territory, as: data type, value mode, test-types, data etc.Interface function externally is provided, can comes information, the value in index different parameters or member territory, for the execution of test case provides the multidate information service by the calling interface function.With respect to prior art, the value of the test case that the present invention realizes is unknowable before not carrying out, and when having only test case to run to, could call the list item interface function according to the index of test case, obtain member's value information, so these data construct dynamically.
Step S32 realizes that labyrinth generates.
By abovementioned steps as can be known, for non-fundamental type data class, it is decomposed base class by certain hierarchical relationship by syntax parsing.Input to non-fundamental type data class user test information, be based on selection and the input of forming the type data class fundamental type, therefore to the value of non-fundamental type data class, before test, necessarily need the fundamental type of the type data class will be formed, by hierarchical relationship and the information that parses, construct the value of non-fundamental type data class with basic data type constructed fuction or value, the result of this process is exactly the constructed fuction code that generates complicated type data class value.
What need further specify is, non-fundamental type such as labyrinth includes but not limited to structure, community, enumerates, pointer, function.
Step S33 generates test code.
This step generates the test package code that measurand is called.Specifically:, finish value structure to tested each parameter of function by the information of resolving; The information data table that the constructed fuction that these constructed fuctions or value provide from data type storehouse, labyrinth generative process, interface or preprocessing process produce provides data by interface function.
Seeing also Fig. 9, is example with the C language, and process and relation that test case (test code) produces are described.
To tested function API _ n (struct c a) carry out grammatical analysis and extract detecting information and receive the detecting information of user's input after, at first carry out the information pre-service, generate detecting information table and list of values etc., and interface function get_data () is provided, be used to finish to the inquiry of detecting information and data with obtain; Secondly, complicated type is handled, obtained detecting information and data, produce constructed fuction test_c_construct (), be used to finish the structure of labyrinth sampling process by list item interface function and data type bank interface function; Subsequently, test code calls constructed fuction and data type bank interface function, finishes the test environment structure, and then calls tested object, implements the test of calling of tested object.
Step S4, the execution of target test code.
The test code, the data that generate must compile, be connected generation and can carry out file destination with measurand, could finally realize test.Concrete is exactly test code, data and measurand one that the present invention generates to be reinstated compiling fastening means (as GCC) generation can carry out file destination (just common be similar to * .EXE file).
At first, finish test macro initialization (as, test control and test execution relevant entries, data, the establishment of control signal, initialization) and the externally registration of test control interface, the establishment of test control task, test execution task dispatching starts.
Subsequently, realize registered test case is called by customer requirements (as test case execution order, call number, method of calling etc.), and the behavior of control and measuring process, record test process data.
In addition, for guarantee that information, data can in time be exported or on request reduction reproduce test process, the present invention also provides the communal space and management, is used for the control information of user's input and test process information, data are carried out recorded and stored.
See also Figure 10, for realizing the software product robustness method of testing, the present invention also provides a kind of software product robustness proving installation.
Described software robustness test comprises automatic generating portion 100 of test case and object code operating part 200.The automatic generating portion 100 of described test case is mainly used in detecting information (comprising the API detecting information of automatic extraction and the detecting information of user's input) is converted to detecting information tables of data and test code, realize the conversion of detecting information, finish the test case design to code; Described object code operating part 200 is used for test code, measurand API compiled with the test record unit with unit of testing and controlling, test execution unit and is connected, generate executable file, and on hardware platform, move, come running status is monitored by external module, calling of realization test case tested and monitoring.
The automatic generating portion 100 of test case comprises detecting information parsing extraction unit 110, user test information input unit 120, data type storehouse 130 and test case generation unit 140.
Detecting information is resolved extraction unit 110 and is realized the parsing of measurand API detecting information is extracted.
Important source as robustness testing instrument detecting information, detecting information is resolved 110 pairs of measurands of extraction unit and is analyzed, therefrom extract relevant information, as: the layer of structure relation in tested function name, number of parameters, parameter type, the interior member territory of parameter etc.
The input object that detecting information is resolved extraction unit 110 is the measurand file that includes aforementioned detecting information.By the syntax parsing device, this class file is compiled, obtain the grammatical analysis result, and then therefrom extract the information of each tested API, and convert to and can be kept in internal memory with certain frame mode by the data message of computer Recognition, use for follow-up unit.
As another source of detecting information, user test information input unit 120 is finished the detecting information input of user to each territory.That is to say, on the basis that measurand API detecting information is extracted, by GUI (graphical user interface, graphics user interface) interface display, whether the detecting information in each territory of measurand of user input is carried out record, and detecting information comprises: test, test-types, test mode, test data.By the detecting information setting of user, finish the input of user test information to the measurand territory.
It is test data, test code that test case generation unit 140 is used for converting tested API detecting information to test case.
Test case generation unit 140 can be divided into three subelements, is respectively: detecting information pretreatment unit 141, labyrinth generation unit 142 and test code generation unit 143.
Detecting information pretreatment unit 141 generates the detecting information tables of data in conjunction with to the extraction description of tested object information and the input of user test information, dynamically obtains relevant information and data when the test case that is used to generate is carried out.
What deposit in the detecting information tables of data that generates is the detecting information of user to each parameter field or member territory, as: data type, value mode, test-types, data etc.Interface function externally is provided, can comes information, the value in index different parameters or member territory, for the execution of test case provides the multidate information service by the calling interface function.
Value to non-fundamental type data class, before test, necessarily need labyrinth generation unit 142 will form the fundamental type of the type data class, by hierarchical relationship and the information that parses, construct the value of non-fundamental type data class with basic data type constructed fuction or value, the result of this process is exactly the constructed fuction code that generates complicated type data class value.
Test code generation unit 143 generates the test package code that measurand is called.Specifically:, finish value structure to tested each parameter of function by the information of resolving; The information data table that the constructed fuction that these constructed fuctions or value provide from data type storehouse, labyrinth generative process, interface or preprocessing process produce provides data by interface function.
Above-mentioned three unit 141,142 and 143 division of labor produce test data, test code, because the complicated diversity of parameter information and the randomness of user test information setting, requirement to test value in test data and the test code is corresponding one by one, to guarantee each domain of function, each parameter field, the correctness of the corresponding value in each nodes domains and each member territory.Just because of this, though being branches, data, code produce at different units, but generate data, with code before, these unit have carried out unified processing according to the rule of interior customization, thereby guaranteed that data type is at construction process, can get corresponding data, guaranteed test data, information accurately.Described rule by interior customization has been carried out unified processing and has been meant that test case is to index test data and information corresponding in the test chart exactly in carrying out, the resolved detecting information that goes out is classified and unified numbering, so just can when dynamically constructing test data, get desired test data, information according to index.
With the C language is example, in the practical operation flow process of whole device, the user at first will finish the selection and the input (being * .h) of tested object on graphical interfaces, and configuration information is input in relevant the setting, starting grammatical analysis extracts, finish the extraction of tested object detecting information, these information can be presented on the terminal with graphics mode afterwards, check and be provided with detecting information for the user.After finishing aforesaid operations, can start the use-case systematic function, can call detecting information pretreatment unit 141, labyrinth generation unit 142, test code generation unit 143 this moment successively, finishes the conversion of final test information to test case (test code, data).
See also Figure 11, object code operating part 200 comprises unit of testing and controlling 210, test execution unit 220 and test record unit 230.Be responsible for the information such as startup, execution and real time record test command, data of target test code, so that control the monitoring test process in real time during test execution.
The test code, the data that generate must compile, be connected generation and can carry out file destination with measurand, could finally realize test.Therefore, test code, data and the measurand one that the present invention is generated reinstated compiling fastening means (as GCC) generation and can be carried out file destination (just common be similar to * .EXE file under the WIN32 or the executable file under other operating system).
Unit of testing and controlling 210 be used to finish test macro initialization (as, test control and test execution relevant entries, data, the establishment of control signal, initialization) and the externally registration of test control interface, the establishment of test control task, test execution task dispatching starts.By test control task and interface, realize the externally parsing of test control command, instruct the test execution task to realize the test of calling to test case realizing higher level control simultaneously on this basis, as functions such as the analysis of test problem, location, reproduction operations by user's requirement.
For realizing complexity, various test function, target operating part 200 is divided into three unit, finishes mutual function and coordination by interface, information table, message etc.; External interface provides the dynamic control operation and the order of test process, is convenient to the user and changes test process at any time in carrying out test process.Information is played in test record and test data writes down, caching function, as required detecting information, data is outputed to the outside at any time.As shown in figure 11, unit of testing and controlling 210 receives outside order need provide the external interface function, and these functions notify unit of testing and controlling 210 to carry out the function of wanting or order exactly; Information then is control module 210 and a memory block of test execution unit 220 interactive testing information, and the dynamic memory test case needs to preserve or mutual information in carrying out, as call number, etc.; Message then two unit is the mutual data packets transmitted of realization with external function.
It more than is the explanation that whole device use-case generates, use-case carries into execution a plan and each Elementary Function is realized, realization by the said units function, the target that the present invention has proposed when having solved design preferably, realize test case design, the flow process that generates, carries out effectively, guarantee in operation, use, safeguard and improve work efficiency, reduced difficulty.
In sum, the present invention is based on the robustness testing of assembly API and a kind of automated testing method and the device thereof of development and Design.This method and device have mainly been realized the automatic generation of measurand detecting information is extracted with automatic, and user interface is provided, and realize detecting information processing, perfect, and then generate MUT module under test robustness testing use-case (test code) automatically.By control, performance element, realize to test case control, call, finish the tested object robustness testing.Whole test case generates and carries out, and does not need the user that measurand is understood in depth, and input test object and detecting information can be finished said process automatically on request, thereby improve the test automation degree greatly, reduce labor workload and difficulty.
The above only is a preferred implementation of the present invention; should be pointed out that for those skilled in the art, under the prerequisite that does not break away from the principle of the invention; can also make some improvements and modifications, these improvements and modifications also should be considered as protection scope of the present invention.
Claims (10)
1. the method for testing of a software product robustness is characterized in that, comprises step:
1) resolves the measurand application programming interfaces, from described measurand application programming interfaces, extract first detecting information;
2) first detecting information of the described extraction of demonstration receives second detecting information that the user imports;
3) described first detecting information and described second detecting information are carried out the information pre-service, generate the interface function of detecting information list item and described detecting information list item;
4) detecting information with the complicated type data of storing in the described detecting information list item generates constructed fuction;
5) first detecting information stored in the described detecting information list item and second detecting information and described constructed fuction are carried out the value structure, generate test case;
6) described test case is compiled, is connected with measurand, generate the test target code;
7) the described test target code of operation.
2. the method for testing of software product robustness according to claim 1 is characterized in that, resolves the measurand application programming interfaces in the described step 1), extracts first detecting information specifically from described measurand application programming interfaces:
By the syntax parsing device measurand application programming interfaces are compiled and grammatical analysis; The grammatical analysis result who obtains is converted to first detecting information; Described first detecting information comprises the layer of structure relation in member territory in tested function name, number of parameters, parameter type, the parameter.
3. the method for testing of software product robustness according to claim 2, it is characterized in that, when the test-types in second detecting information of described user's input was the acquiescence mode, the data type storehouse provided second test data of basic universal test data as user's input.
4. the method for testing of software product robustness according to claim 1, it is characterized in that, described step 2), on the graphical user interface interface, shows first detecting information of described extraction, and on described graphical user interface interface, receive second detecting information of user's input; Described second detecting information comprises whether testing, test-types, test mode and test data.
5. according to the method for testing of claim 3 or 4 described software product robustness, it is characterized in that in the described step 4), the data type storehouse generates identification and the value that constructed fuction provides the fundamental type test data for the detecting information of described complicated type data.
6. the method for testing of software product robustness according to claim 5 is characterized in that, the described test target code of the described operation of described step 7) comprises:
71) finish the initialization of test macro and the externally registration of test control interface, the establishment of test control task, test execution task starts;
72) by pre-provisioning request registered test case is called the behavior of control and measuring process, the control information of user's input in data that produce in the record test process and the test process.
7. the method for testing of software product robustness according to claim 6 is characterized in that, the described test target code of the described operation of described step 7) also comprises: the detecting information of storing in the described detecting information list item of dynamic queries.
8. the proving installation of a software product robustness is characterized in that, comprising:
Detecting information is resolved extraction unit, is used to resolve the measurand application programming interfaces, extracts first detecting information from described measurand application programming interfaces;
The user test information input unit is used to show the detecting information of described extraction, receives second detecting information of user's input;
The test case generation unit comprises detecting information pretreatment unit, labyrinth generation unit and test code generation unit;
The detecting information pretreatment unit is used for described first detecting information and described second detecting information are carried out the information pre-service, generates the interface function of detecting information list item and described detecting information list item;
The labyrinth generation unit, the detecting information that is used for complicated type data that described detecting information list item is stored generates constructed fuction;
The test code generation unit is used for first detecting information that described detecting information list item is stored and second detecting information and described constructed fuction and carries out the value structure, generates test case;
The object code performance element is used for described test case is compiled, is connected with measurand, generates the test target code, and moves described test target code;
The data type storehouse is used to store basic universal test data, for the generation and the operation of test case provides basic universal test data and the information service of fundamental type data identification.
9. the proving installation of software product robustness according to claim 8 is characterized in that, described data type storehouse comprises universal class storehouse, expansion type storehouse and private type storehouse.
10. according to Claim 8 or the proving installation of 9 described software product robustness, it is characterized in that described object code performance element comprises:
Unit of testing and controlling is used to finish the registration of the external test control interface of the initialization of tested software system and described tested software system, and the establishment of test control task, test execution task starts;
The test execution unit is used for by pre-provisioning request registered test case being called, the behavior of control and measuring process;
The test record unit is used for writing down the data that produce in the control information of test process user input and the test process.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB2004100886733A CN100351811C (en) | 2004-11-15 | 2004-11-15 | Method and device for testing software product robustness |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB2004100886733A CN100351811C (en) | 2004-11-15 | 2004-11-15 | Method and device for testing software product robustness |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN1776643A CN1776643A (en) | 2006-05-24 |
| CN100351811C true CN100351811C (en) | 2007-11-28 |
Family
ID=36766160
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNB2004100886733A Expired - Fee Related CN100351811C (en) | 2004-11-15 | 2004-11-15 | Method and device for testing software product robustness |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN100351811C (en) |
Families Citing this family (27)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101571798B (en) * | 2008-04-29 | 2013-01-09 | 环达电脑(上海)有限公司 | Method for generating and using automatic test script |
| CN101593145B (en) * | 2008-05-29 | 2012-08-08 | 福建升腾资讯有限公司 | Designing method of high-efficiency high-coverage-rate function test case |
| CN101876939B (en) * | 2009-11-20 | 2011-12-21 | 北京信息科技大学 | System test method and device |
| CN101859276B (en) * | 2010-05-14 | 2012-02-22 | 东南大学 | Regression testing case selection method based on hierarchical slicing |
| CN101894058B (en) * | 2010-05-31 | 2011-11-16 | 北京航空航天大学 | Method and device for analyzing test coverage automatically aiming at automatic test system |
| CN101866317B (en) * | 2010-06-29 | 2012-07-25 | 南京大学 | Regression test case selection method based on cluster analysis |
| CN102300118B (en) * | 2011-09-06 | 2015-03-25 | 博威科技(深圳)有限公司 | Testing system and testing method for monitoring system |
| CN102629227B (en) * | 2012-02-28 | 2015-07-22 | 中科开元信息技术(北京)有限公司 | Method for assisting function verification of interface of embedded system |
| CN103810171B (en) * | 2012-11-06 | 2017-02-08 | 深圳市金蝶天燕中间件股份有限公司 | Method and system for generating random test data within limited range |
| CN104731698A (en) * | 2013-12-20 | 2015-06-24 | 深圳中兴力维技术有限公司 | Abnormal test case automatic generation processing method and device based on automatic tests |
| CN103777949A (en) * | 2014-01-17 | 2014-05-07 | 浪潮电子信息产业股份有限公司 | Design method of automated testing script |
| JP6316120B2 (en) * | 2014-06-30 | 2018-04-25 | 日立オートモティブシステムズ株式会社 | Test case generation system and recording medium recording test case |
| CN104035873A (en) * | 2014-06-30 | 2014-09-10 | 青岛海信电器股份有限公司 | Method and device for generating testing codes |
| CN105446870B (en) * | 2014-08-25 | 2019-04-05 | 北京畅游天下网络技术有限公司 | The method and apparatus that automatic test is carried out to app application |
| CN105630656B (en) * | 2014-11-06 | 2019-01-18 | 阿里巴巴集团控股有限公司 | System robustness analysis method and device based on log model |
| CN104407976B (en) * | 2014-11-28 | 2018-08-07 | 联动优势电子商务有限公司 | A kind of interface robustness testing case generation method and device |
| CN105808420B (en) * | 2014-12-31 | 2018-12-28 | 阿里巴巴集团控股有限公司 | The implementation method and device of robustness testing process |
| CN107608873B (en) * | 2016-07-11 | 2021-05-25 | 珠海优特电力科技股份有限公司 | Software testing method and system and testing platform |
| TWI637275B (en) | 2017-01-10 | 2018-10-01 | 廣達電腦股份有限公司 | Cloud service hosting system and methods for handling abnormal states of cloud services |
| CN106874202A (en) * | 2017-02-14 | 2017-06-20 | 网易无尾熊（杭州）科技有限公司 | Method, device and readable storage medium storing program for executing for unit testing |
| CN108228965B (en) * | 2017-12-06 | 2022-01-11 | 北京物芯科技有限责任公司 | Simulation verification method, device and equipment for memory cell |
| CN109614329B (en) * | 2018-12-06 | 2021-11-02 | 中国航空工业集团公司洛阳电光设备研究所 | Software test case aided design method based on interface control file |
| CN109933521A (en) * | 2019-01-23 | 2019-06-25 | 平安科技（深圳）有限公司 | Automated testing method, device, computer equipment and storage medium based on BDD |
| CN110865892A (en) * | 2019-11-18 | 2020-03-06 | 北京锐安科技有限公司 | Detection method and device for call interface, storage medium and electronic equipment |
| CN113806211A (en) * | 2020-06-17 | 2021-12-17 | 北京同邦卓益科技有限公司 | Data processing method and device for interface test |
| CN111857692A (en) * | 2020-07-30 | 2020-10-30 | 中国民航信息网络股份有限公司 | Code generation method and device and electronic equipment |
| CN116561014B (en) * | 2023-07-07 | 2023-09-29 | 国电南瑞科技股份有限公司 | Device and method for generating test case of electric power secondary equipment |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1286432A (en) * | 1999-11-27 | 2001-03-07 | 深圳市中兴通讯股份有限公司 | Automatic example generation method for analyzing state transition diagram and processing and testing navigation loop |
| US6269150B1 (en) * | 1998-06-11 | 2001-07-31 | Lucent Technologies, Inc. | Reliable, unattended, automated testing system and method for complex telecommunication systems |
| US6301701B1 (en) * | 1999-11-10 | 2001-10-09 | Tenfold Corporation | Method for computer-assisted testing of software application components |
| DE10137297A1 (en) * | 2001-08-01 | 2003-02-20 | Abb Patent Gmbh | Method for automated testing of software or software component robustness in which the number of test cases to be run is reduced by not running test cases containing known critical test values |
-
2004
- 2004-11-15 CN CNB2004100886733A patent/CN100351811C/en not_active Expired - Fee Related
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6269150B1 (en) * | 1998-06-11 | 2001-07-31 | Lucent Technologies, Inc. | Reliable, unattended, automated testing system and method for complex telecommunication systems |
| US6301701B1 (en) * | 1999-11-10 | 2001-10-09 | Tenfold Corporation | Method for computer-assisted testing of software application components |
| CN1286432A (en) * | 1999-11-27 | 2001-03-07 | 深圳市中兴通讯股份有限公司 | Automatic example generation method for analyzing state transition diagram and processing and testing navigation loop |
| DE10137297A1 (en) * | 2001-08-01 | 2003-02-20 | Abb Patent Gmbh | Method for automated testing of software or software component robustness in which the number of test cases to be run is reduced by not running test cases containing known critical test values |
Non-Patent Citations (2)
| Title |
|---|
| 内核同步对象异常处理健壮性测试 段川，蒋凡.计算机工程，第30卷第8期 2004 * |
| 软件测试的自动化 孙惠杰，杨晓红.哈尔滨师范大学自然科学学报，第19卷第5期 2003 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN1776643A (en) | 2006-05-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN100351811C (en) | Method and device for testing software product robustness | |
| CN107807878B (en) | Keyword-based universal test resource driving and execution management method | |
| CN1928816A (en) | Model drive for embedded system software and component development method | |
| US9032361B2 (en) | Agile unit and regression testing framework for domain specific languages | |
| Paige et al. | The design of a conceptual framework and technical infrastructure for model management language engineering | |
| US9098633B2 (en) | Application testing | |
| CN1513145A (en) | System and method of mapping between software objects and structure languige element-based documents | |
| CN109254905B (en) | Distributed parallel automatic test system based on workflow | |
| EP3296866B1 (en) | Method implemented by a computer that presents software-type applications based on design specifications | |
| CN1728094A (en) | Smart gui activity recording and playback framework | |
| CN101311903A (en) | Method and device for reusing component of software system based on component | |
| Larsson | Predicting quality attributes in component-based software systems | |
| CN1949187A (en) | Program testing system and method | |
| CN110928783A (en) | RobotFramework-based automatic test datamation reconstruction platform | |
| CN103116513B (en) | A kind of heterogeneous multi-nucleus processor compiler | |
| CN1786925A (en) | TTCN-3 testing system basedon C++ mapping and its testing method | |
| CN1908895A (en) | System and method for application program globalization problem verification and computer products | |
| Martinez et al. | Recovering sequence diagrams from object-oriented code: An ADM approach | |
| CN1836271A (en) | Speech input interface for dialog systems | |
| Selic | Tutorial: an overview of UML 2 | |
| WO2009116748A2 (en) | Reserved component container-based software development method and apparatus | |
| CN1835450A (en) | Mesh operation supervisor | |
| CN1900905A (en) | Realizing method and its device for detecting descriptive language | |
| CN1746694A (en) | Automatic tester and test thereof | |
| JP2007115155A (en) | Program structure management device and program structure management program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20071128 Termination date: 20161115 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |