CN104407977A - Model detection based automatic joint debugging test method for staged task system - Google Patents
Model detection based automatic joint debugging test method for staged task system Download PDFInfo
- Publication number
- CN104407977A CN104407977A CN201410742936.1A CN201410742936A CN104407977A CN 104407977 A CN104407977 A CN 104407977A CN 201410742936 A CN201410742936 A CN 201410742936A CN 104407977 A CN104407977 A CN 104407977A
- Authority
- CN
- China
- Prior art keywords
- test
- state
- model
- sut
- window
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Abstract
The invention discloses a model detection based automatic joint debugging test method for a staged task system. The model detection based automatic joint debugging test method is completed through matching of an automatic testing system and a testing frame and comprises the steps of dividing stages for a tested system, determining behavioral agents, elements and state sets of the behavioral agents of the tested system at all stages; abstracting a window tree model and a state graph model of the tested system; automatically generating testing cases to form a testing case set according to the window tree model in a batch mode; conducting executable operation on the testing cases through the testing frame, feeding the testing cases into the tested system for execution and catching and sending state change information to the automatic testing system; resolving messages through the automatic testing system to obtain state transfer sequences and utilizing the state graph model to obtain a detection result through comparison. By means of the model detection based automatic joint debugging test method, the problems of incomplete coverage and low efficiency caused by manual editing testing cases are effectively solved, a modular design thought is adopted for design of the automatic testing system and an environment, and the method has good expandability.
Description
Technical field
The present invention relates to the automated testing method based on model inspection, specifically the robotization uniting and adjustment method of testing of a kind of task system stage by stage based on model inspection (Phased-Mission System, referred to as PMS).
Background technology
Along with informatization carrying out in a deep going way in every field, software systems become and become increasingly complex.Task system has Time Continuous and the basic task stage mutually do not covered forms by a series of stage by stage, and there is different mission reliability requirements in each basic task stage, is a kind of typical complex task system.
Task system is widely used in the safety-critical such as disaster assistance, Aeronautics and Astronautics field, emergency disposal is current important research content both at home and abroad, comprise emergency response and emergency management and rescue, can be applicable to the emergency disposal field of the various Strong time constraints such as macroseism rescue, tsunami rescue, anti-terrorism disposal, and the promptness realizing emergency disposal needs sending in time of all kinds of statistics, to realize decision support, implement efficient disposal activity.Such as, along with the frequent generation of earthquake disaster in recent years, as Wenchuan earthquake, Yaan earthquake etc.Earthquake rescue emergency disposal has become the problem that people are extremely concerned about, the application handling problems after research calamity not only has very important theory significance, and has extremely important practical significance, and this field belongs to the task system problem based on space-time data.For another example, how all kinds of attacks of terrorism of some area generations in recent years, carry out the research contents that emergency disposal also belongs to this project timely and effectively.Especially for the operation of empty sky task system, its credibility is " safety concerns (safety-crucial) " especially.For the Trusting eBusiness of this type of stage fill, model inspection (Model Checking) is a kind of effective means.
Summary of the invention
The technical problem to be solved in the present invention is the demand for the uniting and adjustment test of task system stage by stage, a kind of robotization uniting and adjustment method of testing of the task system stage by stage based on model inspection is proposed, by carrying out abstract modeling to the window tree-model of system under test (SUT) and state graph model, building automatization test system and environment, complete the uniting and adjustment test job to task system stage by stage, current the method can be applied to the so large-scale task system stage by stage of Emergency Response to Earthquake system practically.
The robotization uniting and adjustment method of testing of the task system stage by stage based on model inspection provided by the invention, build automatization test system and environment and carry out uniting and adjustment test to task system stage by stage, whole test process has been coordinated by automatization test system and test frame.
Part I is automatization test system, and function comprises:
(1) window tree modeling is carried out and automatic generating test use case to system under test (SUT);
(2) according to system running state transfer and theory state model, the ruuning situation of system is verified;
(3) according to detecting percent of pass evaluation algorithms, system under test (SUT) is assessed.
Part II is test frame, and function comprises:
(1) system under test (SUT) testing results use-case is allowed;
(2) the state transinformation in capture systems;
(3) communication work between system under test (SUT) and test macro is responsible for.
The robotization uniting and adjustment method of testing of the task system stage by stage based on model inspection provided by the invention, comprises the steps:
Step one: divide the system under test (SUT) stage, defines the behavioral agent treating each stage system under test (SUT) that uniting and adjustment are tested and the element comprised, determines the state set of element and the state set of behavioral agent.In conjunction with the state set of the exhaustive behavioral agent element of concrete system in this step, finally determine the state set of behavioral agent.
Step 2: according to window tree-model and state graph model definition, the window tree-model of abstract system under test (SUT) and state graph model; Window tree-model interior joint is window, and the limit of connected node is event, realizes situation carry out the modeling of window tree to system under test (SUT); Carry out constitutional diagram modeling to task system design conditions, described state graph model is for describing the state transition information of system under test (SUT);
Step 3: after window tree-model structure, automatic generating test case module is according to the information of window tree-model, automatic batch generating test use case, and the test case of generation is put into test case concentrate, after use-case to be tested all generates, test use cases is sent to test frame;
Step 4: after test frame receives test use cases, carries out execution operating to each test case wherein; Described performed change operation refer to for each API in test case select to match type supplemental characteristic and substitute into, API is performed in system under test (SUT);
Step 5: each test case test case concentrated, is brought in system under test (SUT) successively and performs, catches system under test (SUT) state change information during this period;
Step 6: according to the data layout of agreement between system under test (SUT) and test frame, is packaged into message bag by the system under test (SUT) state change information obtained in step 5, is sent in automatization test system does further process by network;
Step 7: after automatization test system receives message bag, according to protocol analysis message bag, extracts system under test (SUT) state change information wherein and leaves in state metastasis sequence according to order.This step cycle is carried out, until after receiving the message that a test case that test frame sends is finished, state metastasis sequence and state graph model are together sent into model checking module process, empty current state metastasis sequence simultaneously, wait for next message bag.
Step 8: model checking module compares the practical operation situation of system under test (SUT) and theoretical ruuning situation.Specific practice is: each state in state metastasis sequence be brought in state graph model successively and detect, and checks in state graph model whether there is the path that meets this state metastasis sequence.After state metastasis sequence all substitutes into, testing result is brought in percent of pass judgement schematics, final output uniting and adjustment test percent of pass assessment report.
The present invention proposes a kind of robotization uniting and adjustment method of testing of the task system stage by stage based on model inspection, its advantage is:
(1) testing efficiency is high: when a large amount of test cases, tester need not hand-coding test case, only need to set up window tree-model to system under test (SUT), being imported in system as input by model just can generating test use case automatically, decreases the work load of tester;
(2) accuracy rate is tested high: automatically working can be avoided in craft or manual test, because the omission that causes of human error, improving the accuracy rate in test process;
(3) visual result: the model checking module that the present invention relates to can be tested to the practical operation situation of system under test (SUT) and assess, the final uniting and adjustment test percent of pass assessment report visual result exported;
(4) extensibility is strong: automatization test system have employed modular design when designing, and the demand that any module can use according to reality makes corresponding adjustment, flexibly and easily.
Accompanying drawing explanation
The automatization test system that Fig. 1 adopts for the inventive method and environmental structure figure;
Fig. 2 is the robotization uniting and adjustment method of testing overall flow figure of the task system stage by stage based on model inspection of the present invention;
In Fig. 3, the window tree-model figure that (A) is macroseism stage of development, the window tree-model figure that (B) is the emergency response stage;
Fig. 4 is the flow chart of steps of automatic generating test case in step 3 of the present invention;
Fig. 5 is that a concrete API can execution procedure chart;
Fig. 6 is the flow chart of steps of model inspection.
Fig. 7 is uniting and adjustment test percent of pass assessment report schematic diagram in the embodiment of the present invention.
Embodiment
Below in conjunction with drawings and Examples, the present invention is described in further detail.
The present invention proposes a kind of robotization uniting and adjustment method of testing of the task system stage by stage based on model inspection, be divided into two parts on the whole.
Part I is automatization test system, and function comprises:
(1) window tree modeling is carried out and automatic generating test use case to system under test (SUT);
(2) according to system running state transfer and theory state model, the ruuning situation of system is verified;
(3) according to detecting percent of pass formula, system under test (SUT) is assessed.
Part II is test frame, and function comprises:
(1) system under test (SUT) testing results use-case is allowed;
(2) the state transinformation in capture systems;
(3) communication work between system under test (SUT) and test macro is responsible for.
As shown in Figure 1, the joint debugging method of the task system stage by stage implementation process that the present invention is based on model inspection completes jointly by based on the automatization test system of model inspection and test frame two parts, wherein input content is window tree-model file and state graph model file, exports as uniting and adjustment test percent of pass assessment report.Automatization test system based on model inspection includes automatic generating test case module, model checking module and percent of pass evaluation module, wherein automatic generating test case module is used for according to window tree-model automatic batch generating test use case to form test use cases, model checking module is used for comparing the practical operation situation of system under test (SUT) and theoretical ruuning situation, percent of pass evaluation module, according to percent of pass judgement schematics, calculates testing result and exports uniting and adjustment test percent of pass assessment report.Test frame then contains test case executable module, message managing module and system under test (SUT), wherein test case executable module is used for carrying out execution operating to test case, and message managing module is for transmitting the packet between automatization test system and test frame.Automatization test system and test frame carry out communication by network according to the data layout of agreement.
Particularly, the robotization uniting and adjustment method of testing of the task system stage by stage based on model inspection that the present invention proposes, as shown in Figure 2, the method comprises five periods, eight steps.
Interval I: system is abstract.
The work that this period mainly completes carries out abstract to system under test (SUT), and target obtains to describe system under test (SUT) and realize and the window tree-model of theoretical case and state graph model, so that these information of computer recognizing the carrying out of robotization process.
Step 1: divide the system under test (SUT) stage, the element determined system under test (SUT) behavioral agent and comprise, and the state set of element, and determine the state set of behavioral agent.
System under test (SUT) i.e. task system, first divide the stage Phase of system under test (SUT), if be divided into N number of stage, the system under test (SUT) after division can with multiple stage { Phase
1, Phase
2..., Phase
nbe described.By dividing multiple stage, modeling can be carried out for system under test (SUT) stage by stage, and then merging on the whole, generating more complicated overall flow.
In the present embodiment, Emergency Response to Earthquake system is divided into the daily stage on duty, macroseism stage of development, system responses stage, emergency disposal stage have four-stage altogether.
Then definition treats that the behavioral agent of the stage fill that uniting and adjustment are tested is task (Mission), is behavioral agent modeling.In the present invention, behavioral agent comprises two elements, i.e. Mission=(Flow, Report), and wherein, Mission is task, and Flow is flow process, and Report is report.Determine the state of the element of behavioral agent simultaneously, behavioral agent state is combined by elementary state to be determined, the present invention refers to the state Mission_State=(Flow_State of task, Report_State), wherein Mission_State is the state of task, Flow_State is the state of flow process, and Report_State is the state of report.
State set in conjunction with the exhaustive behavioral agent element of concrete system:
Flow_State＝{Flow_State
1,Flow_State
2,…,Flow_State
m}
Report_State＝{Report_State
1,Report_State
2,…,Report_State
n}
Wherein, m represents the state number of flow process, and n represents the state number of report.
By combination producing behavioral agent state set Mission_State_Set:
Here, behavioral agent state set Mission_State_Set is the subset of Flow_State and Report_State cartesian product, because the possibility of result in cartesian product exists redundancy or do not meet the kind of system design aspect.
In addition, state change in behavioral agent state set is due to API (Application Programming Interface, application programming interface) operation change, therefore carried out associating with API by element concentrated for subjective situation and can obtain orderly tlv triple, wherein the API set of system is API-Set:
(Mission_State
i, Mission_State
j, API
m) and Mission_State
i, Mission_State
j∈ Mission_State_Set ∪ API
m∈ API_Set
The set of these tlv triple composition is the description to Systems Theory transfer case, and this information is for building the state graph model of system under test (SUT).
In the present embodiment, by analyzing the Theoretical Design of system under test (SUT) and sum up, the flow state collection obtained and report condition collection as follows:
Flow state collection={ null, creates, and circulation is waited for, suspends, dies young, and terminates }
Report condition collection={ null, creates, instantiation, editor }
Therefore through combining the behavioral agent state set obtained, namely task status collection is as follows:
(null creates), (create, create), (circulation, instantiation), (circulation, editor), (waiting for, editor), (suspend, create),
(suspending, instantiation), (suspending, editor), (dying young, null), (terminating, null) }.
Step 2: according to the situation that realizes and the theoretical case of system under test (SUT), the window tree-model of abstract system under test (SUT) and state graph model.
The first step of system under test (SUT) being carried out to automatic test generates the test case for testing.The present invention proposes the concept of window tree-model for Mass production test case.Window tree-model is abstract to system actual functional capability, and it both can be described GUI (Graphical User Interface, graphical user interface), also can be described internal system simultaneously.Although internal system does not exist the such its visibility window of GUI, the behavior of internal system is but have similarity with GUI.Generally, the implementation of task system can be regarded as and is made up of a large amount of window, is generally through one or more event from a window to another window.And may cause calling API difference due to the difference of parameter, different branches will be produced, the final form forming similar one tree, be referred to as window tree.Node in window tree is window, and the limit of connected node is event.
Window tree is defined as four-tuple (W, w
0, E, R
w,E), wherein:
● W is the set of window, and window w ∈ W is a node;
● w
0∈ W is home window, is the root node of window tree, i.e. the home window of system under test (SUT);
● E is event intersection, the API set namely circulated between window and window;
● R
w,E∈ W × W × E, represents the circulation relation between window, if there is (w
i, w
j, e) ∈ R
w,Eand e ∈ E, then represent window node w
iwith window node w
jbetween there is set membership, and w
ifor father node, w
jfor child node, w
iwindow to circulate w by calling e
jwindow.
In the methods of the invention, window tree-model adopts XML to describe, and each node on tree is described by multiple field value, as follows:
Wherein, the field of concrete definition is:
● sons: child node title, can comprise multiple child node title;
● parent: father node title, represents the father node of this node;
● api: event title, refers to and can arrive this window by what runs, namely father node and this node fillet;
● states: state set, represents the state of this node, can store a more than state variable.
In the present embodiment, carry out dividing the four-stage obtained according to the step a pair system under test (SUT) stage and carry out the modeling of window tree respectively, obtain the window tree of four different phases, (A) and (B) wherein in Fig. 3 is respectively the window tree-model in macroseism stage of development and emergency response stage, fillet is depicted as run API.Next union operation is carried out to the window tree of four-stage, the window tree-model representing whole system can be obtained.
Abstract except to window tree-model, this step also needs to carry out abstract to constitutional diagram.The constitutional diagram of the inventive method definition features the normal behaviour of system under test (SUT) theory, therefore can judge whether system running is normal by corresponding state graph model and running situation.Node in constitutional diagram is the running status of system under test (SUT), and directed edge represents the required API performed of state transition.
Constitutional diagram is defined as five-tuple (S, s
0, E, R
s,E, F), be constructed as follows:
● S represents the finite aggregate of all states in constitutional diagram;
● s
0represent original state, s
0∈ S;
● E presentation of events or sequence of events set;
● R
s,Estate transfer set function, δ: S × E → S;
● F is final state set,
Similarly, state graph model also adopts XML to be described, and each node in figure is described by multiple field value, as follows:
Wherein each field is defined as follows:
● states: state set, represents the state of this node, can store a more than state variable;
● edges: limit is gathered, all directed edge set that to represent with this node be starting point, every bar limit have recorded failover events API and peripheral node Id.FILLIN_TEMPLATE represents the API of instantiation.
Interval I I: generating test use case.
It is generate test macro test case used that this period mainly finishes the work, this method adopts the mode of automatic generating test case, according to all API sequences that window tree-model traversal generation system under test (SUT) may occur, and by can execution operation these API sequences can be directly brought in system under test (SUT) run, carry out data encasement for during system cloud gray model.
Step 3: automatic generating test case module reads window tree-model file, the generating test use case of mass.
Automatic generating test case is according to window tree-model for the actual description realizing situation of system, according to the automatic generating run sequence of possible operating process and test case.According to window tree-model information illustrated in step 2, find the root node of window tree-model can to travel through all possible operations, just can complete the robotization generation work of test case according to this thinking.
This flow chart of steps is shown in Fig. 4, specifically, is divided into following a few sub-steps:
Step 3.1, data initialization, reads the window tree-model of XML, is loaded into each nodal information and according to relationships between nodes structure one tree, and to arrange present node NowRoot be tree root Root, clearing buffers queue APIList simultaneously, jump procedure 3.2;
Step 3.2, judges whether NowRoot is empty, if be empty jump procedure 3.7, if be not sky, jump procedure 3.3;
Step 3.3, judges whether present node is leaf node, if not then present node being joined in APIList, and jump procedure 3.4, if it is jump procedure 3.6;
Step 3.4: judge whether present node also exists the child node of not accessing, if existed, is set to present node NowRoot, jump procedure 3.2, there is not then redirect 3.5 by child node;
Step 3.5: NowRoot is removed from buffer queue APIList, and to arrange its father node be NowRoot, jump procedure 3.4;
Step 3.6: joined by this leaf node in buffer queue APIList, is then saved in test case using the content in buffer sequence as a test case and concentrates, then jump procedure 3.5;
Step 3.7: obtain test use cases, flow process terminates.
By above-mentioned steps and automatable generating test use case collection, these test use cases will process, to ensure that system under test (SUT) can directly run these test cases in next step performed change operation.
In the present embodiment, the emergency response stage shown in Fig. 3 (A) is according to above-mentioned steps, two API sequences can be produced, because whole tree only has Liang Tiao branch, they are null->FILLIN_TEMPLATE->SEND_ FORM->VIEW_FORM and null->FILLIN_TEMPLATE->SEND_ FORM->EDIT_FORM respectively, the null of most beginning is the original state of representative system, does not at this time need to perform API.
Step 4: the performed change process of test case.
The test case that test frame receives can not directly apply to system under test (SUT), because they are API sequence sets at present, needing to bring suitable parameter into can run, and therefore will carry out the performed change of test case.This needs the data getting out series of different, each API is selected and its parameter type match data that must go up be updated to wherein generate can execution test case and save and wait for next step operation.
In the present embodiment, once concrete performed change process as shown in Figure 5.Concentrate the API of a FILLIN_TEMPLATE by name to carry out to test case can in the process of execution, can learn that the method needs two character string types by the parameter analyzing API, and be EQID and TEMPLATEID, therefore from the parameter list registered in advance, find two of this API applicable parameters to add to wherein complete and can execution operate, the API like this after process just can be brought in system under test (SUT) and directly run.
Interval I II: system under test (SUT) testing results use-case.
The groundwork of this period allows system under test (SUT) testing results use-case, and obtain the state transfer case of system under test (SUT), these information will be used for judging whether the operation of system under test (SUT) meets expection.
Step 5: system under test (SUT) testing results use-case.
Each test case that test case is concentrated, after the performed change process of step 4, is updated in system under test (SUT) successively and performs.The execution of each API can have an impact to the state of internal system, and the tested framework of state change information is caught, and treats subsequent step process.
When carrying out uniting and adjustment test to earthquake emergency system, when a name is called that the API of FILL_IN_TEMPLATE performs, by the state of system under test (SUT) from (circulation, create) become (circulation, instantiation), this changes after tested framework captures and up-to-date state is extracted with corresponding API Name, i.e. (circulation, instantiation, FILL_IN_TEMPLATE).
Interval I V: message is disposed.
The groundwork of this period is carried out encapsulating, transmit and analyzing at the message captured in interval I II.The state transinformation of system under test (SUT) needs to be admitted to test macro and just can process.Therefore need to be encapsulated as message bag according to specific data form in test frame, be sent in test macro by network, just can carry out follow-up analytical work.
Step 6: status information is packaged into message bag according to the data layout consulted and is sent to automatization test system.
After the state change capturing system under test (SUT), the data layout that basis and automatization test system consult by test frame, state and other relevant information (as time, type etc.) are packaged into a message bag, are sent to outside automatization test system by network.Here, in order to ensure that multiple test macro monitors the different message bag of process simultaneously, test frame allows test macro to register, the type of message of accreditation system address and concern, whenever there being new message to contract for fixed output quotas life, message just can be distributed according to this registration table by test frame.
In the present embodiment, automatization test system based on model inspection is registered to test frame, register the address of this system and the type of message of concern, test frame also saves the time of this automatization test system registration in the process of registration message simultaneously, these field informations are kept in the registration table safeguarded by test frame, and its form is as shown in the table:
The registration table form that table 1 test frame is safeguarded
| RegAddress | TimeStamp | Favorites |
| 192.168.1.102 | 2014-07-2216:30:44.0 | MSG_STCH |
Each meaning of parameters is as follows:
● RegAddress: registered address, String type, the address information used when preserving Accreditation System communication is generally IP address;
● TimeStamp: timestamp, String type, preserve the time of exotics system registration;
● Favorites: focus, String array type, preserve the type of message that Accreditation System is wanted to receive.
In the present embodiment, test frame is after capturing the change of system under test (SUT) state, generating with protocol format the message bag communicated according to the message packet format of specifying, and message distributed by sending module, is wherein that the message packet format that 192.168.1.102 sends is as shown in table 2 to address:
Table 2 test frame and the message format communicated between automatization test system
Wherein each parameter declaration is as follows:
MsgType: type of message, String type, this field value is MSG_STCH in the present embodiment;
TimeStamp: timestamp, String type, represents that this message is contracted for fixed output quotas the raw time;
Description: describe, String type, carry out additional description to message, generally when normal configuration, this field is regular;
Destination: target, String type, represents that this message will be sent to intended target address;
DataFormat: data layout, String type, what kind of mode the data segment after the type represents adopts preserve;
Data: data segment, String type, preserve the data that message bag will send, the form of data is determined by the protocol type ProtocolType set, and in the present embodiment, Data section is made up of two state variables and corresponding API Name.
In addition, after a test case is run, test frame can generate a message bag be finished and outwards send, to guarantee that automatization test system processes the message received before.
Step 7: automatization test system receives and processing messages, generates state metastasis sequence, and it is pending that bonding state graph model sends into model checking module etc. in the lump.
Automatization test system receives the message bag that test frame is sent, and is first read according to specific data encapsulation format by message bag, and extract the state of system under test (SUT) and be saved in a queue, this queue is the state transition queues of system under test (SUT).The behavior performs until receive end always, after receiving, this queue is kept at a new position and state metastasis sequence, and empties the new message bag of queue wait-receiving mode.This state metastasis sequence and state graph model are together sent into model checking module simultaneously, be for further processing.
In the present embodiment, automatization test system based on model inspection receives the message bag from test frame, carrying out analyzing the Data field can known wherein to the fixed field of message is the encapsulation adopting the data layout of called after P_3 to carry out, namely data be by "; " carry out splitting, therefore according to the description of this data layout, Data field is disassembled, and this state is deposited in queue.
Period V: model inspection.
The groundwork of this period is the state metastasis sequence of system under test (SUT) interval I V process obtained, compare with state graph model, whether coincidence theory is expected in the operation of inspection system under test (SUT), and in combination with rate formula, export uniting and adjustment test percent of pass assessment report, complete the uniting and adjustment test job of system under test (SUT).
Step 8: model checking module bonding state graph model detects system under test (SUT) practical operation situation, and generate corresponding testing result.
Model inspection is the state graph model according to the theoretical ruuning situation of descriptive system, is brought into by the state metastasis sequence of running and wherein checks.According to state graph model information illustrated in step 2, model detection system meeting reading state graph model XML file information, and adopt the structure of figure to store, then sequence is brought in figure and checks whether that to there is legal path corresponding with it, just can complete model inspection work by such process.
The treatment step of model checking module as shown in Figure 6, is divided into following sub-step specifically:
Step 8.1, read the state graph model fileinfo of XML, digitized description is carried out to the data of each node described in file, become a node in constitutional diagram, to couple together with this node other node directed line segments related simultaneously, until whole XML file is disposed, obtain constitutional diagram G;
Step 8.2, writ state metastasis sequence is S={s
0, s
1..., s
n, the N state owS of current matching is s
i, initial i=0, simultaneously corresponding API sequence is API={api
0, api
1..., api
n-1;
Step 8.3, if (the s in state metastasis sequence
i, s
i+1, api
i) in constitutional diagram G, the match is successful, then (s is described
i, s
i+1, api
i) legal, record this api
icorrect execution once, otherwise records this api
imistake performs once, then performs step 8.4;
Step 8.4, after record behavior terminates, upgrades i=i+1, if the i>n-1 after upgrading, then jumps to step 8.5, otherwise jump to step 8.3;
Step 8.5, if there is API mistake to perform record in statistics, then illustrate that system under test (SUT) actual motion state does not meet expection, namely corresponding test case performs incorrect, and instead then illustrate that system under test (SUT) actual motion state meets expection, test case is normally run.
Can be detected the state metastasis sequence of system under test (SUT) by above-mentioned steps, and count detailed operation result and API implementation status.By being further analyzed these statisticses, the percent of pass of system under test (SUT) can be obtained.
For a test use cases TS, suppose that it comprises K test case TC, then each TC contains many API.Make different API all have different weight M here, this considers that the significance level of each API in whole system under test (SUT) is different.
When an API performs, finite state machine will check whether state transition meets expection, for state S
i, when the transfer of system under test (SUT) state is shifted inconsistent with theory after API performs, then record this API once mistake, otherwise it is once correct to record this API.So, after test use cases TS runs, there are two values for each API in whole system, correct execution number of times G
inumber of times B is performed with mistake
i.
In test case operational process, API is more important, and the weight ratio M occupied by it is larger, therefore weighs system under test (SUT) percent of pass formula P
esas follows:
Wherein, K represents the number of API, M
irepresent the weight of i-th API, G
iand B
irepresent that correct execution number of times and the mistake of i-th API perform number of times respectively.
In the present embodiment, two test cases generated in step 3 are after being finished, by the state transinformation of generation successively according to above-mentioned steps process, the information in Fig. 7 can be obtained, which includes the ruuning situation of each test case, API execution Information Statistics and test percent of pass.
The uniting and adjustment test percent of pass assessment report of final generation contains the concrete implementation status of test case, detection case and percent of pass assessed value in model inspection process, these specifically, intuitively information have larger help by improving system for help developer.
Claims (6)
1., based on a robotization uniting and adjustment method of testing for the task system stage by stage of model inspection, it is characterized in that, the method has been coordinated by automatization test system and test frame, and performing step is as follows:
Step one: divide the system under test (SUT) stage, the behavioral agent defining each stage system under test (SUT) and the element comprised, determine the state set of element and the state set of behavioral agent;
Step 2: the window tree-model of abstract system under test (SUT) and state graph model; Described window tree-model interior joint is window, and the limit of connected node is event; Described state graph model is for describing the state transition information of system under test (SUT);
Step 3: according to the window tree-model obtained, test use cases, to form test use cases, is sent to test frame by automatic batch generating test use case;
Step 4: after test frame receives test use cases, carries out execution operating to each test case wherein; Described performed change operation refer to for each API in test case select to match type supplemental characteristic and substitute into, API is performed in system under test (SUT);
Step 5: each test case test case concentrated, is brought in system under test (SUT) successively and performs, catches system under test (SUT) state change information during this period;
Step 6: according to the data layout of agreement between automatization test system and test frame, is packaged into message bag by the system under test (SUT) state change information obtained in step 5, is sent to automatization test system by network;
Step 7: after automatization test system receives message bag, resolves message bag, extracts system under test (SUT) state change information wherein and leaves in systematic state transfer sequence according to order; Step 7 circulation is carried out, until after receiving the message that a test case is finished, systematic state transfer sequence and state graph model are together sent into automatization test system process, empty current systematic state transfer sequence simultaneously, wait for next message bag;
Step 8: the practical operation situation of system under test (SUT) and theoretical ruuning situation are compared, specifically: each state in state metastasis sequence be brought in state graph model successively and detect, check in constitutional diagram whether there is the path that meets this state metastasis sequence; After all state metastasis sequences all substitute into, testing result is brought in percent of pass judgement schematics, final output uniting and adjustment test percent of pass assessment report.
2. the robotization uniting and adjustment method of testing of the task system stage by stage based on model inspection according to claim 1, is characterized in that, the window tree described in step 2 is defined as four-tuple (W, w
0, E, R
w,E), wherein:
(1) W is the set of window;
(2) w
0∈ W is home window, is the root node of window tree;
(3) E is event intersection, is the API set circulated between window and window;
(4) R
w,E∈ W × W × E, represents the circulation relation between window, if there is (w
i, w
j, e) ∈ R
w,E, e ∈ E, w
i, w
j∈ W, then represent window w
iwith window w
jbetween there is set membership, and w
ifor father node, w
jfor child node, window w
ito circulate window w by calling e
j.
3. the robotization uniting and adjustment method of testing of the task system stage by stage based on model inspection according to claim 1, it is characterized in that, the constitutional diagram described in step 2 is defined as five-tuple (S, s
0, E, R
s,E, F), be constructed as follows:
(1) S represents the finite aggregate of all states in constitutional diagram;
(2) s
0represent original state, s
0∈ S;
(3) E presentation of events or sequence of events set;
(4) R
s,Estate transfer set function, δ: S × E → S;
(5) F is final state set,
4. the robotization uniting and adjustment method of testing of the task system stage by stage based on model inspection according to claim 1, it is characterized in that, the automatic batch generating test use case described in step 3, concrete steps are as follows:
Step 3.1, reads window tree-model, is loaded into each nodal information and according to relationships between nodes structure one tree, and to arrange present node NowRoot be tree root Root, clearing buffers queue APIList simultaneously, jump procedure 3.2;
Step 3.2, judges whether NowRoot is empty, if be empty jump procedure 3.7, if be not sky, jump procedure 3.3;
Step 3.3, judges whether present node is leaf node, if not then present node being joined in APIList, and jump procedure 3.4, if it is jump procedure 3.6;
Step 3.4: judge whether present node also exists the child node of not accessing, if existed, the child node of not accessing is set to present node NowRoot, jump procedure 3.2, there is not then jump procedure 3.5;
Step 3.5: NowRoot is removed from buffer queue APIList, and to arrange its father node be NowRoot, jump procedure 3.4;
Step 3.6: joined by this leaf node in buffer queue APIList, is then saved in test case using the content in buffer sequence as a test case and concentrates, then jump procedure 3.5;
Step 3.7: obtain test use cases, flow process terminates.
5. the robotization uniting and adjustment method of testing of the task system stage by stage based on model inspection according to claim 1, it is characterized in that, the automatic batch generating test use case described in step 8, concrete steps are as follows:
Step 8.1, reading state graph model, forms constitutional diagram G according to each nodal information;
Step 8.2, if state metastasis sequence is S={s
0, s
1..., s
n, API={api
0, api
1..., api
n-1it is corresponding API sequence; If the N state owS of current matching is s
i, initial i=0;
Step 8.3, if state transfer (s
i, s
i+1, api
i) in constitutional diagram G, the match is successful, then (s is described
i, s
i+1, api
i) legal, record this api
icorrect execution once, otherwise records this api
imistake performs once, then performs step 8.4;
Step 8.4, i, from increasing 1, judges whether i is greater than n-1, if so, jumps to step 8.5 and perform, otherwise continue to go to step 8.3 execution;
Step 8.5, if counted on API mistake to perform record, then illustrate that system under test (SUT) actual motion state does not meet expection, corresponding test case has performed incorrect, and instead then illustrate that system under test (SUT) actual motion state meets expection, test case is normally run.
6. the robotization uniting and adjustment method of testing of the task system stage by stage based on model inspection according to claim 1, it is characterized in that, described automatization test system comprises automatic generating test case module, model checking module and percent of pass evaluation module; Automatic generating test case module is used for according to window tree-model automatic batch generating test use case to form test use cases; Model checking module is used for comparing the practical operation situation of system under test (SUT) and theoretical ruuning situation; Percent of pass evaluation module is used for testing result to be brought in percent of pass judgement schematics, exports uniting and adjustment test percent of pass assessment report.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410742936.1A CN104407977B (en) | 2014-12-08 | 2014-12-08 | Based on the automatization uniting and adjustment testing method of the task system stage by stage of model inspection |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410742936.1A CN104407977B (en) | 2014-12-08 | 2014-12-08 | Based on the automatization uniting and adjustment testing method of the task system stage by stage of model inspection |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104407977A true CN104407977A (en) | 2015-03-11 |
| CN104407977B CN104407977B (en) | 2016-06-01 |
Family
ID=52645610
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410742936.1A Expired - Fee Related CN104407977B (en) | 2014-12-08 | 2014-12-08 | Based on the automatization uniting and adjustment testing method of the task system stage by stage of model inspection |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104407977B (en) |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106326732A (en) * | 2015-07-03 | 2017-01-11 | 阿里巴巴集团控股有限公司 | Application programming interface (API) protection method and device |
| CN107066382A (en) * | 2017-03-09 | 2017-08-18 | 北京航空航天大学 | A kind of Space Vehicle System automated testing method based on model |
| CN107544906A (en) * | 2017-09-07 | 2018-01-05 | 杨瑞 | A kind of Web automated testing methods based on the judgement of equal value of content non-sensibility DOM skeletons |
| CN107590160A (en) * | 2016-07-08 | 2018-01-16 | 阿里巴巴集团控股有限公司 | A kind of method and device for monitoring radix tree internal structure |
| CN108509339A (en) * | 2018-03-22 | 2018-09-07 | 京北方信息技术股份有限公司 | Method for generating test case, device based on browser and mind map and equipment |
| CN108628727A (en) * | 2018-04-19 | 2018-10-09 | 山东省计算中心（国家超级计算济南中心） | It is a kind of based on mode operation when feature mode operations running state analysis method |
| CN109213681A (en) * | 2018-08-30 | 2019-01-15 | 北京三快在线科技有限公司 | Test macro, method, equipment and readable storage medium storing program for executing |
| CN110286882A (en) * | 2019-05-24 | 2019-09-27 | 北京航空航天大学 | A kind of foreground system design and verification method based on model inspection |
| CN111104329A (en) * | 2019-12-20 | 2020-05-05 | 光大兴陇信托有限责任公司 | Self-service system docking case guide test method |
| CN111679960A (en) * | 2020-05-30 | 2020-09-18 | 北京航空航天大学 | Reliability, elasticity and brittleness system state evaluation method |
| CN111934948A (en) * | 2020-07-20 | 2020-11-13 | 浙江理工大学 | EFSM executable test sequence generation method based on Monte Carlo tree search |
| CN112270110A (en) * | 2020-11-16 | 2021-01-26 | 国家工业信息安全发展研究中心 | Compatibility testing method and system for industrial internet platform assembly |
| CN113608092A (en) * | 2021-06-28 | 2021-11-05 | 臻驱科技（上海）有限公司 | Double-pulse test system |
| CN113704093A (en) * | 2021-08-05 | 2021-11-26 | 卡斯柯信号(成都)有限公司 | Universal interlocking software test case generation method |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103235756A (en) * | 2013-04-22 | 2013-08-07 | 北京航空航天大学 | Simulation testing method for partition application software of embedded system |
| US8510596B1 (en) * | 2006-02-09 | 2013-08-13 | Virsec Systems, Inc. | System and methods for run time detection and correction of memory corruption |
| CN104021070A (en) * | 2013-03-01 | 2014-09-03 | 腾讯科技（深圳）有限公司 | Method, device and system for testing intelligent terminal software packages |
-
2014
- 2014-12-08 CN CN201410742936.1A patent/CN104407977B/en not_active Expired - Fee Related
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8510596B1 (en) * | 2006-02-09 | 2013-08-13 | Virsec Systems, Inc. | System and methods for run time detection and correction of memory corruption |
| CN104021070A (en) * | 2013-03-01 | 2014-09-03 | 腾讯科技（深圳）有限公司 | Method, device and system for testing intelligent terminal software packages |
| CN103235756A (en) * | 2013-04-22 | 2013-08-07 | 北京航空航天大学 | Simulation testing method for partition application software of embedded system |
Cited By (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106326732A (en) * | 2015-07-03 | 2017-01-11 | 阿里巴巴集团控股有限公司 | Application programming interface (API) protection method and device |
| CN107590160A (en) * | 2016-07-08 | 2018-01-16 | 阿里巴巴集团控股有限公司 | A kind of method and device for monitoring radix tree internal structure |
| CN107590160B (en) * | 2016-07-08 | 2021-08-27 | 阿里巴巴集团控股有限公司 | Method and device for monitoring internal structure of radix tree to realize test |
| CN107066382A (en) * | 2017-03-09 | 2017-08-18 | 北京航空航天大学 | A kind of Space Vehicle System automated testing method based on model |
| CN107544906B (en) * | 2017-09-07 | 2020-12-25 | 杨瑞 | Web automatic testing method based on content-insensitive DOM framework equivalence judgment |
| CN107544906A (en) * | 2017-09-07 | 2018-01-05 | 杨瑞 | A kind of Web automated testing methods based on the judgement of equal value of content non-sensibility DOM skeletons |
| CN108509339A (en) * | 2018-03-22 | 2018-09-07 | 京北方信息技术股份有限公司 | Method for generating test case, device based on browser and mind map and equipment |
| CN108628727A (en) * | 2018-04-19 | 2018-10-09 | 山东省计算中心（国家超级计算济南中心） | It is a kind of based on mode operation when feature mode operations running state analysis method |
| CN108628727B (en) * | 2018-04-19 | 2021-06-01 | 山东省计算中心（国家超级计算济南中心） | Pattern operation running state analysis method based on pattern running characteristics |
| CN109213681A (en) * | 2018-08-30 | 2019-01-15 | 北京三快在线科技有限公司 | Test macro, method, equipment and readable storage medium storing program for executing |
| CN110286882A (en) * | 2019-05-24 | 2019-09-27 | 北京航空航天大学 | A kind of foreground system design and verification method based on model inspection |
| CN111104329A (en) * | 2019-12-20 | 2020-05-05 | 光大兴陇信托有限责任公司 | Self-service system docking case guide test method |
| CN111104329B (en) * | 2019-12-20 | 2021-03-16 | 光大兴陇信托有限责任公司 | Self-service system docking case guide test method |
| CN111679960A (en) * | 2020-05-30 | 2020-09-18 | 北京航空航天大学 | Reliability, elasticity and brittleness system state evaluation method |
| CN111934948A (en) * | 2020-07-20 | 2020-11-13 | 浙江理工大学 | EFSM executable test sequence generation method based on Monte Carlo tree search |
| CN111934948B (en) * | 2020-07-20 | 2022-04-12 | 浙江理工大学 | EFSM executable test sequence generation method based on Monte Carlo tree search |
| CN112270110A (en) * | 2020-11-16 | 2021-01-26 | 国家工业信息安全发展研究中心 | Compatibility testing method and system for industrial internet platform assembly |
| CN113608092A (en) * | 2021-06-28 | 2021-11-05 | 臻驱科技（上海）有限公司 | Double-pulse test system |
| CN113704093A (en) * | 2021-08-05 | 2021-11-26 | 卡斯柯信号(成都)有限公司 | Universal interlocking software test case generation method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104407977B (en) | 2016-06-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104407977A (en) | Model detection based automatic joint debugging test method for staged task system | |
| KR102483025B1 (en) | Operational maintenance systems and methods | |
| US20090271351A1 (en) | Rules engine test harness | |
| Xhafa et al. | Processing and analytics of big data streams with yahoo! s4 | |
| CN107526676B (en) | Cross-system test method and device | |
| CN104539053A (en) | Power dispatching automation polling robot and method based on reptile technology | |
| CN108377255A (en) | Information processing unit, information processing method and recording medium | |
| US20150286555A1 (en) | System and method for converting the business processes to test-centric activity diagrams | |
| CN101639803A (en) | Exception handling method and exception handling device for multithread application system | |
| US11347620B2 (en) | Parsing hierarchical session log data for search and analytics | |
| CN110895506A (en) | Construction method and construction system of test data | |
| US8918765B2 (en) | Auto-documenting based on real-time analysis of code execution | |
| Xu et al. | Quantitative analysis of variation-aware Internet of Things designs using statistical model checking | |
| Sebu et al. | Similarity of business process models in a modular design | |
| Barisic et al. | Systematic literature review on multi-paradigm modelling for cyber-physical systems | |
| US20080168102A1 (en) | Method, system, and computer program product for reconstructing a data stream | |
| D’Ambrogio et al. | A method for the prediction of software reliability | |
| Tonjes et al. | Automated testing of context-aware applications | |
| Pitakrat | Hora: Online Failure Prediction Framework for Component-based Software Systems Based on Kieker and Palladio. | |
| Kaupp et al. | „From Raw Data to Smart Documentation: Introducing a Semantic Fusion Process for Cyber-Physical Systems.“ | |
| Cheng et al. | A distributed event-centric collaborative workflows development system for IoT application | |
| Parri et al. | FaultFlow: A tool supporting an MDE approach for Timed Failure Logic Analysis | |
| CN113010331A (en) | Abnormal data processing method and device and computer readable storage medium | |
| Herbert et al. | Workflow fault tree generation through model checking | |
| Rahim et al. | Modular and distributed verification of SysML activity diagrams |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20160601 Termination date: 20201208 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |