Summary of the invention
The invention provides a kind of protocol automatic test method and system thereof, overcome the deficiency that prior art can not automatic test, and need not installation of additional equipment and test.
In order to address the above problem, the invention provides a kind of protocol automatic test system, comprise server end and at least one user terminal;
Comprise between described server end and any one user terminal: the test data passage of the control channel of Connection Service device end first Network Interface Module and user side second Network Interface Module and Connection Service device end first protocol stack module and user side second protocol stack module;
Described server end also comprises:
The test case memory cell is used to preserve test case;
First configuration module is used for pre-configured test parameter and server end channel parameters; Described channel parameters comprises the channel parameters of the control channel between described first, second Network Interface Module, and the channel parameters of the test data passage between described first, second protocol stack module;
The test dispatching module is used to control described first Network Interface Module and sends initial signal to described user terminal by described control channel; Receive the affirmation signal of described user terminal feedback when described first Network Interface Module after, according to described test parameter, call corresponding test case successively and send to described first protocol stack module, and control of the execution of described first protocol stack module by test case between described test data passage and the described user terminal;
Described user terminal also comprises: second configuration module is used for the pre-configured channel parameters identical with server end;
Scheduler module is used to control described second Network Interface Module and receives described initial signal, and controls described second Network Interface Module and give described server end by described control channel feedback acknowledgment signal.
Further, described test case memory cell comprises:
The first test case memory cell, first test case that is used to store the different testing procedures that correspond respectively to agreement to be tested;
The second test case memory cell, be used to store the correct execution result's who corresponds respectively to described different testing procedures second test case;
Described test dispatching module is called is first test case in the described first test case memory cell.
Further, described server end also comprises: the comparative analysis module;
Described first protocol stack module sends to described comparative analysis module with each execution result;
Described comparative analysis module generates a test report by corresponding correct execution result in this execution result relatively and the described second test case memory cell, comprising: judge whether there is mistake in the agreement; If there is mistake, then further judge this wrong source, and concrete wrong content.
Further, described server end also comprises: the sequential parsing module;
Described first protocol stack module sends to described sequential parsing module with each execution result;
Described sequential parsing module is analyzed described first protocol stack module and described second protocol stack module sequential in the process of implementation by corresponding correct execution result in this execution result relatively and the described second test case memory cell, and the generation sequential chart.
Further, described server end also comprises:
The test result display unit is used to show described test report;
Agreement sequential display unit is used to show described sequential chart.
Further, the test parameter of described first configuration module configuration comprises:
Test case kind and test mode;
Described test case kind comprises: consistency, interactivity, pressure;
Described test mode comprises: single use-case is carried out, all use-case is carried out or the combination in any execution.
The present invention also provides a kind of protocol automatic test method, is used for comprising the system of server end and at least one user terminal, may further comprise the steps:
Described server end storage test case, and configuration testing parameter;
Described server end and the identical channel parameters of described user terminal configuration; Described channel parameters comprises the channel parameters of the control channel between described server end and described user terminal, and the channel parameters of the test data passage between described server end and described user terminal;
Described server end sends initial order to user terminal on described control channel;
Described user terminal receives behind the described initial order that the feedback acknowledgment signal is given server end on described control channel;
After described server end receives described confirmation signal, call described test case successively, and carry out the test case of being called jointly by described test data passage with user terminal according to described test parameter.
Further, described test case comprises:
First test case corresponds respectively to the different testing procedures of agreement to be tested;
Second test case corresponds respectively to the correct execution result of described different testing procedures;
The test case that described server end calls is described first test case.
Further, described method also comprises:
Corresponding correct execution result in the execution result of more described first test case of described server end and described second test case;
According to relatively generating a test report, described report comprises: judge whether there is mistake in the agreement; If there is mistake, then further generate this wrong source, and concrete wrong content.
Further, described method also comprises:
Corresponding correct execution result in the more described first test case test result of described server end and described second test case;
According to the sequential in the comparative analysis implementation, and generate sequential chart.
Further, described method also comprises:
Show described test report and described sequential chart.
Further, described test parameter comprises:
Test case kind and test mode;
Described test case kind comprises: consistency, interactivity, pressure;
Described test mode comprises: single use-case is carried out, all use-case is carried out or the combination in any execution.
Technical scheme of the present invention provides a kind of protocol automatic test method and system thereof,, can finish automatically the data that produce in the measuring executing process are collected, analyze, put in order in advance at the good test case of design of protocol to be measured in the system by the user; The user need not manual operation, thereby has simplified test process, difficulty and can reduce the testing time, has improved the efficient of development﹠ testing greatly, has reduced cost.A prioritization scheme of the present invention is divided into testing procedure with test case and to test result that should testing procedure, with convenient in test accurate location mistake; Another prioritization scheme of the present invention promptly can be according to the automatic test of projects such as the consistency of agreement in server and the client/server end in user's the test execution order start-up system, interactivity, pressure after server end is configured at the project of needs tests by the user; Another prioritization scheme of the present invention can be in the process of test by the test result of testing procedure in to test case, with this test case in to the compare of analysis of test result that should testing procedure, can helper person or the tester develop and test phase accurately to orient in the agreement specifically be what problem where to have occurred; Can also further use the dynamic display protocol sequential chart of form and the test result of gui interface, can make developer or tester find out the problem that the agreement execution exists in the system intuitively, simplified the step of analysis and loaded down with trivial details operation, thereby realized visual automatic test system protocol.
Embodiment
Below in conjunction with drawings and Examples technical scheme of the present invention is described in detail.
Protocol automatic test of the present invention system, an embodiment comprise server end (server) 100 and at least one user terminal (client) 200 as shown in Figure 2;
Wherein, described server end (server) 100 comprises: first configuration module 102, test dispatching module 103, test case memory cell, first Network Interface Module 105 and first protocol stack module 106 that are used for configuration testing parameter and channel parameters;
Described user terminal 200 comprises: second Network Interface Module 201, scheduler module 202, second configuration module 203 that is used for the collocation channel parameter and second protocol stack module 205;
Comprise between described server end and any one user terminal: the test data passage of the control channel of Connection Service device end first Network Interface Module and user side second Network Interface Module and Connection Service device end first protocol stack module and user side second protocol stack module; The channel parameters of first, second configuration module configuration is identical; Described channel parameters comprises the channel parameters of the control channel between described first, second Network Interface Module, and the channel parameters of the test data passage between described first, second protocol stack module.
Optionally, server end can further include: first user interactive module 101, and during test, the user can be configured by first user interaction unit, 101 indication 102 pairs of test parameters of described first configuration module and/or channel parameters;
Optionally, user terminal can further include: second user interactive module 204, and during test, the user can be configured by 203 pairs of channel parameters of second user interaction unit, 204 described second configuration modules of indication;
Described first, second user interactive module can but be not limited to GUI (graphic user interface).
In actual applications, do not get rid of the situation that first, second configuration module is configured automatically yet,, be configured according to this configuration file then etc. such as reading a default or default configuration file.
Optionally, the test parameter of described first configuration module 102 configurations comprises: test case kind and test mode; This test case kind comprises: consistency, interactivity, pressure or other tests; This test mode comprises: single use-case is carried out, all use-case is carried out or the combination in any execution.Described first configuration module 102 can save as a configuration file with described test parameter.
Optionally, described first, second configuration module can also be used for configuration device information, and promptly this equipment is server end or user terminal; Same like this equipment both can have been worked as server end and used, and also can work as user terminal and use; Certainly also do not get rid of the way that equipment is fixed as server end or user terminal in the practical application, can dispose described facility information this moment.
Optionally, described test case memory cell can comprise the first test case memory cell 104 and the second test case memory cell 109.
Wherein, the described first test case memory cell 104, the second test case memory cell 109 are respectively applied for storage first test case and second test case;
Each described first test case corresponds respectively to the different testing procedures of agreement to be tested;
Each described second test case corresponds respectively to the correct execution result of different testing procedures;
As seen, first, second test case also is one to one.
When comprising several packets in one first test case, also will comprise several correct execution result in second test case corresponding with this first test case; For one first test case, the correct execution result of its each packet should come to the same thing with corresponding correct execution in second test case of correspondence; If different, then explanation is carried out and is made mistakes.Because packet is carried out chronologically, therefore can find the correct execution result of each packet correspondence chronologically; Certainly do not get rid of utilization yet and in packet and correct execution result, add the correct execution result that corresponding sign is searched each packet correspondence.
Described test dispatching module 103 starts test according to the order of user's input, or starts test in the time automatically, controls first Network Interface Module 105 and sends second Network Interface Module 201 of initial signal to client terminal 200 by described control channel; Second Network Interface Module 201 receives and sends this initial signal to scheduler module 202, and described scheduler module 202 controls second Network Interface Module 201 is given described first Network Interface Module 105 by described control channel feedback acknowledgment signal;
After described test dispatching module 103 is received the affirmation signal of user terminal feedback when described first Network Interface Module, test parameter according to 102 configurations of described first configuration module, call in the first test case memory cell 104 corresponding first test case successively and give first protocol stack module 106, control between described first protocol stack module 106 and second protocol stack module 205 by of the execution of described test data passage to described first test case.
Described first, second configuration module is identical to the channel parameters of control channel configuration, and is also identical to the channel parameters of test data channel arrangement.
In one embodiment, first, second configuration module can just be finished whole configurations before test, in other embodiments, first, second configuration module also can be configured when being triggered, such as: server end can be the channel parameters that triggers first configuration module, 102 configuration testing parameters and test data passage after receiving described confirmation signal; User terminal can be after receiving described initial signal, trigger second configuration module, 203 configuration testing data channel channel parameters (at this moment, return confirmation signal after the configuration of second configuration module 203 finishes and give scheduler module 202, scheduler module 202 receives behind the affirmation signal of second configuration module 203 that the feedback acknowledgment signal is to server end again).But when beginning to test, first, second configuration module has all configured the channel parameters of control channel.
Optionally, server end can further include: the monitoring analysis module; This monitoring analysis module comprises comparative analysis module 107 and sequential parsing module 108, or wherein arbitrary.
Optionally, server end is corresponding comprising further: test result display unit 110 and agreement sequential display unit 111, or wherein arbitrary.
Described first protocol stack module 106 also is used for each execution result is sent to comparative analysis module 107 and/or sequential parsing module 108, comparative analysis module 107 generates a test report by content corresponding in this execution result relatively and the second test case memory cell 109, comprise: judge whether there is mistake in the agreement, if there is mistake, then further judge this wrong source, and concrete wrong content; And this test report exported to test result display unit 110 to show; Sequential parsing module 108 is by corresponding content in this execution result relatively and this second test case memory cell 109, analyze this first protocol stack module 106 and second protocol stack module 205 sequential in the process of implementation, and the generation sequential chart is exported to agreement sequential display unit 111 to show.
The a certain testing procedure that described test dispatching module 103 is called in this first test case memory cell 104 is given first protocol module 106, to realize the test between its execution and second protocol stack module 205, when first protocol module of testing 106 or second protocol module, 205 appearance mistakes, comparative analysis module 107 can be by judging this wrong source to correct execution result that should a certain testing procedure in relatively this execution result and second test case, and concrete wrong content.
Wherein, be connected by control channel between described first Network Interface Module 105 and second Network Interface Module, this control channel can be: wired and wireless mode.Wherein wired mode can be: serial ports, parallel port, USB, Ethernet etc.; Wireless mode can be: infrared, WIFI, bluetooth etc.
Wherein, the pass test data passage is connected between described first protocol module and second protocol module, and this test data passage can be: wired and wireless mode.Wherein wired mode can be: serial ports, parallel port, USB, Ethernet etc.; Wireless mode can be: infrared, WIFI, bluetooth etc.
Wherein, described agreement comprises: TCP/IP (TCP, UDP, IP, ICMP, ARP, SIP, HTTP, TFTP, WAP or the like), 802.11 (WIFI), 802.3 etc.
Seeing also Fig. 3, is the flow chart of the embodiment of visualization protocol automatic test approach of the present invention, and its step comprises:
Step 800, the user imports configuration information by config option, and this configuration information can comprise:
Configuration device information is as server or client;
Configuration testing use-case kind of information, as uniformity test, interactivity test, pressure test, other tests wait options, dispose the directory path of the test case of each kind correspondence simultaneously, can the combination in any selection;
Configuration testing mode information, as carry out single test case, or carry out whole test cases, or carry out the combination in any use-case;
The collocation channel parameter is as information such as the baud rate in the serial communication, port numbers, the control channel between apparatus for establishing;
Step 802, the configuration module of two equipment forms configuration file according to the configuration information of user input, and this configuration file can be TXT form or other types form, and as the file of * .doc type, or XML document etc. can.For example configuration file 1: equipment: server; The test case kind: uniformity test, f: ConformanceTestSuite, interactivity test, f: IOPTtestSuite ...; Test mode: all cases carries out; Communication mode is selected: serial ports, 9600, COM1 ...
Step 804 judges whether the user requires to revise configuration information, if revise then enter step 800, otherwise, enter step 806;
Step 806 judges whether to start the automatic test of agreement, if do not start, then continues to enter step 804 and judges whether to need to revise configuration information, otherwise enter step 808;
Step 808, the scheduler module of server end and user terminal (test dispatching module 103 and scheduler module 202 in promptly above installing) reads configuration file; Click the button of " beginning test " as the user after, server end sends initial order to user terminal; User terminal returns confirms order; Control channel between server end and the user terminal is set up;
The scheduler module of server end reads " facility information ", " the test case kind of information " and " test mode information " in the configuration file then, to form querying condition;
Step 810, the scheduler module of server end is extracted the test case that all will be used according to configuration file; From test case library, obtain the test case that satisfies condition according to the querying condition that forms, be extracted up to all test cases that will use and finish.
Step 812, the scheduler module of server end send the test case of writing in advance extracted successively to first protocol stack module, send a test case at every turn; Set up the execution of the protocol test between first and second protocol stack module;
Step 814, the scheduler module of server end judge whether the execution of current test case finishes, if not then carry out step 818 simultaneously and finish up to carrying out; Otherwise, enter step 816.
Step 818, actual execution result of comparative analysis module and correct execution result, this step are preferred steps; The sequential parsing module is resolved the sequential in first protocol stack module execution test process successively, and this step is a preferred steps; Can also further this sequential be converted to sequential chart and export demonstration, referring to Fig. 4;
Step 816 judges that whether described all test cases that will use all carry out end, carry out next test case if continue to enter step 812, otherwise enter step 820.
Step 820, the test report of output display protocol; This step is a preferred steps.This test report content comprises:
Pass Number: relatively the back is consistent to represent the correct execution result of test result and expection, represents that tested content is correct;
Fail Number: relatively the back is inconsistent to represent the correct execution result of test result and expection, represents that tested content is incorrect;
ErrorNumber: represent to occur unusually the number of termination in the measuring executing process;
TestCases Total: statistics collection amounts to has carried out for how many bar test cases.
The test case of wherein writing in advance is to adopt the prel language, also can adopt language such as C, C++, JAVA.
This shows, the user is by the configuration of gui interface in the protocol automatic test process, convenient test or developer can rapid extraction select for use for different testing schemes, also test case is classified simultaneously, accelerated tester or developer extraction, improved operating efficiency test case.
In addition, the user is by configuration, finish the setting of the communication modes of server (server) and user terminal (client) equipment room, as: the basic configuration (baud rate of serial ports, port etc.), realize for dispatching data transmission by wired or wireless mode, thus the tester separately coding can be implemented in transfer of data between distinct device, analysis and collection test result; Same set of such system is installed on two kinds of dissimilar equipment simultaneously, simplifies job step, reduce cost.
State conversion process when carrying on an agreement test automatically below in conjunction with server among the embodiment and user terminal further is elaborated to protocol automatic test method of the present invention.
Seeing also Fig. 5-1, is server end state exchange schematic diagram in the test process of embodiment.
State 401: initial state, this moment, the test dispatching module was in inactive state;
After the test dispatching module receives that the user starts the order of test, enter init state 402.
State 402: the test dispatching module at first reads the configuration information that is generated by GUI; Read in test case and test environment parameter configuration according to configuration information; Initialization is also opened control channel (as: serial ports); Send user terminal (client) Paging and order, inquire about available Client, and timer is set to control channel; If this moment, timer did not have overtimely, then begin to wait for the response of Client, if timer expired then finishes the execution of this module, and returns mistake, enter done state 405; If receive the wrong responses (i.e. abnormal responses among the figure) of Client, then resend order to control channel; After scheduler module is received the response (i.e. normal response among the figure) that Client returns, enter executing state 403.
State 403: after the test dispatching module receives that correct Client responds, enter executing state.
After entering this state, initialization test data channel (as: Ethernet card) at first; The Server end is carried out loaded test case, and this moment, test case should be in blocked state; Send the order of startup test case to control channel, and start timer; If it is overtime that timer does not have, then wait for replying of Client; If the overtime execution that then finishes this module, and return mistake, enter done state 405; If receive the wrong responses (i.e. abnormal responses among the figure) of Client, then resend order to control channel; If scheduler module is received correctly the replying of Client (i.e. normal response among the figure), this moment, test data began to transmit on the test data passage so, entered monitoring analysis state 404.
State 404: under this state the execution of the main monitoring analysis test case of monitoring analysis module whether correct, and the synchronous execution by control Server such as judgement time and execution result and Client.
After entering this state, start timer, judge whether overtimely, if the overtime execution of then ending test case, and return mistake, enter done state 405; If there be not overtime execution the finish command and the test report of beaming back by Client of then waiting for; If receive the execution the finish command and the test report that send by Client, then finish the execution of Server end test case; Enter done state 405.
State 405: done state, in this state, control channel and the test data passage set up before the test dispatching module will be removed, arrival state 406.
Remove test data passage (as: Ethernet card); Remove control channel (as: serial ports); Remove the environment that test case is carried out, discharge used resource; The test report that the monitoring analysis module is held the test report of Server end and Client is sent to output unit and submits to GUI and show.
See also Fig. 5-the 2nd, user terminal state conversion schematic diagram in the test process of embodiment,
State 501: scheduling of user terminals module initial state, this moment, scheduler module was in inactive state.
When the scheduling of user terminals module is received user's starting command, enter init state 502.
State 502: initialization control channel, open this passage so that the order of accepting to hold from Server; Start timer, and judge whether overtimely,, enter done state 505 if overtime this module that then withdraws from is carried out; Overtime as if not having, then waiting for server sends user terminal Paging order; If receive user terminal (client) Paging order (being equivalent to the above initial order of server end), then send response; Start timer, and judge whether overtimely,, enter done state 505 if overtime this module that then withdraws from is carried out; Overtime as if not having, then waiting for server sends and starts test case (Test Case) order; If receive to start the test case order, then enter executing state 503.
State 503: scheduler module is received and is started the test case order, the beginning implementation of test cases.
The startup test case order that parsing is received; According to analysis result, judging needs to carry out which test case and how to carry out, and test case is loaded; Send and carry out response, the beginning implementation of test cases; Start timer and wait for that Server returns the affirmation order,, enter done state 505 if overtime this module that then withdraws from is carried out; If Server sends response, to return and confirm to order, control module receives and confirms that order then enters monitor state 504 that the control and measuring use-case is carried out.
State 504: monitor state, judge whether test case carries out end.
Start timer, judge whether overtimely,,, enter done state 505 if overtime this module that then withdraws from is carried out if do not have overtimely to judge then whether use-case carries out end; Finish if test case is carried out, then send and finish the use-case order, and hold test report to be recycled to Server Client to control channel; Enter done state 505.
State 505, scheduler module will be removed link, the cleaning test environment; Remove test data passage (as: Ethernet card); Remove control channel (as: serial ports); Remove test environment, discharge used resource.
Seeing also Fig. 6, is server side to monitor analysis module workflow diagram among the embodiment.
Step 701, after receiving the order that begins to test, after reading configuration file " facility information " and being server, the data of initialization monitoring analysis module.
Step 702,703, according to " channel parameters " in the configuration file, select to open relevant control channel and carry out policer operation, data read to the monitoring analysis module that is about on the passage is handled, and does not influence the normal running of test dispatching module and test case simultaneously.
Step 704,705, do not receive exit command and do not receive overtime sign in, will execution in step 706, judged whether the test data process, if having then execution in step 707,708,709 successively; Otherwise return step 704; Step 707, the process here will send to Server end test case to test data according to the content of test data and test case, be used for the record and the judgement of test case external data; Step 708, record protocol sequential and offer output interface to the corresponding protocols sequential and show with the mode of GUI in document.
Exit command or overtime sign the time when receiving, will execution in step 710, whether there is the content of test report to judge in the information that provides according to data and test case, if when not having the content of test report to come, execution in step 712 is closed behind the test data channel monitoring execution in step 713 closing control channel monitorings again, finishes up to whole flow process.If also have the test report then execution in step 711 of coming, preserve test report, collect up to all test report and finish, enter into step 712,713, finish up to whole flow process.
The data content of the test report of collecting simultaneously deposits in the file, and the GUI that offers output interface shows report content.
Certainly; the present invention also can have other various embodiments; under the situation that does not deviate from spirit of the present invention and essence thereof; those of ordinary skill in the art work as can make various corresponding changes and distortion according to the present invention, but these corresponding changes and distortion all should belong to the protection range of claim of the present invention.