KR20010000267A - Method and apparatus for dynamic protocol conformance testing - Google Patents

Abstract

PURPOSE: The apparatus and the method for testing the suitability of a dynamic protocol are provided to increase the correctness in the suitability test for the protocol implemented by vendors, by finding out the reason that makes an incorrect examination of the implementation under test(IUT), and thereby resolving this problem. CONSTITUTION: An input processing module(100) divides the information about the certain protocol, which has been changed to the state of a transition set and will be tested, into the information based on transitions and states. A test sequence generating module(200) creates a plurality of preamble sub-sequences that include possible paths except the loop which contains more than two initial states, for the preamble that is a path to a test target transition, referring to the tree information processed by the module(100). A test sequence generating module(300) creates a plurality of post-amble sub-sequences for which all possible paths are considered, for the post-amble that verifies the arrival state of the test target transition. A module(400) combines the sequences from the modules(200,300) to form a test sequence tree. A module(500) applies each sequence of the tree to the IUT, and determine the defect in each transition of the IUT.

Description

METHOD AND APPARATUS FOR DYNAMIC PROTOCOL CONFORMANCE TESTING}

The present invention relates to a method for testing dynamic protocol conformance, and more particularly, it is enacted by the International Standards Organization ISO / IEC JTC1 (International Standard Organization / International Electrotechnical Commission Joint Technical Committee1), which checks whether an implemented protocol product conforms to the protocol specification. Protocol conformance test methods.

Most protocol specifications are standardized by international standards bodies, and because of the ambiguity inherent in protocol specifications, vendors can interpret differently so that interoperability between protocol implementations implemented by vendors from protocol standards can be avoided. It can't be. To prevent this problem, a protocol conformance test is established as an international standard through ISO / IEC JTC1, which tests how well a protocol implementation is implemented in its protocol standards or specifications.

In this case, the protocol implemented by the vendor under test is referred to as an implementation under test (IUT), which is generally regarded as a black box having only an input port and an output port. Conventional test methods test the implementation of the protocol specification by comparing the results of applying input sequences obtained from the protocol specification to the IUT with outputs already obtained from the specification.

The procedure of this protocol conformance test proceeds as described in the flowchart illustrated in FIG. 1.

First, one test sequence for testing an IUT, that is, a test sequence generated to test all transitions constituting the protocol FSM, is generated (step 110). In this case, the test sequence is a sum of all the sub test sequences for testing each transition of the IUT, and the sub test sequence exists as many as the number of transitions constituting the protocol FSM. At this time, the sub test sequence is made through the following process.

(1) Place the state of the IUT from the initial state to the state where you want to begin testing along the shortest path.

(2) Apply the input for the transition under test to the IUT.

(3) Observe the results and verify that the IUT has finished in the expected state.

The test sequence is then applied to the IUT for testing (steps 120 and 130) and proceeded to derive the test results (step 140).

As described above, a typical protocol conformance test method generates a sub test sequence through three steps, which is a fixed sequence since only one shortest path is considered from an initial state. For this reason, if the path includes a mis-implemented transition, it may affect the test result of the transition to be tested, resulting in a problem of inaccurate diagnosis.

In other words, the above-described protocol conformance test method of the prior art often fails to derive accurate test results because it uses a fixed test sequence even though the transition to be tested is correctly implemented. This problem is due to the fact that several transitions that make up the protocol finite state machine (FSM) are included in the test sequence, which affects the testing of the transition under test.

Therefore, an object of the present invention is to provide a method of improving the accuracy of conformance testing for protocol implementations implemented by actual vendors by finding and solving the causes of inaccurate diagnostic results of the IUT in the conventional protocol conformance test method. do.

The protocol conformance test method according to a preferred embodiment of the present invention for achieving the above object, preamble sub-test sequences of all possible paths from the finite state machine (FSM) of the protocol under test having a set form of transition and all Collecting the postamble subtest sequences to construct a test sequence tree; And repeatedly applying each test sequence constituting the test sequence tree to the protocol to determine whether there is a defect in each transition of the test target protocol.

According to another embodiment of the present invention, an apparatus for testing protocol conformance may include: an input processing module configured to divide a test object protocol having a set of input transitions into a tree in the form of a linked list by dividing the transition-oriented information into state-oriented information; ; A first preamble subsequence for generating a plurality of preamble subsequences including all other possible paths except for a loop including an initial state two or more times with respect to a preamble that is a path to a transition under test with reference to the transition information and state information; A subsequence generation module; A second sub-test sequence generation module generating a plurality of post-amble subsequences in consideration of all other possible paths with respect to the postamble for verifying the arrival state of the transition to be tested by referring to the transition information and state information; A test sequence tree generation module for constructing a test sequence tree by collecting preamble and postamble subtest sequences generated by the first and second subtest sequence generation modules; And a test module for repeatedly applying each test sequence of the test sequence tree to the test target protocol to determine whether there is a defect in each of the transitions.

1 is a flowchart illustrating a protocol conformance test method of the prior art;

2A is a flowchart illustrating a process of performing a dynamic protocol conformance test method according to the present invention;

FIG. 2B is a schematic block diagram of a conformance test apparatus suitable for performing the dynamic protocol conformance test method illustrated in FIG. 2A;

3A to 3E are diagrams illustrating a process of configuring and testing the TST of the present invention with an arbitrary protocol finite state machine (FSM),

4 is a diagram illustrating a configuration for comparing the test method of the present invention with the test method of the present invention using the dynamic protocol conformance test apparatus illustrated in FIG. 2;

5a to 5d are views comparing the results obtained through the configuration shown in FIG.

<Explanation of symbols for main parts of drawing>

100: input processing module

200, 300: first and second sub-test sequence generation module

400: test sequence tree generation module 500: test module

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

First, in order to help the understanding of the present invention, a brief description is as follows. Unlike the conformance test of the prior art, the protocol conformance test method according to the present invention uses a data structure representing a set of test sequences called TST to dynamically reconstruct the test sequence during the test, and sequentially sequence each test sequence through a feedback process. Is to apply the method to the IUT.

The test sequence consists of a set of sub-test sequences. The sub-test sequence consists of a preamble, which is a path from the initial state of the IUT to the transition under test, and a postamble for verifying the arrival state of the transition under test. It is composed.

In the present invention, instead of using the preamble portion and the postamble portion as a fixed sequence when generating a test sequence, the test sequence is generated using a TST (Test Sequence Tree) including a plurality of sequences in consideration of all possible paths. Dynamically select and test.

This TST consists of a set of Test Sub Sequence Trees (TSST), as can be seen from FIG. 3 described below. In addition, the TSST is a set of PTSs that a single transition can have, and includes a plurality of PTSs because the sequence includes both an alternative path in the preamble part and all sequences considering another path in the preamble part. It will be included.

Subsequently, a local determination of whether a failure is performed while applying each transition of all test sequences to the IUT, wherein the term local verdict is to determine whether a transition is defective If the local determination of any transition is a failure, it is determined that the TSST is 'defect' as shown in Figure 3d below, and the need to test it by removing the PTSs containing this transition and reconstructing the tree structure. Can be.

That is, in the dynamic conformance test method of the present invention, since a plurality of PTSs can be included in the subtest sequence, a relatively accurate diagnosis result can be derived as compared with the method of taking only one fixed subtest sequence as in the conventional conformance test method. It can be.

In addition, if one TSST consists of only one PTS, it has a set of transition in unique path (STU), which helps to quickly determine whether or not to include an incorrectly implemented transition in the test sequence.

Referring now to FIG. 2A, a flow diagram illustrating the protocol conformance test procedure of the present invention, which solves the problems of prior art protocol conformance test methods as mentioned above, is shown.

First, the protocol conformance test procedure of the present invention begins with a step 210 of deriving a test suite for an abstract test for an IUT and setting up a dynamic environment.

In a next step 220, a test sequence tree composed of test sequences is generated by the test sequence tree generation module 400, and a test path to be tested is selected with reference to the test sequence tree.

Thereafter, the test module 500 applies the test sequence to the test protocol along the selected test path and tests the test sequence (step 230), and performs a determination on whether there is a local defect for each transition of the test sequence (step 240). .

At this time, if a determination result of a failure (fail), the process proceeds to step 220, and the above-described test and determination process is repeatedly performed for the next test sequence, this repeated test and determination is repeated for all the tests of the test target protocol Repeated for the sequence.

At the end of the test for the last test sequence, an overall decision is made on the protocol under test as a test result.

FIG. 2B is a schematic block diagram of a protocol conformance test apparatus according to the present invention suitable for performing the protocol conformance test procedure illustrated in FIG. 2A. The apparatus of the present invention includes an input processing module 100, a first sub test sequence generation module 200, a second sub test sequence generation module 300, a test sequence tree generation module 400, and a test module 500. do.

The input processing module 100 separates any protocol to be converted as a set of transitions into transition center information and state center information. The transition center information and the state center information separated by the input processing module may be a test sequence tree (hereinafter referred to as TST) in the form of a linked list having a data structure representing a set of test sequences. 3) are stored in the transition storage database 110 and the state storage database 120, respectively.

The first sub-test sequence generation module 200 may refer to the tree information processed by the input processing module 100 and, with respect to the preamble that is the path to the transition to the test object, may be possible except for a loop including the initial state more than once. Generate a plurality of preamble subsequences containing all paths.

Similarly, the second sub test sequence generation module 300 generates a plurality of postamble subsequences in consideration of all other possible paths for the postamble for verifying the arrival state of the transition to be tested. In the second sub-test sequence generation module 300, a plurality of UIOs having a maximum length of 2, that is, multiple UIOs, are generated in consideration of unique input outputs (UIOs) among various postamble generation methods.

The test sequence tree generation module 400 collects the preamble and postamble subtest sequences generated by the first and second subtest sequence generation modules 200 and 300 to form a test sequence tree.

The test module 500 applies and tests each test sequence constituting the test sequence tree in the test sequence tree generation module 400 to the IUT, and determines, as an output, whether there is a defect for each transition of the IUT.

The test method according to the present invention performed in the test module 500 repeatedly determines whether there is a defect for one transition, and if the local verdict of any transition is a failure, a defect in the TSST. And eliminates the need to test by removing the PTSs that contain this transition. When the test of any one transition in the test module 500 ends, the test module 500 repeatedly requests the test sequence tree generation module 400 for a new next PTS to be tested to determine whether each transition is defective. Determine.

As described above, the present invention uses a TST including a plurality of sequences in consideration of all possible paths, rather than using a single fixed sequence of the preamble portion and the postamble portion as in the prior art. By dynamically selecting and testing a test sequence, more accurate diagnostic results can be generated for the IUT.

3 is a diagram illustrating a process of configuring and testing the TST of the present invention with an arbitrary protocol finite state machine (FSM).

FIG. 3A is a diagram illustrating an arbitrary protocol FSM, and FIG. 3B is a representation of a multiple postamble set (MPS) for FIG. 3A.

FIG. 3C is a TST considering another path of the preamble with respect to FIG. 3A, and FIG. 3D is a diagram illustrating managing a TST when a local determination of "fail" is received during a test by applying a test sequence. The TST is dynamically reconfigured by removing the PTSs including the received transition. As can be seen in FIG. 3E, the TST is composed of a set of a plurality of subtest sequences (TSST). In addition, the TSST is a set of path test sequences (PTSs) that one transition can have, and includes a sequence that considers different paths in the preamble part and a sequence that considers other paths in the postamble part. It will be included.

The term illustrated in FIG. 3 is defined as follows.

Set of Transition (ST) is a set of transitions, and refers to a set of all transitions in the FSM M as represented by Equation 1 below, where t _i denotes the i-th transition.

(t _i = 〈a head state, an input/output, a tail state〉, n = total number of transitions in Machine M)

Unique path (UP) is the only unique path existing from the initial state to t _i , and is represented by Equation 2 below.

In Equation 2, @ means to combine the heat in the concatenation (concatenation).

Set of Transition in UP (STU) is a set of UP, and STU _i is a set of all transitions of UP _i as shown in Equation 3 below.

PTS (Path Test Sequence) is a path test sequence, which is a test sequence for transition t _i . PTS _i ^q is defined as in Equation 4 below, and is generated in the following order.

(1) Bring the IUT from the initial state to the starting state of t _i .

(2) Apply t _i .

(3) Apply DS, UIO, W to verify arrival status of t _i .

In Equation 4, Path _i ^q represents a sequence of transitions of the q th path from the initial state of t _i .

A tree sub-sequence tree (TSST) is a sub test sequence tree, and TSST _i is a set of all PTS _i of t _i as shown in Equation 5 below.

In Equation 5, j represents the number of all possible paths in the initial state of t _i .

TST (Test Sequence Tree) is a test sequence tree, TST is a structure representing all the test sequences for the FSM M as a tree, expressed as shown in Equation 6 below.

MPS (Multiple Postamble Set) is a set of a plurality of postambles, that is, MPS is a set of sequences constituting a postamble that can have a certain state, if the element is UIO, multiple UIO if DS, multiple DS, W multiple It is W and is represented by the following formula (7).

In the above equation, MPS _i refers to the MPS for the i-th state, and a _j refers to the j-th postamble sequence. In addition, n represents the number of postamble sequences that can exist in one state.

Figure 4 illustrates a configuration for contrasting the defect range with the prior art using the dynamic conformance test apparatus of the present invention shown in Figure 2, the defect machine generation module 600 is added, the test sequence tree generation module As 400 is the same as that shown and described in FIG. 2A except for generating the test sequence according to the present invention as described above, and also generating the test sequence according to the prior art, a detailed description thereof is omitted. do.

4, the fault machine generation module 600 generates a machine with any fault based on the protocol under test, and allows the user to select the number of fault transitions of the machine to be tested, thereby selecting the fault machine selected. To the next test module 500. In addition, the test sequence tree generation module 400 selectively provides a test sequence generated according to the present invention and a test sequence generated according to the prior art to the test module 500.

Thus, the test module 500 has the present invention and the prior art test sequences generated by the defect machine generated by the defect machine generation module 600 and the test sequence tree generation module 400, and the method of the present invention and the prior art. Tests are based on algorithms by method, and their output yields a range of errors for each method.

FIG. 5A is a diagram illustrating an FSM of Transmission Control Protocol (TCP), which is an actual protocol to be tested in the dynamic conformance test apparatus of the present invention shown in FIG. 2, and FIG. 5B is a string of the TCP FSM of FIG. 5A for convenience of testing. The figure illustrates the modification by replacing the value with a natural number.

FIG. 5C is a table comparing the results of an ideal test apparatus using a defect machine, and FIG. 5D is a graph of the test results of FIG. 5C. As can be seen from FIG. 5C and FIG. 5D, it can be seen that the dynamic conformance test method of the present invention yields better results than the prior art method in terms of defect coverage. In particular, the IUT with a few poorly implemented transitions shows higher error coverage than the conventional conformance test method. Therefore, the present invention can be very helpful when considering that there is a high probability that there are a few defects in protocol implementations implemented by real vendors. Could be.

While preferred embodiments of the invention have been described, the invention should not be construed as limiting the scope of the invention. It is to be understood that the present invention may have various forms of embodiments within the scope of the following claims.

Therefore, since the dynamic conformance test method of the present invention can have a plurality of PTSs as its subtest sequences, it is possible to obtain relatively accurate diagnosis results as compared to the method of using only one fixed subtest sequence as in the conventional conformance test method. Can be derived. Therefore, since the result shows an improved result in terms of error coverage compared to the protocol conformance test method of the prior art, it is possible to derive a more accurate diagnosis result for the IUT.

Claims (5)

In the protocol conformance test method, The test sequence tree is constructed by collecting all preamble subtest sequences of all possible paths from the finite state machine (FSM) of the protocol under test as a set of transitions and all postamble subtest sequences verifying the arrival state of the subject transition. Constructing; And repeatedly applying each test sequence of the generated test sequence tree to the protocol to determine whether there is a defect for each transition of the test target protocol. The method of claim 1, wherein the method is And if it is determined in the defect determination step that any transition is a defect, removing the test sequence including the transition of the defect and reconstructing the test sequence tree. In the protocol conformance test apparatus, An input processing module for dividing a test object protocol having a set form of an input transition into a tree in the form of a linked list by dividing the transition-oriented information and the state-oriented information; A first preamble subsequence for generating a plurality of preamble subsequences including all other possible paths except for a loop including an initial state two or more times with respect to a preamble that is a path to a transition under test with reference to the transition information and state information; A subsequence generation module; A second sub-test sequence generation module generating a plurality of post-amble subsequences in consideration of all other possible paths with respect to the postamble for verifying the arrival state of the transition to be tested by referring to the transition information and state information; A test sequence tree generation module for constructing a test sequence tree by collecting preamble and postamble subtest sequences generated by the first and second subtest sequence generation modules; And a test module for repeatedly applying each test sequence of the test sequence tree to the test target protocol to determine whether there is a defect for each of the transitions. The method of claim 3, wherein the second sub-test sequence generation module generates a plurality of UIOs having a maximum length of 2 in consideration of only a unique input output (UIO). A computer-readable recording medium having recorded thereon a program for realizing the function described in claim 1.