JPH06205083A - Specification describing system, preparing system, and analyzing system for test data - Google Patents

Abstract

PURPOSE:To provide a method to prepare test data from specification in which only an item to be tested is described and a method to inspect only the item to be tested in the analysis of the test data by performing only specification description on the item to be tested in the specification description of the test data used in the test of communication software. CONSTITUTION:Specification descriptive language describable a setting condition at every parameter is introduced, and the specification description of the test data is performed by using the language. The preparation of the test data is performed by a tdgen based on the specification description. Also, the analysis of the test data is performed by converting received data into the specification descriptive language by a tdgen, and inspecting whether or not the item to be tested is satisfied. Thereby, the description of test data description can be suppressed to a minimum. Also, ambiguity between the item to be tested and the item other than that can be eliminated. Issuance data can be generated by the specification description of only the item to be tested, and the analysis of reception data for only the item to be tested can be performed.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a specification description system of test data to be applied in a test to be applied to communication software, a test data creating system, and an analyzing system.

[0002]

2. Description of the Related Art Generally, a test for communication software is performed by a black box test (functional test) method. In this method, the test result is judged by monitoring the test data exchanged between the program to be tested and the tester (test system). At this time, the test data used for the test can be classified into two types. One is test data (issued data) given to the test target program by the tester, and the other is test data (received data) expected to be received by the tester from the test target program.

FIG. 11 shows a general test embodiment for communication software. 13 in the figure
Is a tester (hereinafter referred to as a test system, also referred to as LT), and 14 is a program to be tested (hereinafter also referred to as IUT). Reference numeral 15 is test data (issued data), and 16 is test data (received data).

For example, as a language for describing data of the OSI application layer protocol, ISO is an international standard ISO8.
824 "Abstract Syntax No. ASN.1 (Abstract Syntax No.
tation 1) ”. FIG. 12 shows ISO882.
4 in ASN. ASN.1 Type Notation
Test data (international standard ISO / DIS1002
6 is an example in which a data structure of PDU of “OSI distributed transaction processing” is defined.

Although a detailed description of FIG. 12 is omitted here, its simple contents will be described. In the figure, TP
The data structure -BEGIN-DIALOGUE-RI has a SEQUENCE of CHOICE and C
HOICE is channel SEQUENCE
And SEQUENCE, which is called dialogue.
Further, the channel and the dialog are shown to be composed of the elements shown in the figure, respectively.

In executing a test on communication software, a specification description method of test data applied to the test, a method of creating test data (issued data) to be given to a test object,
The problem is how to analyze the test data (received data) received from the test object. FIG. 13 shows a conventional ISO88 for the data structure defined in FIG.
24 in ASN. ASN.1 Value Notatio
This is an example in which the specifications of test data are described using n). In the figure, 0, 3, 2, and {circle around (1)} 0B indicate cases where each constituent element of SEQUENCE, which is dialogue, specifically takes these values. FIG. 14 shows the ASN. It is a flowchart which describes the specification of test data using a one-valued notation and creates test data (issued data). FIG. 15 shows the ASN. It is a flowchart which describes the specification of test data using a one-valued notation, and analyzes test data (received data).

Next, a conventional test data specification description method will be described. The test data is ASN. It is created by setting concrete values for the respective parameters of the data structure defined using the type 1 notation. Conventional ASN. In the specification description method using the one-value notation, all parameter values for the item under test and other items are determined and the specification of the test data is described. At this time, in the case where the parameter value of the item under test can take a plurality of values, it is necessary to determine to a specific value among them, and to set some parameter values other than the item to be tested.

That is, in the conventional test data creating method, all parameter values are determined at the stage of describing the specifications of the test data, and the parameter values are set to the ASN. What was described using the one-valued notation was used as test data.

Further, in the conventional test data analysis method,
All the parameter values for the test data expected to be received are clarified, and this is set to ASN. Describe as test data specifications using one-value notation. Then, the received test data is subjected to a simple coincidence test with the prepared test data specification to determine the test result.

[0010]

Generally, in a test for communication software, not all parameters included in the test data are items to be tested. Further, the parameter value is not limited to a specific value and may be any of a plurality of values. In the conventional test data specification description method, the values of all parameters have to be described. Therefore, in order to carry out the test, it was necessary to describe up to parameter values other than the item under test and to determine a specific value from the parameter values of the item under test that can take a plurality of values. That is, the test data was created by describing the data outside the range of the original test data specifications. Also, in the analysis of the test data, it was necessary to clarify all the parameter values including those other than the item under test and then inspect the entire test data. Furthermore, since such specifications are described, it is not possible to unambiguously describe the tested items and other items in the test data specifications.

In addition, according to the complicated protocol of the application layer, the behavior of the IUT that cannot be controlled by the UT (upper tester) or the LT (not shown) causes test data corresponding to the behavior of the LT and its test data. May require parameters. For example, in the test scenario of OSI transaction processing, in the case of a test scenario in which a dialog is established from the IUT, the dialog ID is determined inside the IUT protocol machine and cannot be controlled externally. The value of this dialog ID must also be used in the test data issued from LT to IUT. Therefore, LT to I
As for the data issued to the UT, it is impossible to create all the test data in advance. In order to deal with such a complicated test scenario, it is necessary to install a module that performs special processing inside the tester (UT, LT) each time the test object changes, which increases the amount of work required for tester development. Has become.

The present invention has been made to solve the above problems, and has an object to describe only the specifications of the test data with respect to the test item, and further to describe the test data including only the test item. A system for creating test data from specifications, and a system for inspecting only items under test in analysis of test data in a test are provided.

Also, depending on the operation result of the test object,
The purpose is to provide a test system that enables testing even when test data cannot be created in advance.

[0014]

In the first invention,
When describing the specifications of test data, a specification description language that allows specification of each setting condition for each parameter included in the test data is introduced instead of the parameter values of the entire test data.

Further, in the second invention, the specification of the test data describes only the item to be tested using this specification description language, and the lack information is appropriately supplemented to the specification of the test data. Determine the parameter value and create test data (issue data).

Further, in the third invention, the test data (reception data) is once converted into a specification description language expressing the setting condition for each parameter, and satisfies the setting condition of the test data specification described in advance. Analyze by checking whether or not there is.

Further, in the fourth invention, the data necessary for the next test is extracted from the test data (received data) so that the test data (issued data) can be created.

[0018]

As described above, in the test data specification description system according to the first aspect of the present invention, by introducing the specification description language capable of describing the setting condition for each parameter, the test data specification for only the item under test in the test is introduced. Can be described. For items under test that can take multiple values,
It is possible to describe specifications that are not limited to specific values.

In the test data creating system of the second aspect of the present invention, a parameter setting condition specifies a specific value, and if the setting condition allows a plurality of values, the value is set. It is possible to create test data (issued data) from the minimum required test data specifications by setting any of the above values and any of all possible values if the setting conditions do not exist. Become.

Further, in the analysis of the test data of the third invention, the test data (reception data) specifications describing only the item under test using the specification description language described above are prepared in advance. Then, the test data received by the tester from the program to be tested is converted into a specification description language that expresses the setting conditions for each parameter, compared with the prepared test data (reception data) specifications, and the specification description obtained by converting the reception data. The inspection is performed on the item under test by inspecting that is included in the test data (reception data) specifications prepared in advance.

In the test data specification description system according to the fourth aspect of the present invention, the data necessary for creating the next test data (issued data) can be extracted from the test data (received data). Changing parameters can also be set automatically.

[0022]

EXAMPLES Example 1. (1) Description of Test Data Specification Description System According to First Invention FIG. 1 shows an embodiment of a test data specification description method according to the present invention. ASN. It is possible to describe all parameter setting conditions for a PDU whose data structure is defined by the type 1 notation, "PSL.1 (Parameter Specification Language f
or ASN.1) ”is introduced. ASN.
For a PDU whose data structure is defined using 1, nondeterminism that requires manual intervention when setting parameters can be classified into the following four types. (1) Parameter value of a certain type (2) Parameter setting / omission for OPTIONAL type (3) Type represented by permutation / set of single type (SequenceOf
Number of elements of type, SetOf type) (4) Selection of one element from multiple elements (Choice type) (1) The answer to the nondeterminism is (a) A specific value should be taken (b) A certain value set Any value included in (c) Any value may be taken. Regarding the non-determinism of (2), either (a) must be set, (b) must be omitted, or (c) either is acceptable. The nondeterminism of (3) is equal to (1) for positive integer values, and the nondeterminism of (4) is equal to (1) for values of a finite set (Enumerated type). .

PSL. 1 is the ASN.1 when creating test data from the test series. It is a language for describing answers to all non-determinism that occurs in one parameter setting. The identification of each parameter of the PDU is specified using the path format. For example, if the value of value should be set as the parameter of the path of type type, describe as type (path) = value ;. Using this syntax, the setting conditions are described for the parameters belonging to (a) with respect to the nondeterminism of the description (1). In addition, PSL. In 1, an inequality sign may be used instead of the equal sign. Accordingly, the enum (diagnostic)> = 1; enum (diagnostic) <= 3; To explicitly specify the omission of parameters for (2) nondeterminism, describe as bit (selected-FUs) = OPT ;. For the specification of the number of elements in (3), describe as seqof (user-data) = 3 ;. For selection of the element in (4), describe as choice (tp-bgn-dlg) ;. In addition, for a parameter that is expressed as a character string type such as PrintableString type or an octet string of OctetString type, create the parameter to be set in another file, and printable (text) <■ doc ■; It is possible to refer to. The example in Figure 1
13 is a specification description of test data for the data structure defined in FIG. As described above, PSL. In 1, the setting condition for each parameter is expressed by using equality or inequality. The test data is created and analyzed based on this specification description method.

Next, the operation of the first embodiment will be described. At the stage of describing the specifications of the test data, PSL. The condition is described using 1. In FIG. 1, ri-bgn-dlg is shown in FIG.
It means TP-BEGIN-DIALOGUE-RI. First, select TP-BEGIN-DIALOGUE-RI in the first line, and select it in the second line.
It indicates to select dialogue. The third line is a dialog ID (ri-bgn-dlg / elem / dialogue) that is an integer type value.
/ dig-id) must be 0, 4,
The fifth and sixth lines also indicate that the respective values must be 3, 2, and 0, and the selected functional unit (ri-bgn-dlg / elem / dialogue / slcted-) which is a bit string type value on the seventh line. FUs) must be ■ 0110 ■. The 8th line is the invocation data (ri-bgn-dlg / elem / dialog
ue / inv-data) must be omitted.

In the example of FIG. 1, the one shown in FIG. Although the case of rewriting by 1 is shown, an inequality sign may be used as shown in FIG. The third line in FIG. 2 indicates that the dialog ID may be 0 or more.
Further, as shown in FIG. 3, the fourth line from FIG. 1 may be deleted. In the case of Figure 3, recipient-tpsu-tit
The value of le (rcp-tpsu-ttl) can be arbitrary, meaning that it has nothing to do with this test. As described above, only the parameter setting conditions that must be observed in the test are PSL. It is described using 1. When there can be a plurality of values to be set for the same parameter, they are expressed using inequalities as shown in FIG. 2, and when they can take arbitrary values, nothing is set as shown in FIG. Is not described.

FIG. 4 shows the PSL. 6 is a flowchart for creating test data (issued data) by describing the specification of test data using 1. In the figure,
1. Clarify the parameter range of the item under test. At this time, if a plurality of values can be taken, only the range thereof needs to be clarified, and it is not necessary to determine a single value. This test specification is PSL. It is described using 1. PS in 3
L. The value of the parameter for which the setting condition is not described in 1 or only the range is specified is specifically determined. This process is described, for example, in "The Art" by Glenford J. Myers.
equivalence partitioning method (equiv
alence partitioning) and boundary valuation method (boundary valuation)
By using e analysis), it can be automatically performed by a computer or the like. The creation of the test data to be issued to the test object is completed by determining all the parameter values. In 4, the test data is issued to the test object.

FIG. 5 shows the PSL. 3 is a flowchart for describing the specification of the test data using 1 and analyzing the test data (received data). In the figure,
The parameter range of the item under test is clarified in 5, and this specification is set in PSL. Described in 1. After receiving the test data from the test object at 7, the received data is sent to PSL. Convert to 1 syntax. By this conversion, all the parameter setting conditions included in the test data (received data) are set to P
SL. The format described using 1 is obtained. 6 in 9
In advance with PSL. Each setting condition of the item under test described using 1 is the PSL. 1
Check that the syntax is satisfied. Based on this inspection result, it becomes possible to determine in 10 whether the test data is correct.

Next, ASN. A specific example of one test data generation / analysis tool will be described. As shown in FIG. 6, this tool is composed of two programs, a test data generation tool tdgen and a test data analysis tool tdana. Each program can be written in C language, and ASN.
For the encoding / combining process of No. 1, the codec library of the existing system can be used.

(2) Description of Test Data Generation System According to Second Invention The test data generation tool tdgen is a test data (PDU).
Of the ASN. 1 Data type definition table (A
SN. 1 pre-combiner output) and PSL. One description is input, and parameters are determined by capturing the lacking information interactively or automatically with the user, and encoded test data (PD
U) is a tool for creating. The setting parameters are (1) PSL. Setting conditions described in 1. (2) Interactive input with the user (3) Non-decision solving algorithm Determine according to the priority order. That is, PSL. If parameter setting conditions and the like (FIGS. 1, 2, and 3) are described using 1, the designation is followed. If the description does not exist, for example, as shown in FIG. 7, an undetermined solution (input of a value, selection, etc.) is requested to the user in the interactive mode, and if the solution is rejected (only the line feed key is input), the predetermined Parameters are automatically determined according to the non-deterministic resolution algorithm.
Further, in the fully automatic mode, as shown in FIG. 8, the interactive input with the user regarding the nondeterminism is omitted. That is, P
SL. Automatic generation of test data is performed by applying the non-deterministic solution algorithm to all non-deterministic ones in which there is no description. In addition, tdgen returns the answer to the nondeterminism obtained by the interaction with the user or the automatic solution to the PSL. It has the function of outputting in the syntax format of 1. This PSL. 1 expression is saved in a file by using the redirection function of the OS, so that PSL. 1 description can be created. In addition, ASN. 1 definition table and PSL. It has the function to execute the analysis of 1 description. When the parameters are automatically determined, they are determined using a nondeterministic solution algorithm that applies the equivalence division method and the boundary value analysis method. For parameters that represent time, etc., implement a special module to select an appropriate value.

(3) Description of Test Data Analysis System According to Third Invention Next, the operation of the test data analysis tool tdana will be described. The test data analysis tool tdana, as shown in FIG. This is a tool that inputs 1 data type definition table and test data (PDU), decodes it, and outputs parameter analysis information. At this time, the analysis information (parameter value, etc.) is set to PSL. It has a function to output in the format of 1 syntax. By combining tdgen and tdana in one expression, it is possible to create test data according to the operation of the test target and execute the test only for the items that need to be inspected. FIG. 9 is a diagram showing a specific operation of tdana.
UT outputs Test Data 91, and tdana outputs it as PSL. It shows an example output in the format 93 of one syntax. P expected to be received by LT from IUT
Not all parameters of the DU are items to be inspected. For example, the dialog ID at the time of dialog probability from the IUT side, and the diagnostic parameter at the time of failure are:
It cannot be predicted on the tester side. Ideally, it is desirable to decode the received PDU and inspect only the necessary items, but this requires a dedicated inspection module in the LT for all the received data. In traditional testing,
For parameter values that are undefined by convention (implementer-dependent),
We take measures such as obtaining the information about the internal operation from the developer who created the program, clarifying the expected values of all the parameters, and then preparing the test data that simply requires a match test. When such a test scenario is created, the distinction between the items to be inspected in the test and the values that may be arbitrary values becomes ambiguous. Therefore, the PSL. Prepare in 1 syntax format. Instead of checking that the received PDU matches the prepared PDU, the PSL. Inverse conversion to 1 representation and PS
L. Compare each expression of one syntax and check that all requirements of test data are included in PDU. That is,
Conventionally, Test Data 92 is prepared in advance as an expected result, and Test Data 92 and Test Data 92 are prepared.
According to the present embodiment, only the items to be tested are PSL. 1 description 94 is prepared in advance, and this and P output by tdana
SL. It is sufficient to compare one expression 93. For example, as shown in the figure, the value of rcp-tpsu-title is 3 and int-
Only if the value of tpsu-title is 2 is compared, and if this comparison is satisfied, the test result is normal.

(4) Description of Test Data Specification Description System According to Fourth Invention FIG. 10 is a diagram for explaining the fourth invention. For example, if a dialog is established from the IUT, the LT will
TB-BEGEN which saves the dialog ID included in TB-BEGEN-DIALOGUE-RI received from T and issues it from LT to IUT
-It is necessary to set the same value to DIALOGUE-RC.
In the conventional test, the test data in which the value of the dialog ID is set to 0 is created, and the module that directly sets the value of the received dialog ID in the coding portion is mounted inside the LT to cope with the problem. . In this method, it is necessary to create a dedicated processing module inside the tester every time the number of items of reuse of analysis information increases, which reduces the versatility of the tester. Therefore, PSL. A necessary condition extraction module (hereinafter, also referred to as CSM) that extracts an expression required for creating test data from the test data analysis information expressed in one syntax is created. PPSL. 1 can be described by dividing it into a plurality of files, and the parameter setting conditions that do not change due to the operation of the IUT are PSL. 1
Create it in the description. Then, the received data is converted into PSL. 1
A part of the expression that has been inversely transformed into the expression and the prepared PSL. By combining 1 description and inputting it to tdgen, test data is created immediately according to the behavior of the test object.

Example 2. The above-described first embodiment is an OS
This is an example in which the test data for the PDU of the I application layer is implemented, but the test data for the OSI application layer service primitive can also have the specification description of the test data using the specification description language capable of describing the parameter setting conditions. At this time, the same test data generation method and analysis method as in Example 1 can be applied.

Example 3. Further, in the above description, the application example in the test for the OSI application layer communication software has been described, but it goes without saying that it can be applied in the test for other software.

[0034]

According to the first aspect of the invention as described above, it is possible to make a specification description for an item under test in a specification description of test data for communication software or the like.
Therefore, it is possible to minimize the work required to describe the specification of the test data, and it is possible to describe the specification in which the item under test and the other items are unambiguous.

According to the second aspect of the invention, the test data (issued data) is created by automatically determining the value of each parameter from the specifications of the test data describing the item under test. Is possible.

According to the third aspect of the invention, in the analysis of the test data (reception data), the test data of only the tested item can be obtained without clarifying all the parameter values including the parameters other than the tested item in advance. It becomes possible to inspect only the item under test by describing the specifications of.

Further, according to the fourth aspect of the present invention, since the data necessary for creating the next test data (issued data) can be extracted from the test data (received data), the parameters that change depending on the operation of the test object are automatically generated. Can be set manually.

[Brief description of drawings]

FIG. 1 is a PSL. 3 is a diagram showing an example of specification description of test data using 1. FIG.

FIG. 2 is a PSL. 3 is a diagram showing an example of specification description of test data using 1. FIG.

FIG. 3 is a PSL. 3 is a diagram showing an example of specification description of test data using 1. FIG.

FIG. 4 is a flowchart of a test data creating method using the specification description method according to the present invention.

FIG. 5 is a flowchart of a test data analysis method using the specification description method according to the present invention.

FIG. 6 is a diagram showing a test data generation / analysis tool using the specification description method according to the present invention.

FIG. 7 is a diagram showing an example of parameter determination of a test data generation tool using the specification description method according to the present invention.

FIG. 8 is a diagram showing an example of parameter determination of a test data generation tool using the specification description method according to the present invention.

FIG. 9 is an operation explanatory diagram of a test data analysis tool using the specification description method according to the present invention.

FIG. 10 is a diagram for explaining a necessary condition extraction module for test data using the specification description method according to the present invention.

FIG. 11 is an example diagram of a test embodiment for general communication software.

FIG. 12: International standard ASN. PD using type 1 notation
It is a definition example figure of U data structure.

FIG. 13 shows a conventional ASN. It is an example figure of one specification description of the test data using a one-value notation.

FIG. 14 is a flowchart of a test data creation method using a conventional specification description method.

FIG. 15 is a flowchart of a test data analysis method using a conventional specification description method.

[Explanation of symbols]

 13 tester (test system) 14 test target program 15 test data (issued data) 16 test data (received data)

Claims (4)

[Claims] 1. A test data specification description system capable of expressing specifications for test items by describing specifications of test data used for a test in the form of setting conditions for each parameter. 2. A test data creation system for creating test data specifications by using the specification description system according to claim 1 and automatically creating test data to be transmitted to a test target. 3. A specification description of test data is performed using the specification description system according to claim 1, and the test data received from the test target is converted into the specification description according to claim 1, thereby inspecting the item under test. An analysis system for test data. 4. The specification of the test data used for the test is described in the form of the setting condition of each parameter, so that the specification for the item under test can be expressed and the test data received from the test object can be used for the test. A test data specification description system that can express the data required for data creation.