JP2017220008A - Test supporting device and test supporting method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a test supporting device and a test supporting method which generate a test case necessary for running of a test by analyzing an input pattern from input information of a test subject in order to reduce labor for a person who runs the test.SOLUTION: In a test supporting device 100, an analysis part 111 analyzes the definition of form information stored in a form information definition storage part 105 and an outside specification stored in an outside specification storage part 101, and generates analyzed parameter information. A conversion part 112 converts the analyzed parameter information with reference to the type information definition and generates test case generation information. A test case generation unit 113 generates a test case with reference to the type information definition. A test case generation part 114 generates test data from the test case and the test case generation information.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a test support apparatus and a test support method, and more particularly, to a test support apparatus and a test support method suitable for generating a test case from an external specification and reducing labor in software testing. .

  In software development, a test for confirming whether the developed software operates as expected has an important significance for ensuring the quality of the software. When conducting tests, it is common to analyze the specifications created during the software design stage and perform test design to create specific input data and corresponding expected results for performing the tests. is there. In such a test design, it is necessary to analyze a design specification in the test design, extract relevant input information without omission, and generate a test case for testing.

  Therefore, for example, in Patent Document 1, when generating a test case, the items described in the design specification and the test specification and the correspondence between the items of the design specification and the test specification should be determined in advance. Discloses a method for preventing omission of extraction from a design specification in a test design stage.

  Further, since Patent Document 2 outputs a value that covers the value output in the specification from the software specification, whether or not the value output in the specification is included in an expected output value that is expected to be output as a result of the input. And a technique for generating a test output value and a test input value corresponding to the output value in accordance with the specifications.

JP 2013-77173 A JP 2014-186407 A

  When conducting a test, the software developer generally analyzes the input software (parameters) related to the software to be tested, or the configuration module and function, and creates test data. It is difficult to test a combination of data. Therefore, it is necessary to perform appropriate classification after analyzing the patterns that each parameter can take and considering the relationship between the parameters. Moreover, not all information necessary for the test is necessarily described in the design specification. However, Patent Document 1 and Patent Document 2 do not disclose how to deal with the necessity of additional input at the test design stage, or how to perform parameter analysis.

  The present invention has been made to solve the above-mentioned problems. The design specification is analyzed, information necessary for the test design stage is extracted, and input from the test target input information obtained from the design specification is input. An object of the present invention is to provide a test support apparatus and a test support method capable of analyzing a pattern, generating a test case necessary for test execution, and reducing the labor of a tester.

  In order to solve the above problems, a test support apparatus according to the present invention includes a CPU, a storage unit, and a display unit, and implements a function by executing a program stored in the storage unit. The storage unit stores external specifications, analyzed parameter information, test case generation information, and test cases, and the external specifications and analyzed parameter information are parameters described by the software program to be tested. The parameter information of the parameter and the parameter information of the external parameter and analyzed parameter information includes the type information of the parameter and the pattern information that the parameter name can take. Test case generation information includes parameter information of parameters described in the software program to be tested. The parameter information of the test case generation information includes type information of the parameter and possible values that the parameter name can take, and the test case is a test target One or more combinations that define parameter values of parameters described in the software program are held. Then, parameter classification is performed from parameter type information, pattern information, and constraint conditions of external specification parameter information, and analyzed parameter information is generated. Parameter type information and pattern information of parameter information of analyzed parameter information , Classify parameters from constraint conditions, generate test case generation information, determine parameter values from parameter type information of parameter information of test case generation information, constraint conditions, and candidate values of parameter information Generated test combinations.

  According to the present invention, the design specification is analyzed to extract information necessary in the test design stage, and the input pattern is analyzed from the input information of the test target obtained from the design specification so as to be necessary for the test execution. It is possible to provide a test support apparatus and a test support method that can generate a test case and reduce labor of a tester.

It is a figure which shows the function structure and data flow of a test assistance apparatus. It is a hardware block diagram of a test support apparatus. It is a schematic diagram which shows the model of a to-be-tested program. 4 is a flowchart showing processing from input of external specifications to test case generation by the test support apparatus 100. It is a figure which shows an example of an external specification. It is a figure which shows an example of analyzed parameter information. It is a figure which shows the complementary input screen of analyzed parameter information. It is a figure which shows an example of test case production | generation information. It is a figure which shows an example of a test case. It is a flowchart which shows the detailed process of external specification analysis process S302. It is a flowchart which shows the detailed process of test case generation information generation process S304. It is a figure which shows the example which produces | generates test case production | generation information from analyzed parameter information. It is a flowchart which shows the detailed process of test case generation information generation process S305. It is the figure which described the outline of the data structure of the external specification 400, the analyzed parameter information 500, and the test case production | generation information 600 in the class diagram like UML (Unified Modeling Language). It is a figure showing basic type information definition and individual type information definition as definition information required for test case generation processing. It is a flowchart explaining the process from the produced | generated test case to test execution.

  A test support apparatus according to an embodiment of the present invention will be described below with reference to FIGS.

First, the configuration of the test support apparatus will be described with reference to FIGS. 1 and 2.
FIG. 1 is a diagram illustrating a functional configuration and a data flow of the test support apparatus.
FIG. 2 is a hardware configuration diagram of the test support apparatus.

As shown in FIG. 1, the test support apparatus 100 includes an external specification storage unit 101 for storing each data, an analyzed parameter information storage unit 102, a test case generation information storage unit 103, a test case storage unit 104, and type information. From the definition storage unit 105, the test data storage unit 106, the storage unit of the execution result storage unit 107, the analysis unit 111, the conversion unit 112, the test case generation unit 113, the test data generation unit 114, the test execution unit 115, and the user The input / output unit 110 is configured to input the information and display the storage information of each storage unit.
Details of the information stored in each storage unit will be described later.

  The analysis unit 111 is a functional unit that analyzes the definition of the type information stored in the type information definition storage unit 105 and the external specification stored in the external specification storage unit 101 to generate analyzed parameter information. The conversion unit 112 is a functional unit that refers to the type information definition, converts the analyzed parameter information, and generates test case generation information. The test case generation unit 113 is a functional unit that generates a test case with reference to the type information definition. The test data generation unit 114 is a functional unit that generates test data from a test case and test case generation information. The test execution unit 115 is a functional unit that inputs test data, generates execution result data by executing a program to be tested, and stores the data in the execution result storage unit 107.

  As shown in FIG. 2, the test support apparatus 100 can be realized by the structure of a general computer (electronic computer). The test support apparatus 100 has a configuration in which a CPU (Central Processing Unit) 201, a main memory 202, a display I / F (Interface) 203, an input / output I / F 204, and an auxiliary storage I / F 205 are connected by a bus.

  The CPU 201 is a processor that controls each part of the computer and performs arithmetic processing. The CPU 201 loads various programs from the HDD 210, which is an external storage device, into the main memory 202, and executes the predetermined programs to realize the above-described processing units. . The main memory 202 is a storage device composed of a semiconductor storage device, and stores programs to be executed and work data. A display I / F 203 is a part to which a display device 240 such as an LCD (Liquid Crystal Display) is connected. The input / output I / F 204 is a part that connects a keyboard 241, a mouse 242, a printer, and the like to perform interface processing. The auxiliary storage I / F 205 is a part that connects an external storage device such as an HDD (Hard Disk Drive) 210 or an SDD (Solid State Drive) and performs interface processing.

  The HDD 210 stores a test support program 220 and a test support DB (Data Base) 230.

  The test support program 220 includes an input / output program 221, an analysis program 222, a conversion program 223, a test case generation program 224, a test data generation program 225, and a test execution program 226, each of which includes the input / output unit 110 described in FIG. This corresponds to the analysis unit 111, the conversion unit 112, the test case generation unit 113, the test data generation unit 114, and the test execution unit 115.

  The test support DB includes an external specification 400, type information definition (basic type information definition 1300, individual type information definition 1310: described later), analyzed parameter information 500, data case generation information 600, test case 800, test data 900. The execution result 1000 is stored.

  The CPU 201 executes each program of the input / output unit 110, the analysis unit 111, the conversion unit 112, the test case generation unit 113, the test data generation unit 114, and the test execution unit 115 by executing a program loaded in the main memory 202. Is embodied as a process.

  The predetermined program may be stored in advance in an external storage device such as the HDD 210 or may be read out as necessary from a portable external storage medium that can be used by a computer via a reader. . Alternatively, it may be downloaded as necessary from another connected computer via a network that can be used by the computer or a communication device on the network and stored in an external storage device.

Next, before describing specific processing of the test support apparatus, a model of a program under test will be briefly described with reference to FIG.
FIG. 3 is a schematic diagram showing a model of the program under test.

  The program to be tested is a model that is an image of equipment control for railway route control, and is a program that determines whether or not a train can be departed according to railway operation control according to the current position of the line and the state of traffic lights. 1R is a traffic signal to be controlled, and 2R and 3R represent traffic signals that compete with 1R, respectively. ST, 1T,..., 4T are track circuits that represent positions on the standing line. In FIG. 3, 2T, 3T, and 4T indicated as “closed” indicate ranges in which the train can enter when the traffic light 1R permits entry.

  In actual facility control, function processing design and implementation are performed under more detailed conditions. In the following description, the present embodiment will be described using a simple example, but may be applied to a specifically implemented function or processing design.

  In addition to the above equipment control, for example, it may be a module for a business system of finance or bank, or setting information such as a combination of system operating environments (for example, OS (Operation System) or browser). It may be.

Next, the process from the input of the external specification to the test case generation by the test support apparatus 100 will be described with reference to FIG.
FIG. 4 is a flowchart showing processing from the input of the external specification to the test case generation by the test support apparatus 100.

  First, the test support apparatus 100 receives the user input 311 and stores the external specification as an external specification in the external specification storage unit 101 (S301). The input of the external specification may be performed in S301 using a dedicated design tool, for example, or a design document created externally may be imported by, for example, file input.

  Next, the analysis unit 111 of the test support apparatus 100 performs an external specification analysis process (S302), presents the analyzed parameter information analyzed on the analyzed parameter information supplement screen (described later), and receives the user input 312. Receive. Thereby, the supplemented analyzed parameter information is stored in the analyzed parameter information storage unit 102 (S303).

  Based on this, the conversion unit 112 of the test support apparatus 100 generates test case generation information (S304). The test case generation information is stored in the test case generation information storage unit 103.

  Next, the test case generation unit 113 of the test support apparatus 100 generates a test case from the test case generation information (S305). The generated test case is stored in the test case storage unit 104.

Next, the data structure handled by the test support apparatus will be described with reference to FIGS.
FIG. 5 is a diagram illustrating an example of external specifications.
FIG. 6 is a diagram illustrating an example of analyzed parameter information.
FIG. 7 is a diagram showing a complementary input screen for analyzed parameter information.
FIG. 8 is a diagram illustrating an example of the test case generation information.
FIG. 9 is a diagram illustrating an example of a test case.

  The external specification 400 is data that represents the external specification of the software with parameters and constraint conditions relating to the parameters. As shown in FIG. 5, the external specification 400 includes an external specification (parameter information) 401 and an external specification (constraint condition) 410. The

  An external specification (parameter information) 401 is a table for describing information on parameters related to software configuration modules and functions to be designed, and includes an ID column 402, a parameter name column 403, a type column 404, an IO column 405, It consists of an array column 406 and a pattern column 407.

  The ID column 401 is a column indicating parameter identifiers. The parameter name column 403 is a column that describes the name of the parameter. The type column 404 is a column that describes parameter type (type) information. The IO column 405 is a column that indicates whether the parameter is used for input, output, or both. The array column 406 is a column indicating whether or not the parameter is an array structure. The pattern column 407 is a column that describes variations that the parameter can take.

  Furthermore, the external specification (constraint condition) 410 is a table for describing a constraint condition that extends over a plurality of parameters, and includes an ID column 411 and a constraint condition constraint condition column 412 as shown in FIG.

  The ID column 411 is a column indicating an identifier for identifying a constraint condition. The constraint condition column 412 is a column for describing a constraint condition body.

  A simple grammar is defined for the description of the constraint body. A statement that determines execution based on a condition is if (condition) then (execution). Here, if and then are reserved words. The parameter value is represented by $ {parameter name}. In the condition part, logical operations can be performed using reserved words such as =, or, and.

  Further, in the parameter name column 401 of FIG. 5, there are items indented in the parameter column, and this represents a parent-child relationship between parameters. For example, “closed” and “competing traffic signal” are listed in a row one level lower than “monitoring target”. This indicates that “close” and “competing signal” are child parameters of “monitoring target”. Such a parent-child relationship between parameters may be implemented, for example, as a member variable of a structure or a class in program implementation.

  The type column 404 indicates the type of type that each parameter has. The type information indicates what value the parameter can take, and an integer, real number, time, and unique type information determined for each software is input. The definition of the type information will be described later with reference to FIGS. In this embodiment, names such as “monitoring target” and “present line state” are used for the type information. However, types “int” and “int” used in programming languages such as C language and JAVA (registered trademark) language are used. char "or the like may be used.

  The IO column 405 describes whether the parameter acts as input information, output information, or both for the design object. In FIG. 5, “I” (Input: input) and “O” (Output: output) are shown, and when used for both input and output, for example, “I / O” may be described.

  In addition, when information other than IO, such as a parameter storage destination or a reference destination, is given, not only “I”, “O”, etc., but also an identifier corresponding to the information to be assigned may be set. Good. For example, in the case of targeting an implemented function, for example, “(P) I”, “(G) I” is used to distinguish information passed as a function parameter and external (global variable) referenced in the function. May be described so that the type information of the reference destination can be understood in the IO information column.

  The array column 405 is information used to indicate whether or not the parameter is an array type, and an item in which “◯” is described indicates that the parameter is an array. When the length of the array is determined in advance, such as a fixed length array, the value may be written in the array column 405.

  The pattern column 407 describes variations of values that each parameter can take, separated by “,”. For example, in the parameter named “closed”, “missing line, {entering, accommodating}” is described in the pattern column 407. This indicates that the pattern of candidate values that can be taken as “close” can be either “absent line” or “{entering, accommodating}”. Note that “{entering, accommodating}” indicates that one of the candidate values described in “{}” (in this case “entering” and “accommodating”) takes one value. Yes. That is, “closed” takes one of the values of “missing line”, “entering”, and “accommodating” as an input pattern. In the external specification 400, “not present line”, “{entering, accommodating” } ”Indicates that the class is divided into two classes, where“ class ”is an“ equivalent class ”used in the concept of software development and is a parameter that can be regarded as the same output as software operation It is a unity. That is, “entering” and “accommodating” mean that any value does not affect the output during the test.

  “@Another”, “@Valid”, and “@Invalid” described in “Contest signal” and “Column number” respectively represent “other value”, “valid value”, and “invalid value”. It is a reserved word and is used to describe a pattern in a test when it cannot be determined at the design stage of external specifications.

  In addition, in the pattern column 407, “@Null” representing a NULL value in program implementation, “@Length” representing a variation in the length of the array, or “@Any” indicating that an arbitrary value is taken. The predetermined information may be described by defining a reserved word in advance.

  First, “all $ x area R” described in the pattern field 407 of the parameter name 403 of the ID field 3 “competitive traffic signal” has values of all elements of the competitive traffic signal 403 being an array as R (red signal). It means that there is.

  In addition, the parameter name 403 in the ID column 8 can represent a range that can be taken by an equal sign or an inequality sign as in the pattern column 407 of the “remaining time”.

  Furthermore, True and False in the pattern column 407 in which the parameter name 403 in the ID column 8 is “output” are reserved words representing true / false.

  The constraint condition column 412 in the external specification (constraint condition) 410 describes the constraint relationship between parameters. In the example of FIG. 5 (when the ID column 411 is “1”), when the length of the track position of the target train is not “1”, the track time of the target train is “0”.

  Note that “.” Indicates the parent-child relationship between the parameters mentioned in the description of the parameter name field 403. For example, “target train.remaining line position. @ Length” is a child parameter of “target train”. It shows that it is the “length” of a certain “present line position”.

  Next, the analyzed parameter information 500 shown in FIG. 6 is information obtained as a result of analyzing the external specification 400, and is composed of analyzed parameter information (parameter information) 501 and analyzed parameter information (constraint conditions) 510. Is done.

  The analyzed parameter information (parameter information) 501 includes an ID column 502, a parameter name column 503, a type column 504, an IO column 505, an array column 506, and a pattern column 507, like the external specification (parameter information) 401. .

  The analyzed parameter information 500 has a data structure similar to that of the external specification 400, but the test support apparatus extracts information necessary for generating a test case from the external specification 400 as a processing result of the external specification analysis processing S303 in FIG. Then, the parameter information complementing screen shown in FIG. 7 is displayed on the result of the class division of the parameter with insufficient class division, and the user is prompted for the complementary input of the parameter information. On the parameter information completion screen, parameter information is displayed in tabular form, and the user is requested to input by displaying a thick frame in the column that asks the user for completion, or by changing the cell background color or blinking the cell. Prompt.

  In the analyzed parameter information 500 shown in FIG. 6, the part for requesting complementary input of parameter information is displayed with a thick frame.

  For example, in the analyzed parameter information 500 of FIG. 6, a record of the parameter name “@Length” of “3a” and “6a” in the ID column 502 is added to the external specification 400 of FIG. This is because the corresponding parent element's “competing signal” and “present line position” are array information, so it is necessary to specify the length of the array and assign specific data when executing the test. Is encouraged.

  In addition, “current time” in which the ID column 502 is “11” is added to the parameter name. However, according to the external specification (constraint condition) 410, the description of the parameter that does not exist in the parameter name column 407 is $ {current This is because it is described in the constraint condition as “time” (see the constraint condition 412 in which the ID column 411 is “2”).

  Also, for example, when an abstract pattern such as “@Another” (parameter field of “competition signal”) is described or blank (“present line position” etc.), pattern input or a specific candidate value Is necessary, and the input value is reflected in the {} part.

  Then, the test support apparatus 100 executes a test case generation information generation process (S304) on the result of the user inputting a specific value or a data pattern.

  6 is different from the description in the pattern column 407 of the record of “remaining time” in the external specification 400 of FIG. 4 in the pattern column 507 of the record whose parameter of the analyzed parameter information (parameter information) 501 in FIG. ing. This is a result of the classification being supplemented by the external specification analysis process S302 when the parameter classification of the external specification 400 is insufficient.

  In the above description, the input of the specific value is prompted at the stage where the analyzed parameter information 500 is generated (after S302), but the input of the specific value is not essential. In this case, an abstract pattern name (@Valid) is used until the test case generation process of S304. Therefore, an executable specific value may be assigned before test execution (after test case generation).

  Next, the test case generation information 600 shown in FIG. 8 is information obtained by converting the analyzed parameter information 500 supplemented by the user, and includes test case generation information (parameter information) 601 and test cases. It consists of generated information (constraint conditions) 610. The test case generation information 600 is information serving as original data for generating a test case 800 that is the next step.

  The test case generation information 601 (parameter information) includes an ID column 602, a parameter name column 603, a type column 604, an IO column 605, and a candidate value column 606.

  An ID column 602, a parameter name column 603, a type column 604, and an IO column 605 are respectively an ID column 502, a parameter name column 503, a type column 504, an IO column 505, an array column 506 of the analyzed parameter information (parameter information) 501. This corresponds to the pattern field 507. The candidate value column 606 is a column that stores candidate values that the parameter can take based on the analyzed parameter information 500.

  The test case generation information (constraint condition) 610 includes an ID column 611 and a constraint condition column 612.

  The ID column 611 is a column indicating an identifier for uniquely identifying the constraint condition identification information. The constraint condition column 612 is a column that describes the constraint condition between parameters used when generating a test case.

  As described above, the test case generation information 600 shown in FIG. 8 has the same format as the external specification 400 and the analyzed parameter information 500. However, in the test case generation information 600, the parameter name field 603 is not hierarchized unlike the external specification 400 and the analyzed parameter information 500, and instead of “.”, A parent-child relationship between parameters is used. (For example, a member variable of a class).

  Further, for example, since the parameter whose ID column 602 is “6-1” and “6-2” and whose parameter name is “position position” is an array, a value is assigned to each element of the array, and the length of the array The parameters are generated. In this example, two parameters are generated with respect to the “present line position”. This is based on the input of the analyzed parameter information by the user input 311 (S304), and the maximum is “2” as “@Length”. This shows a case where a pattern having “” is input.

  The above conversion to test case generation information corresponds to handling each parameter as non-hierarchical information in the test case generation processing S305, and for the array data, specific data for each element of the array This is a process for assigning. Note that the parameter with the parameter name “competitive signal” is also an array, but due to the difference in type information (because it is a set type), data for the length is not generated in this example.

  In addition, a constraint condition with ID (ID column 611) “3” is added to the test case generation information (constraint condition) 610. These are constraints for generating an appropriate test case considering the parent-child relationship of parameters and the length of the array. For example, for a parameter having a parent-child relationship, if the parent parameter is a NULL value, the child parameter has no value, or, similarly, if there is a variation in the length information of the array, the length is greater than the length. Do not give values to the elements of the array (for example, when the array length takes two values “1” and “2”, if the length is “1”, the second array data This is to prevent invalid test cases from being generated.

  Next, the test case 800 shown in FIG. 9 is data obtained as a result of the test case generation processing S305, and is the original data of the actual software test. In the test case generation process S305, a test case is generated by generating a combination of input parameter candidate values from the test case generation information 600.

  As shown in FIG. 9, the test case 800 includes an ID column 801, a parameter name column 802, a type column 803, an IO column 804, and a test case column 805.

  The ID column 801 is a column indicating parameter identifiers. The parameter name column 802 is a column that describes the name of the parameter. The type column 803 is a column that describes parameter type information. The IO column 804 is a column indicating whether the parameter is input or output. The test case column 805 is a column that describes each test case (a value determined for each parameter).

  For example, in the sub-field 1 of the test case column 805, the parameter value of the parameter name “competing signal” is @Notnull, the parameter value of the parameter name “monitoring target. It means that.

Next, details of the external specification analysis processing S302 will be described with reference to FIG.
FIG. 10 is a flowchart showing detailed processing of the external specification analysis processing S302.

  The external specification analysis process S302 is a process executed in the analysis unit 111 as described above.

  First, all parameters are analyzed sequentially for the external specification (parameter information) 401, parameter unit analysis processing (S902), parameter unit class division processing (S903), pattern information analysis processing (S904). (S901 to S905).

  Here, in the parameter unit analysis process (S902), when “◯” is described in the array column 506 (in the case of a variable length array), “@Length” representing the length information of the array is set as the parameter information. Add to. Further, when the IO column 405 is I / O (parameter used for both input and output), the parameter is divided into two parameters of “I” (input) and “O” (output).

  Further, in the class division processing in parameter units (S903), when the class division of the pattern described in the pattern column 407 is insufficient, the class division processing is performed based on the type information described in the type column 404.

  Further, the pattern information analysis process (S904) analyzes whether or not an abstract pattern such as “@Valid” or “@Invalid” is described in the pattern column 407, for example.

  Next, the external specification (constraint condition) 410 is analyzed, the constraint condition column 412 is sequentially read, and if a parameter not described in the external specification (parameter information) 401 is described, the item is extracted. (S907) (S906 to S908).

  Finally, the analyzed parameter information 500 is shaped and generated from the processing results up to S908 (S909). In that case, the parameter column 507 of the parameter extracted in S904, the parameter added in S907, and the column that is not described in the pattern column 507 are recognized as data that needs to be complemented. When the parameter information complement screen shown in FIG. 7 is displayed, highlighting is performed.

Next, details of the test case generation information generation processing S304 will be described with reference to FIGS.
FIG. 11 is a flowchart showing detailed processing of the test case generation information generation processing S304.
FIG. 12 is a diagram illustrating an example of generating test case generation information from analyzed parameter information.

  The test case generation information generation process S304 is a process executed by the conversion unit 112 of the test support apparatus 100, and is a process for generating the test case generation information 600 from the analyzed parameter information 500.

  First, in the test case generation information generation process S304, all parameters are read sequentially, parameter information conversion (S1002), array data generation (S1003), parameter unit class division process (S1004), and addition of constraint conditions (S1005) is performed (S1001 to S1006).

  In the parameter information conversion (S1002), the parameter name is converted from the relationship between the parent and child of the parameter. That is, the parameter name of the test case generation information 600 is converted by connecting the parent parameter to the parameter with “.”.

  Next, candidate values are added to the pattern based on the relationship between the parent and child. More specifically, when it is clear that the child parameter has some value, “@NotNull” indicating that it is not a NULL value is added. If the parent parameter includes a pattern having no value such as “@Null”, “@Vain” indicating that the parameter does not have a value is added. Next, in the generation of array data (S1003), when the parameter is array data, the parameter is generated for the maximum length given to the pattern column 507.

  The parameter unit class division processing (S1004) is the same as the parameter unit class division processing (S903) in the external specification analysis processing S302. The class division process is also performed in this test case generation information generation process. The class division process is performed for the pattern additionally input to the pattern column 507 when the user input 311 is input to the analyzed parameter information (S304). This is for processing.

  In addition of the constraint condition (S1005), when the parent parameter does not have a value such as “@Null”, “@Vain” indicating that the child parameter does not have a value is taken, or an array longer than the length of the array It is added to the constraint that the parameter is “@Vain” for the element.

  In the example of the test case generation information (constraint condition) 610 shown in FIG. 8, when the length of the “remaining line position” is “1” for the record whose ID column 611 is 3, the second existing line position is displayed. An example is shown in which a constraint condition in which the array element “target train. Existing line position [1]” is “@Vain” is added.

  In the next processing from S1007 to S1009, the constraint conditions are converted in accordance with the processing results from S1001 to S1006. In this process, when the constraint condition is described for the array data, the constraint condition for the array data is copied.

  Next, class division processing between parameters is performed (S1010) (details will be described later).

  Finally, information obtained as a result of conversion from the analyzed parameter information 500 is output as test case generation information 600 (S1011).

  In the class dividing process between parameters in S1010, parameters are replaced so that candidate values in the candidate value column 606 of the test case generation information (parameter information) 601 can be generated based on the pattern 507 of the analyzed parameter information 500. A specific example will be described below taking the case of FIG. 12 as an example.

  In the class division processing between parameters in the example shown in FIG. 12 (S1010), analyzed parameter information (constraint conditions) 702 is applied to P01, P02, P03, and P04 that are child parameters of the parameter P0 of the analyzed parameter information 700. The example which produces | generates the test case production | generation information 710 based on the information described in this is shown.

  As shown in FIG. 12, no value is defined in the pattern field of the analyzed parameter information (constraint condition) 702. In this case, analyzed parameter information (constraint condition) 702 is analyzed, related constraint conditions are extracted, and pseudo parameters (A, B, and C in FIG. 12) are generated from the constraint conditions for analysis. The pattern obtained from the parameter information (constraint condition) 702 and the constraint relation between the parameters are shown in the pseudo parameter candidate value column of the test case generation information (parameter information) 711 and the test case generation information (constraint condition) 712, respectively. Add to the constraint field.

  In the example shown in FIG. 12, there is a conditional expression $ {P1} + $ {P3} <10 at the beginning of the analyzed parameter information (constraint condition) 702, but the pattern field and parameter of the record with the parameter name P0 No value is defined in the pattern field of the record of name P3. Therefore, as the ID column of the test case generation information (constraint condition) 712 is shown in the record 2, A = P 0. P1 + P0. P3 is generated. Thus, the conditional expression equivalent to the conditional expression in question is A <10, and as a candidate value in the test case generation information for A, a case of ≧ 10 is generated when <10 (test case generation information) (Parameter information) Record whose ID column of 711 is 1). Similarly, B = P0. P1 + P0. P2, C = P0. P2 + P0. P3 is introduced, and the candidate value in the test case generation information is generated as in the conditional expression.

  As described above, if no specific value is set for the parameter when generating test case generation information, a warning message is output before the test case is generated. You may ask no. Even in the above process, if there is a parameter whose candidate value is undetermined, an error message is displayed, the process is stopped, or a warning message is displayed. Generation may be performed.

Next, details of the test case generation process S305 will be described with reference to FIG.
FIG. 13 is a flowchart showing detailed processing of the test case generation information generation processing S305.

  The test case generation process S305 is a process executed in the test case generation unit 113 of the test support apparatus 100.

  In the test case generation process 305, candidate values are selected from the test case generation information (parameter information) 601 for each parameter whose IO column 605 is input (“I”), and a combination of parameter candidate values is generated (S1101). . Next, it is checked whether the combination of candidate values of the selected parameter satisfies the constraint condition of the test case generation information (constraint condition) 610 (S1102). When the constraint condition is satisfied (S1103: Satisfaction), the combination of parameter candidate values generated in S1101 is registered as a test case, and it is checked whether or not the test case generation criteria are complied with (S1104).

  When the generated test case group satisfies the test case criteria (S1105: Satisfies), the test case 800 is generated (S1106).

  If the constraint condition is not satisfied (S1103: not satisfied), and if the generated test case group does not satisfy the test case criteria (S1105: not satisfied), the process returns to S1101, and a combination of parameter candidate values is generated again. To do.

  Here, the constraint condition can be checked by checking whether the constraint condition is satisfied using, for example, SAT (SATisfiability problem) Solver or SMT (Satisfiability Modulo Theories) Solver.

  Test case generation criteria may be set in advance according to test design criteria such as the number of test cases generated and the coverage of combinations of parameter candidate values. Note that n-wise coverage is known as a test case reference in consideration of combinations of parameter candidate values. A test case generation method satisfying n-wise coverage is called an n-wise method, and various test case generation methods are known. The test case generation process S305 shown in FIG. 13 may use these n-wise test case generation methods. In addition, n-wise methods are known in which test cases are generated in consideration of the priority of parameters, and application methods such as grouping parameters and setting a coverage ratio for each group are known. These application methods may be used by providing columns for adding parameter priority and grouping information to the analyzed parameter information (parameter information) 501. Furthermore, in the description of the test case generation process S305 and the general n-wise method, output parameters are not considered, but for example, a method (such as Patent Document 2) that considers output parameters may be used. .

Next, details of the data structure of the external specification 400, the analyzed parameter information 500, and the test case generation information 600 employed in the description of the present embodiment will be described with reference to FIGS.
FIG. 14 is a diagram in which the outline of the data structure of the external specification 400, the analyzed parameter information 500, and the test case generation information 600 is described in a UML (Unified Modeling Language) class diagram-like.
FIG. 15 is a diagram showing basic type information definition and individual type information definition as definition information necessary for the test case generation process.

  The data of this embodiment shown in FIG. 14 includes a parameter structure 1200, type information 1210, and constraint conditions 1221.

  The parameter structure 1200 represents the relationship between parameters. The type information 1210 is information indicating a type relationship that the parameter can take. The constraint condition 1221 is information describing a parameter constraint condition.

  The parameter structure 1200 includes a parameter 1201, a composite parameter 1202 for storing a parent parameter, and a simple parameter 1203 that requires specific data allocation. Here, the upper half described in the box of the parameter structure 1200 corresponds to the class of the UML class diagram, and the lower half corresponds to the attribute. For example, “length” as a component of the parameter 1201 is an attribute for storing the length of the array when the parameter is array data.

  The composite parameter 1202 is data for storing parent parameter information such as a structure and a class, and the type name represents the type name of the structure or class. The pattern is used to indicate that the composite parameter 1202 itself has a NULL value or no value (corresponding to “@Null”, “@Vain”, etc. in FIGS. 5, 6, and 8). .

  Further, the simple parameter 1203 includes “patterns” for storing parameter patterns as candidate values, and stores information on which type information the parameter belongs to in relation to the type information 1210.

The type information 1210 is data for storing parameter type information (corresponding to the type column of the external specification 400, the analyzed parameter information 500, and the test case generation information 600).
As shown in FIG. 14, the type information 1210 includes a group type 1212, an enumeration type 1213, a set type 1214, and a comparable group type 1215 based on a basic base type 1211.

  Here, the group type 1212 and the comparable group type 1215 are generic names of types for parameters that can perform arithmetic operations, such as a group in which operations in mathematical terms are defined. The comparable group 1215 is an integer, a real number, or the like. This is a generic term for types for parameters that can be compared in size. The enumeration type 1213 is a generic name of types for parameters for which possible values of parameters such as days of the week and sex are determined, and the set type 1214 is used when parameters need to be handled as a set. For example, the external specification of FIG. It consists of a plurality of items such as 400 “competing traffic signal”, but “all XX are □□” (reserved word all described in the pattern field 407 of the record whose ID field is 3 in FIG. 5). , “At least one of OO” (reserved word “some” is not shown in the figure), but the items in the parameters need to be handled together as one set.

  Next, the constraint condition 1221 represents data described in the external specification (constraint condition) 410, analyzed parameter information (constraint condition) 510, and test case generation information (constraint condition) 610.

  FIG. 14 is a schematic diagram of the structure of the external specification 400, the analyzed parameter information 500, and the test case generation information 600, and may include components other than those described above. Also, the type information 1210 may include a type other than the type information shown in FIG.

Next, definition information necessary for the test case generation process will be described with reference to FIG.
As shown in FIG. 15, the definition information includes a basic type information definition 1300 and an individual type information definition 1310. The basic type information definition 1300 includes an ID column 1301, a type type column 1302, a constraint condition description column 1303, and a constraint condition analysis check 1304.

  An ID column 1301 is a column indicating an identifier of type information. The type classification column 1302 is a column indicating the type of company. The constraint condition description column 1303 is a column for describing a constraint condition description method for the type type. The constraint condition analysis check 1304 is a column in which a notation corresponding to the constraint condition description column 1303 used in the constraint condition check process S1102 shown in FIG. In particular, the constraint condition description column 1303 is intended to be used by a test tool such as SAT Solver or SMT Solver.

  The individual type information definition 1310 stores type information used in the actual external specification 400, analyzed parameter information 500, and test case generation information 600, and notation and related information. As shown in FIG. 15, the individual type information definition 1310 includes an ID column 1311, a type name column 1312, a type type column 1313, a format column 1314, and a supplementary information column 1315.

The ID column 1311 is a column indicating an identifier of definition information. The type name column 1312 is a column indicating the name of the type. The type classification column 1313 is a column indicating the type information as a base. The format column 1314 is a column that defines the description method in the pattern column. The supplementary information column 1315 is a column for storing supplementary information for each type information such as a minimum value, a maximum value, and a bit width.
Based on the type information defined above, class division is performed for each parameter and for each parameter (S903 in FIG. 10, S1004 in FIG. 11).

  The outline of class division processing for each type will be described below.

  In the case of group types and enumeration types, class division is performed from variations of values that parameters can take. For example, there are enumerated parameters that can take six values “e1, e2, e3, e4, e5, e6”, and “{e1, e2, e3}”, “e4”, “@Another” are used as pattern fields. When it is described that there are four variations “” and “e1”, the given variations are sequentially read and processed as follows. “@Another” represents “others” and indicates that any one of the remaining values from the given variation is taken.

  First, from the pattern “{e1, e2, e3}”, “e1, e2, e3, e4, e5, e6” are changed to “{e1, e2, e3}” and “{e4, e5, e6}”. Divide into two classes. Next, from the pattern “e4”, “{e4, e5, e6}” is divided into “e4” and “{e5, e6}”. For “@Another”, “{e5, e6}” that is not assigned to the pattern is assigned. Finally, “{e1, e2, e3}” is divided into “e1” and “{e2, e3}” for “e1”, and finally, “e1, {e2, e3” }, E4, {e5, e6} ".

  On the other hand, in the case of a comparable group type, class division is performed based on the magnitude relation of data and the comparison operator with numerical values in a given pattern. For example, when the integer parameter pattern is “> = 0”, “> 10”, “3 <x <5”, the following processing is performed.

  First, numerical data in a given variation are arranged in ascending order (or descending order). For the results obtained in ascending order (“> = 0”, “3 <”, “<5”, “> 10”), classify as follows, considering the characteristics of the comparison operator .

  First, in order to interpolate classes between “> = 0” and “3 <” from 0 to more than 3, a “<= 3” is interpolated from a description of “3 <”, and “0 < = X <= 3 "division is obtained. Next, “3 <x <5” is generated from the descriptions “3 <” and “<5”. From the description of “<5” and “> 10”, in order to interpolate a class representing a value of 5 or more and less than 10 as in the first division process, “5 <=” is interpolated from “<5”, “5 <= x <10” is generated. Finally, from the description of “0 <”, the class of “> = 10” is interpolated from “> 10”. From the above analysis results, “0 <= x <= 3”, 3 <x <5 ”, 5 <= x <10”, and “> = 10” are obtained. The above division can be easily implemented by mapping each divided region on a numerical line.

  In the above, class division is performed only from given variations, but areas other than designation may be included in the class division result. In the above example, “<0” may be added to the class division for “> = 0”.

  Finally, in the collective type, each pattern described in the pattern column may be used as a variation as it is. The above class division for each type information is an example of a class division method, and may be performed in a different form from the above.

  Through the above processing, generation of analyzed parameter information 500, test case generation information 600, and generation of test case 800 is realized from external specification 400.

  In this embodiment, the analyzed parameter information 500 and the test case generation information 600 are expressed in a tabular format from the external specification 400, and the constraint condition includes a constraint condition including only input parameters such as a precondition, and an input condition. Although the restriction conditions representing the output parameters are mixedly described, they may be divided according to the contents of the restriction conditions such as the precondition and the postcondition.

  In addition, in the description of the constraint condition and the constraint condition generation process in the test case generation information generation process S304, the constraint condition may be supplemented from related specifications. For example, a constraint condition may be added by extracting the multiplicity between parameters and the relationship between parameters from a class diagram in UML, an ER (Entity Relationship) diagram used for DB design, or the like.

Next, processing from the test case generated to the test execution will be described using FIG.
FIG. 16 is a flowchart for explaining processing from the generated test case to the test execution.

  As described above, test cases are generated from external specifications, and when actually performing tests, test execution is performed using the generated test cases for the software configuration modules or functions to be tested. Confirm the result.

The test execution uses specific test data to operate the software configuration module or function to be tested, and obtains the result, but the process to operate the test target is not necessarily the test support apparatus 100. There is no need to execute it. For example, when the software configuration module to be tested is connected via a network, it may be executed via the network.
As shown in FIG. 16, the process flow from external specification storage (S1501) to test case generation (S505) is the same as the process flow of FIG.

  In the process of FIG. 16, the test case generation (S1505) of FIG. 4 includes the correction of the generated test case (S1506), the test data generation process (S1507), the correction of test data (S1508), the test execution (S1509), A procedure for confirming the test execution result (S1510) is added.

  First, the test case is corrected for the generated test case (S1506). For example, when an abstract candidate value such as “@Valid” is set as a parameter candidate value, a specific value that can be tested is assigned, or a test case that is desired to be individually added is added.

  Next, test data is generated (S1507). For example, when the test pattern is an abstract pattern (for example, “@Valid”, “0 <x <100”, etc.), a specific value that can be actually executed by the test is generated, or the test data is input. Emphasis is displayed to prompt the user to input the test data 900 by the user input 1523. In the generation of the specific value, a random specific value satisfying a given condition may be set, or when a specific range such as “0 <x <100” is assigned, A boundary value or an intermediate value may be assigned according to the test standard.

  Next, specific data allocation to the generated test data 900, and when the test executor determines that correction is necessary, the test data 900 is further corrected (S1508), and thereafter The test is executed (S1509).

  As a result of test execution, test results corresponding to all test cases were obtained, or the test execution results 1000 of all (or more than the specified number of tests) were as expected. If so (S15011: Satisfies), the test is terminated. If not (S15011: Not Satisfies), the process returns to the input procedures of S1501, S1503, S1506, and S1508 according to the result (S1511).

  For example, when there are not enough descriptions in the external specification 400, such as when there is a description omission in the constraint condition from the test result and an invalid input value is entered, the process returns to the creation of the external specification 400 (S1501). It is sufficient to review 400 descriptions.

  In addition, if the test execution result is insufficient in the number of tests or coverage at the time of execution (such as black box test coverage standards such as execution step coverage and branch step coverage), the test case is corrected (S1506). Return and add test cases. At this time, for example, the execution path during the test execution may be recorded, and the execution path to which each input data corresponds may be included in the test execution result. As a result, when the test case correction (S1506) is performed, a combination of unselected patterns (which are likely to pass through different paths) is added to the test case generation, and the test case correction (S1506) is used as a reference. It becomes possible.

  In addition, when including the correspondence with the execution path in the execution result of the test, the coverage rate of the execution step coverage and the branch step coverage satisfies the acceptance criteria even though the test execution is performed for all the combination patterns. If the path does not satisfy or is different from the intended path, the correspondence between the generated test case and the execution path may be referred to such as reviewing the external specification 400 or the source code of the test target.

  These can be confirmed visually by the user based on the test execution result, or the test pass criteria can be obtained by acquiring the test execution path during test execution and managing the correspondence with the test case on the test case. (S1511) may be automatically determined.

  Further, in the test case generation (S1505), when the execution result 1000 cannot be specified (only the combination of input parameters is generated), the execution result is as expected from the execution result 1000 at the first execution time of each test case. After confirming whether or not, it may be used as an expected result by reflecting it in the correction of the test case (S1506) or the correction of the test data (S1508). This facilitates automatic test execution including confirmation of the results of the regression test during software renovation.

  DESCRIPTION OF SYMBOLS 100 ... Test support apparatus, 101 ... External specification storage part, 102 ... Analyzed parameter information storage part, 103 ... Test case generation information storage part, 104 ... Test case storage part, 105 ... Type information definition storage part, 106 ... Test data Storage unit 107 ... Execution result storage unit 110 ... Input / output unit 111 ... Analysis unit 112 ... Conversion unit 113 ... Test case generation unit 114 ... Test data generation unit 115 ... Test execution unit

Claims (14)

A test support apparatus that has a CPU, a storage unit, and a display unit, and implements a function by executing a program stored in the storage unit,
The storage unit stores external specifications, analyzed parameter information, test case generation information, and test cases,
The external specification and the analyzed parameter information hold parameter information of a parameter described in a software program to be tested and a constraint condition indicating a relationship between the parameters of the parameter,
The parameter information of the external specification and the analyzed parameter information includes type information of the parameter and pattern information that can be taken by the parameter name,
The test case generation information holds parameter information of parameters described in the software program to be tested, and constraint conditions representing the relationship between the parameters of the parameters,
The parameter information of the test case generation information includes type information of the parameter and candidate values that can be taken by the parameter name,
The test case holds one or more combinations that define parameter values of parameters described in the software program to be tested,
From the parameter type information of the parameter information of the external specification, the pattern information, and the constraint conditions, classify the parameters, generate the analyzed parameter information,
From the parameter type information of the parameter information of the analyzed parameter information, the pattern information, the constraint conditions, classify the parameters, generate the test case generation information,
Generating a test case by generating a combination that defines the parameter value from the type information of the parameter of the parameter information of the test case generation information, the constraint condition, and a candidate value of the parameter information. Test support device. When generating the analyzed parameter information from the external specification,
Extracting parameter deficiency information described in the external specification, displaying a screen for highlighting parameter deficiency information for each parameter on the display unit, and receiving complementary input of the analyzed parameter information, The test support apparatus according to claim 1. When generating the test case generation information from the analyzed parameter information,
The parameter described in the test case generation information is generated by converting a parameter described in the analyzed parameter information based on a parameter constraint condition of the analyzed parameter information. Test support equipment.   The parameter classification of the parameter information in the external specification and the parameter information in the analyzed parameter information is characterized by performing class division processing determined for each parameter type information on the parameter described in the pattern information. The test support apparatus according to claim 1. The process of converting the parameter described in the analyzed parameter information and generating the parameter described in the test case generation information includes:
When the parameter described in the analyzed parameter information has a parent-child relationship, when the parent parameter does not have a value, pattern information indicating that the child parameter does not have a value, or a value for the child parameter. Processing to add pattern information indicating that the parent parameter has a value when
When the parameter described in the analyzed parameter information is an array composed of a plurality of items, a process of generating a parameter according to the number of items in the given array;
4. The processing according to claim 3, further comprising: adding a constraint condition between parameters generated according to the parent-child relationship of the parameters and the length of the array to the constraint condition described in the test case generation information. Test support device. Extracting parameter deficiency information described in the external specification, displaying a screen for highlighting parameter deficiency information for each parameter on the display unit, and accepting complementary input of the analyzed parameter information,
When the parameter described in the external specification is an array composed of a plurality of items and the data of the array is variable, a process of adding a parameter for inputting the length of the array;
3. The test support apparatus according to claim 2, further comprising a process of extracting pattern information for which information necessary for test execution is insufficient from the parameter pattern information described in the external specification. The storage unit further includes test data for test execution storing specific values that can be executed based on a combination that defines the parameter values described in the test case, and the test data. Memorize test results,
Generating test data from the test case,
The test support apparatus according to claim 1, wherein a test execution is performed using the test data, and the execution results are collected. A test support method for a test support apparatus having a CPU, a storage unit, and a display unit, and realizing a function by executing a program stored in the storage unit,
The storage unit stores external specifications, analyzed parameter information, test case generation information, and test cases,
The external specification and the analyzed parameter information hold parameter information of a parameter described in a software program to be tested and a constraint condition indicating a relationship between the parameters of the parameter,
The parameter information of the external specification and the analyzed parameter information includes type information of the parameter and pattern information that can be taken by the parameter name,
The test case generation information holds parameter information of parameters described in the software program to be tested, and constraint conditions representing the relationship between the parameters of the parameters,
The parameter information of the test case generation information includes type information of the parameter and candidate values that can be taken by the parameter name,
The test case holds one or more combinations that define parameter values of parameters described in the software program to be tested,
Classifying the parameter from the parameter type information of the parameter information of the external specification, the pattern information, and the constraint condition, and generating the analyzed parameter information;
Classifying the parameters from the parameter type information of the parameter information of the analyzed parameter information, the pattern information, and the constraint conditions, and generating the test case generation information;
Generating a combination defining the parameter values from the parameter type information of the parameter information of the test case generation information, the constraint conditions, and candidate values of the parameter information, and generating the test case. A test support method characterized by comprising: In the step of generating the analyzed parameter information from the external specification,
Extracting parameter deficiency information described in the external specification, displaying a screen for highlighting parameter deficiency information for each parameter on the display unit, and receiving complementary input of the analyzed parameter information, The test support method according to claim 8. In the step of generating the test case generation information from the analyzed parameter information,
9. The parameter described in the analyzed parameter information is converted based on a parameter constraint condition of the analyzed parameter information to generate a parameter described in the test case generation information. Test support method.   The parameter classification of the parameter information in the external specification and the parameter information in the analyzed parameter information is characterized by performing class division processing determined for each parameter type information on the parameter described in the pattern information. The test support method according to claim 8. The step of converting the parameter described in the analyzed parameter information to generate the parameter described in the test case generation information includes:
When the parameter described in the analyzed parameter information has a parent-child relationship, when the parent parameter does not have a value, pattern information indicating that the child parameter does not have a value, or a value for the child parameter. Processing to add pattern information indicating that the parent parameter has a value when
When the parameter described in the analyzed parameter information is an array composed of a plurality of items, a process of generating a parameter according to the number of items in the given array;
The process according to claim 10, further comprising: adding a constraint condition between parameters generated according to the parent-child relationship of the parameters and the length of the array to the constraint condition described in the test case generation information. Test support method. Extracting parameter deficiency information described in the external specification, displaying a screen for highlighting parameter deficiency information for each parameter on the display unit, and accepting complementary input of the analyzed parameter information,
When the parameter described in the external specification is an array composed of a plurality of items and the data of the array is variable, a process of adding a parameter for inputting the length of the array;
The test support method according to claim 9, further comprising: extracting pattern information for which information necessary for test execution is insufficient from the pattern information of the parameters described in the external specification. The storage unit further includes test data for test execution storing specific values that can be executed based on a combination that defines the parameter values described in the test case, and the test data. Memorize test results,
Generating test data from the test case;
The test support method according to claim 8, further comprising: performing a test execution using the test data and collecting the execution result.

Images