JP5509992B2 - Test data generation apparatus and program evaluation support method - Google Patents

Description

  The present invention relates to a test data generation apparatus and a program evaluation support method for generating test data for use in a dynamic test of a program executed on a computer.

  Conventionally, in order to verify that the operation of the program is normal, the program quality is evaluated by providing test input values to the program and verifying whether the values that should be output are output correctly. The method of doing is used.

  This verification work may be performed by a human system, but the work efficiency decreases as the number of verification data increases. For this reason, techniques for automatically executing verification work have been proposed. (For example, see Patent Document 2 and Patent Document 3.)

  Further, in the verification work, evaluating output values for all input values increases man-hours and decreases efficiency. For this reason, a method for efficiently testing by reducing the number of verification work steps has been proposed. For example, Patent Document 4 discloses a technique for automatically analyzing a branch of a program and creating a test item that passes through all paths of the source code.

  Patent Document 1 discloses a technique for efficiently creating a test specification by concealing an internal I / O subsystem signal pattern by a graphical box pattern in order to proceed with a large amount of man-hours. It is disclosed.

  Patent Document 5 discloses a technique for graphically displaying waveforms and creating test data in order to efficiently proceed with program verification.

JP-A-11-119821 JP-A-9-222302 Japanese Patent Laid-Open No. 9-259006 JP-A-5-210537 JP 2006-99518 A

  By the way, as pointed out in Patent Document 1, it is not efficient to execute tests for all test cases in the test stage after the unit test. With respect to this problem, Patent Document 1 divides state transitions into processing units and pays attention to values at the time of input / output. However, when this method is adopted, the disadvantages of covering all cases are eliminated, but when time series data such as waveform data is to be handled, it is possible to visually distinguish the waveform shape such as trapezoidal shape and triangular wave shape. It becomes difficult. That is, the method of Patent Document 1 has a new problem that it becomes difficult to grasp the shape of the entire waveform, and the test points cannot be efficiently specified or narrowed down.

  The present invention has been made in view of the above-described circumstances, and test data that can generate test data by extracting data effective for dynamic testing of a program from waveform data such as time-series data. It is an object to provide a generation apparatus and a program evaluation support method.

  The outline of the present invention is to display the waveform diagram of each of the data to be input to the verification program and the output data to be output so that the input / output correspondence and time-series changes can be easily seen. Display on the device. Then, the designer attaches an extraction mark to the input / output data used for the test data while visually confirming the input / output relationship. Thereafter, test data is generated from input / output data with extraction marks.

  In addition, the present invention enables the test data to be output directly as a program or a part of the program, so that the test data can be directly coupled to the verification program or taken into the verification program. The test data is generated separately for an input value to the verification program and an output value expected when the input value is input to the verification program (hereinafter referred to as “output expected value”). The expected output value is compared with an output value output from the verification program when an input value of test data is given to the verification program, and is used for evaluation of the verification program.

  Normally, it is only necessary to verify whether a correct value is calculated from the extracted set of input values and output values, so it is sufficient to output only the extracted set of input values and output values. However, in the case where the program to be verified stores the previous state, a correct value cannot be calculated only from the set of the extracted input value and output value. For this reason, when such verification is performed, all data is extracted in time series, while only output values designated for extraction are used for output. However, data position information that identifies the input time point is attached to the output value so that it can be determined at which time point the data is.

  Specifically, the test data generation device according to the present invention inputs an input value to the verification program and an output expected value for the input value, and is input to the combination of the input value and the output expected value. Data input means for storing data as an input data in the storage device with a data position indicating the order, and a graph display for graphically displaying the input value and the output expected value on the display device based on the input data stored in the storage device Means for displaying a cursor indicating a relationship between an input value and an expected output value corresponding to the input value on each graph, an input value indicated by the cursor in response to a data extraction instruction command, and the cursor Extraction data designation means for attaching an extraction mark to the data position of the output expected value corresponding to the input value, and extracting the input value and the output expected value of the data position to which the extraction mark is attached Characterized in that and a test data output means for generating test data for verification program Te.

  In the present invention, the relationship between the input value and the output expected value is displayed in a graph, and the designer can extract the data he / she wants to extract arbitrarily by looking at the graph. Therefore, effective test data is selected efficiently. be able to.

  In the test data generation device according to the present invention, the test data output means extracts all input values input by the data input means for the input values, and adds an extraction mark for the output expected value. The output expected value is extracted by attaching the data position of the output expected value, and test data is generated using the extracted data.

  In the present invention, test data can be appropriately generated even in the case where the history of input values affects the output value.

  Further, in the test data generation device according to the present invention, the extraction data specifying means further determines whether or not the expected output value satisfies a predetermined extraction condition, and if so, data of the expected output value. An extraction mark is displayed at the position. As an extraction condition, for example, it is determined by whether or not a change in output expected value is larger than a predetermined threshold, and an extraction mark is displayed at a data position where the change in output expected value is larger than the threshold. The verification part can be narrowed down to.

  The program evaluation support method according to the present invention operates a verification program on a computer, inputs test data generated by a test data generation device to the verification program, and outputs the calculation result of the verification program to a display device. A program evaluation support method for supporting evaluation of a verification program by displaying the test data by inputting test data input values to the verification program in a predetermined time interval in the order of data positions, and from the verification program It is characterized in that it is determined whether or not the output value and the expected output value of the test data match, and the determination result is displayed on the display device.

  In the present invention, the verification program is evaluated by inputting test data to the verification program and comparing the output value with the expected output value for each input value stored in the test data. Whether or not they match is determined by whether or not the difference is equal to or less than a predetermined threshold.

  Preferably, the response time of the verification program is set in the computer, and after the input value of the test data is input to the verification program, the output value after the response time, the expected output value for the input value, and Are compared to determine whether the output value matches the expected output value. Thereby, the expected value of the response time of the program can be evaluated simultaneously.

  In addition, for each data input position, the difference between the expected output value and the output value of the verification program is calculated, and if the difference is equal to or greater than a predetermined threshold value, an alarm is output, thereby allowing the designer to enter the verification program. You can be sure that there is an anomaly or that there is a problem with the specification.

  According to the present invention, the input value and the expected output value of the test data are displayed in a waveform in time series, and the designer can extract the data to be extracted by looking at the waveform. Data can be selected. In the present invention, test data can be appropriately generated even in the case where the history of input values affects the output value.

It is a device block diagram of the test data generation device 1 by embodiment of this invention. It is a functional block diagram of the test data generation device 1 by embodiment of this invention. The example of a structure of the screen displayed on the display apparatus 50 of FIG. 2 is shown. It is a flowchart which shows the process sequence of the cursor display means 13 of FIG. It is a flowchart which shows the process sequence of the test data output means 15 of FIG. It is an example of the test data file output by the test data output means 15 of FIG. It is another example of the test data file of FIG. It is the other example of a screen structure of FIG. It is an example of the test data file which the test data output means 15 of FIG. 2 output in the C language format in the all data output mode. It is a flowchart which shows the process sequence in the all data output mode of the test data output means 15 of FIG. It is a block diagram of the program verification apparatus by embodiment of this invention. It is an example of the interface program 22 of FIG. It is a flowchart which shows the process sequence of the extraction data designation | designated means 14 of FIG. It is a table of extraction conditions used for the process of the extraction data designation | designated means 14 of FIG. It is a flowchart which shows the process sequence of the test data output means 15 of FIG.

  Embodiments of the present invention will be described below. FIG. 1 is a device configuration diagram of a test data generation device 1 according to the present embodiment. As shown in this figure, the test data generation device 1 includes an input device 40 for inputting data, an arithmetic device 10 for executing various arithmetic processes, a main storage device 20 used for program execution of the arithmetic device 10, etc. The auxiliary storage device 30 stores the program and the display device 50 displays the calculation result.

  FIG. 2 is a functional block diagram of the test data generation apparatus 1 according to this embodiment. In this figure, the test data generating device 1 includes a data input means 11 for inputting an input value and an expected output value to the verification program via the input device 40, and a graph for displaying a graph on the display device 50 based on the input data. Display means 12, cursor display means 13 for performing cursor display and cursor movement processing on the displayed graph, extracted data designation means 14 for accepting designation of data to be used for test data out of input data, designated data Test data output means 15 for extracting and generating test data of the verification program and outputting it to the auxiliary storage device 30 is provided. Each means 11-15 can be realized by a program as a function of the CPU.

  The function of the test data generation device 1 shown in FIG. 2 will be described in relation to FIG. 1. A program for realizing each means 11 is stored in the auxiliary storage device 30, and this program is the main program at the start of the test data generation process. It is loaded into the storage device 20 and executed by the arithmetic device 10. The generated test data is stored in the auxiliary storage device 30.

Next, the operation of the test data generation apparatus 1 having the above configuration will be described.
<Data input processing>
FIG. 3 shows a configuration example of a screen displayed on the display device 50 by the data input unit 11 of the test data generation device 1. The left side of the screen is a screen for inputting data (data input unit 2), and the right side of the screen is a screen (waveform display unit 3) for displaying a graph based on the input value. In addition, on the upper side of the waveform display unit 3, a group of buttons (instruction data change button display unit 4) for moving the cursor is arranged.

  A designer (one who generates test data) sets and inputs an input value and an expected output value via the data input unit 2 on this screen. The data input unit 2 is a spreadsheet-like input interface that can be found in spreadsheet software and the like, and numerical values can be individually input into each cell in the “IN” column of the column 2b and the “OUT” column of the column 2c. It is like that. An input value to the program can be input in the “IN” column, and an expected output value can be input in the “OUT” column. A serial number is automatically displayed in the “No.” column of the column 2a. This “No.” column represents the order of data, that is, time series. In the following description, “data position” means the order of data. When data is input to the bottom line, a new line is scrolled.

  In FIG. 3, the input value and the output expected value are directly input. Instead of setting the input / output value, the numerical value is input and the input value or the output expected value is designated. The output expected value or the input value may be automatically calculated from the above.

  Further, in the example of FIG. 3, the input data string is one series, but may be a plurality of series. Similarly, the output data string may be a plurality of series when the program has a plurality of output points. At this time, it is preferable to automatically adjust the display range of the cell width of each column according to the number of data columns.

  In addition, when calculating mathematical formulas such as spreadsheet software, it may be necessary to calculate intermediate values so that the mathematical formulas are not overly complex. In such a case, although not shown in FIG. 3, a column for calculating an intermediate value may be provided, and the input value or the output expected value may be calculated based on this column.

  The input data is stored in the main storage device 20 in the same table format as the data input unit 2 of FIG.

<Graph display processing>
The graph display unit 12 displays a graphical display on the waveform display unit 3 using the data input via the data input unit 2. By the graph display means 12, the time series waveform of the input value is displayed on the input waveform display section 3a. The horizontal axis of the input waveform display portion 3a corresponds to the “No.” column, and serial numbers are displayed at regular intervals. The vertical axis corresponds to the “IN” field, and the range of the vertical axis of the graph is determined by the maximum value and the minimum value of the values input in the “IN” field, and scales are displayed at regular intervals between them. The output waveform display section 3b displays a time series waveform of expected output values. Similar to the input waveform display section 3a, the horizontal axis corresponds to the “No.” field, and the same number as the “IN” field is displayed at equal intervals. The vertical axis of the output waveform display portion 3b corresponds to the “OUT” field, and the range of the vertical axis of the graph is determined by the maximum value and the minimum value input in the “OUT” field, and the scale is displayed at regular intervals between Is done.

  Although not shown, a column is provided in parallel with the data columns 2b and 2c so that data for drawing an auxiliary line on the graph can be set and input, and the input data is input to the input waveform display unit 3a, You may make it display on the output waveform display part 3b.

<Cursor display processing>
The cursor display means 13 displays vertical line cursors 9a and 9b on the input waveform display section 3a and the output waveform display section 3b, respectively. When any one of the buttons 4a to 4d of the instruction data change button display unit 4 is pressed, the cursor display unit 13 detects the pressing and moves the cursor position corresponding to the pressed button. . For example, if the left movement button 4b that is second from the left in the frame of the instruction data change button display unit 4 is pressed, the cursor display means 13 can move after determining the limit position in the procedure of FIG. For example, move the cursor to the left by 1. Similarly, for the other buttons of the instruction data change button display unit 4, the cursor is moved according to each button. The cursors of the input waveform display unit 3a and the output waveform display unit 3b always move in the same manner, and are displayed at the same data position on the horizontal axis. For this reason, the designer can easily recognize the correspondence between the input value and the expected output value at the data position.

  The cursor position display unit 7 displays the position where the cursor is currently located as a numerical value in the “No.” column. That is, there is a relationship of “cursor position = data position”. The cursor position can also be set by directly inputting a numerical value in the “No.” column to the cursor position display unit 7. In this case, when the input value exceeds the movable range of the cursor, it is limited within the movable range of the cursor.

  Next, the processing procedure of the cursor display means 13 for realizing the cursor processing will be described with reference to FIG. It should be noted that pressing each button in the display screen saves the pressed status when each button is pressed, and activates the cursor display means 13 when a button pressing event occurs. The cursor display means 13 is described as detecting which button is pressed by reading the status and resetting the status, but the activation method is not limited to this.

  When the cursor display means 13 is activated by the occurrence of a button press event, it determines whether or not the left move button 4a has been pressed (S101). If the result of this determination is that the left move button 4a has been pressed, the cursor display means 13 next determines whether or not the cursor position is at least 10 before this from the left (S102). The current cursor position is moved to the left by 10 (S103). Specifically, the numerical value in the “No.” column stored as the current cursor position is decremented by 10. When the cursor position is less than 10 from the left, the cursor display means 13 moves the cursor position to the leftmost end (S104). That is, the numerical value in the “No.” field stored as the current cursor position is set to 1.

  If the left movement button 4a is not pressed in step S101, the cursor display means 13 next determines whether or not the left movement button 4b has been pressed (S105). If the result of the determination is that the left move button 4b has been pressed, the cursor display means 13 determines whether or not the current cursor position is one or more before the left (S106). Is moved by one to the left (S107). That is, the numerical value in the “No.” field stored as the current cursor position is decremented by one.

  If the left move button 4b is not pressed in step S105 or if the current cursor position is not one or more from the left in step S106, the cursor display means 13 determines whether or not the right move button 4d is next pressed. Is determined (S108). If the result of this determination is that the right move button 4d has been pressed, the cursor display means 13 next determines whether or not the cursor position is 10 or more before the right (S109). The current cursor position is moved to the right by 10 (S110). Specifically, the numerical value in the “No.” field stored as the current cursor position is incremented by 10. When the cursor position is less than 10 from the right, the cursor display means 13 moves the cursor position to the right end (S110). That is, the numerical value in the “No.” field stored as the current cursor position is set to the final value.

If the right movement button 4d is not pressed in step S108, the cursor display means 13 next determines whether or not the right movement button 4c has been pressed (S112). As a result of the determination, if the right movement button 4c is pressed, the cursor display means 13 determines whether or not the current cursor position is one or more before the right (S113). The cursor position is moved to the right by one (S114). That is, the numerical value in the “No.” field stored as the current cursor position is incremented by one.
The cursor display means 13 operates according to the above procedure.

<Extracted data specification processing>
After moving the cursor to the data position to be extracted, the designer presses the instruction data selection button 5. The extraction data specifying means 14 detects the pressing of the selection button 5 and sets an extraction mark in the “extraction” column of the column 2d of the data input unit 2. When the instruction data selection button 5 is pressed at a data position where an extraction mark has already been set, the extraction data specifying means 14 cancels the extraction mark by a toggle operation. In the example of FIG. 3, the extraction mark is displayed in the column 2d, but another table dedicated to the extraction mark is displayed on the screen, and each time the instruction data selection button 5 is pressed, the instruction is displayed. The recorded data may be copied to the table.

<Test data output processing>
When the above operation is repeated and all the extraction marks are added to the necessary data, the designer next presses the save button 6. When the save button 6 is pressed, the test data output means 15 is activated. FIG. 5 shows a processing procedure of the test data output means 15.

  When activated by pressing the save button 6, the test data output means 15 determines whether or not an extraction mark is set for each input data (S402), and if it is set, its input value and output. The value and the data position are output to a file to generate a test data file (S403).

  FIG. 6 is an example of the output test data file. In this example, the data shown in FIG. 3 is output, and the output format is a text format (CSV: Comma Separated Values) with a comma as a delimiter.

  FIG. 7 shows another example of the test data file. This example is also an example in which the data shown in FIG. 3 is output, but adopts a program format (C language format). Thus, if data is output in advance in a program format, it can be easily taken into a test program and can be handled efficiently.

  In the example of the graph shown in the input waveform display unit 3 of FIG. 3, the input value changes from −100 to +100, but the output value is limited to a change from −60 to +60. Test data for the lower limiter. In this way, in the case of an algorithm such as an upper / lower limiter, the calculated value of the current output does not depend on the previous calculated value, and therefore, the extraction point is set to NO. 4, NO. 7, NO. Even if the data is extracted independently at 10 data positions, the verification can be performed using only the data.

On the other hand, in the case of waveforms as shown in the input waveform display section 3a and the output waveform display section 3b in FIG. 8, they cannot be taken out independently. In this case, the output of the test target program is calculated by the following equation.
OUT [k] = (IN [k] + IN [k-1]) / 2 (k: an integer greater than or equal to 2), OUT [1] = 0 (1)
Here, OUT [k] is an expected output value at the data position K, and IN [k] is an input value at the data position K.

  The above equation (1) represents that the input / output values have a moving average relationship. In this case, in order to calculate the current output value, the previous input value is also required in addition to the current input value. For example, in FIG. 3, no. In the data of 4, the input values are the same, but the output values are different. In this case, if the test data extraction method as shown in FIG. 6 or FIG. 7 is employed, the previous input value may not be included in the test data, which is insufficient as data for verification. For this reason, the test data output means 15 has a mode (normal output mode) for outputting only the data with extraction marks as test data, and a mode for outputting all input data (all data). Two data output modes) may be provided. In both modes, only the data with the extraction mark is output for the expected output value.

  When the current output value depends on the past input value as shown in FIG. 8, the all data output mode is selected, the input value outputs all the data, and the output expected value indicates the position of the data. Only the data with the extraction mark is output together with the information. As a result, it is possible to obtain test data necessary and effective for verifying the program in accordance with the operation characteristics of the program.

  FIG. 9 is an example of a test data file output in C language format in the all data output mode. Since the number of data of the input value and the expected output value is different, they are generated as separate arrays. The expected output value is attached with a data position so that the correspondence relationship with the input value can be understood.

  FIG. 10 is a flowchart showing a data output procedure to a file in the all data output mode. The data output procedure of the expected output value is the same as that shown in FIG. 5, but all data for the input value is output to a file.

<Program verification using test data>
Next, an example of how to use this test data will be described.
As shown in FIG. 11, the verification program 23 is incorporated in a program verification device (emulator) 60 composed of a computer and executed. In this program verification device 60, the test data 21 generated in the format of FIG. 6, 7 or 9 is given as an input of the verification program 23 via the interface program 22. The output from the verification program 23 may be transferred to the interface program 22 and displayed on the display device 24, or may be output through an actual output device.
When displaying on the display device 24, the output value from the verification program and the expected output value in the test data are displayed in association with the input value of the test data, and the difference between the output value and the expected output value is determined in advance. An abnormal mark is displayed when a predetermined condition is satisfied, such as when the value exceeds a certain value.

FIG. 12 shows an example of the interface program 22, which is used for verification of a C language program.
In FIG. 12, first, test data generated by the test data generation device 1 is taken into the program in the form of #include “TestLimData1.c”. In this case, as shown in FIG. 7, it is assumed that data is generated in a program format and output to a file called TestLimData1.c.

  In the for loop, the test program body passes the test data generated by the test data generation device 1 to the function Limiter () to be tested, and executes the calculation. After that, this calculation result is verified by ASSERT_EQUAL () macro. This macro is reported as an error when the two arguments have different values. This is verified in the for loop for all the extracted data generated by the test data generation device 1, and the ASSERT_EQUAL () macro calculates the output (output value) and the output generated by the test data generation device 1. If no mismatch with the expected value is detected, the test passes.

  As described above, the test data is input to the verification program through the interface program as shown in FIG. 12, and the output value of the verification program is compared with the expected output value of the test data, thereby enabling efficient verification. Is possible.

  The test data is set so that a delay time (response time of the verification program) can be set, and the interface program inputs the input value to the verification program and outputs the output value after the delay time. It is also possible to compare the input expected value with respect to the input value and determine whether it is a match or a mismatch. In this way, evaluation including the processing speed of the program becomes possible. In this case, it is effective to verify with an actual machine.

  As described above, according to the present embodiment, it is possible to graphically extract test data effective for a verification program, perform dynamic testing of the program using the extracted test data, and automatically determine the result This improves the efficiency of the verification work.

  Note that the program verification device 60 is not limited to an emulator or an actual device, and an existing verification device may be arbitrarily selected and used depending on the evaluation contents. For example, if the evaluation of the program is not included in the response time, and the program is evaluated with the object code of the target CPU, the program verification device 60 is configured integrally with the test data generation device 1, and the CPU simulator is executed on the program verification device 60 It is also possible to use.

  Alternatively, the program verification device 60 may be integrated with the test data generation device 1 so that the program is evaluated at the logic level of the C language.

  For example, the test data generation apparatus 1 is configured using a general-purpose computer, a verification program is compiled using a C compiler that generates an execution program for the computer, and is executed natively on the computer. Then, by comparing the execution result with respect to the input value of the test data and the expected output value of the test data and determining the match or mismatch, the program can be verified efficiently.

(Another example of extraction data specification processing)
Next, another embodiment of the extraction data specifying process will be described with reference to FIGS.
In the present embodiment, test data extraction conditions are set through a display screen, and thereafter, the extraction data specifying means 14 is activated to automatically extract test data.

  FIG. 14 shows the extraction conditions. The extraction condition is stored in the main storage device 20 of the test data generation device 1, and the extraction data designation unit 14 displays the extraction condition for each item on the display device 50. The designer selects conditions and inputs threshold values for each item through this screen. When there is a selection input, the extraction data designation unit 14 sets a selection flag for the extraction condition stored in the main storage device 20 and stores the threshold value.

  When the input of all necessary data is completed (“YES” in S701 and S702), the extracted data specifying means 14 determines whether or not the selected extraction condition is satisfied for the data at all the data positions (S704). If the extraction condition is satisfied, an extraction mark is set at the data position (S705).

The following formula (2) is an example of an extraction condition calculation formula.
Change [k] = OUT [k]-OUT [k-1] (k: integer greater than or equal to 2) (2)

  The change amount [k] is a calculation formula for obtaining the change amount of the output expected value at the data position [k], and when there is a change amount equal to or greater than a threshold value, an extraction mark is set.

  Instead of the equation (2), points may be extracted by comparing the change rate calculated by the change amount [k] / OUT [k] with a threshold value.

  As shown in FIG. 3, an extraction mark is displayed at a data position that satisfies the extraction condition. When the instruction data selection button 5 is pressed with the cursor positioned at the data position, the extraction mark is canceled by a toggle operation.

  According to the present embodiment, marks can be automatically displayed at important data positions under the selected extraction condition, so that the efficiency of test data generation work is improved.

(Another example of test data output processing)
FIG. 15 shows a processing procedure of the test data output means 15 according to another embodiment. In this embodiment, the format of the test data is set in advance through the display screen. The format of the test data is that the input value is an extraction pattern with an extraction flag only (individual input) or all the inputs (all inputs), and the output type is an output type that is only data or program embedded Means. The test data output means 15 generates test data based on this setting. In FIG. 15, the individual output algorithms (S804 to S807) follow the procedures in FIGS.

  According to the present embodiment, since the extraction pattern and output type of output data can be set, the convenience when generating test data is improved.

DESCRIPTION OF SYMBOLS 1 ... Test data generation apparatus 2 ... Data input part 3 ... Waveform display part 3a ... Input waveform display part 3b ... Output waveform display part 4 ... Instruction data change button 4a, 4b ... Left move button 4c, 4d …… Right move button 5 …… Instruction data selection button 6 …… Save button 7 …… Cursor position display portion 10 …… Calculation device 11 …… Data input means 12 …… Graph display means 13 …… Cursor display means 14… ... Extracted data designation means 15 ... Test data output means 20 ... Main storage device 21 ... Test data 22 ... Interface program 23 ... Verification program 30 ... Auxiliary storage device 40 ... Input device 50 ... Display device 60 …… Program verification device

Claims (5)

An input value to the verification program and an expected output value for the input value (hereinafter referred to as “expected output value”) are input, and the input order is input to the combination of the input value and the output expected value. Data input means for attaching the data position shown and storing it in the storage device as input data;
Graph display means for displaying the input value and the output expected value in a graph on a display device based on input data stored in the storage device;
Cursor display means for displaying a cursor indicating a relationship between an input value and an expected output value corresponding to the input value on each graph;
An extraction data specifying means for receiving an instruction of data extraction instruction and attaching an extraction mark to the data position of the input value indicated by the cursor and the output expected value corresponding to the input value;
Test data output means for generating test data of a verification program by extracting an input value and an output expected value of the data position with the extraction mark;
A test data generating apparatus comprising:   The test data output means extracts all the input values input by the data input means for the input value, and for the output expected value, the output expected value with the extraction mark is set to the expected output value. 2. The test data generating apparatus according to claim 1, wherein the test data is extracted by attaching a data position, and test data is generated using the extracted data.   The extracted data specifying means determines whether or not the expected output value meets a predetermined extraction condition, and if so, adds an extraction mark to the data position of the expected output value. The test data generation device according to claim 1 or 2. A verification program is operated on a computer, test data generated by the test data generation device according to claim 1 is input to the verification program, and a calculation result of the verification program is displayed on the display device A program evaluation support method for supporting evaluation of a verification program by displaying the
The input value of the test data is input to the verification program at a predetermined time interval in the order of the data positions attached to the input value, and the output value from the verification program and the test data A program evaluation support method comprising: determining whether output expected values match, and displaying the determination result on the display device.

The computer sets a response time of the verification program,
After the input value of the test data is input to the verification program, the output value after the response time is compared with the expected output value for the input value, and the output value matches the expected output value. The program evaluation support method according to claim 4, wherein it is determined whether or not.

Images