JPH10275094A - Program evaluation system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To restart an evaluation test at a time when the test is once interrupted without executing the test items again from the first one in a second test mode after the test is discontinued by loading the execution information on the test into a test device when the second test is executed. SOLUTION: When a test discontinuation instruction is given from an input means 1 by some reason, an execution situation acquisition part 22 of a retesting execution part 21 acquires the program execution situation from an emulation memory part 28. Then an execution situation store part 24 of the part 21 stores the acquired data in a storage means 25 as the execution information. When a retesting execution instruction is given from the means 1, an execution situation loading part 23 of the part 21 reads the program execution situation out of the means 25 and loads it into the part 28. A host computer 4 controls a test device 26 to execute a program to be evaluated and starts again a test at a time when the test is once interrupted.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to a program evaluation system for evaluating a built-in program of a microcomputer for controlling devices. In particular, the present invention relates to a method of performing an evaluation test focusing on input information and output information on a memory used by an evaluation target program, and a program evaluation system capable of automatically executing an evaluation test.

[0002]

2. Description of the Related Art A conventional program evaluation system will be described. For example, JP-A-6-161824 discloses a dedicated evaluation system for generating and tracing input / output signals to / from a pin of a microcomputer,
Actually operate peripheral devices or visually confirm operation by tracing signal patterns generated when operating input devices in the evaluation system main body, tracing the resulting output signals and comparing with expected values The test is running automatically.

[0003] Such a conventional program evaluation system performs a test (a test consisting of a plurality of test items) for confirming a series of operations from the start to the end of a program. A retest was performed by starting over from the beginning.

In Japanese Patent Application Laid-Open No. 64-58027, only the execution state at the time of interruption is confirmed by the function of storing and displaying the register contents at the time of interruption in a memory.

Further, as a program evaluation system which not only evaluates a variable value of a program to be evaluated but also evaluates a range of the variable value, as disclosed in Japanese Patent Application Laid-Open No. 6-148283, an address scan of a memory evaluation pattern is performed. This is realized by a memory tester that specifies a range and displays data in the specified range, and the evaluation tester visually evaluates the displayed data.

[0006]

In the above-described conventional program evaluation system, when a test such as a test sequence including a plurality of test items for performing a test from the start to the end of the program is interrupted, In order to perform the test items after the interruption, the test items must be performed again from the beginning of the test items, so that there is a problem that unnecessary evaluation time is generated.

Another conventional program evaluation system has a function of storing and displaying register contents at the time of test interruption in a memory, but this information cannot be used for re-execution of a test. There was a point.

Further, another conventional program evaluation system can evaluate a variable value of a program to be evaluated as an expected value of a test, but evaluation of a range of the variable value is performed only visually by a data display function. Therefore, there is a problem that evaluation errors are likely to occur.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-described problem. In performing a retest after a test is stopped halfway, the test items are not executed from the beginning but from the time of the test interruption. The purpose is to obtain a program evaluation system that can perform tests.

Another object of the present invention is to provide a program evaluation system capable of evaluating not only the variable values of a program to be evaluated but also the range of the variable values.

[0011]

A program evaluation system according to the present invention executes an emulation operation of an evaluation target program stored in a test apparatus based on pseudo input information from a host computer, and outputs a result. In a program evaluation system for performing a program evaluation test by acquiring as information, the host computer is designated from a test case group describing a plurality of test cases describing pseudo input / output information for executing the evaluation test. A test case analysis unit that analyzes the test case, a test device control unit that receives the pseudo input / output information of the specified test case, and controls the test device, and executes execution information at the time of interruption of the evaluation test. And loading the execution information at the time of interruption into the test apparatus when re-executing the evaluation test. It is obtained by a re-test execution unit.

Further, in the program evaluation system according to the present invention, when the evaluation test is interrupted, the retest execution unit obtains execution information at the time of interruption of the evaluation test, and stores the execution information in execution information storage means. An execution status storage unit,
An execution status loading unit that loads the stored execution information at the time of interruption when the evaluation test is re-executed.
When the evaluation test is re-executed, the test items are performed from the test items after the evaluation test was interrupted.

Further, in the program evaluation system according to the present invention, the test apparatus includes a host computer communication I / F for communicating with the host computer, and an emulation memory unit for storing the evaluation target program.

Further, in the program evaluation system according to the present invention, the test device is an ICE of a target microcomputer.

Further, in the program evaluation system according to the present invention, the host computer may further control a range of the output information acquired by controlling the test apparatus and a range of an expected value described in the specified test case. And a test result range judging unit for judging pass / fail of the evaluation test.

In the program evaluation system according to the present invention, the host computer may further generate an interrupt based on a trace timing described in the specified test case, acquire trace information at that time, and report It has a data trace unit to perform.

Further, in the program evaluation system according to the present invention, the data trace unit may include an interrupt control unit that generates an interrupt based on a trace timing of output information described in the specified test case, A data acquisition unit for acquiring traced data; a data storage unit for storing the traced data as trace information in a trace information storage unit; and a data report unit for reporting the stored trace information.

Further, in the program evaluation system according to the present invention, the host computer further compares the output information obtained by controlling the test apparatus with an expected value described in the specified test case. A test result determination unit for performing an evaluation test; and a test history information generation unit for generating and accumulating the results of the evaluation test as a test history, and reporting the accumulated test history as necessary.

Further, in the program evaluation system according to the present invention, the test history information generating section generates a test history as a test history based on a result of the evaluation test, and a test history storage for storing the generated test history. And a test history report section for reporting the stored test history.

[0020]

BEST MODE FOR CARRYING OUT THE INVENTION

Embodiment 1 FIG. The configuration of the program evaluation system according to the first embodiment of the present invention will be described with reference to FIG. FIG. 1 is a block diagram showing a configuration of the first embodiment of the present invention. In the drawings, the same reference numerals indicate the same or corresponding parts.

In FIG. 1, reference numeral 1 denotes input means, 2 denotes display means, and 3 denotes a printer. The input unit 1 executes a combination and execution of a test case group described below in which input information change timing, change address, change value, output information acquisition timing, reference address, expected value, and memory size are described, and a test case to be executed. Enter the number of times. Display means 2
Displays the execution result of the evaluation test from the host computer described later. Further, the printer 3 outputs information on the result of the evaluation test.

In the same figure, 4 is a host computer, 5 is a test case group, 6 is a test case analysis unit,
Reference numeral 7 denotes a test result determination unit, and reference numeral 8 denotes a test result range determination unit.
The test case group 5 includes a set of a memory change timing of input information corresponding to an input of an evaluation test, a change address and a change value, an acquisition timing and a reference address of output information corresponding to an output generated by the input, an expected value, and a memory. Describes a plurality of test cases consisting of sizes. The test case analysis unit 6 analyzes a test case specified from the test case group 5 and controls execution of an evaluation test. The test result determination unit 7 receives output information obtained by controlling a test device described later and an expected value described in a test case from a memory of an evaluation target device described later, and compares the two by comparing the two. Is determined. Further, the test result range determination unit 8 determines not only the evaluation value but also the value range of the result data.

In the same figure, 9 is a test history information generation unit, 10 is a test history creation unit, 11 is a test history storage unit,
Reference numeral 12 denotes a test history report section. Test history creation unit 10
Acquires the result determined by the test result determination unit 7 or the test result range determination unit 8 and creates test history information that summarizes the test execution date and time and the test result. The test history storage unit 11 stores the created test history information. Further, the test history report unit 12 displays a convergence report of a bug or a past test history from the accumulated test history information.

In FIG. 1, reference numeral 13 denotes storage means,
Reference numeral 4 denotes a test device control unit. The storage unit 13 stores test history information in the test history storage unit 11. The test apparatus control unit 14 receives from the test case analysis unit 6 the timing for changing the input information, the address, the changed value, the timing and address for obtaining the output information, the expected value, and the memory size. Control.

In the same figure, 15 is a data trace unit, 16 is an interrupt control unit, 17 is a data acquisition unit, 18
Denotes a data report unit, 19 denotes a data storage unit, and 20 denotes a storage unit. The interrupt control unit 16 traces variables and labels of the specified evaluation target program at regular intervals.
The data acquisition unit 17 acquires trace information when an interrupt occurs. The data report unit 18 reports an average value, a maximum value, a minimum value, and the like from the acquired trace information. The data storage unit 19 stores the acquired trace information. Further, the storage unit 20 stores the trace information.

In the same figure, reference numeral 21 denotes a retest execution unit, 22 denotes an execution status acquisition unit, 23 denotes an execution status loading unit,
24 is an execution status storage unit, and 25 is a storage unit. The execution status acquisition unit 22 holds the execution status (memory and register contents) at the time of interruption of the test. Further, the execution status loading unit 23 loads information at the time of test interruption into an emulation memory unit to be described later when the test is re-executed. The execution status storage unit 24 stores the execution status. Further, storage means 2
5 stores the execution status as execution information.

In FIG. 2, reference numeral 26 denotes a test device connected to a device to be described later, reference numeral 27 denotes a host computer communication I / F, and reference numeral 28 denotes an emulation memory unit. The host computer communication I / F 27 communicates with the host computer 4. Emulation memory unit 2
Reference numeral 8 emulates a memory that stores an evaluation target program and variables used by the program in an evaluation target device described later.

Further, in FIG.
0 is output information, 31 is an evaluation target program, 32 is an input driver unit, 33 is a program main unit, and 34 is an output driver unit. The input driver unit 32 converts a signal from an input device, which will be described later, into input information 29 that can be used by the program main unit 33 and arranges the input information 29 on a memory. The program main unit 33 implements the essence (specification part) of the function of the program. Further, the output driver unit 34 converts the output information arranged on the memory as a processing result of the program main unit 33 into a signal to be output to an output device described later.

Further, in the figure, reference numeral 35 denotes a device to be evaluated, 36 denotes a microcomputer, 37 denotes an input device, and 38 denotes an output device.

Next, the operation procedure of the operation in the program evaluation system according to the first embodiment will be described with reference to FIG. FIG. 2 is a flowchart showing an operation procedure relating to the retest execution of the evaluation test execution in the first embodiment of the present invention.

First, in step S101, the evaluation test implementer determines the memory change timing of the input information corresponding to the input of the evaluation test, the change address and the change value, and the acquisition timing of the output information corresponding to the output generated by the input. A test case group 5 including a reference address, a set of expected values, and a memory size is input to the input unit 1 of the host computer 4.
Enter from.

Next, in step S102, the host computer 4 analyzes the input test case group 5.

Next, in step S103, the host computer 4 executes the test device 26 based on the test case.
To change the input information 29 in the memory and output information 3
The comparison of 0 is performed, and the evaluation test is executed.

Next, in step S104, the host computer 4 determines whether or not an instruction to cancel the test has been issued from the input means 1 for some reason. When the instruction to stop the test is not issued, the process is terminated, and when the instruction to stop the test is issued, the process proceeds to the next step.

Next, in step S105, if the execution status acquisition unit 22 in the retest execution unit 21 of the host computer 4 gives an instruction to stop the test from the input unit 1 for some reason, the execution status acquisition unit 22 specifies it. The program execution status is acquired from the emulation memory unit 28 based on the memory size thus set.

Next, in step S106, the execution status storage unit 24 in the retest execution unit 21 of the host computer 4 executes the acquired data (execution status) in a secondary storage device (storage means) 25 such as a hard disk. Store as information.

Next, in step S107, the host computer 4 determines whether or not an instruction to re-execute the test has been issued from the input means 1. If the instruction to re-execute the test is not issued, the process ends. If the instruction to re-execute the test is issued, the process proceeds to the next step.

Next, in step S108, the execution status loading unit 23 in the retest execution unit 21 of the host computer 4 checks the execution status of the program when the instruction to execute the test is issued from the input unit 1. The data is read from the storage unit 25 and loaded on the emulation memory unit 28.

Finally, in step S109, the host computer 4 controls the test apparatus 26 to execute the program to be evaluated, and starts a retest.

The program evaluation system according to the first embodiment processes a signal from the input device 37, outputs the result to the output device 38, and performs device control.
The host computer 4 instructs the test apparatus 26 to change and refer to the memory contents based on the reference timing and the change timing of the memory contents, and the test apparatus 26.

The host computer 4 stores the memory change timing, change address, and change value as input information corresponding to the input of the evaluation test, the memory reference timing and reference address of the output information output from the input information, and the expected value. , A test case analysis unit 6 for analyzing a test case group 5 describing a memory size, a test device control unit 14 for controlling a test device 26, etc., and the test device 26 communicates with the host computer 4. A host computer communication I / F 27 for performing the operation, and an emulation memory unit 28.

The emulation memory unit 28 is provided with input / output information 29 and 30 which are input and output between the programs to be evaluated via the host computer communication I / F 27 and an input device 37 of the microcomputer device (the device 35 to be evaluated). An input driver unit 3 for converting a signal into input information that can be identified by the program main unit 33 and arranging it on a memory
2, an output driver unit 34 that converts a processing result of the program main unit 33 into a signal for output control in the output device 38, and a program main unit 33 of a microcomputer device to be evaluated.

The host computer 4 has an execution status acquisition unit 22 for holding the execution status (memory and register contents) when the test is interrupted, and an execution status storage unit 24 for storing the execution status. A retest execution unit 21 including an execution status loading unit 23 for loading execution information at the time of test interruption into the emulation memory unit 28 when the test is re-executed.

That is, in the first embodiment, when the test is interrupted, the execution state (contents of memories and registers) at the time of interruption is held, and the execution state held earlier when the retest is executed is performed by the emulation memory unit 28. , It is not necessary to execute the test items from the beginning when performing the retest, the test can be started from the test item at the time of the test interruption, and the evaluation time can be shortened.

Embodiment 2 The program evaluation system according to the second embodiment of the present invention is obtained by replacing the test apparatus 26 of the first embodiment with an ICE (In Circuit Emulator).

In this case, the host computer 4 analyzes the input test case group 5, controls the ICE to change the input information in the memory and collate the output information, and executes an evaluation test. Thereby, the operation and operation confirmation test of the evaluation target device 35 can be performed by a commercially available tool.

In the second embodiment, an ICE of a target microcomputer is used as the test apparatus 26, the execution state (memory and register contents) at the time of interruption of the test is held when the test is interrupted, and the By loading the execution state stored in the emulation memory unit 28 into the emulation memory unit 28, it is not necessary to execute the test item from the beginning when performing the retest, and the test from the test item at the time of the test interruption can be performed.

According to the second embodiment, the test device 26
By using ICE instead of, the execution state (contents of memories and registers) at the time of a test interruption is retained even when the test is interrupted, and the information at the time of the test interruption is emulated by the emulation memory at the time of retesting. By loading the data in the section 28, an evaluation test can be performed from the time of the test interruption. Therefore, a flexible program evaluation system can be constructed.

Embodiment 3 A program evaluation system according to Embodiment 3 of the present invention will be described with reference to FIG. FIG. 3 is a flowchart showing an operation procedure relating to the evaluation of a variable value range of execution of an evaluation test according to Embodiment 3 of the present invention. The configuration is the same as in the first embodiment.

First, in step S201, the evaluation tester determines the memory change timing of the input information corresponding to the input of the evaluation test, the change address and the change value, and the acquisition timing of the output information corresponding to the output generated by the input. A test case group 5 describing a plurality of test cases composed of reference addresses, expected value ranges, and the like is input from the input unit 1 of the host computer 4.

Next, in step S202, the host computer 4 analyzes the input test case group 5.

Next, in step S203, the host computer 4 controls the test apparatus 26 based on the test case group 5 to change the input information 29 in the memory.
Execute the program to be evaluated.

Next, in step S204, the test apparatus control section 14 of the host computer 4 acquires the output information generated by the input by controlling the test apparatus 26 based on the acquisition timing and the reference address.

Next, in step S205, the test result range determination section 8 of the host computer 4 performs evaluation by comparing the acquired data with the range of the expected value.

The evaluation performed here is as follows:
One or two values are specified, and the range is compared with the execution result. For example, if the expected values are A and B
If the output information is specified as being within the range, the output information generated by the input of the evaluation test is determined to be a test pass if the output information is within the range, otherwise the test is determined to have failed.

In addition to the contents of the evaluation, in addition to the specification in the range, the evaluation contents are outside the range of A and B, smaller than A, smaller than or equal to A, equal to A, larger than or equal to A,
A range can be specified, such as greater than A or not equal to A.

As described above, by specifying the range of the variable value in the evaluation test, it is possible to evaluate the upper limit value and the lower limit value which are the variable value ranges specified when the evaluation target program is executed.

In the third embodiment, the test result range determining unit 8 for verifying the range of the variable value specified when the evaluation target program is executed by specifying the range of the variable value of the evaluation target program in the evaluation test is executed by the host computer. 4 is provided.

According to the third embodiment, by specifying the range of the variable value in the evaluation test, it is possible to evaluate the upper limit value and the lower limit value which are the specified variable value range of the evaluation target program. .

Embodiment 4 A program evaluation system according to Embodiment 4 of the present invention will be described with reference to FIG. FIG. 4 is a flowchart showing an operation procedure relating to an evaluation for tracing variables and labels of a program for executing an evaluation test at regular intervals according to the fourth embodiment of the present invention. The configuration is the same as in the first embodiment.

First, in step S301, the evaluation tester performs a memory change timing of input information corresponding to an input of an evaluation test, a change address and a change value, and a timing of tracing output information corresponding to an output generated by the input. A test case group 5 describing a plurality of test cases composed of addresses to be traced and the like is input from the input unit 1 of the host computer 4.

Next, in step S302, the host computer 4 analyzes the input test case group 5.

Next, in step S303, the host computer 4 controls the test apparatus 26 based on the test case group 5 to change the input information 29 in the memory.
Execute the program to be evaluated.

Next, in step S304, the interrupt control unit 16 in the data trace unit 15 of the host computer 4 generates an interrupt based on the output information trace timing described in the test case group 5.

Next, in step S305, the data acquisition unit 17 in the data trace unit 15 of the host computer 4 acquires data to be traced.

The timing of tracing can be specified in units of time or in units of tracing each time a variable to be obtained is accessed by the program to be evaluated.

Next, in step S306, the data storage unit 19 in the data trace unit 15 of the host computer 4 stores the traced data in a secondary storage device (storage means) 20, such as a hard disk, as trace information.

Then, in step S307, the data report unit 18 in the data trace unit 15 of the host computer 4 converts the trace information stored in the storage unit 20 into a value traced from the display unit 2 or the printer 3, an average value, Display and print the maximum and minimum values.

In the fourth embodiment, an interrupt control unit 16 for tracing variables and labels of a specified evaluation target program at regular intervals, a data acquisition unit 17 for acquiring data when an interrupt occurs, The host computer 4 is provided with a data trace unit 15 including a data storage unit 19 for storing data and a data report unit 18 for reporting acquired trace information.

According to the fourth embodiment, by tracing the values of variables and labels of a specified evaluation target program at regular time intervals, it is possible to report the average value, maximum value, and minimum value of the traced values. it can.

Embodiment 5 FIG. A program evaluation system according to Embodiment 5 of the present invention will be described with reference to FIG. FIG. 5 is a flowchart showing an operation procedure for creating a test history of execution of an evaluation test according to the fifth embodiment of the present invention. The configuration is the same as in the first embodiment.

First, in step S401, the evaluation tester determines the memory change timing of the input information corresponding to the input of the evaluation test, the change address and the change value, and the acquisition timing of the output information corresponding to the output generated by the input. A test case group 5 describing a plurality of test cases composed of a set of a reference address, an expected value, and the like is input from the input unit 1 of the host computer 4.

Next, in step S402, the host computer 4 analyzes the input test case group 5.

Next, in step S403, the host computer 4 controls the test apparatus 26 based on the test case group 5 to change the input information 29 in the memory.
Execute the program to be evaluated.

Next, in step S404, the test apparatus controller 14 of the host computer 4 acquires the output information generated as a result of the input based on the acquisition timing and the reference address described in the test case group 5, and The evaluation test is executed by comparing the expected value described in the test case group 5 with the obtained output information by the result determination unit 7 or the test result range determination unit 8.

Next, in step S405, the test history creating section 10 in the test history information creating section 9 of the host computer 4 creates a test result as a test history.

The test history is information in which date and time of test execution, test pass / fail information, test contents, and error information (expected value, result) are described. The test history information 13 is updated (added) every time the test is executed. The history of tests performed in the past is accumulated.

Next, in step S406, the test history storage unit 11 in the test history information generation unit 9 of the host computer 4 stores the created test history in the secondary storage device (storage means) 13 such as a hard disk. Store as information.

Next, in step S407, the test history report unit 12 in the test history information generation unit 9 of the host computer 4 displays the stored test history information on the display unit 2.
Alternatively, it is displayed or printed by the printer 3.

Therefore, in steps S408 to S409, the evaluation tester judges whether the displayed or printed test result is OK. That is, the evaluation tester can confirm whether the program is operating normally by looking at the test results. If the output information is different from the expected one, modify the evaluation target program.

The program evaluation test is executed by repeating such a procedure for a plurality of test items. Also, by accumulating the created test execution date and time, it can be used for creating documents such as bug convergence report and test report.

In the fifth embodiment, a test history creating unit 10 for creating a test execution result as a test history, a test history storage unit 11 for storing the created test history, and a bug convergence report based on the accumulated test history information. And a test history information generating unit 9 including a test history report unit 12 for outputting a past test history.

According to the fifth embodiment, a test execution result is created as a test history, and the test history information is accumulated, so that a convergence report of a bug of the program to be evaluated and the output (display, printing) of the past test history )It can be performed. In addition, the test history information can be used to create a document such as a test report by outputting the test history.

[0084]

As described above, the program evaluation system according to the present invention executes the emulation operation of the evaluation target program stored in the test apparatus based on the pseudo input information from the host computer, and outputs the result. In a program evaluation system for performing a program evaluation test by acquiring as output information, the host computer specifies a test case describing a plurality of test cases describing pseudo input / output information for executing the evaluation test from a test case group describing a plurality of test cases. A test case analysis unit for analyzing the specified test case, a test device control unit for receiving the pseudo input / output information of the specified test case, and controlling the test device, and execution information at the time of suspension of the evaluation test And the execution information at the time of interruption is stored in the test apparatus when the evaluation test is re-executed. The re-test execution section that performs the re-test eliminates the need to execute the test items from the beginning when performing re-tests. Play.

Further, in the program evaluation system according to the present invention, as described above, the retest execution unit obtains execution information at the time of interruption of the evaluation test when the evaluation test is interrupted, and executes the execution information. An execution status storage unit for storing in the information storage means; and an execution status loading unit for loading the stored execution information at the time of re-execution of the evaluation test into the test apparatus. Since the test is performed from the subsequent test items, it is not necessary to execute the test items from the beginning when performing the retest, and the test can be started from the test item at the time of the test interruption, and the evaluation time can be shortened.

Further, as described above, in the program evaluation system according to the present invention, the test apparatus includes a host computer communication I / F for communicating with the host computer, and an emulation memory unit for storing the evaluation target program. Therefore, it is not necessary to execute the test item from the beginning when performing the retest, and the test can be started from the test item at the time of the test interruption, and the evaluation time can be shortened.

Further, as described above, the program evaluation system according to the present invention has an effect that a flexible system can be constructed because the test apparatus is an ICE of a target microcomputer. .

Further, in the program evaluation system according to the present invention, as described above, the host computer further controls the test apparatus to describe the range of output information and the specified test case. The test result range determination unit that compares the expected value with the range of the evaluation test to determine whether the evaluation test is successful or not has the effect that the upper limit and the lower limit of the variable value range of the evaluation target program can be evaluated. Play.

In the program evaluation system according to the present invention, as described above, the host computer further generates an interrupt based on the trace timing described in the specified test case, and the trace information at that time is generated. Since the data tracing unit is provided to acquire and report the values of the variables, the value, average value, maximum value, minimum value, and the like of the traced variable can be reported.

Further, in the program evaluation system according to the present invention, as described above, the data trace unit generates the interrupt based on the trace timing of the output information described in the specified test case. A data acquisition unit that acquires data traced when an interrupt occurs, a data storage unit that stores the traced data as trace information in a trace information storage unit, and a data report unit that reports the stored trace information. The value of the traced variable,
It is possible to report an average value, a maximum value, a minimum value, and the like.

Further, in the program evaluation system according to the present invention, as described above, the host computer further controls the test apparatus to obtain the output information and the expected value described in the specified test case. And a test history information generating unit that creates and accumulates the result of the evaluation test as a test history and reports the accumulated test history as necessary. , Reports the convergence of bugs in the target program and displays past test history,
This has the effect that it can be used to create documents such as test reports.

Further, as described above, in the program evaluation system according to the present invention, the test history information generating unit generates a test history based on a result of the evaluation test as a test history; Since it includes a test history storage unit for storing and a test history report unit for reporting the stored test history, it is possible to display a convergence report of a bug of a program to be evaluated and to display a past test history. This has an effect that it can be used for creating documents such as books.

[Brief description of the drawings]

FIG. 1 is a diagram showing a configuration of a program evaluation system according to Embodiment 1 of the present invention.

FIG. 2 is a flowchart showing a work procedure for performing an evaluation test according to the first embodiment of the present invention.

FIG. 3 is a flowchart showing a work procedure for performing an evaluation test according to a third embodiment of the present invention.

FIG. 4 is a flowchart showing a work procedure for performing an evaluation test according to a fourth embodiment of the present invention.

FIG. 5 is a flowchart showing a work procedure for performing an evaluation test according to a fifth embodiment of the present invention.

[Explanation of symbols]

1 input means, 2 display means, 3 printer, 4 host computer, 5 test case group, 6 test case analysis section, 7 test result judgment section, 8 test result range judgment section, 9 test history information generation section, 10 test history creation Department,
11 Test history storage unit, 12 Test history report unit, 1
3 storage means (test history information), 14 test apparatus control units,
15 data trace section, 16 interrupt control section, 17
Data acquisition unit, 18 data report unit, 19 data storage unit, 20 storage means (trace information), 21 retest execution unit, 22 execution status acquisition unit, 23 execution status loading unit, 24 execution status storage unit, 25 storage unit ( Execution information), 26 test equipment, 27 host computer communication I / F, 28 emulation memory unit, 29 input information, 30 output information, 31 evaluation target program, 32
Input driver unit, 33 program main unit, 34 output driver unit, 35 evaluation target device, 36 microcomputer, 37 input device, 38 output device.

Claims (9)

[Claims] 1. A program for performing an emulation operation of an evaluation target program stored in a test apparatus based on pseudo input information from a host computer and acquiring a result as output information to perform an evaluation test of the program. In the evaluation system, the host computer includes: a test case analysis unit configured to analyze a specified test case from a test case group describing a plurality of test cases describing pseudo input / output information for executing an evaluation test; A test apparatus control unit that receives pseudo input / output information of a specified test case and controls the test apparatus; and acquires execution information at the time of interruption of the evaluation test, and executes the interruption at the time of re-execution of the evaluation test. And a retest execution unit for loading information into the test apparatus. System. 2. An execution status acquisition unit that acquires execution information at the time of interruption of an evaluation test, an execution status storage unit that stores the execution information in execution information storage means, and an evaluation test. 2. An execution status loading section for loading the stored execution information at the time of re-execution into the test apparatus, wherein the re-execution of the evaluation test is performed from the test items after the interruption of the evaluation test. The described program evaluation system. 3. The program evaluation according to claim 2, wherein the test apparatus includes a host computer communication I / F that communicates with the host computer, and an emulation memory unit that stores the evaluation target program. system. 4. The program evaluation system according to claim 1, wherein the test device is an ICE of a target microcomputer. 5. The host computer further compares a range of output information acquired by controlling the test apparatus with a range of expected values described in the specified test case to determine whether the evaluation test is successful or not. The program evaluation system according to claim 1, further comprising a test result range determination unit for determining. 6. The host computer further includes a data trace unit that generates an interrupt based on a trace timing described in the designated test case, and acquires and reports the trace information at that time. The program evaluation system according to claim 1 or 4, wherein: 7. The data trace unit includes: an interrupt control unit configured to generate an interrupt based on a trace timing of output information described in the designated test case; and a data acquisition unit configured to acquire data traced when the interrupt occurs. The program evaluation system according to claim 6, further comprising: a data storage unit that stores the traced data as trace information in a trace information storage unit; and a data report unit that reports the stored trace information. . 8. The host computer further includes: a test result determination unit that compares an output information obtained by controlling the test apparatus with an expected value described in the specified test case to perform an evaluation test. And a test history information generating unit that creates and accumulates the result of the evaluation test as a test history, and reports the accumulated test history as needed.
The described program evaluation system. 9. The test history information generation unit, a test history creation unit that creates a result of the evaluation test as a test history, a test history storage unit that stores the created test history, and the stored test history 9. The program evaluation system according to claim 8, further comprising: a test history report section for reporting.