JPH0546435A - Method and device for automatic test - Google Patents

Abstract

PURPOSE:To automatize the system test of the same software operated in plural computers. CONSTITUTION:Operation procedures of test object soft are preliminarily stored in an operation storage means 1, and an operation generating means 2 reads out operation procedures from the means 1 to send them to an operation procedures sending means 3. This means 3 sends these operation procedures to the test object software operated in plural computers. Execution results of test object software are recorded by a result recording means 4 and are compared with one another by a result comparing means 5 to output the verification result to a picture or the like. Thus, effects of the reduction of the time and persons required for the system test, the improvement of accuracy of the test, the improvement of the fault detection rate in various conditions, etc., are obtained.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an automatic test device and an automatic test method for automatically performing a system test of interactive software.

[0002]

2. Description of the Related Art In recent years, it has become more and more important to improve quality as the scale of software increases. In particular, the quality of interactive application software typified by a word processor is greatly affected by the quality and quantity of a test process (hereinafter referred to as a system test) in an actual use environment according to external specifications. However, for the reasons listed below, conducting system tests has required a great deal of time and personnel.

Every time the version of the application software itself is revised or a bug is corrected, it is necessary to repeat the same test contents as the previous one in order to guarantee the previous quality of the function that has not been changed. When porting to another model or upgrading the system such as the OS, it is necessary to re-execute the test having exactly the same content as the previous test, although the function of the application software is not changed.

Whether the test result is correct or not is determined by
In many cases, humans visually compare whether the execution results are the same as the external specifications. Therefore, an automatic test device for automating these system tests has been proposed. FIG. 14 shows the structure of a conventional automatic test apparatus. In the figure, 121 is an operation storage means,
122 is an operation generating means, 123 is a correct answer file comparing means, 124 is a result recording means, and 125 is a correct answer file storing means. Further, FIG. 15 shows a form in which a conventional automatic test device is used. In the figure, 131 is a conventional automatic test apparatus, and 132 is a computer that executes software to be tested. Note that FIG. 15 shows a general form, and in an environment in which a plurality of processes can be executed by one computer such as UNIX, the test target software may be the same computer as the automatic test device.

The automatic test apparatus configured as described above proceeds with processing in the order listed below. (1) First, the test target software is activated on the computer 132. (2) The operation generation unit 122 reads the operation procedure from the operation storage unit 121 and executes the test target software. (3) The result recording means 124 records the execution result of the software to be tested. (4) The correct answer file comparing means 123 is the result recording means 1
Execution result recorded in 24 and correct answer file storage means 125
The corresponding correct execution result of is compared and the comparison result is output to a screen or the like. (5) If there is an operation that has not been executed yet in the operation storage unit 121, the above processes (2) to (4) are repeated.

[0006]

However, in the above configuration, the correct execution result of the software to be tested must be stored in advance in the correct answer file storage means, which is troublesome and requires a large storage device. Become. Further, there is a problem that the test cannot be performed in a situation where the correct answer file storage means is not stored.

In view of the above problems, it is an object of the present invention to provide an automatic test apparatus and an automatic test method which do not require the correct answer file to be stored in advance and enable a test at any time. ..

[0008]

In order to solve the above-mentioned problems, the automatic test apparatus of the present invention comprises an operation storage means for storing an operation procedure of software to be tested, and an operation generation for reading the operation procedure from the operation storage means. Means, an operation procedure sending means for receiving the operation procedure from the operation generating means, sending the operation procedure to the test target software operating in a plurality of environments, and recording the execution result of the operation procedure for each test target software. The result recording means and the inter-result comparison means for comparing the execution results of the respective test target software recorded by the result recording means.

Further, the automatic test method of the present invention comprises a step of reading out the operating procedure from an operation storing means for storing the operating procedure of the software to be tested, and a test in which the read operating procedure is operating in a plurality of environments. The method is characterized by including a step of sending the target software to be executed, a step of recording the execution result of the operating procedure in each test target software, and a step of comparing the recorded execution results of the respective test target software.

[0010]

According to the present invention, with the above configuration, the correct answer file storage means can be omitted by comparing the execution results among a plurality of software under test at the time of testing.

[0011]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 is a diagram showing a usage pattern of the present invention.
In the figure, 111 is an automatic test device, and 112 to 114.
Is a computer that executes the software to be tested. The automatic test equipment 111 and each of the computers 112 to 114 are connected by a line 115 capable of data transmission. Of course, a publicly known method such as a bus and a LAN can be adopted to connect the automatic test device and each computer. In addition, the figure shows a general usage pattern. In an environment where multiple processes can be executed by one computer such as UNIX, the test target software runs on the same computer as the automatic test equipment. May be Further, in the figure, the case where the software to be tested is executed by three computers is shown, but the number may be two, or four or more.

As shown in FIG. 2, the automatic test apparatus 111 comprises an operation storage means 1, an operation generation means 2, an operation procedure sending means 3, a result recording means 4 and a result comparison means 5. The operation storage unit 1 stores the operation created by the operator in advance. Here, the operation means, for example, when the software to be tested is the software of the word processor, as shown in the operation storage means 2 of FIG.
It means that character codes such as "A" and "I" and control codes such as "copy" and "print" are arranged in a predetermined order. A code such as a character code or a control code is generally used as a format for storing the operation in the operation storage means 1, but a voice signal or the like may be used.

The operation generation means 2 reads out the operation stored in the operation storage means 1 and supplies it to the operation procedure sending means 3. In this case, the language format stored in the operation storage unit 1 and the computer 112 that is the destination of the operation procedure transmission unit 3
If the language that can be processed by the computer is different, the process of converting to a language that can be processed by the computers 111 to 113 is also performed. The operations can be read from the operation storage means 1 in the order in which they are stored, but they can also be read by any method that makes it easy to find a failure in the software under test. Such an embodiment will be described later.

The operation procedure sending means 3 sends the operation procedure received from the operation generating means 2 to each of the computers 111 to 113. In this case, the operation procedure is transmitted in time sharing for each operation content or for each unit (for example, several bits to several tens of bits) so that the computers 111 to 113 can simultaneously execute the operation procedure. The result recording unit 4 exists in correspondence with the number of the computers 111 to 113 in order to record the processing result of each of the computers 111 to 113. The results to be recorded are, for example, when the software to be tested is a word processor, bit data corresponding to a screen image, print data to be printed, a data file of the software to be tested, or the type of failure to be detected. Can be selected according to.

The result comparison means 5 compares the results recorded by the result recording means 4. As a comparison method, as shown in FIG. 3, the results A, B, and C recorded by the result recording means 4a, 4b, and 4c are converted into common formats A ′, B ′, and C ′ for post-comparison. The method and the inter-result comparison means 5 as shown in FIG. 4 have information on the difference in the format among the results A, B, and C, so that the results are immediately converted without conversion into a common format. A method of making a comparison can be implemented.

As a result of the comparison between the results, for example, B and C are the same result, but A is different.
The comparison status of B and C is output. From this output, the operator determines that the software under test that has produced the result of A is a failure based on the principle of majority voting, for example. If there is software that has no faults in advance among multiple software to be tested, comparison between results can be performed by comparing other results based on the processing results of the software that does not have faults. In that case, the principle of majority voting You don't have to.

FIG. 5 is a flow chart for explaining the operation of the above automatic test device. In # 1, first, the software to be tested is started on the computers 111, 112, 113, and the operation generating means 2 reads the operation procedure from the operation recording means 1 (#
2). Then, the operation procedure sending means 3 causes the operation generating means 2 to
Operation procedure from a plurality of computers 111, 11
The operation procedure is sent to the software to be tested operating at 2 and 113 to be executed at the same time (# 3). Next, the result recording means 4 records the execution result of each test target software executed by each of the computers 111 to 113 (# 4). Finally, the result comparison unit 5 compares the execution results of the test target software recorded by the result recording unit 4 and outputs the comparison result on a screen or the like (# 5). The above processing is executed until there is any operation that has not been executed yet in the operation storage means 1 (#
6). It is possible to detect the difference in execution results by sending the same operation to each test target software operating on multiple computers and executing them simultaneously. Then, in this case, the test target software that operates correctly is moved on one computer, and the test target software under test is moved on the remaining computers to test the test target software that has been ported to multiple computers at once. Therefore, it is possible to significantly reduce the time.

Next, FIGS. 6 and 7 show other processing contents that can be adopted in the operation generating means 2. Among them, FIG. 6 shows a flow of the operation generating means 2 having a random address generating function capable of randomly specifying an address for the operation storing means 1. In # 21, an arbitrary operation is randomly selected from the operation storage means 1 and sent to the operation procedure sending means 3. In # 22, a preset termination condition, for example, the number of times or elapsed time is checked, and if the conditions are met, the process is terminated. If the conditions are not met, the process returns to # 21.

As in this embodiment, if the operation procedure is read out randomly and executed on each piece of software to be tested, there is an advantage that a fault can be more easily found. On the other hand, the processing content of FIG. 7 reads a specific operation from the operation storage means 1 (# 3
1) Until the predetermined end condition (for example, the number of times or the elapsed time) is reached (# 31), it is sent to the operation procedure sending means 3 (# 32).

The operation procedure sending means 3 is composed of the computers 111 to 1
Since the data is sent to 13, the test target software operating on each computer repeatedly executes a specific operation a predetermined number of times or for a predetermined time. In this way, by repeatedly executing a specific operation on each piece of software to be tested, it is possible to increase the possibility of finding a failure in a harsh usage environment.

8 and 9 show other processing contents that can be adopted in the operation procedure sending means 3. Of these, Figure 8
Is a computer running the software under test 111-
11 shows an operation procedure sending operation in consideration of the machine speed of 113. The time to wait in # 43 is adjusted to the machine speed of the slowest one of all the computers. By considering the machine speed in this way, it is possible to avoid the inconvenience that some of the operating procedures are not executed by any of the test target software, and it is possible to preset the time interval for sending the operation to the test target software, It is possible to carry out the test in consideration of the processing time of the result recording means 4 and the inter-result comparison means 5.

On the other hand, FIG. 9 shows a procedure for monitoring the completion of execution of each test target software and transmitting the operation data. In # 53, the method of confirming the end of execution of the software to be tested may be polling or sensing. According to such processing contents, it is possible to synchronize the timing of sending an operation to the software to be tested and the end of the previous operation execution, and the software to be tested reliably executes the next operation in an acceptable state. It becomes possible.

Next, FIGS. 10 to 12 show three examples of processing contents in the result recording means 4. The process of FIG. 10 is a reading of a screen image of the test target software.
Here, the screen image refers to screen bitmap information, a screen description language, or the like. According to the method of recording with a screen image in this way, it is possible to compare even differences that human eyes tend to overlook, and it is possible to detect a fault occurring as a screen image with higher accuracy.

FIG. 11 shows the processing for reading the print image of the software to be tested. Here, the print image refers to print-required bitmap information or page description language of software having a printing function such as a word processor. According to this processing content, as compared with the conventional method in which the print result is output to a printer on paper and the comparison and verification with external specifications are performed manually, there are subtle differences in printing, such as character fonts. It is possible to easily find differences in layout and layout. Also, a great saving of paper is possible.

FIG. 12 shows the processing when reading the data file of the software to be tested. Here, the data file refers to a word processing document file, a database of database software, or the like. According to this processing content, compared to the conventional method of dumping the contents of the data file and comparing and verifying whether it is correct by hand, the time required for comparison can be greatly saved and the accuracy can be increased. Is possible.

In the above embodiment, the case where only the operation procedure is stored in the operation storage means 1 has been described, but the time interval between the operation procedures may be stored together with the operation procedure. In that case, it is rational that the operation procedure sending means 3 does not read the time interval but the operation generating means 2. FIG. 13 is a flow chart showing the processing of the operation procedure sending means 3 when the time interval is also stored in the operation storage means 1. In this figure, the time multiplier of # 104 is a value inversely proportional to the machine speed of each computer. That is, for example, a computer with a double machine speed does not need the same waiting time as other computers, but may have half the waiting time. Therefore, it is 1/2 as a time multiplier for a computer with double machine speed.
Is prepared, and each operation procedure is transmitted after waiting for the time obtained by multiplying the read time interval by the time multiplier (# 105).

[0027]

As described above, according to the present invention, the operation procedure is read from the operation storage means for storing the operation procedure of the test target software, and the read operation procedure is sent to the test target software operating in a plurality of environments. As it is executed and records the result executed by the software to be tested and compares it, it reduces the time and personnel required for the system test, increases the accuracy of the test, and improves the fault detection rate under various conditions. There is an effect that the effect such as can be obtained.

[Brief description of drawings]

FIG. 1 is a system diagram showing a usage pattern of an automatic test apparatus according to an embodiment of the present invention.

FIG. 2 is a diagram showing an automatic test apparatus as one embodiment of the present invention.

FIG. 3 is a diagram showing an example of a result comparison means.

FIG. 4 is a diagram showing another example of the result comparison means.

FIG. 5 is a flow chart illustrating the operation of the automatic test device.

FIG. 6 is a flowchart showing processing contents that can be adopted by the operation generating means.

FIG. 7 is a flowchart showing another processing content that can be adopted by the operation generating means.

FIG. 8 is a flowchart showing processing contents that can be adopted by the operation procedure sending means.

FIG. 9 is a flowchart showing another processing content that can be adopted by the operation procedure sending means.

FIG. 10 is a flowchart showing the processing contents of the result recording means.

FIG. 11 is a flowchart showing the processing contents of the result recording means.

FIG. 12 is a flowchart showing the processing contents of the result recording means.

FIG. 13 is a system diagram showing a usage pattern of a conventional automatic test apparatus.

FIG. 14 is a diagram showing a configuration of a conventional automatic test device.

FIG. 15 is a diagram showing a usage pattern of a conventional automatic test device.

[Explanation of symbols]

 1 Operation Storage Means 2 Operation Generation Means 3 Operation Procedure Sending Means 4 Result Recording Means 5 Result Comparison Means 111 Automatic Test Equipment 112 Calculator 113 Calculator 114 Calculator 121 Operation Storage Means 122 Operation Generating Means 123 Correct Answer File Comparison Means 124 Result Recording Means 125 Correct answer file storage means 131 Automatic test device 132 Computer

Claims (14)

[Claims] 1. An operation storage unit that stores an operation procedure of software to be tested, an operation generation unit that reads the operation procedure from the operation storage unit, and an operation procedure that is received from the operation generation unit, and operates in a plurality of environments. Operation procedure sending means for sending the operation procedure to the test target software to be executed, result recording means for recording the execution result of the operation procedure for each test target software, and execution result of each test target software recorded by the result recording means And an inter-result comparison means for comparing the above. 2. The automatic test apparatus according to claim 1, wherein the operation generation means reads out an operation from the operation storage means at random and sends it to the operation procedure sending means. 3. The automatic test apparatus according to claim 1, wherein the operation generating means reads out a specific operation from the operation storing means and repeatedly sends it to the operation procedure sending means. 4. The operation procedure sending means sends the operation procedure received from the operation generating means to the test object software at a fixed time interval from the time when the operation procedure was sent to the test object software last time. The automatic test device according to claim 1. 5. The operation procedure sending means sends the operation procedure received from the operation generating means to the test object software after waiting for completion of execution of the test object software associated with the operation procedure sent to the test object software last time. The automatic test apparatus according to claim 1, which is characterized in that. 6. The automatic test apparatus according to claim 1, wherein the result recording means reads a screen image, a print image or a data file of the software to be tested and sends it to the result comparison means. 7. The automatic test apparatus according to claim 1, wherein the operation recording means stores an operation procedure of the software to be tested together with a time interval between the operation procedures. 8. The operation means sending means has the operation procedure received from the operation generating means at a time interval of the operation procedure stored in the operation storage means from the time when the operation procedure was sent to the software to be tested last time. 8. The automatic test apparatus according to claim 7, wherein the automatic test apparatus sends the test software at a time interval multiplied by a fixed number. 9. A step of reading an operating procedure from an operating storage means that stores the operating procedure of the software to be tested,
A step of sending the read operation procedure to a test target software operating in a plurality of environments for execution, and a step of recording the execution result of the operation procedure in each test target software,
An automatic test method comprising: a step of comparing execution results of the recorded software to be tested. 10. The automatic test method according to claim 9, wherein in the step of reading the operation procedure from the operation storage means, a read address is randomly designated. 11. The automatic test method according to claim 9, wherein in the step of reading the operation from the operation storage means, the operation is read from a specific address and repeatedly sent to the software to be tested. 12. The step of sending the operating procedure to the test target software for execution includes sending the operating procedure received from the operation generating means at a fixed time interval from the time when the operating procedure was sent to the test target software last time. The automatic test method according to claim 9, which is characterized in that. 13. The step of sending the operation procedure to the test target software for execution, after waiting for the operation procedure received from the operation generating means to wait for the execution end of the test target software associated with the operation procedure previously sent to the test target software, The automatic test method according to claim 9, wherein the automatic test method is sent. 14. The automatic test method according to claim 9, wherein the step of recording the execution result of the operation procedure includes printing or reading a screen image of the software to be tested.