JP2007026306A - Program test device and method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a program test device for raising readability in a test scenario which is used in the test of a program to be tested. <P>SOLUTION: A part 110 for simulating a service to be called simulates the external service to be called from the program to be tested. A part 102 for performing the program to be tested performs the program 160 to be tested. A test scenario analysis performing part 101 transmits message reception to the part 110 for simulating the service to be called when it is described in the test scenario 150 that a message is to be received from the program to be tested. When a message reception receiving part receives the message reception from the test scenario analysis performing part 101 and also receives the message from the part 102 for performing the program to be tested, a message waiting part 112 transmits the received message to the test scenario analysis performing part 101. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a program test apparatus, method, and program, and more particularly to a program test apparatus, method, and program for testing a program whose execution proceeds by exchanging messages.

  There are programs that call other services and modules, exchange messages with the called services, etc., and proceed with processing. As a technique for testing such a program, there is a technique described in Non-Patent Document 1. FIG. 17 shows a state of the test in the program test system described in Non-Patent Document 1. In this technique, a program under test 502 is tested using a test program 501 that executes a test and a simulated object 503 that simulates the operation of an object called by the program under test.

  FIG. 18 shows an operation flow during the test of the program test system described in Non-Patent Document 1. First, prior to test execution, a behavior such as what value is returned to the simulated object 503 when called from the program under test is set (step M1). Next, the program under test 502 is executed (step M2), and it is verified whether the simulated object 503 has been correctly called and an expected value has been passed (step M3).

  By the way, as a technique for testing a test target program using a test scenario, there is a technique described in Patent Document 1. FIG. 19 shows a schematic configuration of the test system described in Patent Document 1. In this technique, a test scenario 504 and a test program 505 that executes the test scenario are used to test the operation of the program under test 506 in the GUI environment.

  FIG. 20 shows an operation procedure of the test system shown in FIG. The test program 505 gives an input such as a mouse operation to the test program 506 in accordance with the test scenario 504 (step G1). The program under test 506 returns an output such as a screen transition to the test program 505 as a result of the input. The test program 505 verifies whether the program under test 506 has returned the expected output according to the test scenario 504 (step G2). Next, it is determined whether or not the test scenario 504 has been executed to the end (step G3). When the test scenario 504 has not been executed to the end, the process returns to step G1 to continue the test.

JP 2002-366387 A Endo Testing: Unit Testing with Mock Objects (URL http://www.connextra.com/about/Us/mockobjects.pdf)

  In the technique described in Non-Patent Document 1, it is necessary to set the behavior of the simulated object called from the program under test before executing the program under test. For this reason, there is a problem that after a certain event occurs, it must be written on the test program before it occurs. In addition, the behavior setting of the simulated object and the verification thereof are written separately to facilitate fragmentation. As described above, in Patent Document 1, the test program is written in a different order from the time sequence of the test procedure or is fragmented even though it is desired to test the flow of a certain procedure, so that the readability of the test program is improved. There is a problem that the maintenance becomes worse.

  In the technique described in Non-Patent Document 1, it is possible to test whether or not the test program has finally completed execution by calling a simulated object and executing the execution. However, when the test program calls a plurality of services (simulated objects), there is a problem that it is difficult to know in what order the simulated objects are called.

  The technique described in Japanese Patent Laid-Open No. 2004-228561 provides an input to a program under test and verifies an output returned for the given input. For this reason, when the program under test accepts input from other services or calls other services before the input is given to the program under test and the output is returned, the order of acceptance and the order of invocation There is a problem that it cannot be verified. That is, in Patent Document 1, when the program under test calls a service or when execution proceeds by being called from a service, it is not possible to test the message exchange order.

  In each of the above prior arts, the order of calls to the program under test is considered, but the order of calls from the program under test is not fully considered. For this reason, when the call of the external service by the program under test and the call of the program under test by the external service are mixed, there is a problem that it is difficult to verify whether the order is as designed. .

  The present invention solves the above-mentioned problems of the prior art and calls a service or the like, or a program under test whose processing progresses by being called by a service or the like, a program with high readability of the test content and high maintainability of the test itself It is an object to provide a test apparatus, method, and program.

  Further, the present invention can easily verify that calls are made in the order according to the test scenario even when calls by the program under test and calls of the program under test by services or the like are mixed. It is an object to provide a program test apparatus, method, and program.

  In order to achieve the above object, the program test apparatus according to the present invention provides a program under test for transmitting / receiving a message to / from an external service and a message to be transmitted / received to / from the program under test. A program test apparatus for testing according to a described test scenario, comprising: a test scenario interpretation execution unit that reads the test scenario from a storage device, interprets the read test scenario, and executes the test scenario. When receiving a message having a test program as a caller, a test scenario interpretation execution unit for transmitting a message acceptance signal for accepting the message, and reading the test target program, and executing the read test target program The program under test execution part A test program execution unit that transmits at least a message having the program under test as a caller; a message reception reception unit that receives and stores the message acceptance signal; and a message transmitted from the test program execution unit. When the message reception unit to store and the message reception reception unit receive the message reception signal and the message reception unit receives the message, the message stored by the message reception unit is transmitted to the test scenario interpretation execution unit. And a called service simulation unit having a message waiting unit.

  The program test method of the present invention interprets and executes a test scenario in which a message to be transmitted / received to / from the test target program is described for a program under test that performs processing while transmitting / receiving messages to / from an external service. A test scenario interpretation execution unit that performs testing using a test apparatus that includes a test service simulation unit that simulates an external service that is called by the test target program, wherein the test scenario interpretation execution unit includes the test scenario interpretation execution unit, A step of reading a test scenario from a storage device and transmitting a message acceptance signal for accepting a message having the program under test as a caller in accordance with the read test scenario; Message sent by the interpreter Receiving and storing a received signal, a step in which the called service simulating unit receives and stores a message having the program under test as a calling source, and the called service simulating unit receives the message receiving signal. Receiving a message acceptance signal in a step and receiving the message in the message reception step, transmitting the stored message to the test scenario interpretation execution unit; and the test scenario interpretation execution unit includes the called service. And a step of checking whether or not the message received via the simulation unit matches the message indicated in the test scenario.

  The program of the present invention performs a process for testing a program under test that proceeds with processing while transmitting / receiving a message to / from an external service in accordance with a test scenario in which a message to be transmitted / received to / from the program under test is described. A program to be executed by a computer, wherein the computer reads the test scenario from a storage device and generates a message acceptance signal for accepting a message with the program under test as a caller in accordance with the read test scenario A process for receiving and storing a message with the program under test as a caller, and when the message acceptance signal is generated and a message with the program under test as a caller is received, the stored message And a test scenario Characterized in that it comprises a processing and written message checks whether match.

  In the program test apparatus, method, and program of the present invention, the test scenario side transmits (generates) a message acceptance signal indicating that the program under test side receives a call source message according to the test scenario, and the program under test When both the reception of the message transmitted by the message and the message acceptance signal are completed, the message transmitted from the program under test side is delivered to the test scenario side. In this way, the test scenario side can actively receive the call source message by the program under test side based on the message acceptance signal. Therefore, the test scenario side controls the reception order of such messages. be able to. Therefore, the call from the test scenario side and the call from the program under test side can be executed in the same scheme without separation, and test scenario fragmentation can be avoided and test scenario readability and maintainability can be improved. it can. Further, since the message transmission / reception order shown in the test scenario is the same as the order of message transmission / reception by the program under test, the configuration of the test scenario is easily understood.

  In the program test apparatus of the present invention, the message queuing unit may receive the message reception signal by the message reception reception unit or the message reception by the message reception unit within a predetermined time. If the other does not occur, a configuration can be adopted in which a test failure is transmitted to the test scenario interpretation execution unit. In this case, for example, a situation in which the progress of the test remains stopped can be avoided.

  In the program test apparatus of the present invention, the message queuing unit can transmit a message that the call by the test program is completed to the test program execution unit when the message is transmitted to the test scenario interpretation execution unit. The program under test execution unit is configured to stop the progress of the program under test until a message indicating that the call is completed is received from the callee service simulation unit after transmitting a message to the callee service simulation unit. Can be adopted. In this case, the program under test execution unit can start the next process after confirming that the call has been completed.

  In the program test apparatus of the present invention, the called service simulation unit receives and stores a reply message sent from the test scenario interpretation execution unit with respect to a message having the program under test as a caller; When the message queuing unit sends a message having the program under test as a caller to the test scenario interpretation executing unit, and the reply message receiving unit receives the reply message, the reply message stored by the reply message receiving unit It is possible to adopt a configuration further including a reply message queuing unit that transmits a message to the test program execution unit. A reply message may be required from the external service side in response to a call from the program under test side. In this case, the called service simulation unit transmits a message to the test scenario interpretation execution unit, and then waits for a reply message transmitted from the test scenario interpretation execution unit, thereby transmitting the reply message to the program under test side. be able to. By doing so, the reply message can be specified in the test scenario, and the readability of the test scenario can be improved as compared with the case where the reply message is set in advance in the called service simulation unit.

  In the program test apparatus of the present invention, the program under test execution unit transmits the message to the called service simulation unit and then proceeds with the progress of the program under test until the reply message is received from the called service simulation unit. A configuration to stop can be adopted. Further, the reply message queuing unit may transmit a message indicating that transmission of the reply message is completed to the test scenario interpretation executing unit when the reply message is transmitted to the program under test executing unit. In this case, the program under test execution unit can complete the call by receiving the reply message and start the next process.

  In the program test apparatus of the present invention, after the test scenario interpretation execution unit transmits a reply message to the called service simulation unit, the test scenario interpretation execution unit receives from the called service simulation unit that the transmission of the reply message is completed. A configuration that stops the progress of the test scenario can be adopted. In this case, the test scenario interpretation execution unit can resume the progress of the test scenario after confirming that the reply message has been transmitted.

  In the program test apparatus according to the present invention, the test scenario interpretation execution unit outputs a test failure when not receiving the completion of the transmission of the reply message within a predetermined time after the transmission of the reply message. Configuration can be adopted. In this case, it is possible to avoid a situation in which the progress of the test scenario remains stopped.

  In the program test apparatus of the present invention, the test scenario interpretation execution unit can output a message indicating that the test is successful when the test scenario is advanced to the end. In the test scenario, a message to be transmitted / received to / from an external service by the program under test can be described according to a message transmission / reception order. For this reason, the fact that the test scenario was successfully executed to the end means that the test was successful.

  In the program test apparatus, method, and program of the present invention, the test scenario side can actively receive a message to be called by the program under test side by transmitting (generating) a message acceptance signal. For this reason, calls from the test scenario side and calls from the program under test side can be implemented with the same scheme without separation, and test scenario fragmentation can be avoided and test scenario readability and maintainability can be improved. Can do. Further, since the message transmission / reception order shown in the test scenario is the order of message transmission / reception by the program under test, the configuration of the test scenario is easy to understand.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 shows the configuration of a program test system according to the first embodiment of the present invention. The test system 10 includes a computer (central processing unit, processor, or data processing unit) 100, a test scenario storage unit 180, and a test program storage unit 190. The computer 100 operates under program control, and includes a test scenario interpretation execution unit 101, a test program execution unit 102, a test program call unit 103, and a called service simulation unit 110.

  The test program storage unit 190 stores a test program 160 to be tested. The program under test 160 proceeds with processing while receiving a message from another service or module or transmitting a message to another service or the like. Examples of the program under test 160 include a program written in a business process execution language such as WS-BPEL (Web Services Business Process Execution Language), a program that operates by combining several commands such as a shell script, or a socket. There are network programs that use and cooperate with several servers and daemons. The program under test execution unit 102 reads the program under test 160 and executes an operation according to the program under test 160 while transmitting and receiving messages to and from the test scenario interpretation execution unit 101.

  The test scenario storage unit 180 stores a test scenario 150 used for testing the program under test 160. In the test scenario 150, the program under test 160 is arranged in the order in which it is desired to confirm that message exchange occurs with what service, what message, and in what method. Yes. The test scenario interpretation execution unit 101 reads the test scenario 150 and proceeds with operations according to the test scenario 150 while transmitting and receiving messages to and from the program under test execution unit 102.

  The message transmission / reception methods include synchronous transmission / reception and asynchronous transmission / reception. Synchronous transmission / reception refers to transmission in which processing is transferred to the other party (transmission destination) after sending a message, and when the other party's processing is completed, if there is a reply message, it is received and the source process resumes -Refers to the reception form. On the other hand, asynchronous message transmission / reception refers to a transmission / reception form in which a message can be transferred to the next process immediately after the message transmission. In the present embodiment, the test scenario interpretation execution unit 101 transmits and receives a message synchronously or asynchronously with the program under test execution unit 102 when the test scenario interpretation execution unit 101 is a caller. When the test program execution unit 102 becomes a caller, a message is asynchronously received from the test program execution unit 102.

  When the test scenario 150 describes that the test program 160 receives a message synchronously or asynchronously from an external service, the test scenario interpretation execution unit 101 uses the test scenario interpretation execution unit 101 as a caller. A message is transmitted to the called program-under-execution unit 102 as a call destination synchronously or asynchronously via the program-under-test calling unit 103. At this time, when the message transmitted by the test scenario interpretation execution unit 101 is synchronous transmission, and the program under test execution unit 102 transmits a reply message synchronously to the received message, the test scenario interpretation execution unit 101 synchronously receives the reply message from the test program execution unit 102 via the test program call unit 103.

  The called service simulation unit 110 simulates an external service called by the test program 160 and is prepared corresponding to the called external service. When the test scenario 150 describes that the external service receives a message asynchronously transmitted by the program under test execution unit 102 as a caller, the test scenario interpretation execution unit 101 responds to the message. To be received via the called service simulation unit 110.

  The called service simulation unit 110 includes a message reception receiving unit 111, a message waiting unit 112, and a message receiving unit 113. When the program-under-execution unit 102 becomes a caller according to the program under test 160 and sends a message asynchronously to an external service, the program-under-execution unit 102 sends a message to the called service simulation unit 110 corresponding to the external service. Send. The message receiving unit 113 receives the message transmitted by the test program execution unit 102. When the message receiving unit 113 receives a message from the program under test executing unit 102, the message receiving unit 113 notifies the message waiting unit 112 to that effect.

  On the other hand, when the test scenario 150 describes that the test program execution unit 102 sends a message to an external service as a call source, the test scenario interpretation execution unit 101 simulates the corresponding called service. The message acceptance is transmitted to the unit 110. The message acceptance receiving unit 111 accepts the message acceptance transmitted by the test scenario interpretation execution unit 101 and notifies the message waiting unit 112 that the message acceptance has been received.

  The message queuing unit 112 waits for a message transmitted by the program under test execution unit 102 and a message reception transmitted by the test scenario interpretation execution unit 101. When the message queuing unit 112 is notified that the message has been received from the message receiving unit 113, and the message reception receiving unit 111 is notified that the message reception has been received, the message waiting unit 112 receives the message from the test scenario interpretation executing unit 101. Send the message. In addition, the program under test execution unit 102 is notified that the message has been transmitted.

  The message queuing mechanism in the message queuing unit 112 includes, for example, Java. util. current. Blocking Queue or the like can be used. In addition, an area for storing a message and message acceptance and a flag for checking whether or not a message or message acceptance is stored in the area are provided. If the flag is not set, the process waits until the flag is set. When is set, a mechanism for transmitting the stored message and message acceptance to the test scenario interpretation execution unit 101 and the program under test execution unit 102 can be employed.

  The program under test execution unit 102 suspends the progress of the program under test 160 after transmitting a message to the called service simulation unit 110. In addition, the test scenario interpretation execution unit 101 temporarily stops the progress of the test scenario 150 after transmitting the message acceptance. When the test scenario interpretation execution unit 101 receives a message from the called service simulation unit 110, the test scenario interpretation execution unit 101 resumes the progress of the test scenario 150, and the test program execution unit 102 receives the test scenario interpretation execution unit from the called service simulation unit 110. When the notification that the message is transmitted to 101 is received, the progress of the program under test 160 is resumed.

  The test system 10 is typically a computer system having a computer 1000 including a CPU 1001, a memory 1002, and an external storage device 1003, an input device 1004, and an output device 1005, as shown in FIG. Can be configured. The test system 10 operates by reading a test creation execution program 1010 for executing a test into the memory 1002. The test scenario 150 and the program under test 160 (FIG. 1) are stored in the memory 1002 or the external storage device 1003, respectively. At the start of the test, the user commands the start of test creation and execution through the input device 1004 such as a mouse, keyboard, or touch panel. The test result is displayed on an output device 1005 such as a display.

  FIG. 3 is a flowchart showing the operation procedure of the test system. When the user instructs the start of the test through the input device 1004 (FIG. 2), the test scenario interpretation execution unit 101 (FIG. 1) reads the test scenario 150 from the test scenario storage unit 180 (step A1). The program under test execution unit 102 reads the program under test 160 from the program under test storage unit 190 (step A2), and starts its execution (step A3). Thereafter, the test scenario interpretation execution unit 101 starts executing the test scenario 150 read in step A1 (step A4).

  The test scenario interpretation execution unit 101 checks whether or not the test scenario 150 has an instruction to send a message to the test program execution unit 102 (step A5). A message is transmitted to the test program execution unit 102 via the unit 103 (step A9). The test scenario interpretation execution unit 101 checks whether or not there is an instruction that the external service receives a message from the program under test execution unit 102 (step A6). (Step A10) and confirms whether the received message is a message indicated by the test scenario 150 or not. When the messages do not match, the test scenario interpretation execution unit 101 determines that the test has failed and outputs a message to that effect.

  In step A 10, when the test scenario interpretation execution unit 101 receives a reply message to the message synchronously transmitted from the test program execution unit 102, the test scenario interpretation execution unit 101 receives the reply message via the test program call unit 103. On the other hand, when the message to be tested is received by the program under test execution unit 102 side, a message acceptance is transmitted to the called service simulation unit 110 corresponding to the external service that should receive the message, and the called program is received. A message is received via the service simulation unit 110.

  FIG. 4 shows an operation procedure of the called service simulation unit 110 when the test program execution unit 102 side transmits a message and the test scenario interpretation execution unit 101 receives the message. The message reception receiving unit 111 receives a message received from the test scenario interpretation execution unit 101 (step B1). When the message reception is received, the message reception unit 111 notifies the message waiting unit 112 of the message reception, and the message waiting unit 112 receives the message reception. It waits until notification that the message is received from the program execution unit 102 (step B2).

  On the other hand, the program under test execution unit 102 transmits a message to the called service simulation unit 110 corresponding to the external service to which the message is to be transmitted, according to the program under test 160. The message receiving unit 113 accepts a message from the program under test executing unit 102 (step B11). When the message is received, the message receiving unit 113 notifies the message waiting unit 112 of the message and notifies the message waiting unit 112 of the test scenario interpretation executing unit 101. Waits until a notification indicating that the message has been accepted is received (step B12).

  Upon receiving the notification that the message has been received and the notification that the message has been accepted, the message waiting unit 112 transmits the message received from the program under test execution unit 102 to the test scenario interpretation execution unit 101. (Step B3). In addition, a notification that the message has been transmitted to the test scenario interpretation execution unit 101 is transmitted to the program under test execution unit 102 (step B13). When receiving the message acceptance, the program under test execution unit 102 completes the call from the program under test 160 and resumes the progress of the program under test 160. When the test scenario interpretation execution unit 101 receives a message from the called service simulation unit 110, the test scenario interpretation execution unit 101 resumes the progress of the stopped test scenario 150.

  Returning to FIG. 3, the test scenario interpretation execution unit 101 determines whether or not the test scenario 150 has been executed to the end (step A7). When the test scenario interpretation execution unit 101 determines that the execution of the test scenario 150 has not been completed, the test scenario interpretation execution unit 101 returns to step A5 and executes the next process described in the test scenario 150. When the test scenario interpretation execution unit 101 determines that the execution of the test scenario 150 is completed, the test scenario interpretation execution unit 101 outputs that the test is successful (step A8), and ends the process.

  Hereinafter, the operation of the test system 10 will be described with a specific example. In the following, consider the case where the program under test 160 (test program execution unit 102) operates by sending and receiving messages to and from a plurality of services (service A to service C) as shown in FIG. In this example, in the test scenario 150, as shown in FIG. 6, message exchange with each service and its method (synchronous reception / transmission or asynchronous reception / transmission) are performed by message exchange. Arranged in the order they should occur.

  In the example of FIG. 5, the service to be called by the program under test 160 is the service C, and therefore the called service simulation unit 110 corresponding to the service C is prepared in the computer 100. The called service simulation unit 110 can be generated by a test (scenario) creator prior to the start of the test, or can be generated mechanically by causing the computer 100 to analyze the test scenario 150. . For example, for a service to be simulated, an interface definition that defines what type of message is exchanged by the service is input, and the message receiving unit 113 is generated based on the definition. The message reception receiving unit 111 and the message waiting unit 112 are combined to automatically generate the called service simulation unit 110.

  The test scenario interpretation execution unit 101 first reads the first item (# 1) of the test scenario 150 and knows that the program under test 160 receives message 1 from the service A asynchronously. Next, the content of the message 1 is acquired, and the message 1 is asynchronously transmitted to the program under test 160 via the program under test calling unit 103. Here, the content of the message 1 can be directly described in the test scenario 150. Alternatively, a pointer may be described in the test scenario 150, and the content of the message 1 may be acquired from the external storage device 1003 or the memory 1002 (FIG. 2) by referring to the pointer. When the test scenario interpretation execution unit 101 receives a response indicating that the message 1 has been received from the program under test execution unit 102, it regards this item as successful.

  When the test scenario interpretation execution unit 101 succeeds in the first item of the test scenario 150, the test scenario interpretation unit 101 reads the second item (# 2), and the program under test 160 synchronously receives the message 2 from the service B, and then the message 3 Knows that it will reply in sync (# 3). The test scenario interpretation execution unit 101 acquires the content of the message 2 and transmits the acquired message 2 to the test program 160 via the test program call unit 103 in synchronization. Further, a message 3 that is a reply message to the received message 2 is received from the test program execution unit 102 via the test program call unit 103.

  When the test scenario interpretation execution unit 101 receives the reply message from the program under test execution unit 102, the test scenario interpretation execution unit 101 acquires the content of the message 3 to be received, compares the received reply message with the message 3, and receives the message 3 correctly. It is judged whether it was able to be done. When the test scenario interpretation execution unit 101 can correctly receive the message 3 from the test program execution unit 102, the test scenario interpretation execution unit 101 regards the second item and the third item of the test scenario 150 as successful.

  When the test scenario interpretation execution unit 101 succeeds in the second item and the third item of the test scenario 150, the test scenario interpretation execution unit 101 reads the fourth item (# 4), and the program under test 160 transmits the message 4 to the service C asynchronously. know. The test scenario interpretation execution unit 101 transmits a message acceptance to the called service simulation unit 110 corresponding to the service C, and temporarily stops the progress of the test scenario 150. On the other hand, the program under test execution unit 102 executes processing according to the program under test 160 after sending the message 3, and calls the message 4 corresponding to the service C at any timing according to the contents of the program under test execution unit 102. 103, and temporarily stops the progress of the program under test 160.

  Either the message 4 transmission by the program under test execution unit 102 or the message acceptance transmission by the test scenario interpretation execution unit 101 may be first. The reason is that the message queuing unit 112 waits for message reception and message reception. The called service simulation unit 110 corresponding to the service C receives a notification that the message waiting unit 112 has received a message from the message reception receiving unit 111 and a message that the message reception unit 113 has received a message. When the message is received, the message 4 is transmitted to the test scenario interpretation execution unit 101, and the fact that the message has been transmitted is transmitted to the program under test execution unit 102. The test scenario interpretation execution unit 101 acquires the content of the message 4 to be received, compares the received message with the message 4, and determines whether the message 4 has been correctly received. When the test scenario 150 is correctly executed to the end, the test result is output to the external output device 1005 (FIG. 2) such as a display or a printer, or the test result is written to the storage device 1003 such as the memory 1002 or the disk.

  The test scenario interpretation execution unit 101 suspends the progress of the test scenario 150 until the message to the service C of the program under test 160 is transmitted, and at what timing the program under test 160 transmitted the message. Can be recognized. That is, assuming that the program under test execution unit 102 transmits a message to another service before transmitting a message to the service C, the test scenario interpretation execution unit 101 reads from the called service simulation unit 110 corresponding to the service C. Since the message cannot be received, it is understood that the message exchange order is incorrect. When the program under test execution unit 102 transmits a message to the service C before message transmission / reception with the service B, the test scenario interpretation execution unit 101 Since the message acceptance is not transmitted to the service C until the transmission / reception of the message between is completed, it is understood that the message exchange order is incorrect.

  In general, when the test target program execution unit 102 sends a message as a caller, the test scenario interpretation execution unit 101 on the call receiving side passively receives the message. The timing cannot be controlled. In this embodiment, the test scenario interpretation execution unit 101 actively sends a message by sending a message acceptance to the called service simulation unit 110 corresponding to the service that should receive the message at the timing when the message should be received. In order to receive, the test scenario interpretation execution unit 101 can actively receive a message even when the test scenario interpretation execution unit 101 becomes a call receiving side.

  In this embodiment, the called service simulation unit 110 waits for the message received by the test scenario interpretation execution unit 101 and the message transmitted by the program under test execution unit 102. Thereby, the test scenario 150 side can control the order of calls from the program under test 160 and calls to the program under test 160. Therefore, the call from the program under test 160 and the call to the program under test 160 can be implemented in the same scheme without separation, and fragmentation of the test scenario 150 can be avoided, thereby improving the readability and maintainability of the test scenario. Can be improved. In addition, since the order shown in the test scenario 150 is the order of message exchange by the program under test 160 as it is, the configuration of the test scenario 150 is easy to understand.

  As described above, the test scenario 150 side can control not only the timing to call the program under test 160 but also the timing to receive the message transmitted by the program under test 160. The progress of message exchange by the program 160 can be controlled. Therefore, the order of message transmission / reception by the program under test 160 can be verified according to the progress of the test scenario 150, and it is not necessary to analyze the log to check the order of message transmission / reception. Become. Also, if the program under test 160 does not exchange messages with the service in the order of the test scenario 150, the test scenario 150 cannot progress, so whether or not the test scenario 150 has been executed to the end. By checking, it is possible to easily verify whether or not the order of message exchange by the program under test 160 has been correctly performed.

  FIG. 7 shows the configuration of a test system according to the second embodiment of the present invention. This embodiment is different from the first embodiment in that the computer 100a includes a called service simulation unit 120 having the configuration shown in FIG. 7 instead of the called service simulation unit 110 (FIG. 1). The called service simulation unit 120 further includes a reply message receiving unit 114 and a reply message waiting unit 115 in addition to the configuration of the called service simulation unit 110 of the first embodiment. In the first embodiment, the program under test 160 is a program for asynchronously transmitting a message to an external service or the like. In this embodiment, the program under test 160 is a program for synchronously transmitting a message to an external service or the like.

  The message receiving unit 113 receives the message transmitted by synchronous transmission from the program under test executing unit 102 and notifies the message waiting unit 112 that the message has been received. The message acceptance receiving unit 111 accepts the message acceptance transmitted by the test scenario interpretation executing unit 101 and notifies the message waiting unit 112 to that effect. When the message and the message acceptance are ready, the message queuing unit 112 transmits the message synchronously transmitted by the program under test execution unit 102 to the test scenario interpretation execution unit 101 and indicates that the message has been transmitted. Notify

  When the message receiving unit 113 receives a notification that the message has been transmitted, the message receiving unit 113 notifies the reply message waiting unit 115 that the reply message is accepted. The test scenario interpretation execution unit 101 confirms whether or not the received message matches the message indicated by the test scenario 150, and after confirming the message, transmits a reply message to the called service simulation unit 120. The reply message accepting unit 114 accepts the reply message transmitted from the test scenario interpretation executing unit 101 and transmits a message to that effect to the reply message waiting unit 115. When the reply message queuing unit 120 completes the reply message and the reply message acceptance, the reply message queuing unit 120 transmits a reply message to the program under test execution unit 102, and interprets that the reply message is transmitted to the program under test execution unit 102. To the unit 101.

  FIG. 8 is a flowchart showing the operation procedure of the test system. The test scenario interpretation execution unit 101 reads the test scenario 150 from the test scenario storage unit 180 and starts executing the read test scenario 150 (step C1). The program under test execution unit 102 reads the program under test 160 from the program under test storage unit 190 and starts its execution.

  The test scenario interpretation execution unit 101 checks whether or not the test scenario 150 has an instruction to send a message to the test program execution unit 102 (step C2). The message is transmitted to the test program execution unit 102 via the unit 103 (step C7). Further, the test scenario interpretation execution unit 101 checks whether or not there is an instruction that the external service receives a message from the program under test execution unit 102 (step C3), and when there is such an instruction, the program under test 160 (Step C8), and confirms whether the received message is a message shown in the test scenario 150 or not.

  In step C 8, when receiving a reply message for the message synchronously transmitted from the test program execution unit 102, the test scenario interpretation execution unit 101 receives the reply message via the test program call unit 103. On the other hand, when the message to be tested is received by the program under test execution unit 102 side, a message acceptance is transmitted to the called service simulation unit 110 corresponding to the external service that should receive the message, and the called program is received. A message is received via the service simulation unit 110.

  The test scenario interpretation execution unit 101 checks whether or not the test scenario 150 has an instruction to transmit a reply message for the synchronously received message (step C4). When there is such an instruction, the test scenario interpretation execution unit 101 transmits a reply message to the test program execution unit 102 via the called service simulation unit 120 (step C9).

  FIG. 9 shows an operation procedure of the called service simulator 120 when the test program execution unit 102 transmits a message synchronously and receives a reply message. The message reception receiving unit 111 receives a message received from the test scenario interpretation execution unit 101 (step D1). When the message reception is received, the message reception unit 111 notifies the message waiting unit 112 to that effect, and sends a message to the message waiting unit 112 to be tested. It waits until it is notified that the message from the program execution part 102 was received (step D2).

  On the other hand, the program under test execution unit 102 transmits a message to the called service simulation unit 110 corresponding to the external service to which the message is to be transmitted, according to the program under test 160. The message receiving unit 113 accepts a message from the program under test execution unit 102 (step D11). When the message reception unit 113 receives the message, the message reception unit 113 notifies the message waiting unit 112 of the message and notifies the message waiting unit 112 of the test scenario interpretation execution unit 101. Waits until a message indicating that the message has been accepted is received (step D12).

  Upon receiving the notification that the message has been received and the notification that the message has been accepted, the message waiting unit 112 transmits the message received from the program under test execution unit 102 to the test scenario interpretation execution unit 101. (Step D3). At this time, the message queuing unit 112 notifies the message receiving unit 113 that the message has been transmitted to the test scenario interpretation executing unit 101.

  The test scenario interpretation execution unit 101 transmits a reply message to the called service simulation unit 120 according to the test scenario 150. The reply message accepting unit 114 accepts the reply message transmitted by the test scenario interpretation executing unit 101 (step D4), notifies the reply message waiting unit 115 that the reply message has been received, and sends a reply to the reply message waiting unit 115. It waits until it is notified that the message is accepted (step D5).

  On the other hand, when the message receiving unit 113 receives a notification from the message waiting unit 112 that the message has been transmitted to the test scenario interpretation executing unit 101, the message receiving unit 113 notifies the reply message waiting unit 115 that the reply message is accepted. Thereafter, it waits until the reply message waiting unit 115 is notified that the reply message from the test scenario interpretation execution unit 101 has been received (step D13).

  When the reply message and the reply message acceptance are complete, the reply message awaiting section 115 transmits a reply message to the program under test execution section 102 (step D14), and indicates that the reply message has been transmitted to the test scenario interpretation execution section 101. Notification is made (step D6). When the test program execution unit 102 receives the reply message, it ends the synchronous transmission / reception processing of the message and resumes the progress of the program under test 160. When the test scenario interpretation execution unit 101 receives the notification that the reply message has been transmitted, the test scenario interpretation execution unit 101 resumes the progress of the test scenario 150.

  Returning to FIG. 8, the test scenario interpretation execution unit 101 determines whether or not the test scenario 150 has been executed to the end (step C5). When the test scenario interpretation execution unit 101 determines that the execution of the test scenario 150 has not been completed, the test scenario interpretation execution unit 101 returns to step C2 and executes the next process described in the test scenario 150. When the test scenario interpretation execution unit 101 determines that the execution of the test scenario 150 is completed, the test scenario interpretation execution unit 101 outputs a message indicating that the test is successful (step C6) and ends the process.

  Next, the operation of the test system 10a of this embodiment will be described using a specific example. FIG. 10 shows a state of message exchange between the program under test 160 and an external service. In FIG. 10, since the service called by the test target program 160 is the service D, the called service simulation unit 120 corresponding to the service D is prepared in the computer 100a. A test scenario 150 shown in FIG. 11 is used for testing the program under test 160 shown in FIG. The first to third items of the test scenario 150 are the same as the test scenario 150 (FIG. 6) used in the description of the first embodiment, and the test scenario interpretation execution unit 101 has been described in the first embodiment. Messages are exchanged with the program under test 160 by the same operation as.

  When the test scenario interpretation execution unit 101 succeeds in the third term (synchronous reply of message 3) of the test scenario 150, the test scenario interpretation execution unit 101 reads the fourth term (# 4), and the program under test 160 synchronously transmits the message 5 to the service D. Know what to do. Thereafter, the test scenario interpretation execution unit 101 transmits a message acceptance to the called service simulation unit 120 corresponding to the service D, and temporarily stops the progress of the test scenario 150. The program under test execution unit 102 synchronously transmits the message 5 to the called service simulation unit 120 corresponding to the service D at an arbitrary timing according to the program under test, and temporarily stops the progress of the program under test 160.

  In the called service simulation unit 120, the message queuing unit 112 waits for the reception of the message and the reception of the message, and when both of them are ready, the received message is transmitted to the test scenario interpretation execution unit 101. After sending the message, the message receiving unit 113 notifies the reply message waiting unit 115 that the reply message is accepted. When the test scenario interpretation execution unit 101 receives a message from the called service simulation unit 120, the test scenario interpretation execution unit 101 confirms that the received message matches the message 5 to be received.

  After confirming the received message, the test scenario interpretation execution unit 101 resumes the progress of the test scenario 150, reads the fifth term (# 5), and the program under test 160 receives the reply message 6 from the service D. Know. The test scenario interpretation execution unit 101 acquires the content of the reply message 6 to be transmitted, and transmits the reply message 6 to the called service simulation unit 120 corresponding to the service D. The reply message reception unit 114 receives the reply message 6 from the test scenario interpretation execution unit 101 and notifies the reply message waiting unit 115 that the reply message has been received.

  When the reply message and the reply message are received, the reply message queuing unit 115 transmits the reply message 6 to the test program execution unit 102 and notifies the test scenario interpretation execution unit 101 that the reply message has been transmitted. When the program under test execution unit 102 receives the reply message 6, it completes the call to the service D and resumes the progress of the program under test 160. When the test scenario interpretation execution unit 101 receives a notification that the reply message has been transmitted, the test scenario interpretation execution unit 101 resumes the progress of the test scenario 150.

  In the present embodiment, as in the first embodiment, the called service simulation unit 120 waits for message reception from the test scenario interpretation execution unit 101 and transmission of a message from the test program execution unit 102. Therefore, if messages are not transmitted and received in the order shown in the test scenario 150, the test scenario 150 does not progress. Therefore, in this embodiment, the message transmission / reception order shown in the test scenario 150 is the order in which the program under test 160 should transmit / receive the message as it is, so that the readability and maintainability of the test scenario can be improved.

  In this embodiment, since the reply message reception unit 114 and the reply message waiting unit 115 wait for the reply message, when the program under test execution unit 101 calls the called service simulation unit 120 synchronously, With respect to the reply message received by the program under test execution unit 102, the creation of the reply message and the delivery to the called service simulation part 120 can be delayed until the reply message is to be returned. In other words, if there is no mechanism for waiting for this reply message, it is necessary to send a reply message from the called service simulation unit 120 immediately after sending the message to the test scenario interpretation execution unit 101. Although it is necessary to set a reply message in the simulation unit 120 in advance, it is not necessary to set a reply message in the called service simulation unit 120 in advance by waiting for the reply message. A message can be specified. As a result, the readability of the test scenario can be improved.

  FIG. 12 shows the configuration of the test system according to the third embodiment of the present invention. In the present embodiment, the computer 100 b includes a called service simulation unit (asynchronous) 110 and a called service simulation unit (synchronous) 120. In the test system 10b of this embodiment, the called service simulation unit (asynchronous) 110 is a message transmitted asynchronously from the test program execution unit 102 by the same operation as the called service simulation unit 110 of the first embodiment. Is transmitted to the test scenario interpretation execution unit 101. The called service simulation unit (synchronization) 120 transmits a message synchronously transmitted from the test program execution unit 102 to the test scenario interpretation execution unit 101 by the same operation as the called service simulation unit 120 of the second embodiment. The message transmitted by the test scenario interpretation execution unit 101 is transmitted to the program under test execution unit 102.

  For example, consider a case where the program under test 160 transmits and receives messages to and from each external service as shown in FIG. In this case, the test scenario 150 is described as shown in FIG. 14 according to the message transmission / reception order. In the fourth item (# 4) of the test scenario 150, the test scenario interpretation execution unit 101 receives the message 4 transmitted by the test program execution unit 102 via the called service simulation unit (asynchronous) 110. In the fifth item (# 5) of the test scenario 150, the test scenario interpretation execution unit 101 receives the message 5 via the called service simulation unit (synchronization) 120 and executes the reply message 6 as the program under test execution. Reply to unit 102.

  In this embodiment, by providing the called service simulation unit (asynchronous) 110 and the called service simulation unit (synchronous) 120, the program under test 160 can perform message transmission by synchronous transmission and message transmission by asynchronous transmission. Even if included, testing is possible. Other effects are the same as those of the first embodiment or the second embodiment.

  In each of the above embodiments, if both the (reply) message reception and the (reply) message reception are not completed (waiting is not completed) in the message waiting unit 112 or the reply message waiting unit 115, the test scenario 150 Progress remains stopped. In order to avoid such a situation, the message queuing unit 112 and the reply message queuing unit 115 start a timer when a message acceptance or message reception is notified, and if the other is not notified within a predetermined time, the message queuing unit 112 and the reply message queuing unit 115 It is possible to adopt a configuration in which the fact that the test has failed is output assuming that the test scenario 150 is not satisfied. Alternatively, if the waiting is not completed even if the polling is performed a specified number of times, a message indicating that the test has failed may be output. In these cases, it is possible to avoid a situation in which the waiting is not completed due to an error in the program under test 160 and the execution of the test stops midway.

  Further, as shown in FIG. 15, a test scenario interpretation execution unit 101, a called service simulation unit 110, 120, a test program call unit 103, and a test program execution unit 102 are respectively connected to different computers 200 ( (Client part), 300 (server part), the structure which connects between the client part 200 and the server part 300 with the network 400, and transmits / receives a message via the network 400 is also employable. In this case, a test can be performed via the network, and a test can be performed on the program under test 160 that operates remotely.

  The test system shown in FIG. 15 can be typically realized by a computer system shown in FIG. The configuration of this computer system corresponds to the configuration in which the configuration shown in FIG. 2 is connected via the network 4000 by the communication devices 2006 and 3006. The network 4000 is configured as a network such as a LAN, a WAN, or the Internet. As the communication devices 2006 and 3006, for example, an Ethernet work card, a model, a terminal adapter, or the like that performs communication via the network 4000 can be used. The computer 2000 operates as the client unit 200 (FIG. 15) by reading the test creation execution client program 2010 and executing it. Further, the computer 3000 reads the test execution server program 3010 and operates as the server unit 300.

  Although the present invention has been described based on the preferred embodiment, the program test apparatus, method, and program of the present invention are not limited to the above-described embodiment example. Various modifications and changes are also included in the scope of the present invention.

  The present invention can be applied to uses such as creation / execution of a test for a workflow describing cooperation with a Web service such as WS-BPEL. Also, it can be applied to the purpose of creating and executing a program test when network programming using a socket is performed.

The block diagram which shows the structure of the program test system of 1st Embodiment of this invention. 1 is a block diagram showing a configuration of a computer system that implements a test system 10. FIG. The flowchart which shows the operation | movement procedure of a test system. 7 is a flowchart showing an operation procedure of the called service simulation unit 110 when the test program execution unit 102 transmits a message and the test scenario interpretation execution unit 101 receives the message. The block diagram which shows the mode of the message exchange between a to-be-tested program and an external service. The figure which shows the test scenario used when testing the to-be-tested program shown in FIG. The block diagram which shows the structure of the test system of 2nd Embodiment of this invention. The flowchart which shows the operation | movement procedure of a test system. 7 is a flowchart showing an operation procedure of the called service simulation unit 120 when the program under test execution unit 102 synchronously transmits a message and receives a reply message. The figure which shows the mode of the message exchange between the to-be-tested program 160 and an external service. The figure which shows the test scenario used when testing the to-be-tested program shown in FIG. The block diagram which shows the structure of the test system of 3rd Embodiment of this invention. The figure which shows the mode of the message exchange between the to-be-tested program 160 and an external service. The figure which shows the test scenario used when testing the to-be-tested program shown in FIG. The block diagram which shows the structure of the test system of the modification of the test system of FIG. The block diagram which shows the structure of the computer system which implement | achieves the test system shown in FIG. The schematic diagram which shows the mode of the test in the program test system described in the nonpatent literature 1. FIG. 9 is a flowchart showing an operation flow at the time of testing the program test system described in Non-Patent Document 1. 1 is a block diagram showing a schematic configuration of a test system described in Patent Document 1. FIG. The flowchart which shows the operation | movement procedure of the test system of FIG.

Explanation of symbols

10: Test system 100: Computer (central processing unit, processor, data processing unit)
DESCRIPTION OF SYMBOLS 101: Test scenario interpretation execution part 102: Test program execution part 103: Test program call part 110, 120: Called service simulation part 111: Message reception reception part 112: Message waiting part 113: Message reception part 114: Reply message Reception unit 115: Reply message waiting unit 150: Test scenario 160: Program under test 180: Test scenario storage unit 190: Test program storage unit 200: Client unit (computer)
300: Server part (computer)
400: Network

Claims (13)

A program test apparatus for testing a program under test that proceeds with processing while transmitting / receiving a message to / from an external service according to a test scenario in which a message to be transmitted / received to / from the program under test is described,
A test scenario interpretation execution unit that reads the test scenario from a storage device, interprets and executes the read test scenario, and receives a message that calls the program under test according to the test scenario. A test scenario interpretation execution unit that transmits a message acceptance signal indicating that the message is accepted;
A test program execution unit that reads the program under test and executes the read program under test, and at least a program under test execution unit that transmits a message having the test program as a caller;
A message reception receiving unit for receiving and storing the message reception signal, a message reception unit for receiving and storing a message transmitted from the program under test execution unit, the message reception reception unit receiving the message reception signal, and A call service simulation unit having a message queuing unit that transmits a message stored in the message reception unit to the test scenario interpretation execution unit when the message reception unit receives the message;
A program test apparatus comprising:   The test scenario interpretation execution unit stops the progress of the test scenario after transmitting the message acceptance signal until receiving a message having the test program as a caller from the called service simulation unit. The program test apparatus described in 1.   The message queuing unit receives the message reception signal by the message reception reception unit and the reception of the message by the message reception unit, and if the other does not occur within a predetermined time, the test The program test apparatus according to claim 1, wherein a message indicating that the test has failed is transmitted to the scenario interpretation execution unit.   The message queuing unit, when transmitting a message to the test scenario interpretation execution unit, transmits to the program under test execution unit that the call by the program under test is completed. Program test device.   The program under test execution unit, after transmitting a message to the called service simulation unit, stops the progress of the program under test until receiving a message that the call is completed from the called service simulation unit. 4. The program test apparatus according to 4.   The called service simulation unit includes a reply message receiving unit that receives and stores a reply message for a message that is transmitted from the test scenario interpretation execution unit and that has the program under test as a calling source, and the message waiting unit includes a test target When a message having a program as a caller is transmitted to the test scenario interpretation execution unit and the reply message receiving unit receives the reply message, the reply message stored by the reply message receiving unit is sent to the program under test execution unit. The program test apparatus according to claim 1, further comprising a reply message waiting unit to be transmitted.   The said program under test execution part stops progress of a program under test until it transmits the said reply message from the said callee service simulation part, after transmitting a message to the said callee service simulation part. Program test equipment.   8. The program test according to claim 6, wherein, when the reply message queuing unit transmits a reply message to the program under test execution unit, the test message interpretation execution unit transmits information indicating that transmission of the reply message is completed. apparatus.   The test scenario interpretation execution unit stops the progress of the test scenario after transmitting a reply message to the called service simulation unit until receiving a notification that the transmission of the reply message is completed from the called service simulation unit The program test apparatus according to claim 8.   The test scenario interpretation execution unit outputs that the test has failed when it does not receive that the transmission of the reply message is completed within a predetermined time after the transmission of the reply message. Program test equipment.   The program test apparatus according to claim 1, wherein the test scenario interpretation execution unit outputs that the test is successful when the test scenario proceeds to the end. A test scenario interpretation execution unit that interprets and executes a test scenario in which a message to be transmitted to and received from the test target program is described, while performing a process to be performed while transmitting and receiving a message to and from an external service; A program test method for testing using a test apparatus including a called service simulation unit for simulating an external service called by a test target program,
The test scenario interpretation execution unit reads the test scenario from a storage device, and transmits a message acceptance signal for accepting a message with the program under test as a caller according to the read test scenario;
The called service simulation unit receiving and storing the message acceptance signal transmitted by the test scenario interpretation execution unit;
The called service simulation unit receiving and storing a message having the program to be tested as a caller; and
A step of transmitting the stored message to the test scenario interpretation execution unit when the called service simulation unit receives a message reception signal in the message reception signal reception step and receives a message in the message reception step; ,
The test scenario interpretation execution unit includes a step of checking whether or not a message received via the called service simulation unit matches a message indicated in the test scenario. . A program that causes a computer to execute a process for testing a program under test that proceeds with processing while transmitting / receiving messages to / from an external service according to a test scenario that describes a message to be transmitted / received to / from the program under test And the computer
A process of reading the test scenario from a storage device and generating a message acceptance signal to accept a message with the program under test as a caller in accordance with the read test scenario;
Processing for receiving and storing a message having the program under test as a caller;
When the message acceptance signal is generated and a message having the program under test as a caller is received, a process of checking whether or not the stored message matches the message described in the test scenario; A program characterized by being executed.

Images