JP6436705B2 - Test execution device, test execution method, and computer program - Google Patents

Description

  The present invention relates to a data processing technique, and more particularly to a technique for testing an information system.

  In order to guarantee the quality of the information system, operational tests before starting the service are indispensable. For example, an information system developer often performs an operation test by manually operating a user interface of the information system and confirming the operation result. However, in recent information systems with abundant operation variations, manual operation tests are limited.

  The present applicant has proposed a technique for automating an operation test of an information system and realizing an efficient operation test (see, for example, Patent Document 1).

International Publication No. 2010/116586

  In an integrated test of an information system that is distributedly developed by a plurality of teams, a test is usually performed across a product (software module) individually developed by a plurality of teams for one test case. However, in the case of an information system in which processes having different functions are connected to form a business flow, the teams developed may differ depending on the functions. In such an information system, it may be necessary to digest one test case by a plurality of teams. The present inventor considered that there is room for improvement in order to improve the efficiency of the test of a system in which a plurality of processes with different development managers are connected to form a business flow.

  The present invention is based on the above-mentioned problem recognition of the inventor, and its main purpose is to improve the efficiency of the test of a system in which a plurality of processes with different development managers form a business flow. It is to provide the technology for.

  In order to solve the above problems, a test execution device according to an aspect of the present invention is a device for testing a system that completes processing of an input business event through a plurality of processes, and holds a test scenario for each process. A scenario holding unit, a test execution unit that automatically executes system tests by processing business events according to a test scenario for each process, and when an error is detected in the test, An error recording unit that records information indicating a test scenario of a process in which an error has occurred.

  Another aspect of the present invention is a test execution method. In this method, an apparatus that tests a system that completes processing of an input business event through a plurality of processes processes the business event over a plurality of processes according to a test scenario for each process stored in a predetermined storage area. By doing so, a step of automatically executing a system test and a step of recording information indicating a test scenario of a process in which the error has occurred when an error is detected in the test are executed.

  Note that any combination of the above-described constituent elements and the expression of the present invention converted between a system, a program, a recording medium storing the program, and the like are also effective as an aspect of the present invention.

  ADVANTAGE OF THE INVENTION According to this invention, it can support improving the efficiency of the test of the system which forms the business flow in which several processes from which the development manager differs are connected.

It is a figure which shows typically the flow of a test of an information system. It is a figure which shows typically the acquisition method of the test trail. It is a figure which shows the system configuration | structure of the comprehensive test in 1st Embodiment. It is a block diagram which shows the structure of the test execution apparatus of FIG. It is a figure which shows the example of a screen which a test object system provides. It is a figure which shows the action sheet produced with respect to the screen of FIG. It is a figure which shows the data sheet produced with respect to the action sheet | seat of FIG. It is a figure which shows the kind of test scenario. It is a figure which shows the example of the scenario sheet classified by process. It is a figure which shows the example of the multiple scenario execution sheet according to process. It is a figure which shows the example of a multi-process test execution sheet. It is a figure which shows the Example of the comprehensive test of a test object system. It is a figure which shows the system configuration | structure of the comprehensive test in 2nd Embodiment. It is a figure which shows the structure of the business event process status table. It is a block diagram which shows the structure of the test execution apparatus of FIG. It is a figure which shows the data which the business event process ID holding | maintenance part of FIG. 15 hold | maintains. It is a figure which shows the example of a screen which a test object system provides. It is a block diagram which shows the function structure of the test execution apparatus of 3rd Embodiment. It is a figure which shows an action sheet. It is a figure which shows a data sheet. It is a figure which shows a scenario sheet.

(First embodiment)
First, the main features of the test execution apparatus according to the first embodiment will be described.
(Feature 1)
As shown in FIG. 1A, in a comprehensive test of an information system in which a plurality of processes with different development managers are connected to form a business flow, one test case (for example, specific business event data) is divided into a plurality of processes. It was common to digest across processes. In other words, a test is performed for the entire business flow across the scope of responsibility of multiple development managers, and the normality of the information system is judged based on the test results, for example, whether the behavior is as specified It was common to do.

  However, in the actual system development, in the method shown in FIG. 1 (a), tests such as which function of each process should be tested and what combination of functions to be tested for each process should be tested. There was a problem that the execution range became unclear. In addition, there is also a problem that it is unclear which trail is the test trail of which process among the trails acquired as a result of executing the test. The trail includes data generated as a result of the information system test, for example, data recorded in a database or a log. As a result, it takes time to divide the trail for each person in charge of development, and it is not easy to isolate the cause of the failure, and it may take time to deal with the failure.

  Based on this problem recognition, the present inventor has standardized the method shown in FIG. 1B as a suitable method in an integrated test of an information system in which a plurality of processes with different development managers are connected to form a business flow. I came up with the idea. In the test by the test execution device of the embodiment, a test scenario for one test case is created in units of development managers, in other words, for each process that is the responsibility range of one unit of development managers. Then, business events are processed over a plurality of processes according to a test scenario for each process, and an information system test is automatically executed. If an error occurs in the test, record the test scenario information of the process in which the error occurred. The test scenario defines, for example, a test case ID, a test target function, input data, and the like.

  In this way, by preparing a test scenario for each information system to be tested with multiple functions as a unit of the person in charge of development, it becomes clear who created the error in the test scenario, and is responsible for handling the error. The decomposition point becomes clear. For example, for a system in which multiple processes with different development managers are connected to form a business flow, it becomes easier to identify the person responsible for the error occurrence location (for example, the team responsible for error analysis / repair) in the comprehensive test. , Can help improve the efficiency of failure handling. The development manager in the embodiment includes a development team and a company composed of a plurality of people. The business event of the embodiment indicates one unit of matter (order, purchase, etc.) to be processed by the information system.

(Feature 2)
Real information systems often require compliance with security standards, and it is necessary to comply with these security standards during comprehensive testing. The security standards of the embodiment stipulate that the client device should access the server via a firewall. Further, the firewall determines that the authentication is performed in units of users and port numbers, and the login time of the user and the command issued by the user are left in the log.

  Conventional test execution apparatuses can acquire trails by the following two methods. In the conventional method 1 shown in FIG. 2, the test execution apparatus directly accesses the server and acquires a test trail from the server. Since this method does not go through the firewall, it does not record who issued what command. Therefore, although the test trail can be obtained at high speed, it does not meet the security standards.

  The conventional method 2 shown in FIG. 2 is a method in which the test execution device is linked with a server connection tool (for example, free software provided by a third party), displays the contents of the trail to the standard output, and converts the display contents into a file. is there. Specifically, in order for the test execution device to log in to the server using the server connection tool, SSH communication is first executed through the firewall. When the login to the server is successful, the trail acquisition shell held by the server is executed and the contents of the trail are displayed on the standard output of the server connection tool. The test execution device saves the contents displayed on the standard output in a file.

  In this method, since the server is accessed via a firewall, it is possible to record who executed which command and satisfy the security standard. However, since the server connection tool exchanges data with the server in an interactive manner, there is a lot of waste of communication. Furthermore, in order to acquire a trail, it is necessary to display the contents of the trail once on the standard output of the server connection tool, which is slow. For these reasons, it takes a long time to obtain a trail.

  Therefore, as shown in FIG. 2, the test execution apparatus according to the embodiment causes the firewall to execute authentication using a protocol specialized for the purpose (specifically, a JDBC port for database access). The firewall forwards the request issued by the test execution device to a specific port of the firewall and forwards it to the server. Also, the response from the server is port forwarded and transferred to the test execution device. When a firewall relays a request, the firewall records the user name and request content in a log.

  In the server access method of the embodiment, unlike the conventional method 1, authentication and logging are performed by a firewall, so that the security standard can be satisfied. Also, unlike the conventional method 2, the firewall can relay communication between the test execution device and the database server by port forwarding to eliminate useless data exchange, thereby speeding up the trail acquisition. Also, there is no need to display the contents of the trailer on the standard output, and the trail acquisition can be speeded up.

  FIG. 3 shows the system configuration of the comprehensive test in the first embodiment. The test target system 12 is a system to be subjected to a comprehensive test, and includes a web application server 14 and a database server 16. The test target system 12 may further include various servers and devices such as a form server (not shown), and various information processing services may be realized by cooperation of these devices.

  The web application server 14 provides various information processing services by the test target system to the client device. The business flow of the information processing service provided by the web application server 14 includes a plurality of steps. The plurality of steps may include, for example, an entry step for registering a business event, a step for checking the contents of the entered business event, and a step for executing a billing process for the checked business event. In the embodiment, a plurality of processes are referred to as a process A, a process B, a process C, and a process D. Typically, processing of one business event starts from the process A and is completed in the process D.

  The web application server 14 holds a plurality of applications in which functions of the processes A to D, in other words, information processing logic is mounted. These applications are implemented by different development managers. For example, application A implements the function of process A, and the development manager is company A. On the other hand, the application B implements the function of the process B, and the development manager is B company. In addition, the application corresponding to each of the processes A to D provides a different screen to the client device for each process. For example, the application of the process A provides a web page for business event registration, and the application of the process B provides a web page for confirming whether there is a mistake in the registered business event contents.

  The web application server 14 periodically executes batch processing between processes when processing the entered business event over a plurality of processes. In other words, for a business event that has completed processing of a certain process, processing of the next process for the business event is started on the condition that periodic batch processing has been completed. Although the execution interval of batch processing may be arbitrary, in the embodiment, it is set to 5 minutes. For example, for the business event that has completed the process A, after executing the batch process for the output data in the process A, the process B for the business event that has completed the process A is started. Similarly, for the business event that has completed the process B, after executing the batch process for the output data in the process B, the process C for the business event that has completed the process B is started.

  The web application server 14 records information indicating the processing result of each process for the entered business event, for example, the processing date and time, the presence / absence of error, business event information, and the like in the database server 16. Specifically, it is stored in a predetermined column of a table predetermined for each process. The database server 16 holds a table in which information indicating the processing result of each process is recorded. The database server 16 starts a listener at a specific port number, and accepts a DB operation command including a connection request and an update request / reference request for a table.

  In the embodiment, the web application server 14 provides a web page to the client device, acquires information input to the web page from the client device, and executes business processing. As a modified example, not only a web page but also a client device displays a dedicated client screen, the client device transmits input information to the screen to the test target system 12, and the test target system 12 executes business processing. Also good.

  The test execution device 10 is an information processing device that executes a comprehensive test of the test target system 12. Specifically, the test execution device 10 operates as a client device of the test target system 12 and inputs the business event information to the test target system 12 via a communication network such as a LAN, WAN, or the Internet. The operation of the system 12 is verified.

  The test execution device 10 acquires a web page from the web application server 14 and reproduces a predetermined operation on the web page, thereby causing business event processing in the web application server 14 to proceed. In addition, the test execution device 10 acquires test trail information from the database server 16 and determines whether the operation of the test target system 12 is normal. In FIG. 3, the firewall 18 is drawn only when accessing the database server 16, but it goes without saying that the firewall may also be used when accessing the web application server 14.

  FIG. 4 is a block diagram showing a configuration of the test execution device 10 of FIG. The test execution device 10 includes a control unit 30 and a data recording unit 20. The control unit 30 controls the operation of the test execution device 10 and executes various data processing related to the test of the test target system 12. The data recording unit 20 is a storage area for storing various types of data that are referred to and updated in data processing by the control unit 30. Although not shown in FIG. 4, a predetermined web browser program is installed in the test execution device 10.

  Each block shown in the block diagram of the present specification can be realized in terms of hardware by an element such as a CPU and a memory of a computer or a mechanical device, and in terms of software, it can be realized by a computer program or the like. , Depicts functional blocks realized by their cooperation. Therefore, those skilled in the art will understand that these functional blocks can be realized in various forms by a combination of hardware and software. For example, an automatic test execution application including a software module corresponding to each functional block of the control unit 30 may be installed in the test execution device 10. Then, the function of each functional block of the control unit 30 may be exhibited by the CPU of the test execution device 10 reading out and executing each software module in the memory. Each functional block of the data recording unit 20 may be realized by storing data in a storage device such as a storage or a memory.

  The data recording unit 20 includes a scenario holding unit 22, an action sheet holding unit 24, a data sheet holding unit 26, a collation data holding unit 27, and a test result holding unit 28.

  The action sheet holding unit 24 holds a plurality of action sheets corresponding to a plurality of processes of the information processing service provided by the test target system 12. An action sheet is a document created for each screen provided by the test target system 12. As already described, since the screen is different at least for each process, an action sheet that is different for at least each process is created. In the action sheet, screen operation information listing operations that can be input on the screen is recorded. The screen operation information is, for example, information indicating a list of operations that can be input on one web page in a certain process, and is input to each screen element such as a button or a text field arranged on one web page. It can be said that it is information that comprehensively shows possible operations.

  FIG. 5 shows an example of a screen provided by the test target system 12. A screen that is an operation input target in the test is also referred to as an “operation target screen” below. FIG. 5 shows a web page that is an operation target screen, and an element locator shows an ID assigned to each screen element (form) arranged on the web page. The IDs of the buttons Menu 1 to 4 are “menu1”, “menu2”, “menu3”, and “menu4”, respectively.

  FIG. 6 shows an action sheet created for the screen of FIG. One record of the action sheet corresponds to one type of operation that can be input on the screen. Specifically, Nos. 1 to 4, 8, and 9 in FIG. 6 indicate operations for the buttons 1 to 4, the new registration button, and the change button in FIG. 5. Further, Nos. 5 to 7 in FIG. 6 indicate operations for the USER text field, the PASS text field, and the PASS re-input text field in FIG. In the action sheet, it can be described that the data stored in the data sheet shown in FIG. 7 is referred to as the value of the parameter passed to the function. For example, FIG. 6 describes that the value of the “menu1” column of the data sheet is set as the value of No1 and parameter 2 (element locator to be clicked). Therefore, when “dom = document.getElementById ('menu1')” is described in the “menu1” column of the data sheet, the Menu 1 button in FIG. 5 is pressed during test execution. Further, FIG. 6 describes that the value of the “user” column of the data sheet is set as the value of No5 / parameter 3 (value to be set in the input field). Therefore, when a value to be set is described in the “user” column of the data sheet, the value is set in the USER text field of FIG. 5 when the test is executed.

  The action sheet holding unit 24 holds each of the plurality of action sheets in association with the identification information of the screen targeted by each action sheet. For example, the URL of the web page in FIG. 5 is held in association with the action sheet in FIG.

  Returning to FIG. 4, the data sheet holding unit 26 holds a plurality of data sheets corresponding to the plurality of action sheets held by the action sheet holding unit 24. One data sheet is a document created for each action sheet, that is, one document created per screen provided by the test target system 12. In the data sheet, an operation to be actually input to the screen in the test is designated, and test operation information defining the content of the operation is recorded.

  FIG. 7 shows a data sheet created for the action sheet of FIG. FIG.7 (b) has shown the continuation of Fig.7 (a). In the data sheet, columns corresponding to each screen element that can be operated on the operation target screen are provided (“menu1” to “update” in FIG. 6). One record in the data sheet corresponds to one operation in the test. Specifically, line 1 in FIG. 7 shows test operation information that defines that the menu 1 button in FIG. 5 is pressed and other screen elements are not executed.

  Line 2 in FIG. 7 indicates that “nomura” is entered in the USER text field, “abcdef” is entered in the PASS text field, “abcdef” is entered in the PASS re-entry text field, and the new registration (insert) button is pressed. Indicates defined test operation information. Line 3 of FIG. 7 defines that “nomura” is entered in the USER text field, “efghij” is entered in the PASS text field, “efghij” is entered in the PASS re-entry text field, and the update button is pressed. Test operation information. Whether or not to execute the operation described in the action sheet is controlled by whether or not a value is set in the corresponding operation column in the data sheet. The test execution unit 34, which will be described later, does not execute an operation on a column for which no parameter is specified in the data sheet during the test.

  Returning to FIG. 4, the scenario holding unit 22 holds a test scenario that defines the contents of the comprehensive test. As shown in FIG. 8, in the test execution apparatus 10 of the embodiment, the test is performed by three types of documents: (1) process-specific scenario sheet, (2) process-specific multiple scenario execution sheet, and (3) multi-process test execution sheet. Configure the scenario. The scenario holding unit 22 holds the documents (1) to (3) as test scenarios.

  (1) The process-specific scenario sheet defines a closed business flow in one process, and one process-specific scenario sheet is created for each process / work event test. FIG. 9 shows an example of a scenario sheet for each process. In one record of the scenario sheet for each process, the operation content for one screen provided in a specific process is defined. Specifically, it corresponds to the test case ID (also referred to as business event ID) which is identification information of the scenario sheet for each process, the action sheet created for one screen provided in a specific process, and the action sheet Specify the data sheet to be tested and the line number of the data sheet that describes the operation to be tested. For example, the action sheet specified in the record 2 in FIG. 9 is the action sheet in FIG. 6, and the data sheet specified in the record is the data sheet in FIG. Since row number 2 is specified in the record, the user registration operation specified in row 2 of the data sheet in FIG. 7 is tested.

  Returning to FIG. 8, the (2) process-specific scenario execution sheet defines the execution mode of a plurality of process-specific scenario sheets. It can be said that this is a document for batch execution of a plurality of process-specific scenario sheets for one specific process. FIG. 10 shows an example of a process-specific multiple scenario execution sheet. The multiple scenario execution sheet for each process in the figure is the process specified by the scenario sheet with the test case ID “A-100” from the test in the process A specified by the scenario sheet with the test case ID “A-001” (FIG. 9). The test of A is to be executed sequentially. That is, it is defined that 100 test scenarios for the process A of the test target system 12 are sequentially executed.

  Returning to FIG. 8, the (3) multi-step test execution sheet defines a test execution mode over a plurality of steps. It can be said that it is a document for batch execution of a plurality of scenario execution sheets for a plurality of processes over a plurality of processes. FIG. 11 shows an example of a multi-step test execution sheet. The multi-process test execution sheet in FIG. 3 sequentially executes a multi-scenario execution sheet for process D (D_multi-scenario execution sheet) from a multi-scenario execution sheet for process A (A_multi-scenario execution sheet shown in FIG. 10). Is stipulated. It also defines the type of web browser used for testing and the mode of error handling.

  The verification data holding unit 27 is data for determining whether or not the test result for the test target system 12 is normal, and holds verification data for comparison with the test trail acquired from the test target system 12. The verification data may be defined as the expected value of the test output data based on the specifications of the test target system 12 and the test input data. When the test target system 12 returns a web page in response to the test, the verification data may be data of a web page expected to be a response (including image data). In the embodiment, collation data is created for each process-specific scenario sheet, and the collation data holding unit 27 holds each collation data in association with the test case ID set in the process-specific scenario sheet.

  The test result holding unit 28 holds information indicating the result of the automatic test performed by the test target system 12. The test result holding unit 28 holds, as error information detected in the automatic test, information indicating an error occurrence date and time, an error content, a scenario sheet, an action sheet, and a data sheet related to the error occurrence test in association with each other.

  Returning to FIG. 4, the control unit 30 includes an operation detection unit 32, a test execution unit 34, a trail acquisition unit 36, a test result determination unit 38, a test result recording unit 40, and a test result display unit 42. The operation detection unit 32 detects an operation input by the user to an input device such as a keyboard or a mouse.

  The test execution unit 34 starts an automatic test for the test target system 12 when an operation for instructing execution of the test is input. In the embodiment, a specific multi-step test execution sheet is specified in an operation for instructing execution of a test. The test execution unit 34 identifies the process-specific scenario execution sheets to be executed in the order of the records of the multi-process test execution sheet, and identifies the process-specific scenario sheets to be executed in the order of the records of the process-specific multiple scenario execution sheets. Then, the action sheet, data sheet, and data sheet row number to be executed are identified in the order of records in the process-specific scenario sheet.

  The test execution unit 34 passes the URL associated with the execution target action sheet in the action sheet holding unit 24 to the web browser. The web browser acquires the operation target web page specified by the URL from the web application server 14 and displays the web page on the display. The test execution unit 34 executes an operation specified by the record of the execution target row number in the execution target data sheet, among the operations specified in the execution target action sheet, on the web page displayed by the web browser.

  For the operation on the web page, a known method such as the method described in Prior Art Document 1 may be used. For example, a web browser can be operated on a web page by calling a well-known API published by a web browser program using the frame locator or element locator value specified in the action sheet or data sheet, an input character string, etc. as arguments. You may let them. The web browser transmits an HTTP-POST request or the like to the web application server 14 in accordance with the web page form operation, and acquires and displays another web page. Typically, when the web page is switched, the test execution unit 34 proceeds to the next record in the process-specific scenario sheet, and identifies another action sheet and data sheet defined by the next record as execution targets.

  The test execution unit 34 sequentially identifies the action sheet and the data sheet to be executed in order of records in the multi-process test execution sheet, record order of the process-specific scenario execution sheets, and record order of the process-specific scenario sheets. The test of the test target system 12 is advanced. For example, when the multi-process test execution sheet of FIG. 11 is designated, the test of the multi-task event is advanced in the order of process A → process B → process C → process D.

  When the processing of one record (for example, for 100 business events of process A) of the multi-process test execution sheet is completed, the test execution unit 34 waits for a period until the batch process that is periodically executed is completed, and then the next record ( For example, processing of 100 business events in process B) is started. The period for waiting for the completion of the batch process may be a fixed period (for example, 10 minutes) determined to be equal to or greater than the interval according to the execution interval (for example, 5 minutes) of the batch process. When information indicating completion of batch processing is recorded in a predetermined storage area of the test target system 12, the test execution unit 34 monitors whether the information is recorded and detects that the information is recorded. In this case, business event processing (for example, business event processing for the next process) by the next record of the multi-process test execution sheet may be started.

  The trail acquisition unit 36 acquires test trail information from the database server 16 when it detects that the test execution unit 34 has completed processing of one process-specific scenario sheet, that is, one business event test in one process. To do. The trail information is information recorded in the database server 16 by the web application server 14 as a result of causing the web application server 14 to execute a business event process by operating a web page. For example, data associated with a test case ID of a test target stored in a predetermined table / column that is a recording destination of a business event processing result in the test target process may be acquired as trail information.

  The trail acquisition unit 36 transmits a database (DB) connection request to the firewall 18 via the JDBC driver installed in the test execution device 10. In the DB connection request, (1) the IP address of the test execution device 10 as the source IP address, (2) the IP address of the firewall 18 as the destination IP address, and (3) the predetermined port used by the JDBC driver as the source port number Specify a number. Furthermore, (4) a predetermined port number for DB connection is designated as a destination port number, and (5) predetermined account information (for example, a user name and password) for DB connection is designated.

  The firewall 18 holds a combination of a port number for DB connection and account information to which DB access should be permitted. When the firewall 18 receives a DB connection request that specifies a DB connection port number, the firewall 18 determines whether or not the account information specified in the DB connection request is consistent with the account information that should be permitted DB access. To do. If they match, access from the test execution device 10 to the firewall 18 is permitted.

  When the firewall 18 permits access from the test execution device 10 to the firewall 18, the firewall 18 transfers the DB connection request to the firewall 18 by port forwarding. Specifically, (1) the source IP address is converted into the address of the firewall 18, (2) the destination IP address is converted into the IP address of the database server 16, and (3) the source port number is set to a predetermined value of the firewall 18. (4) The DB connection request in which the destination port number is converted into a predetermined port number activated by the DB listener of the database server 16 is transferred to the database server 16.

  In response to the DB connection request transmitted from the firewall 18, the database server 16 transmits a DB connection response indicating that it has connected to the database to the firewall 18. Specifically, (1) the IP address of the database server 16 as the source IP address, (2) the IP address of the firewall 18 as the destination IP address, and (3) the DB listener activated as the source port number A DB connection response specifying the port number and (4) a predetermined port number of the firewall 18 specified by the DB connection request is transmitted.

  The firewall 18 transfers the DB connection response transmitted from the database server 16 to the test execution device 10 by port forwarding. Specifically, (1) the source IP address is converted into the address of the firewall 18, (2) the destination IP address is converted into the IP address of the test execution device 10, and (3) the source port number is used for DB connection. (4) The DB connection response in which the destination port number is converted into a predetermined port number used by the JDBC driver of the test execution device 10 is transferred to the test execution device 10.

  After receiving the DB connection response, the trail acquisition unit 36 transmits a DB operation command (for example, an SQL query) in which (1) to (4) are set to the firewall 18 in the same manner as the DB connection request. The firewall 18 transfers the DB operation command to the database server 16 by port forwarding in the same manner as the DB connection request. After receiving the DB operation command, the database server 16 transmits the command execution result (query response) in which (1) to (4) are set to the firewall 18 in the same manner as the DB connection response. The firewall 18 transfers the command execution result transmitted from the database server 16 to the test execution device 10 by port forwarding in the same manner as the DB connection response.

  The firewall 18 sequentially records the DB connection request and the contents of the DB operation command received at the DB connection port number together with the account information and the reception date and time in a predetermined log file. Authentication results and other items required by security standards may be further recorded in the log file.

  The test result determination unit 38 compares the test trail information acquired by the trail acquisition unit 36 with the verification data stored in advance in the verification data storage unit 27 to determine the presence / absence of an error in each test case. Specifically, the test trail information acquired at the completion of the test using the specific scenario sheet for each process is compared with the collation data associated with the test case ID set in the scenario sheet for each process. If the two match, it is determined that the test result is normal, that is, it is determined that the operation of the test target system 12 is normal. On the other hand, if they do not match, it is determined that the test result is abnormal, that is, an operation error of the test target system 12 is detected. When the test target system 12 provides a web page and the test execution unit 34 automatically executes data input to the web page to test the operation of the test target system 12, the test result determination unit 38 After the input data to the page is transmitted to the test target system 12, it is determined by comparing the response data with the verification data whether the response data (response web page, etc.) from the test target system 12 is as expected. May be.

  The test result recording unit 40 records information indicating the result of the comprehensive test for the test target system 12 in the test result holding unit 28. Specifically, for a specific test case for the test target system 12, a test scenario that specifies the test case ID, test trail information, normal / abnormal determination result by the test result determination unit 38, and the test to be determined Is recorded as test result information. This test scenario information includes at least identification information (path name, file name, etc.) for each process scenario sheet in order to clarify the person responsible for the location of the error, that is, the developer responsible for error analysis and modification. )including. In addition, it includes identification information of action sheets and data sheets defined in the scenario sheet for each process.

  The test result display unit 42 displays the test result information held in the test result holding unit 28 on the display when an operation for instructing display of the test result is input. For example, if the test result is an error, the fact of the error is displayed in association with information on the scenario sheet, action sheet, and data sheet for each process in which the error has occurred.

The operation during the test with the above configuration will be described below.
A development manager of each of the plurality of processes in the information processing service of the test target system 12 creates an action sheet, a data sheet, a scenario sheet for each process, a plurality of scenario execution sheets for each process, and collation data. The person in charge of development and testing of the entire test target system 12 creates a multi-process test execution sheet. The created various sheets are stored in the data recording unit 20 of the test execution device 10.

  The person in charge of the test inputs a test start instruction designating a specific multi-step test execution sheet to the test execution apparatus 10. The test execution unit 34 drills down in the order of a multi-process test execution sheet, a multi-process multi-scenario execution sheet, and a multi-process scenario sheet, and specifies an execution target action sheet, data sheet, and operation target web page. The test execution unit 34 performs an operation specified by the execution target action sheet and the data sheet on the operation target web page displayed on the web browser. The test execution unit 34 sequentially switches a web page, an action sheet, and a data sheet in accordance with a multi-process test execution sheet, a multi-process multi-scenario execution sheet, and a multi-process scenario sheet, and performs business event processing over a plurality of processes. By proceeding, the comprehensive test of the test target system 12 is automatically executed.

  As shown in FIGS. 1, 8, and 11, in the embodiment, operation tests for a plurality of business events are collectively performed in a single process. Specifically, the operation test for 100 business events in the process A is sequentially executed, and the operation test of the process A is completed until the operation test of the process A for the last business event among those business events is completed. Even if it is a business event, the start of the operation test in the process B is suppressed. Then, on the condition that the operation test for 100 business events in the process A is completed and the regular batch processing is completed, the operation test for the same 100 business events for the process B is started. Also after the process B, after the operation test for 100 business events is completed in a single process, the process proceeds to the next process.

  When the test using one process-specific scenario sheet is completed, the test execution unit 34 passes the identification information and the test case ID of the process that has completed the test to the trail acquisition unit 36. The trail acquisition unit 36 accesses the database server 16 through authentication of the firewall 18. The trail acquisition unit 36 acquires the test trail information identified by the process and the test case ID from the database server 16 through the port forwarding of the firewall 18. When the firewall 18 receives a message designating a specific port number for database connection as a destination port number from the test execution device 10, the firewall 18 is designated by the message while transferring the message to the database server 16 according to the authentication result. The user name and operation details (commands, etc.) are recorded in the log. The test result determination unit 38 compares the test trail information and the collation data, and determines whether or not the operation of the test target system 12 is normal for a specific test case.

  The test result recording unit 40 stores the test result including the test trail information and the normality determination result regarding the operation of the test target system 12 in the test result holding unit 28. The test result display unit 42 displays the test result on the display device in response to a user request. When an error is detected in the operation of the test target system 12, the test result recording unit 40 associates the process-specific scenario sheet, action sheet, and data sheet corresponding to the test case in which the error has occurred with the test result holding unit 28. Store. The test result display unit 42 causes the display device to display the process-specific scenario sheet, action sheet, and data sheet corresponding to the test case where the error has occurred.

  FIG. 12 shows an embodiment of the comprehensive test of the test target system 12. Each of “process A to process D test execution sheet 1”, “process A to process D test execution sheet 2”,... In FIG. In the embodiment, one multi-step test execution sheet is created for each cycle of step A to step D.

  For example, in the case of a test case (business event) in which rework occurs such as process A → B → C → B → C → D, first, in the execution of the sheet 50, the test of process A → B → C (actually A test based on three process-specific scenario sheets for each process is executed. Then, in the execution of the sheet 52, the process B → C → D test is executed.

  In addition, a cycle test that repeats the workflow many times can be realized by executing a multi-process test execution sheet a plurality of times. As a specific example, a cycle test in which steps A → B → C → D are repeated five times includes, for example, a sheet 50, a sheet 52, a sheet 54, a test using 400 process-specific scenario sheets for 100 business events × 4 processes, This can be realized by sequentially executing the sheet 56 and the sheet 58.

  If an error of the test target system 12 is detected during the test, the test is performed again at the responsibility of the development manager of the location where the error occurred. As already described, the information of the process-specific scenario sheet is presented as error information. The retest is a scenario sheet for each process related to the test case in which an error occurred in the previous test. By using the scenario sheet for each process presented in the error information, the problem can be identified and repaired efficiently. .

  According to the test execution device 10 of the embodiment, the process of the input business event is processed based on the characteristics of the test target system 12 in which the development manager completes the process over a plurality of different processes. The test is executed for each scenario unit divided into each. When an error is detected in a test based on each scenario, the scenario of the process in which the error has occurred is recorded, and error information including scenario information is disclosed. By recording trails and normality judgment results in scenario units for each process in this way, it is possible to avoid taking time to divide the trails for each development manager and to isolate the cause of the failure. Can be realized. In addition, it becomes easy to identify the person responsible for the location where the error occurred, and it is possible to support the efficiency of the failure handling.

  Further, according to the test execution apparatus 10, based on the characteristics of the test target system 12 that should execute a scheduled batch process between business event processes of different processes, the process A starts from the start of the process A to the end of the process D. The test of a plurality of business events is collectively executed within one process in units of one process, not the test that continues until. Thereby, it is possible to reduce the time required for testing a plurality of business events. For example, in the case of a test that continues from the start of process A to the end of process D with one business event as a unit, batch processing between process A and process B is performed in addition to the processing time of each process for the test of one business event. A waiting time of 5 minutes, a batch processing waiting time of 5 minutes between process B and process C, and a batch processing waiting time of 5 minutes between process C and process D will be required. On the other hand, in the system according to the embodiment, the batch processing waiting time is generated only once in a test of 100 business events, and an efficient test can be realized by reducing the batch processing waiting time.

  Further, according to the test execution device 10, the division of roles between the action sheet and the data sheet is clarified, and one action sheet is created per screen, and the operation actually executed in the test can be standardized so as to be defined in the data sheet. For example, as operations of a web page including a text field, a radio button, button A, and button B, (1) input a text and press button A, and (2) select a radio button and press a button B. However, the action sheets are common, and the operations (1) and (2) can be reproduced simply by changing the data on the data sheet (designated line number). By such standardization, in the test target system 12 in which a plurality of functions are distributedly developed by a plurality of development managers, it is possible to achieve homogenization of the results of comprehensive tests such as action sheets, data sheets, and process-specific scenario sheets. In addition, improvement in maintainability can be realized by homogenizing the deliverables.

  The present invention has been described based on the first embodiment. This embodiment is an exemplification, and it will be understood by those skilled in the art that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are also within the scope of the present invention. is there.

(Second Embodiment)
In the second embodiment, an information system having a dependency between online processing executed in real time for a request and a regular batch executed at a predetermined time interval independently of the request is automatically tested. The technology for this will be described. The online processing in the embodiment means processing executed in real time by the test target system when data input to the web page is transmitted from the test execution device to the test target system.

  The system under test in the second embodiment does not allow online operation unless a regular batch that is executed periodically is completed, in other words, accepts an online processing request input via a web page. Absent. In the conventional test execution device, before the test target system enters the online processing acceptance state, the web page is automatically operated to request online processing, and the test is abnormally terminated. As a result, the original test item may not be confirmed. It is also conceivable that the test execution device waits for a certain period of time until the test target system is allowed to perform online processing, but the time required to reach that state is not necessarily constant, and an inherently unnecessary waiting time occurs. End up. Even in the case of waiting, if the online processing is not permitted within the designated waiting time, the test will end abnormally.

  Therefore, the test execution apparatus according to the second embodiment monitors a table in which the execution result of the periodic batch in the test target system is stored after the online processing of a certain process is completed, and the test target system performs the online processing of the next process. It is determined whether or not it is in a state of permitting. As a result, when automatically testing an information system in which completion of batch processing that is executed periodically is an on-line processing start condition, it is possible to avoid a test from ending abnormally and suppress a decrease in test efficiency. .

  Hereinafter, the configuration of the second embodiment will be described in detail with reference to the drawings. Components that are the same as or correspond to the devices and functions in the first embodiment are denoted by the same reference numerals. Below, the description which overlaps with 1st Embodiment is abbreviate | omitted suitably. Needless to say, the configuration of the first embodiment and the configuration of the second embodiment can be appropriately combined and replaced.

  FIG. 13 is a diagram showing the system configuration of the comprehensive test in the second embodiment, and corresponds to FIG. The test target system 12 according to the second embodiment includes a batch processing server 13 and an external connection server 15 in addition to the web application server 14 and the database server 16.

  The batch processing server 13 holds an online batch 60 and a regular batch 62 for processing business event information input to the web application server 14. The online batch 60 is a batch processing function (batch file) that processes in real time the data input by the client for the web page of each process. In the embodiment, the test execution apparatus 10 transmits data automatically input to the web page of each process to the web application server 14, and the web application server 14 requests the batch processing server 13 for online processing. The processing server 13 executes the online batch 60. The online batch 60 executes, for example, a check process for business event information input to a web page of each process and a registration process in a database.

  The regular batch 62 is a batch processing function that is periodically executed at a predetermined time interval within a time period in which the external connection server 15 can be used (for example, from 9:00 to 18:00). The regular batch 62 is repeatedly executed periodically independently of the online batch 60, in other words, regardless of whether the test execution apparatus 10 has operated the web page of each process. The execution interval of the regular batch 62 is typically several minutes. For example, the regular batch 62 passes the business event information input to the web page of each process to the external connection server 15 and causes the external connection server 15 to execute various processes.

  The web application server 14 does not accept the online process request for the next process while the regular batch is not executed for the data of the business event after executing the online process of the certain process for one business event. Specifically, a web page of a certain process for inputting information relating to a business event is converted into a client apparatus (in the comprehensive test, the test execution apparatus 10 on the condition that a regular batch as a premise of the process is completed). ).

  The database server 16 holds a business event process status table 64 in which the batch processing status in units of business events in the batch processing server 13 is stored. FIG. 14 shows the configuration of the business event process status table 64. The business event process status table 64 stores a business event process ID, a business event number, and a regular batch completion date and time in association with each other. The business event process status table 64 can also be said to be a table that holds a change history of the progress status of business event processing in the test target system 12.

  The business event process ID is processing identification information in the test target system 12, and is uniquely assigned to each business event unit and each online batch 60 unit. When a plurality of online batches 60 are executed in one business event process in a certain process, the business event process ID of the online batch 60 executed last is stored in the business event process status table 64. In FIG. 14, the business event with the business event number “003” and the business event with the business event number “455” are assumed to have executed the same online batch 60 of the same process, and are associated with the same business event process ID. Shows the situation.

  The business event number is a unique identification number for each business event to be processed in the test target system 12. If they are the same business event, the same number is assigned from process A to process D. The business event number corresponds to the test case ID in the process-specific scenario sheet of FIG. 9, and hereinafter, the test case ID in the scenario sheet is also referred to as a business event number. The regular batch completion date and time is the regular batch completion date and time for the business event, and the initial value is a specific initial value. If the periodic batch completion date / time column is not a specific initial value, it is considered that the periodic batch has been completed, that is, the online processing of the next process can be accepted.

  FIG. 15 is a block diagram showing the configuration of the test execution apparatus of FIG. 13 and corresponds to FIG. The test execution apparatus 10 according to the second embodiment further includes a business event process ID holding unit 29 and a business event state monitoring unit 33. Other configurations are the same as those of the first embodiment.

  The business event process ID holding unit 29 holds a business event process ID for each process of business event processing in the test target system 12. Specifically, as shown in FIG. 16, the business event process ID holding unit 29 includes a business event pre-assigned to the identification information of each process and the online batch 60 executed last in the online processing of each process. The process ID is associated and held.

  Returning to FIG. 15, the business event state monitoring unit 33 has executed a periodic batch to be executed after online processing of the previous process before the test execution unit 34 requests online processing of a certain process for the test business event. It is determined whether or not. Before requesting online processing is before the web application server 14 requests processing that permits the termination of a regular batch, for example, before acquiring a web page, before inputting data to the web page, or It may be before input data transmission processing. For example, the same business event number (test case ID) is assigned to the test business event in the process-specific scenario sheet of FIG. The test execution unit 34 inputs an operation for requesting the online process of the next process (the above “certain process”) on the condition that it is determined that the regular batch has been executed after the online process of the previous process for the test work event. To do.

  Specifically, the business event state monitoring unit 33 reads the business event process ID associated with the previous process in the business event process ID holding unit 29. Then, the business event process status table 64 of the database server 16 is searched using the combination of the business event number of the test business event (the business event number described in the process-specific scenario sheet) and the business event process ID as a key. The business event status monitoring unit 33 has a record that matches the combination of the specified business event number and the business event process ID in the business event process status table 64, and the date and time is set in the periodic batch completion date and time column of the record. In the case where the initial batch value is not a specific initial value in the embodiment, it is determined that the regular batch after the online process in the previous process has been executed.

  The business event state monitoring unit 33 may perform polling of the business event process status table 64 of the database server 16 once every 3 seconds, for a total of 20 times. The polling interval and the number of executions may be set to appropriate values according to the execution interval of the regular batch in the batch processing server 13. For example, the longer the regular batch execution interval, the more polling times may be set. In addition, in order to suppress unnecessary polling, a longer polling interval may be set as the regular batch execution interval is longer, and a waiting time equal to or shorter than the regular batch execution interval is set before the start of regular polling. May be.

  Even if the business event status monitoring unit 33 executes polling a predetermined number of times, the business event process status table 64 does not have a record that matches the combination of the specified business event number and business event process ID. Alternatively, if the periodic batch completion date / time column of the record has a specific initial value, an error message indicating that the test start of the test business event has failed is recorded in the log. At the same time, the subsequent test execution by the test execution unit 34 is stopped, for example, the process of acquiring the web page of the next process from the web application server 14 is stopped.

  As a specific implementation, the test execution apparatus 10 may be provided with a function (herein referred to as “monitoring method”) for starting the monitoring process of the business event process status table 64 by the business event state monitoring unit 33. . Then, the person in charge of the test may describe the monitoring method at the head of the action sheet for the first screen operation of each process so that the monitoring method is called at the beginning of the test of each process. The person in charge of the test may describe the polling interval and the number of times on the data sheet.

  When executing the monitoring method, the test execution unit 34 sets the polling interval and the number of times described in the data sheet, the name of the process (or the previous process determined from the process) described in the scenario sheet, and the business event number of the test business event. You may pass to the test execution part 34. The business event status monitoring unit 33 starts monitoring processing of the business event process status table 64 according to each parameter passed from the business event status monitoring unit 33, and whether or not the periodic batch processing has been completed for the test business event. May be determined.

  The operation during the test with the above configuration will be described below. In the following, one test case common to a plurality of process-specific scenario sheets defining tests over a plurality of processes is called a test work event, and an operation when testing the test work event will be described.

  The test execution unit 34 of the test execution apparatus 10 executes the test according to the process-specific scenario sheet, action sheet, and data sheet for the process A in which the business event number of the test business event is set. Specifically, the API of the web browser is called as appropriate, the web page of process A is acquired from the web application server 14 and displayed on the web browser, information on the test work event is automatically input to the web page, and the web Manipulate forms such as page buttons. Then, an online processing request including the business event number of the test business event and information input to the web page is transmitted to the web application server 14 via the web browser.

  When receiving the online processing request from the test execution device 10, the web application server 14 transmits information on the test business event (including the business event number) to the batch processing server 13 so that the online batch 60 for the process A is executed. Instruct the batch processing server 13. A batch execution unit (not shown) of the batch processing server 13 executes the online batch 60 for the process A for the test job event. After executing the online batch 60 for the process A, the batch execution unit stores the combination of the business event number of the test business event and the business event process ID of the executed online batch 60 in the local storage area. It is also stored in the business event process status table 64. At this time, the periodic batch completion date / time column is set to a specific initial value.

  When the batch execution unit of the batch processing server 13 detects that a predetermined time has elapsed since the previous execution of the regular batch 62, it executes the regular batch 62 again. For the test work event, the process of the regular batch 62 for the process A is executed. Typically, if there are a plurality of business events that have been processed online, the batch processing unit processes those business events in a batch by executing the regular batch 62 once. When the batch execution unit completes the processing of the periodic batch 62 for process A for the test work event, it reads the combination of the work event number and the work event process ID related to the test work event stored in the predetermined storage area. Then, the scheduled batch completion date and time is stored in the record of the business event process status table 64 that matches the combination.

  The test execution unit 34 of the test execution device 10 executes the test according to the process-specific scenario sheet, action sheet, and data sheet for the process B in which the business event number of the test business event is set. At this time, the test execution unit 34 first instructs the business event state monitoring unit 33 to confirm the completion of the regular batch processing for the test business event. The business event state monitoring unit 33 accesses the database server 16 and uses the combination of the business event number of the test business event and the business event process ID of the process A held in the business event process ID holding unit 29 as a key. The situation table 64 is searched. The business event state monitoring unit 33 repeats the search process at intervals and times specified in the data sheet.

  There is a record in the business event process status table 64 that matches the combination of the business event number and the business event process ID specified as the search key, and the periodic batch completion date / time column is other than a predetermined value (for example, other than a specific initial value). Then, the business event state monitoring unit 33 determines that the periodic batch processing of the process A for the test business event is completed, and notifies the test execution unit 34 to that effect. For example, when the business event process status table 64 is in the state shown in FIG. 14, it is determined that the regular batch processing has been completed for the business event number “003”, and the regular batch processing has not been completed for the business event number “455”. judge. The business event state monitoring unit 33 records an error if the above record does not exist even after executing the search processing specified number of times in the data sheet, and stops the process B test for the test business event. As a modification, the search process may be repeated until the record is recorded in the business event process status table 64.

  When the business event state monitoring unit 33 notifies the test execution unit 34 that the periodic batch processing of the process A for the test business event has been completed, the test execution unit 34 sets the business event number of the test business event for each process B. Execute the test according to the scenario sheet, action sheet, and data sheet. Thereafter, the tests of the process B, the process C, and the process D are similarly executed according to the settings of the process-specific multiple scenario execution sheet and the multi-process test execution sheet.

  In the comprehensive test, instead of processing a plurality of test business events for each process at once, the second implementation is also performed when it is desired to test the processes A to D as quickly as possible for one test business event. The test execution device 10 in the form of is useful. In this case, when the test execution unit 34 controls the web browser and transmits the online processing request for the process A related to the test business event to the web application server 14, the business event state monitoring unit 33 polls the business event process status table 64. Processing may be started immediately. When the business event state monitoring unit 33 detects the end of the regular batch processing for the test business event, the test execution unit 34 may immediately transmit an online processing request for the process B related to the test business event to the web application server 14. As described above, according to the test execution device 10 of the second embodiment, when an information system that automatically sets completion of batch processing that is periodically executed as an online processing start condition is automatically tested, the test is terminated abnormally. It is possible to suppress a decrease in test efficiency while avoiding it.

  The present invention has been described based on the second embodiment. This embodiment is an exemplification, and it will be understood by those skilled in the art that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are also within the scope of the present invention. is there.

  For example, the business event state monitoring unit 33 accesses the database server 16 through authentication by the firewall 18 and port forwarding, similarly to the trail acquisition unit 36, and records each business event recorded in the business event process status table 64. Batch processing status may be monitored. As a result, it is possible to achieve both compliance with security standards and database access performance.

  This configuration can also be expressed as follows. The business event state monitoring unit 33 may transmit a database connection request designating a predetermined port number and account information for connection to the database server 16 to the firewall 18. As a result, when the account information in the database connection request for designating the port number for database connection matches the account information that should allow the database connection in advance, various commands for the database server 16 are sent to the database server 16. The firewall 18 may be caused to execute the process of transferring to

(Third embodiment)
In the third embodiment, a technique for extending a test automation range for a system that provides a document in which information is electronically described will be described. In the embodiment, a web page is shown as an example of a document in which information is electronically described. However, the technique described in the embodiment is not limited to a web page, and can be applied to various forms of documents such as a dedicated screen displayed in a specific application and plain text. Further, the present invention is not limited to documents, and can be applied to digital contents including various images and videos displayed on the screen.

  In the description of the third embodiment, elements constituting a web page are collectively referred to as “elements”. The element is an element constituting the screen of the web page, and includes various objects such as a button, a text field, and a radio button. Among elements, an object (for example, JAVA applet or PDF data) expressed in other than HTML is particularly called “content” (JAVA is a registered trademark). The actual content data may be binary data (for example, JAVA applet or PDF data) surrounded by a predetermined tag in web page data such as an HTML file. Hereinafter, unless otherwise specified, content is included when simply referred to as an element. The web application server 14 in the test target system 12 according to the third embodiment is drawn by a Java applet in addition to elements drawn by HTML code and JavaScript (registered trademark) (hereinafter also referred to as “JAVA script”) code. A web page including the content to be transmitted is transmitted to the client device and displayed.

  FIG. 17 illustrates an example of a web page provided by the test target system 12 according to the third embodiment. The USER text field and the login button are elements described by HTML code and Java script code and drawn on the screen of the web browser. Until now, the automatic test execution application records the operation contents on the screen for these elements in association with the ID of each element, and executes the automatic test using the recorded data.

  On the other hand, the text field of PASWORD in FIG. 17 is content written in the Java applet code and drawn on the screen of the web browser. Conventional test automatic execution applications cannot acquire the ID of the content drawn by the Java applet, and as a result, cannot execute the automatic test for the content.

  Therefore, the test execution apparatus according to the third embodiment tries to record the operation at the element level and records the operation at a lower level, more specifically, at the level of the operation position on the screen of the web browser. Thereby, an operation record at least at the level of the operation position is secured. The test execution device reproduces the operation on the web page by selectively using operation records at two levels. This enhances the reproducibility of operations on web pages and extends the scope of automated testing.

  Hereinafter, the configuration of the third embodiment will be described in detail with reference to the drawings. Components identical or corresponding to those in the other embodiments are denoted by the same reference numerals. Below, the description which overlaps with other embodiment is abbreviate | omitted suitably. Of course, the configuration of the third embodiment and the configuration of the other embodiments can be appropriately combined and replaced.

  FIG. 18 is a block diagram illustrating a functional configuration of the test execution apparatus 10 according to the third embodiment, and corresponds to FIG. 15. The test execution device 10 according to the third embodiment further includes an operation recording unit 44. Other configurations are the same as those of the first and second embodiments. Although not explicitly mentioned in the first and second embodiments, the test execution apparatus 10 has many applications such as input / output functions such as keyboard / mouse input and screen output, and disk / memory management. An operating system (OS) that provides basic functions commonly used by software and manages the entire computer is installed.

  The operation recording unit 44 automatically generates an action sheet according to the operation of the person in charge of the test on the web page. As preparation for the automatic test, the person in charge of the test causes the web page to be tested to be displayed on the screen of the web browser (also referred to as a web browser window displayed on the display, hereinafter referred to as “browser window”). Various operations via an input device such as a mouse or a keyboard are input to the elements of the web page displayed in the browser window. For example, a character input operation for a text field, a click operation for a button object, and a drag operation for an icon are input. The operation recording unit 44 generates an action sheet storing data indicating the operation details of the tester on the web page displayed in the browser window.

  The operation recording unit 44 includes a high level recording unit 45 and a low level recording unit 46. The high-level recording unit 45 is an element arranged on the web page in association with the operation identification information input to the web page of the browser window, and an element locator that is the identification information of the element operated by the test subject Is recorded in the action sheet. Hereinafter, the information of the element locator including the element ID recorded on the action sheet by the high level recording unit 45 is also referred to as high level operation information.

  Specifically, the high-level recording unit 45 obtains the operation type in the browser window and the ID of the operation target element by calling a publicly known API provided by COM (Component Object Model). When the high-level recording unit 45 is implemented in the Java language, a publicly known method for acquiring the ID of the operation target element predetermined by COM may be called through the com4j. Alternatively, an event listener or an action listener may be registered with a browser window as a target, and an operation type and an ID of an operation target element may be acquired by calling a callback function from the COM component side when an element is operated.

  The low level recording unit 46 records coordinate information indicating the operation position in the browser window in the action sheet in association with the operation identification information input to the web page of the browser window. The coordinate information may be a combination of an X coordinate and a Y coordinate in a coordinate system with the upper left corner in the client area of the browser window as the origin. Hereinafter, the coordinate information recorded on the action sheet by the low-level recording unit 46 is also referred to as low-level operation information.

  Specifically, the low-level recording unit 46 calls the well-known API provided by the OS (Windows API if the OS is Windows (registered trademark)), thereby obtaining the operation type in the browser window and the coordinates indicating the operation position. get. When the low-level recording unit 46 is implemented in the Java language, the native code of the OS may be called through JNI (Java Native Interface). Further, the operation type and the coordinate of the operation position may be acquired by calling a callback function for acquiring the operation type and the coordinate of the operation position for the browser window from the OS side.

  The operation recording unit 44 adds a Java script code for acquiring the operation type and the coordinates of the operation position to the code of the test target web page that the tester operates in advance, and then displays the test target web page on the web browser. You may let them. The low level recording unit 46 may acquire the operation type and the coordinates of the operation position output by the Java script code and record them in the action sheet.

  Similarly to the first and second embodiments, the test execution unit 34 executes an automatic test for the test target system 12 according to an action sheet, a data sheet, and a scenario sheet. The action sheet of the third embodiment always stores low-level operation information by the low-level recording unit 46, but the high-level operation by the high-level recording unit 45, such as when the operation target is a Java applet content. Information may not be stored. The test execution unit 34 displays the web page to be tested in the browser window, and reproduces the operation input to the web page by selectively using the high-level operation information and the low-level operation information.

  The test execution unit 34 selects the high-level operation information when both the high-level operation information and the low-level operation information are stored in one record of the action sheet indicating one operation on the web page. At this time, the test execution unit 34 inputs an operation associated with the high-level operation information to the operation target element indicated by the high-level operation information. For example, even if an element in a web page displayed in a browser window is operated by calling a well-known API for element operation provided by COM using information specifying the operation content and the ID of the operation target element as arguments, Good.

  The test execution unit 34, when only low level operation information is stored in one record of an action sheet indicating one operation on a web page, or when an element operation based on the high level operation information fails, Select operation information. At this time, the test execution unit 34 inputs an operation associated with the low-level operation information at the coordinate position indicated by the low-level operation information in the browser window. For example, an operation is input at a specific position of a web page displayed in a browser window by calling a well-known API for input operation provided by the OS with information specifying the operation content and the coordinates of the operation position as arguments. May be.

  The test execution unit 34 detects an element operation failure by not accepting a predetermined response even when a predetermined waiting time has elapsed after calling the COM API during element operation based on high-level operation information. May be. Further, after a COM API call, when a predetermined error is returned from the call destination COM component, an element operation failure may be detected.

  When the low-level operation information is selected, the test execution unit 34 notifies the high-level recording unit 45 to that effect. When the high-level recording unit 45 detects that the test execution unit 34 operates the web page based on the low-level operation information, the high-level recording unit 45 tries to acquire information on the operation target element. For example, when the callback function is called from the COM side during the test execution by the test execution unit 34, the operation content and the ID of the operation target element are acquired. The high level recording unit 45 additionally records the ID of the operation target element in the action sheet in association with the operation content.

  The operation during the test with the above configuration will be described below. In the following, an operation for recording an operation on the web page shown in FIG. 17 and reproducing the operation will be described.

  Before starting the automatic test of the test target system 12, the person in charge of the test causes the test execution device 10 to automatically generate an action sheet that records operations on the web page to be executed in the test. Specifically, the person in charge of the test starts an automatic test execution application installed in the test execution device 10 and displays a test target web page in a browser window. At this time, the operation recording unit 44 records the record of the action sheet number 1 shown in FIG. 19 (hereinafter referred to as “record 1”, the same applies to other records).

The person in charge of the test inputs a character string into the USER text field drawn by the HTML code. At this time, the low level recording unit 46 acquires the operation type information indicating the character string input and the coordinates of the operation position via the OS API, and records them in the record 2 of the action sheet.
The coordinate of the operation position is a position on the screen where the test person clicked for the character input operation, and can be said to be a character input start position, and is (300, 600) here.

  In parallel, the high-level recording unit 45 acquires the operation type information indicating the character string input and the ID (in this case, user) of the operation target component via the COM API, and records them in the record 2 of the action sheet. In the example of FIG. 19, the parameter 2 is recorded in the format of high level operation information || low level operation information. The high level operation information is an element locator including the element ID acquired by the high level recording unit 45, and the low level operation information is coordinate information acquired by the low level recording unit 46.

  Note that the low-level recording unit 46 holds a correspondence relationship between operation type information acquired via the OS API and a function predetermined by the test automatic execution application. For example, when operation type information indicating character string input is acquired via the API of the OS, a function “typeText” associated with the operation type is recorded in the action sheet. Similarly, the high-level recording unit 45 holds a correspondence relationship between operation type information acquired via the COM API and a function predetermined by the test automatic execution application. For example, when the operation type information indicating character string input is acquired via the COM API, the function “typeText” associated with the operation type is recorded in the action sheet.

  The person in charge of the test inputs a character string into the text field of the PASWORD drawn by the Java applet. At this time, the low-level recording unit 46 acquires the operation type information indicating the character string input and the coordinates of the operation position (300, 800 in this case) via the OS API, and records them in the record 3 of the action sheet. Since the PASWORD text field is drawn with Java applet code, the high-level recording unit 45 cannot acquire the operation type information indicating the character string input and the ID of the operation target component via the COM API. Therefore, high level operation information is not recorded.

  The person in charge of the test causes the web application server 14 to post the input data to the USER text field and the PASSWORD text field by clicking the login button drawn by the HTML code. Accordingly, the user authentication process executed by the web application server 14 (or an authentication server (not shown)) is tested.

  At this time, the low-level recording unit 46 acquires the operation type information indicating the click operation and the coordinates of the operation position (click position, here, 600, 900) via the OS API, and stores them in the record 4 of the action sheet. Record. The low-level recording unit 46 records the function “clickButton” previously associated with the operation type information “click operation” returned by the OS API in the action sheet.

  In parallel, the high-level recording unit 45 acquires the operation type information indicating the click operation and the ID (here, login) of the operation target component via the COM API, and records them in the record 4 of the action sheet. The high-level recording unit 45 records the function “clickButton” previously associated with the operation type information “click operation” returned by the COM API in the action sheet. Through the above steps, the action sheet of FIG. 19 for reproducing the operation of the person in charge of the test is automatically generated.

  Note that the value of parameter 2 “element locator” in FIG. 19 may be set in the data sheet as described in the first embodiment, and the presence or absence of the test may be defined depending on whether or not the value is set in the data sheet. . In this case, the high level recording unit 45 and the low level recording unit 46 store the element locator and coordinates in the data sheet. Then, the column name of the data sheet storing the element locator and coordinates is set in the data sheet column name column of parameter 2 of the action sheet.

  Further, parameter 3 in FIG. 19 defines that the input data to the text field is defined by a data sheet. The parameter 3 may be set manually by the person in charge of the test in consideration of the column name of the data sheet, but may be set automatically by the operation recording unit 44. For example, when the operation recording unit 44 records a function indicating a predetermined operation such as character input on the action sheet, the parameter for which input data is to be set among parameters predetermined by the function is set so that the value is determined on the data sheet. A predetermined value (such as an element ID acquired by the high level recording unit 45) may be set in the sheet row name column.

  The tester then creates a data sheet. FIG. 20 shows a data sheet. The column names “user” and “password” of the data sheet correspond to the data sheet column name of parameter 3 in the action sheet of FIG. The person in charge of testing sets the value to be set in the USER text field of FIG. 17 in the user column of the data sheet, and sets the value to be set in the PASSWORD text field of FIG. 17 in the password column of the data sheet.

  The tester then creates a scenario sheet. FIG. 21 shows a scenario sheet. In the figure, the file name of the action sheet in FIG. 19 and the file name of the data sheet in FIG. 20 are designated. “1-3” in the row column specifies that the data stored in the rows 1 to 3 of the data sheet are used sequentially. In this example, a scenario in which the login authentication of three users is continuously tested is shown. As shown in FIG. 9 of the first embodiment, when another action is to be executed after login, a combination of an action sheet and a data sheet is further specified after record 2 of the scenario sheet.

  The person in charge of the test inputs a test start instruction specifying the scenario sheet in FIG. 21 to the test automatic execution application. The test execution unit 34 starts the automatic operation of the web page according to the action sheet of FIG. 19 and the data sheet of FIG. The test execution unit 34 acquires the test target web page from the web application server 14 according to the record 1 of the action sheet, and displays it on the browser window.

  Record 2 of the action sheet stores both high-level operation information and low-level operation information, but the test execution unit 34 selects the high-level operation information. The test execution unit 34 calls the typeText function with the element locator of the high-level operation information set in the record 2 of the action sheet and the user character string set in the row 1 of the data sheet as arguments. When high-level operation information (that is, element locator) is specified as parameter 2, as the processing of the typeText function, (1) the USER text field indicated by the element locator is focused, and (2) the user character string is set in the text field. (3) An operation for releasing the focus of the USER text field (focus out) is input to the browser window.

  Since only low level operation information is stored in the record 3 of the action sheet, the test execution unit 34 selects the low level operation information. The test execution unit 34 calls the typeText function with the coordinates of the low-level operation information set in the record 3 of the action sheet and the password character string set in the row 1 of the data sheet as arguments. When low-level operation information (that is, coordinates) is specified as parameter 2, as the processing of the typeText function, (1) click the specified coordinates, (2) set the password string, and (3) perform the focus-out operation In the browser window.

  A plurality of types of processing using such a typeText function may be realized by overloading (in the case of Java language or the like). In addition, the typeText function may be implemented so as to identify whether the argument is high-level operation information or low-level operation information within the typeText function and execute different processing depending on the identification result.

  When the test execution unit 34 executes a web page operation based on the low-level operation information, the high-level recording unit 45 tries to acquire information on the operation target element via the COM API. When acquiring the ID of the operation target element, the high level recording unit 45 records the high level operation information in the action sheet in association with the executed operation. In this example, an element locator that is high-level operation information is additionally stored in the record 3 of the action sheet. When the high-level operation information is recorded, the web page operation based on the high-level operation information is executed at the next and subsequent tests.

  The record 4 of the action sheet stores both high-level operation information and low-level operation information, but the test execution unit 34 selects the high-level operation information. The test execution unit 34 calls the clickButton function with the element locator of the high-level operation information set in the record 4 of the action sheet as an argument. When high-level operation information (that is, element locator) is specified as parameter 2, an operation of clicking the specified element locator is input to the browser window as the clickButton function. When low-level operation information (that is, coordinates) is specified as parameter 2, an operation of clicking the specified coordinates is input to the browser window as the clickButton function.

  When the test execution unit 34 detects a failure in the web page operation based on the high-level operation information, the test execution unit 34 executes the web page operation based on the low-level operation information. The above processing is repeated for row 2 and row 3 of the data sheet. As described above, the operation test of various variations can be efficiently realized by repeating the operation of the web page by the test person recorded in the action sheet a plurality of times according to the settings of the data sheet and the scenario sheet.

  Here, as the operations for the web page, the character input and the button click are shown, but other types of operations are also recorded and can be reproduced. For example, when the tester performs a drag operation of an element (icon or the like), the high level recording unit 45 records high level operation information in which the drag operation and the element locator are associated with each other. The low level recording unit 46 records low level operation information in which the drag operation is associated with the X and Y coordinates of the drag start position and the X and Y coordinates of the drag end position. The test execution unit 34 reproduces the drag operation by calling the COM API based on the high-level operation information or by calling the OS API based on the low-level operation information.

  According to the test execution device 10 of the third embodiment, the operation on the electronic document such as a web page is recorded at the level (high level) of the element to be operated, and the level of the operation position on the screen (low level). Record with. Then, the operation record at each level is selectively used to reproduce the operation on the document. As a result, even when high-level operation information cannot be recorded, such as when content drawn by a Java applet is operated, low-level operation information can be ensured and reproduced in an automatic test. In the embodiment, an example of a Java applet has been described. However, the same applies when an object drawn by another type of element that cannot record high-level operation information, for example, PDF or Flash (trademark or registered trademark) is included in the document. As a result, the range in which automatic testing can be performed can be expanded.

  Further, according to the test execution device 10, the high-level operation information is preferentially used over the low-level operation information when reproducing the operation on the document. Since the low-level operation information is obtained by recording coordinates on the display, there is a possibility that the reproducibility may be lowered when the display type or resolution is changed. Since the high-level operation information is less dependent on the user interface environment such as the display type and resolution, the reproducibility of the operation can be improved by using the high-level operation record with priority.

  Further, according to the test execution device 10, when the document operation based on the high level operation information fails, the document operation based on the low level operation information is tried. As a result, even if the element ID in the document is changed, the reproducibility of the operation is improved, and the failure of the operation test is easily avoided. In addition, the burden on the test personnel for regenerating the action sheet can be reduced.

  Further, according to the test execution device 10, when a document operation based on low-level operation information is performed, if information on an operation target element (content) can be acquired, high-level operation information is newly recorded. As a result, when the document is changed or the function of the automatic test execution application is expanded, high-level operation information can be additionally recorded, and the action sheet can be automatically optimized.

  The case where the document is changed is, for example, a case where an element that can record only low-level operation information is replaced with an element that can record high-level operation information. Further, when the function of the test automatic execution application is expanded, for example, the application corresponds to the Java applet, that is, the identification information of the Java applet object can be acquired, and the operation specifying the Java applet object is performed. This is when it becomes possible. In addition, for example, the test automatic execution application corresponds to Flash, that is, the identification information of the ActionScript object can be acquired, and the operation specifying the ActionScript object becomes possible.

  The present invention has been described based on the third embodiment. This embodiment is an exemplification, and it will be understood by those skilled in the art that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are also within the scope of the present invention. is there.

  In the third embodiment, the low-level operation information is recorded in the action sheet regardless of whether the high-level operation information is acquired. However, if the high-level operation information can be acquired, the low-level operation information is acquired. Recording may be skipped. In the third embodiment, the element ID is acquired via COM as a method for acquiring the high-level operation information. However, another function of the high-level operation information can be obtained by expanding the function of the test automatic execution application. An acquisition method may be added. In that case, it may be determined whether or not high-level operation information can be acquired by each method. For example, the following processing may be executed.

(1) High-level operation information acquisition 1: Acquire element ID via COM. If information cannot be acquired, proceed to (2).
(2) High-level operation information acquisition 2: High-level operation information is acquired by the method A added by function expansion. If it cannot be acquired, the process proceeds to (3).
(3) High-level operation information acquisition 3: High-level operation information is acquired by the method B added by function expansion. If it cannot be acquired, the process proceeds to (4).
(4) Acquisition of low-level operation information: Since high-level operation information cannot be acquired, low-level operation information is acquired. As in the embodiment, the low-level operation information may always be acquired regardless of whether the high-level operation information can be acquired.
Thereby, the certainty of acquiring high-level operation information can be further enhanced.

  Although not mentioned in the third embodiment, a “level” column may be provided in the action sheet as shown in FIG. High or low may be set as the value of the “level” column (hereinafter referred to as “level value”), and the default value may be high. The test execution unit 34 may operate the web page based on high-level operation information or low-level operation information that matches the level value. For example, if the level value is high, the operation is performed in the same manner as in the embodiment. On the other hand, if the level value is low, even if high-level operation information is recorded in the action sheet, the web page is based on the low-level operation information. May be operated. According to this configuration, the person in charge of the test can instruct the test using the low-level operation information as required.

  If the level value is high when the test execution unit 34 operates the web page based on the low level operation information, the high level recording unit 45 obtains the high level operation information as described in the embodiment. Recording may be attempted. On the other hand, if the level value is low, the acquisition and recording of high-level operation information may be skipped. As a result, acquisition and recording of high-level operation information that is judged unnecessary by the person in charge of the test can be arbitrarily skipped.

  Although not mentioned in the third embodiment, the test execution device 10 deletes the operation record that deletes the high-level operation information from the action sheet when the document operation based on the high-level operation information fails. A part may be further provided. As a result, it is possible to avoid the failure of the operation and the occurrence of an error each time the test is performed, in other words, every time the operation associated with the high-level operation information is executed. Even if the high-level operation information is once deleted, the high-level recording unit 45 tries to acquire and record the high-level operation information during document operation based on the low-level operation information. Therefore, even if the previous high-level operation information becomes invalid, such as when the element ID is changed, the latest high-level operation information is automatically reflected in the action sheet, and the latest high-level operation information is used in the next test. Information-based testing can be resumed.

  Any combination of the above-described embodiments and modifications is also useful as an embodiment of the present invention. The new embodiment generated by the combination has the effects of the combined embodiment and the modified examples. In addition, it should be understood by those skilled in the art that the functions to be fulfilled by the constituent elements described in the claims are realized by the individual constituent elements shown in the embodiments and the modified examples or by their cooperation. .

  10 test execution device, 12 test target system, 18 firewall, 22 scenario holding unit, 24 action sheet holding unit, 26 data sheet holding unit, 28 test result holding unit, 33 business event state monitoring unit, 34 test execution unit, 36 trail Acquiring unit, 38 Test result determining unit, 40 Test result recording unit, 44 Operation recording unit, 45 High level recording unit, 46 Low level recording unit.

Claims (6)

A device for testing a system that completes processing of input business events through a plurality of processes,
A scenario holding unit for holding a test scenario for each process;
A test execution unit for automatically executing a test of the system by processing a business event over the plurality of steps according to a test scenario for each step;
When an error is detected in the test, an error recording unit that records information indicating a test scenario of a process in which the error has occurred;
Equipped with a,
The plurality of steps include a first step and a second step in which the processing order of business events is after the first step,
Both test scenarios of the first process and the second process define processing of a plurality of business events,
The test execution unit, after completing the processing of the first step to the last business event from the first business event of the plurality of business events, Rukoto to begin processing of the second step to the plurality of business event A test execution device characterized by the above. A device for testing a system that completes processing of input business events through a plurality of processes,
A scenario holding unit for holding a test scenario for each process;
A test execution unit for automatically executing a test of the system by processing a business event over the plurality of steps according to a test scenario for each step;
When an error is detected in the test, an error recording unit that records information indicating a test scenario of a process in which the error has occurred;
Equipped with a,
The system should start processing the business event in the second step after executing a predetermined batch processing to be executed at regular intervals on the data output by the business event processing in the first step. ,
Both test scenarios of the first process and the second process define processing of a plurality of business events,
The test execution unit completes the processing of the first step for the first business event to the last business event of the plurality of business events and executes the batch process, and then executes the second processing for the plurality of business events. test execution and wherein the Rukoto processing step is started. A device for testing a system that completes processing of input business events through a plurality of processes,
A scenario holding unit for holding a test scenario for each process;
A test execution unit for automatically executing a test of the system by processing a business event over the plurality of steps according to a test scenario for each step;
When an error is detected in the test, an error recording unit that records information indicating a test scenario of a process in which the error has occurred;
A test result acquisition unit for acquiring a test result of the system from a predetermined database through a firewall;
Equipped with a,
The test result acquisition unit transmits a database connection request designating a predetermined port number and account information for connecting to the database to the firewall, whereby an account included in the database connection request designating the port number is included. information, if consistent with the account information should be permitted database connections predetermined test execution device according to claim Rukoto the process of transferring a command to the database is executed in the firewall for the database. A screen operation information holding unit for holding screen operation information for each screen listing operations that can be input on the screens of the plurality of steps;
A test operation information holding unit for holding test operation information for each screen for designating an operation to be actually input to the screen in the test,
The test scenario specifies screen operation information and test operation information corresponding to a screen of a specific process,
The test execution unit, according to the test scenario, among the operation of the screen operation information indicates, claims 1 to 3, characterized in that inputting an operation of the test operation information is specified in the screen of the particular process The test execution device according to any one of the above. A device that tests a system that completes the processing of input business events through multiple processes.
A test execution step of automatically executing a test of the system by processing a business event over the plurality of steps according to a test scenario for each step stored in a predetermined storage area;
When an error is detected in the test, recording information indicating a test scenario of a process in which the error has occurred;
The execution,
The plurality of steps include a first step and a second step in which the processing order of business events is after the first step,
Both test scenarios of the first process and the second process define processing of a plurality of business events,
Said test execution step, after completing the processing of the first step to the last business event from the first business event of the plurality of business events, Rukoto to begin processing of the second step to the plurality of business event Test execution method characterized by A device that tests a system that completes the processing of input business events through multiple processes.
A test execution function for automatically executing a test of the system by processing a business event over the plurality of processes according to a test scenario for each process stored in a predetermined storage area;
A function of recording information indicating a test scenario of a process in which an error has occurred when an error is detected in the test;
Realized ,
The plurality of steps include a first step and a second step in which the processing order of business events is after the first step,
Both test scenarios of the first process and the second process define processing of a plurality of business events,
The test execution function starts processing of the second process for the plurality of business events after completing the processing of the first process for the first business event to the last business event of the plurality of business events. A featured computer program.

Images