JP2014035595A - Testing device for communication system, testing program for communication system, and testing method for communication system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a testing device for a communication system, a testing program for a communication system, and a testing method for a communication system, which are able to maintain the system quality without increasing testing steps.SOLUTION: A testing device 1 for a communication system comprises: a data collection system 5 configured to capture operational telegraphic data circulating between an operating system 3 and an opposite system 13; a data creating section 7 configured to create testing data for circulation between a verification system 9 and a pseudo opposite system 15; an initial state setting section 2 configured to set the verification system 9 on the basis of the testing data so that an environment similar to the operational system 3 is reproduced; and a data comparison and determination section 11 configured to determine whether the verification system 9 is normal or abnormal by comparing the operational telegraphic data and the verification data captured when the testing data is input to the verification system 9 set to the initial state.

Description

  The present invention relates to a communication system test apparatus for testing the operation of a communication system, a communication system test program, and a communication system test method.

Conventionally, a software comprehensive test accompanying a system renewal of a communication system that transmits and receives data via a predetermined network such as a mobile phone, the developer determines the system influence range in the normality confirmation test, and the test item and test message IN / OUT information is created and implemented.
Patent Document 1 discloses a call connection test program for testing a call connection in an IP telephone switching system for the purpose of improving test efficiency and the like, and a process for creating a schedule as a collection of test items based on designation by a user And a process for executing a scenario set by a test item based on designation for each executable element, simulating a plurality of telephone terminals for an IP telephone switching system, and a process for recording a scenario execution result as a log Is executed by a computer of a test terminal.

JP 2011-151696 A

  In the call connection test disclosed in Patent Document 1, the start and end of a session between a telephone and an exchange are clear and the call connection of a telephone with a relatively short period from the start to the end of the session is actually The same test as when the call is connected can be performed. However, this test method is applied when the start and end of the session between the telephone and the exchange is unclear, or when the period from the start to the end of the session is relatively long (for example, several hours). It has the problem that it is difficult to do.

In recent years, the diversification of system functions has led to an increase in the number of normality / abnormality confirmation test man-hours. For this reason, the conventional test method has a problem that the number of test steps for the software scale becomes enormous. In the conventional test method, the process from creation of test items to confirmation of test results is performed manually. For this reason, for example, when creating test items, the program creator narrows down the test items based on his / her own judgment in order to shorten the test time. There is. Therefore, the conventional test method has a problem that the software quality may be lowered, and the quality of the system using the software is also lowered.
An object of the present invention is to provide a communication system test apparatus, a communication system test program, and a communication system test method that can maintain system quality without increasing the number of test steps.

  A communication system test apparatus according to an aspect of the present invention captures, for a predetermined period, operational message data distributed between an operational system and a counterpart system in which a session is currently established, and captures the operational message data. A data collection unit that collects start time data indicating a start time and operation recording data recorded in an operation data storage unit provided in the operation system, and has the same function as at least a part of the operation system A data generation unit that generates test data to be distributed between the verification system and a pseudo-opposite system that realizes an operation equivalent to the counter system based on the operation message data captured by the data collection unit; Based on the test data, the start time data, and the operation recording data, the data collection unit performs the operation. An initial state setting unit for setting the verification system to an initial state so that the environment similar to the operation system before capturing the telegram data is reproduced in the verification system; and the initial state by the initial state setting unit The verification system obtained by comparing the verification data acquired when the test data generated by the data generation unit is input with the operation telegram data and the operation recording data is compared with the verification system set to And a data comparison / determination unit for determining normality / abnormality of the function. According to such a configuration, it is possible to maintain the system quality without increasing the number of test steps.

In addition, the test data may include a dummy message for generating a pseudo start of the session. According to such a configuration, the operation system 3 at the start of capturing the message data and the operation difference of the verification system 9 at the start of the operation test and the result mismatch due to erroneous transmission of the response message are prevented. it can.
The operation recording data includes first operation recording data recorded in the operation data storage unit before the start of capture and second operation recording data stored in the operation data storage unit after the capture ends. The verification data is recorded in a verification data storage unit provided in the verification system after the initial state is set and before the operation test of the function of the verification system is started. And the first verification recorded data that is recorded in the verification data storage unit and is compared with the second operational recorded data after the operation test is completed. According to such a configuration, the normality / abnormality of the verification data storage unit provided in the verification system can be determined.

The verification data may include output message data that is output when the test data is distributed between the verification system and the pseudo-opposite system and is compared with the operation message data. According to such a configuration, it is possible to determine normality / abnormality of data transmission / reception between the verification system and the pseudo-opposite system.
The data comparison / determination unit collects the operation operation log data indicating the operation history in the operation system collected by the data collection unit, and is output after the operation test of the function of the verification system is completed. The verification operation log data indicating the operation history in the system may be compared. According to such a configuration, it is possible to realize operation confirmation as the entire system.

The data collection unit may include a capture function unit that captures the operation message data and a data output function unit that outputs the captured operation message data to a predetermined file. According to such a configuration, it is possible to cope with a network system having a high data amount.
In addition, the data generation unit includes a file acquisition function unit that acquires the predetermined file, and a test data generation function unit that generates the test data from the operation message data stored in the acquired predetermined file. May be. According to such a configuration, the operation message data distributed to the operation system can be directly distributed to the verification system.

  A test program for a communication system according to an aspect of the present invention captures, for a predetermined period, operation message data distributed between an operation system and a counter system in a state where a session is currently established. A data collection unit that collects start time data indicating a time when data capture is started and operation recording data recorded in an operation data storage unit provided in the operation system, and is the same as at least a part of the operation system A data generation unit that generates test data to be distributed between a verification system having a function and a pseudo counter system that realizes an operation equivalent to the counter system based on the operation message data captured by the data collection unit , Based on the test data, the start time data and the operation recording data, An initial state setting unit for setting the verification system to an initial state so that the environment similar to the operation system before the data collection unit captures the operational message data is reproduced in the verification system; and Verification data acquired when the test data generated by the data generation unit is input to the verification system set to the initial state by the initial state setting unit, the operation message data, and the operation recording data And a data comparison / determination unit that determines normality / abnormality of the function of the verification system. According to such a configuration, it is possible to maintain the system quality without increasing the number of test steps.

  The communication system testing method according to one aspect of the present invention captures the operation message data distributed between the operation system and the opposite system in which the session is currently established for a predetermined period, and captures the operation message data. A verification system that collects start time data indicating a start time and operation recording data recorded in an operation data storage unit provided in the operation system, and has a function identical to at least a part of the operation system; , Generating test data to be distributed between the pseudo-opposite system that realizes an operation equivalent to the counter system based on the operation message data captured by the data collection unit, and the test data, the start time data, and Based on the operation recording data, the data collection unit captures the operation message data The verification system is set to an initial state so that the same environment as the previous operation system is reproduced by the verification system, and the data generation unit generates the verification system set to the initial state. The verification data acquired when the test data is input is compared with the operation message data and the operation recording data to determine normality / abnormality of the function of the verification system. According to such a configuration, it is possible to maintain the system quality without increasing the number of test steps.

  According to the present invention, system quality can be maintained without increasing the number of test steps.

1 is a block diagram illustrating a schematic configuration of a test apparatus 1 for a communication system according to an embodiment of the present invention. 1 is a diagram illustrating a communication system test apparatus 1 according to an embodiment of the present invention, and is a block diagram illustrating a schematic configuration of an operation system 3 together with a counter system 3. FIG. 1 is a diagram for explaining a communication system test apparatus 1 according to an embodiment of the present invention, and is a block diagram showing a schematic configuration of a verification system 9 together with a pseudo-opposite system 15. It is a block diagram which shows schematic structure of the data collection part 5 and the data generation part 7 with which the test apparatus 1 of the communication system by one embodiment of this invention was equipped. It is a figure which shows typically the test method of the communication system by one embodiment of this invention. It is a figure explaining the test method of the communication system by one embodiment of this invention, Comprising: The state of the operation system 3 just before a capture is shown typically. It is a figure explaining the test method of the communication system by one embodiment of this invention, Comprising: The data flow from acquisition of operation message data to creation of an action definition file is shown typically. It is a figure explaining the creation method of the dummy message | telegram in the test method of the communication system by one embodiment of this invention. It is a figure explaining the test method of the communication system by one embodiment of this invention, Comprising: The state of the verification system 9 set to the initial state is shown typically. It is a figure explaining the test method of the communication system by one embodiment of this invention, Comprising: It is a block diagram which shows typically the operation test of the verification system 9, and a data comparison and determination process.

(Schematic configuration of communication system test equipment)
A communication system test apparatus, a communication system test program, and a communication system test method according to an embodiment of the present invention will be described with reference to FIGS. First, a schematic configuration of a test apparatus for a communication system according to the present embodiment will be described with reference to FIGS. The communication system in the present embodiment is a system including a monitoring device that monitors a predetermined network, for example. The present embodiment can also be applied to a communication system that transmits and receives data via a predetermined network such as a mobile phone. FIG. 1 is a block diagram showing a schematic configuration of a test apparatus 1 (hereinafter simply referred to as “test apparatus 1”) of a communication system according to the present embodiment. As shown in FIG. 1, the test apparatus 1 includes a data collection unit 5, a data generation unit 7, an initial state setting unit 2, and a data comparison / determination unit 11.

  The data collection unit 5 captures a predetermined period of telegram data (hereinafter also referred to as “operation telegram data”) that circulates between the operation system 3 and the opposite system 13 in which the session is actually established. The start time data indicating the time when the capture of the operation message data is started and the operation recording data recorded in the operation data storage unit (not shown) provided in the operation system 3 are collected. The operation system 3 is connected to the opposite system 13 via the router network 17. The operation system 3 is a system in which data is continuously transmitted to and received from the opposite system 13 via the router network 17 and is always in operation, and is also referred to as a commercial system. Capturing refers to collecting the telegram data distributed between the operation system 3 and the opposing system 13 in real time. Based on the operational message data captured by the data collection unit 5, the data generation unit 7 performs a simulation that realizes an operation equivalent to that of the verification system 9 having the same function as at least a part of the operational system 3 and the opposing system 13. Test data to be circulated with the opposing system 15 is generated. The test apparatus 1 according to the present embodiment captures operation message data distributed between the operation system 3 and the opposite system 13 in a state where a session is actually established, that is, in a state where the session is established, at a predetermined timing for a predetermined period. It is supposed to be. For this reason, the data generation unit 7 may generate halfway test data that cannot form a series of sequences from the start to the end based on the acquired operational message data. In this case, the data generation unit 7 creates a dummy telegram for pseudo generation of the start of the session and assigns it to the test data. For this reason, the test data may include a dummy message.

  The initial state setting unit 2 is based on the test data generated by the data generation unit 7, the start time data collected by the data collection unit 5, and the operation recording data before the data collection unit 5 captures the operation message data. The verification system 9 is set to an initial state so that the same environment as that of the operation system 3 is reproduced by the verification system 9. The data comparison / determination unit 11 includes the verification data acquired when the test data generated by the data generation unit 7 is input to the verification system 9 set to the initial state by the initial state setting unit 2, and the operation message. The normality / abnormality (normal or abnormal) of the function of the verification system is determined by comparing the data and the operation recording data. The verification system 9 is connected to a pseudo counter system 15 that realizes an operation equivalent to that of the counter system 13. As a result, transmission / reception of data between the operation system 3 and the opposing system 13 is reproduced between the verification system 9 and the pseudo opposing system 15.

  In the present embodiment, the system requirements for the operation system 3 and the verification system 9 to be satisfied are substantially equal. The test apparatus 1 captures the telegram data distributed on the operation system 3 side, extracts the system state and the application log, which are all elements that hold the state in the operation system 3, and captures and extracts them. These data are distributed as they are to the verification system 9 using the pseudo counter system 15. The operation system 3 is a system that is actually operating and operating normally. For this reason, when the operation message data distributed between the operation system 3 and the counter system 13 is distributed between the verification system 9 and the pseudo counter system 15, the same function as the operation system 3 of the verification system 9 is exhibited. The principle / principle that the function group operates in the same manner as the operation system 3 if it is normal is used in the test apparatus 1. The test apparatus 1 uses the operation of the operation system 3 that is actually operating as a comparison target of the operation of the verification system 9. For this reason, functions that are not installed in the operation system 3 such as newly developed items in the verification system 9 are not tested in this embodiment. The test apparatus 1 can execute a test of an existing function group similar to that installed in the operation system 3. If the operation message data distributed between the operation system 3 and the opposite system 13 is distributed to the verification system 9, the function system of the verification system 9 is configured so as to operate normally. The same operation as that of the function group is executed. As a result, the verification system 9 outputs the same data as the data output from the operation system 3. For this reason, the test apparatus 1 compares the output message data output from the verification system 9 with the output data output from the operation system 3 to determine whether or not both data are the same. Whether or not there is a system operation error of the system 9 is detected. Further, the test apparatus 1 compares the system state of the operation system 3 immediately after the end of capture with the system state immediately after the end of the operation test of the verification system 9, and further, operation log data indicating the operation history of the operation system 3 and the verification system The operation of the verification system 9 as a whole is confirmed by comparing with the operation log data indicating the operation 9.

(Schematic configuration of operation system)
FIG. 2 is a block diagram showing a schematic configuration of the operation system 3 together with the opposing system 3. FIG. 2 (a) shows a schematic configuration of the entire operation system 3, and FIG. 2 (b) shows an application function unit 3b. A schematic configuration is shown. As shown in FIG. 2A, the operation system 3 includes an adapter function unit 3a connected to the opposite system 13 via the router network 17 (not shown in FIG. 2A), and an application on which the application operates. It includes a function unit 3b, a state holding function unit 3c that holds the states of the adapter function unit 3a and the application function unit 3b, and a log file storage unit 3d that discharges the operation contents of the application function unit 3b. The state holding function unit 3c may be a memory or a database, for example. In addition, a plurality of opposing systems 13 may be connected to the operation system 3. In FIG. 2 and FIG. 3 to be described later, the fact that data is transmitted and received in both directions is indicated by a linear thick arrow Y1, and that data is transmitted in one direction is indicated by a straight one-way thick arrow Y2 (arrow The direction of the data indicates the direction in which the data moves), and one functional unit acquires data from the other functional unit. One-way thick arrow Y3 (the direction of the arrow indicates the direction in which the data moves) It is expressed with).

  As shown in FIG. 2B, the application function unit 3b includes an application execution unit 3e and a data storage unit 3f that stores, holds, or stores an execution result of the application. The data storage unit 3f has an input / output function for inputting and outputting an application execution result, a system operation definition file, and the like. The test apparatus 1 according to the present embodiment is adapted to a system having a function (details will be described later) that can extract stored data from the data storage unit 3f and extract interface distribution information with the opposing system 13. The data holding unit 3f may be configured with a memory or a database.

(Schematic configuration of the verification system)
FIG. 3 is a block diagram showing a schematic configuration of the verification system 9 together with the pseudo-opposite system 15. FIG. 3A shows a schematic configuration of the entire verification system 9. FIG. 3B shows an application function unit 9b. The schematic structure of is shown. As shown in FIG. 3A, the verification system 9 has the same configuration as the operation system 3 and can realize the same application operation. The verification system 9 is different from the operation system 3 in that, instead of the opposing system 13, a pseudo opposing system 15 which is a simulator capable of realizing an operation equivalent to the opposing system 3 is arranged. The pseudo-opposite system 15 is arranged so that the telegram information captured from the operation system 3 side has the same telegram distribution on the verification system 9 side. The pseudo-opposite system 15 is configured such that the message distribution timing on the verification system 9 side and the message distribution timing on the operation system 3 side are equivalent. In the present embodiment, instead of the pseudo counter system 15, a counter system may be connected to the verification system 9.

  Although details will be described later, the adapter function unit 9a has a plurality of function groups. Further, the same number of the pseudo opposing systems 15 as the opposing systems 13 connected to the operation system 3 may be prepared, or if the same operation as the plurality of opposing systems 15 connected to the operation system 3 can be realized. The number may be different from that of the opposing system 13. When a plurality of pseudo-opposite systems 15 are connected to the verification system 9, test data (hereinafter referred to as “test message”) that is automatically generated by capturing telegram data flowing between the operation system 3 and the counter system 13. Is set for each pseudo-opposing system 15. Thereby, the distribution timing of the test data with respect to the distribution timing of the message data is compensated. The pseudo-opposite system 15 has a function of returning a response message to a request message between systems or a function of discharging a request message.

  The verification system 9 includes an adapter function unit 9a connected to the pseudo-opposite system 15, an application function unit 9b in which an application operates, a state holding function unit 9c that holds the states of the adapter function unit 9a and the application function unit 9b, And a log file storage unit 9d from which the operation content of the application function unit 9b is discharged. The state holding function unit 9c may be a memory or a database, for example.

  As illustrated in FIG. 3B, the application function unit 9b includes an application execution unit 9e and a data storage unit 9f that stores, holds, or stores an execution result of the application. The data storage unit 9f has an input / output function for inputting and outputting an application execution result, a system operation definition file, and the like. The test apparatus 1 according to the present embodiment is adapted to a system having a function (details will be described later) that can extract stored data from the data storage unit 9f and extract interface distribution information with the pseudo-opposite system 15. The data holding unit 9f may be configured with a memory or a database.

(Schematic configuration of the data collection unit and data generation unit provided in the test apparatus)
FIG. 4 is a block diagram illustrating a schematic configuration of the data collection unit 5 and the data generation unit 7. As shown in FIG. 4, the data collection unit 5 is a local area connected to the router network 17 that is transmitted and received between the operation system 3 (not shown in FIG. 4) and the opposite system 13 (not shown in FIG. 4). Capture function unit 5a that captures operation message data (for example, transmission control protocol (TCP) data) distributed on a network (Local Area Network: LAN), and user data in the captured operation message data, for example, And a data output function unit 5b for outputting to a predetermined file (for example, the dump file 19). Further, the data collection unit 5 is a database indicating the status of the adapter function unit 3a and the application function unit 3b held in the status holding function unit 3c (see FIG. 2) of the operation system 3 based on a request described in SQL. A function of extracting and storing recorded data (hereinafter sometimes referred to as “operational recorded data”) or the like is exhibited. By having the data collection unit 5, the test apparatus 1 can exhibit a function of capturing operation message data and a function of acquiring operation recording data and start time data indicating a capture start time of operation message data. .

  The data collection unit 5 includes the number of opposing systems 13 that transmit / receive data to / from the operation system 3, the number of communication devices that are included in one opposing system 13 and transmit / receive data to / from the operation system 3, or A plurality may be provided so as to cope with a high load when traffic on the LAN increases. In this case, the data collection unit 5 registers the IP address of the opposite system or communication device from which data is to be collected. For example, when there are the data collection units α and β and the communication devices a to m, the data collection unit α is registered with information for outputting the data of the communication device a and the communication device b to the dump file, and the data collection unit Information that the data from the communication device c to the communication device m is output to the dump file is registered in β.

  As illustrated in FIG. 4, the data generation unit 7 includes a file acquisition function unit 7 b that acquires the dump file 19 of the data collection unit 5. The file acquisition function unit 7b acquires the dump file 19 from the data collection unit 5 by, for example, a file transfer protocol (FTP) or remote copy (rcp). In addition, the data generation unit 7 has a screen interface function unit 7a that receives a request from the operation terminal 29, a file acquisition request to the file acquisition function unit 7b, and returns a processing result for the request to the operation terminal 29. doing. Further, the data generation unit 7 includes a test data creation function unit 7c that creates test data from the operation message data stored in the dump file 19 acquired by the file acquisition function unit 7b. The test data creation function unit 7c outputs only the information necessary for sequence creation from the dump file 19 as a consolidated file 31, and the sequence creation function unit 7e creates a sequence file 33 from the contents of the consolidated file 31. And have. The sequence file 33 is a file including test data. The sequence creation function unit 7d outputs the created sequence file 33 to the screen interface function unit 7a. The created sequence file 33 is input to the initial state setting unit 2 (not shown in FIG. 4) via the screen interface function unit 7a.

  The operation terminal 29 sends a sequence display condition (for example, “period for which a sequence is to be created”, “opposite system or communication for which a sequence is to be created” to the screen interface function unit 7 a provided in the data generation unit 7. "Device", "keyword for which a sequence is to be created", etc.) are designated, the execution request and the result for the request are displayed, and the sequence file generated by the data generation unit 7 is displayed. In FIG. 4, the interface between the operation terminal 29 and the data generation unit 7 is exemplified by a request / response. However, for example, an interface based on http (HyperText Transfer Protocol) or a locally defined telegram interface may be used. .

(Outline of test method for communication system)
Next, a communication system test method according to this embodiment will be described with reference to FIGS. 1 to 4 and FIGS. FIG. 5 is a diagram schematically illustrating a test method for a communication system according to the present embodiment. Operation data 21 collected by the data collection unit 5 from the operation system 3 is shown in the rectangular frame on the left side of the figure, and a data generation unit is shown in the rectangular frame on the right side of the rectangular frame. 7 shows the test data 23 generated and the verification data 25 acquired as a result of the operation test of the verification system 9. Below the rectangular frames, the operation data 21 to be compared in the data comparison / determination unit 11 Verification data 25 is shown.

  As shown in FIG. 5, in the communication system test method according to the present embodiment, the test apparatus 1 first operates the data collection unit 5 and the operation system 3 and the opposite system 13 that are actually in a session established state. The operation message data 21c distributed between and a predetermined period is captured, start time data 21d indicating a time when the operation message data 21c starts to be captured, and an operation data storage unit (for example, a state) provided in the operation system 3 The first operation recording data 21a and the second operation recording data 21b recorded in the holding function unit 3c), and operation log data (hereinafter referred to as “operation”) indicating the operation history of the operation system 3 in a predetermined period from the start of capture to the end of capture. 21e) (referred to as “operation log data”). The operational message data 21c includes system IN information transmitted from the operational system 3 to the opposing system 13, and system OUT information transmitted from the opposing system 13 to the operational system 3 as a response to the system IN information. The first operation recording data 21a is data collected by the state holding function unit 3c before the capture is started (for example, immediately before), and is the environment (for example, the adapter function unit 3a and the application function) before the capture is started. Information indicating the state of the unit 3b). The second operation recording data 21b is data collected by the state holding function unit 3c after the capture ends (for example, immediately after), and includes information indicating the environment of the operation system 3 after the capture ends. The operation operation log data 21e is output to a log file that is automatically generated when a scenario is executed in the operation system 3. Hereinafter, the first and second operation recording data 21a and 21b, the operation message data 21c, the time information data 21d, and the operation operation log data 21e may be collectively referred to as “operation data 21”. As described above, in this embodiment, the data collection unit 5 stores the operation message data 21c distributed between the operation system 3 and the opposite system 13, and the state holding function unit 3c before and after the capture starts. The first and second operation recording data 21a and 21b, operation operation log data 21e, and start time data 21d are acquired.

  Next, the test apparatus 1 operates the data generation unit 7 and distributes between the verification system 9 and the pseudo-opposite system 15 in the operation test of the verification system 9 based on the operation message data 21 c captured by the data collection unit 5. Test data 23 to be generated is generated. The test data 23 includes operation definition data output to the operation definition file 23a and intersystem IN data output to the intersystem IN data file 23b. Moreover, the data generation part 7 adds a dummy message | telegram to the action definition data output to the action definition file 23a as needed. Hereinafter, the data output to each of the operation definition file 23a and the inter-system IN data file 23b may be collectively referred to as “test data 23”. The action definition file 23a is a file that defines actions to be executed by the pseudo-opposite system 15, and is created for each sequence.

When the generation of the test data 23 is completed, an automatic comprehensive test is performed in the verification system 9 using the test data 23. The automatic comprehensive test is roughly divided into two processes, an initial state setting process and a test data distribution process. The test data distribution process includes the process from the distribution of the test data 23 to the comparison / determination process of the acquired verification data 25.
First, the test apparatus 1 operates as follows in order to set the verification system 9 to an initial state. The test apparatus 1 operates the initial state setting unit 2, and based on the test data 23, the start time data 21d, and the operation recording data 21a, the operation system 3 before the data collection unit 5 captures the operation message data 21c. The verification system 9 is set to an initial state so that the same environment as that shown in FIG. The operation recording data 21a input to the verification system 9 is stored in the state holding function unit 9c. The test data 23 input to the verification system 9 is stored in a data storage unit 9f provided in the application function unit 9b. Furthermore, when the start time / time data is input to the verification system 9, the verification system 9 interrupts the time synchronization by the NTP (Network Time Protocol) server and captures the time of the management server (not shown) in the operation system 3. Set to the time just before the start.

  Next, the test apparatus 1 operates as follows in order to distribute the test message. The test apparatus 1 executes the operation test by inputting the test data 23 generated by the data generation unit 7 to the verification system 9 set to the initial state by the initial state setting unit 2, and as a result of the operation test. The verification data 25 is acquired. The verification data 25 includes first verification recording data 25a, second verification recording data 25b, output message data 25c, and operation log data (hereinafter referred to as “verification operation log data”) 25d. The first verification recorded data 25a has the same function as the operation system 9 of the verification system 9 after the initial state is set in the verification data storage unit (for example, the state holding function unit 9c) provided in the verification system 9. Data recorded before the start of the operation test. The second verification recorded data 25b is data recorded in the state holding function unit 9c after the end of the operation test. The output telegram data 25 c is data that is output as the test data 23 circulates between the verification system 9 and the pseudo-opposite system 15. For example, the output message data 25c is inter-system OUT data output from the pseudo-opposite system 15 to the verification system 9 based on, for example, the action definition file 23a. The verification operation log data 25d is data that is output after the end of the operation test and indicates an operation history in the verification system 9.

  Next, the test apparatus 1 operates the data comparison / determination unit 11 to compare the verification data 25 with the operation data 21 and determine normality / abnormality of the same function as the operation system 3 provided in the verification system 9. . The data comparison / determination unit 11 compares the first operation recording data 21a with the first verification recording data 25a, compares the second operation recording data 21b with the second verification recording data 25b, and outputs the operation telegram data 21d. The message data 25c is compared, and the operation operation log data 21e and the verification operation log data 25d are compared. The data comparison / determination unit 11 determines that the comparison target data is normal when there is no difference between the comparison target data, and determines that the comparison target data is abnormal when there is a difference between the comparison target data.

(Operation data acquisition process in communication system test method)
Next, each step of the communication system testing method according to the present embodiment will be described more specifically. First, the acquisition process of the operational data 21 that is mainly executed in the data collection unit 5 will be described with reference to FIGS. 4 and 5 and FIG. FIG. 6 schematically shows the state of the operation system 3 immediately before capture. As shown in FIG. 6, the application function unit 3b of the operation system 3 has a plurality of function groups (N function groups # 1 to #N in FIG. 6) to be verified in the verification system 9 (N is 2 The above-mentioned natural number) and a plurality of function groups (not shown) that are not subject to verification have a plurality of functions. The adapter function unit 3a of the operation system 3 has the same number of counter adapters # 1 to #N as the function group provided in the application function unit 3b to be verified in the verification system 9 (N is a natural number of 2 or more). And a plurality of opposed adapters (not shown) that are not to be verified. The opposing adapter # 1 is provided corresponding to the function group # 1, and the opposing adapter #N is provided corresponding to the function group #N. The inter-system IN data file and the action definition file output from the function group # 1 are input to the counter system 13 via the counter adapter # 1. The inter-system OUT data file from which operational message data that is response message data of the opposing system 13 is output is input to the function group # 1 via the opposing adapter # 1. Similarly, the function group #N is configured to execute data transmission / reception with the opposing system 13 via the opposing adapter #N.

  The states of the function groups # 1 to #N and the opposing adapters # 1 to #N are held in the state holding function unit 3c. For this reason, the data collection unit 5 accesses the state holding function unit 3c immediately before the start of capturing the operation message data, and records the operation of each of the function groups # 1 to #N and the opposing adapters # 1 to #N. Data 21a is acquired. As illustrated in the sequence diagram on the right side of FIG. 6, for example, the function group #N immediately before the start of capture (the capture start time is indicated by a broken line in the sequence diagram) receives data from the counter adapter #N. Is in a state.

First, the data collection unit 5 accesses the state holding function unit 3c to acquire information on the state of the operation system 3 immediately before the start of capturing the operation message data 21c, and functions groups # 1 to #N and the counter adapter. The first operation recording data 21a indicating the respective states of # 1 to #N is acquired.
Next, the data collection unit 5 captures the operation message data 21c distributed on the LAN connected to the router network 17 in a predetermined period that is a period to be verified by the verification system 9, and the operation message data 21c is, for example, a TCP dump. Thus, the dump file 19 is generated and acquired. More specifically, the data collection unit 5 activates the capture function unit 5a and starts capturing the operational message data 21c distributed on the LAN.

First, when the capture function unit 5a captures the operation message data 21c in the data collection unit 5, the data output function unit 5b is activated.
Next, the data output function unit 5b filters the captured operation message data 21c with data related to the IP address of the opposing system 13 registered in the setting information, and discards unregistered data.

If the data output function unit 5b determines that the captured data is SYN, the data output function unit 5b determines that the establishment of the TCP connection is started, and creates connection information for managing the state of the TCP connection according to the data contents as an internal table.
If the data output function unit 5b determines that the captured data is FIN or RST, the data output function unit 5b determines that the TCP connection is released and deletes the connection information.

  When the data output function unit 5b determines that the captured data is other than SYN, FIN, and RST and user data exists, the data output function unit 5b stores the user data in the dump file 19. The dump file 19 stores, for example, a transmission source IP address, a transmission destination IP address, a transmission source port number, a transmission destination port number, a captured time, and the like so that the data generation unit 7 can extract data. Since the memory has an upper limit on the amount of data that can be stored and may not be compatible with a network system with a high amount of data, a method of saving user data in the dump file 19 is employed in the present embodiment. In the present embodiment, it is of course possible to employ a system in which user data is stored in the memory by using a memory having a sufficient storage capacity that can cope with a network system having a high data amount.

When the capture of the operation message data 21c in the predetermined period is completed, the data collection unit 5 accesses the state holding function unit 3c and acquires the function group # 1 in order to obtain information on the state of the operation system 3 immediately after the capture ends. The second operation recording data 21b indicating the states of the #N and the counter adapters # 1 to #N is acquired.
In the test method for the communication system according to the present embodiment, the operation recording data is extracted before and after the operation message data capture is started. Thereby, when executing the test / verification of the verification system 9, it can be confirmed whether the verification system 9 records the first operation recording data 21 a in the state holding function unit 9 c in the same manner as the operation system 3. If the verification system 9 cannot record the first operation recording data 21a as in the operation system 3, and there is a difference between the first verification recording data 25a and the first operation recording data 21a, the data comparison / determination unit 11 It can be determined that there is an error in the recording result, and a defect (bug) in the verification system 9 can be detected.

(Test data creation process in communication system test method)
Next, the creation process of the test data 21 mainly executed in the data generation unit 7 will be described with reference to FIGS. 7 and 8 with reference to FIGS. FIG. 7 is a diagram schematically illustrating a data flow from acquisition of operation message data to creation of an operation definition file.
As illustrated in FIG. 7, the data generation unit 7 acquires the dump file 19 in which the operation message data 21 c captured by the data collection unit 5 from the operation system 3 is stored. The data generation unit 7 receives a test data creation request from the operation terminal 29, refers to the dump file 19, analyzes the data, and creates test data 23.

  For example, the data generation unit 7 first performs message distribution for each sequence unit (message division for each session id). Next, the data generation unit 7 generates the sequence-specific operation definition file 23a by performing the request message and response message distribution. Next, the data generation unit 7 generates an entire sequence file and an inter-system IN data file 23 b that is a response definition file and is used in the pseudo-opposite system 15. Note that the data generation unit 7 is stored in the dump file 19 when the test data 23 is created, and deletes operation message data that is unnecessary for the operation test in the verification system 9.

  Next, a method for generating the test data 23 will be described more specifically. First, a sequence creation condition is input from the operation terminal 29. Next, a predetermined period from the start to the end of the capture, which is a period for displaying (creating) the sequence, is input. Next, for example, a list of communication devices included in the opposing system 13 is displayed, and a communication device to be sequenced is selected. In the present embodiment, a plurality of communication devices can be selected. For example, communication devices A to C are selected. Next, by pressing the execution button, the data generation unit 7 is requested to create a sequence file under the above conditions. When the processing result is returned from the data generation unit 7, the sequence file is acquired from the data generation unit 7, and the content of the processing result is displayed on the display screen of the display device provided in the operation terminal 29. FIG. 7 illustrates a sequence diagram displayed on the display screen in a rectangular frame on the right side of the operation terminal 29 in the drawing.

  Next, the generation process of the test data 23 in the data generation unit 7 will be described more specifically. The screen interface function unit 7a (see FIG. 4) provided in the data generation unit 7 requests the file acquisition function unit 7b to perform a file acquisition process using the sequence creation condition designated from the operation terminal 29 as a parameter. When the screen interface function unit 7 a receives the file acquisition process completion information from the file acquisition function unit 7 b, the screen interface function unit 7 a returns the processing result, the sequence file 33, and the storage path of the consolidated file 31 to the operation terminal 29.

  The file acquisition function unit 7b stores the IP addresses of the communication devices A to C input from the operation terminal 29 in the collection target list. When the test apparatus 1 has a plurality of data collection units, the file acquisition function unit 7b collects the user data of each communication device from the list of communication devices input from the operation terminal 29. Search whether the part is saved. The file acquisition function unit 7b stores the IP address of the target communication device obtained from the search result in the collection target list. Next, the file acquisition function unit 7b acquires the dump file 19 of the sequence creation target period from the data collection unit 7 based on the contents stored in the collection target list.

  The data aggregation function unit 7d receives the list of communication devices input from the operation terminal 29 from the dump file 19 acquired from the data collection unit 5 by the file acquisition function unit 7b, the data numbers of individual units within the sequence creation period, and the capture time. And extract. Next, the data aggregation function unit 7d sorts the extracted data numbers and capture times in time series based on the capture times. Next, the data aggregation function unit 7 d extracts all the individual units including the data unit from the dump file 19 based on the sorted result, and stores them as an aggregated file 31. The format of the individual part is not changed from the dump file 19.

  The sequence creation function unit 7 e reads the aggregate file 31 created by the data aggregation function unit 7 d and outputs the sequence to the sequence file 33. More specifically, first, the sequence creation function unit 7e reads the aggregated file 31, determines whether each unique part is a sequence creation target, and if it is a creation target, stores information necessary for the sequence. To extract. Next, the sequence creation function unit 7e writes vertical lines to the sequence file for each communication device name (communication devices A to C in this example) that becomes vertical lines. Next, the sequence creation function unit 7e writes the sequence between the communication devices A, B, and C in the sequence file.

Here, the sequence creation target extraction will be described. The sequence creation function unit 7e reads the aggregated file 31 in units of individual units, and repeats the next process until the last individual unit is reached.
First, when “narrow down by keyword” is not designated on the operation terminal 29, the sequence creation function unit 7e adds an IP address (Source) and an IP address (Source) to the sequence vertical line element table used for drawing a sequence vertical line. (Destination) is recorded, and the data number is recorded in the sequence message number table used for drawing the horizontal line of the sequence. When “narrow down by keyword” is designated on the operation terminal 29, the sequence creation function unit 7e searches for the keyword from the relative position and checks whether or not it matches the keyword. If the sequence creation function unit 7e determines that the keywords match, the sequence creation function unit 7e executes the same processing as when “narrow down by keyword” is not specified on the operation terminal 29, and determines that the keywords do not match. Read the next individual part.

  Next, a sequence vertical line creation method will be described. The sequence creation function unit 7e first describes the communication device names (communication devices A to C in this example) for the number of IP addresses registered in the sequence vertical line element table in the first line of the file. Next, the sequence creation function unit 7e describes vertical lines (three in this example) for the number of rows of data numbers registered in the sequence message number table.

  Next, a sequence horizontal line creation method will be described. The sequence creation function unit 7e first reads the individual units in the consolidated file 31 in the order of the data numbers in the sequence number table. The sequence creation function unit 7e specifies the vertical line position of the communication device that is the base point of the horizontal line from the IP address (Source). Next, the sequence creation function unit 7e identifies the vertical line position of the communication device that is the end point of the horizontal line from the IP address (Destination). Next, the sequence creation function unit 7e describes an arrow from the base point to the end point. Next, the sequence creation function unit 7e describes the IP address (Source) beside the base point on one line of the arrow, and describes the IP address (Destination) beside the end point of the one line below the arrow.

(Dummy message creation method in communication system test method)
Next, creation of a dummy message will be described with reference to FIG. FIG. 8 is a diagram for explaining a method of creating a dummy message. The left side of FIG. 8 shows the flow from dump file output to dummy message creation, and the upper right side of the figure shows the range of operational message data stored in the dump file. FIG. 4 shows sequence diagrams of a data transmission / reception state between the opposed adapter # 1 and the opposed system # 1, and a data transmission / reception state between the opposed adapter #N and the opposed system #N. In FIG. 8, the counter adapter is abbreviated as “ADP”.

  As described with reference to FIG. 7, the data generation unit 7 starts generating the test data 23 based on the dump file 19 output from the data collection unit 5, and the opposing system that is the operation test target in the verification system 9. A list of sessions at the start of capture for each of # 1 to #N is created. As shown in FIG. 8, the operation message data stored in the dump file 19 includes only information of a predetermined period from the start to the end of the message capture, and data of the period from the start of the session of the operation system 3 to the start of the message capture. Is not included. For this reason, the sequence created by the above method indicates the operation after the start of capture. When the operation test of the verification system 9 is executed without reproducing the state from the session start of the operation system 3 to the start of message capture, the test data input environment of the verification system 9 is different from the environment at the start of capture in the operation system 3. End up. For this reason, when a difference occurs between the operation data 21 and the verification data 25, the difference is caused by a failure of the verification system 9, or the environment at the start of capture in the operation system 3 is not reproduced. It is unclear whether it is caused by this, and an accurate operation test of the verification system 9 is not achieved. Therefore, in the present embodiment, as shown in FIG. 8, when the data generation unit 7 determines that there is no sequence in the period from the session start to the capture start when creating the sequence at the start of capture, Using the message creation tool, a dummy message 27 for reproducing the state of the period is automatically generated. As a result, the test method according to the present embodiment can perform the accurate operation test of the verification system 9 by eliminating the difference caused by not reproducing the environment at the start of capture in the operation system 3. It has become.

  The dummy telegram is a request (response) telegram necessary for starting a session. The data generation unit 7 generates a dummy message 27 according to the specification using a dummy message generation tool based on a definition specification regarding data distribution between the operation system 3 and the opposing system 13. As shown in FIG. 8, the data generation unit 7 adds the generated dummy message 27 to the header portion of the sequence of the action definition file 23a based on the operation message data. The communication apparatus 1 according to the present embodiment sets the verification system 9 to the initial state using the test data 23 to which the dummy message is added, thereby determining the state of the state holding function unit 3c of the operation system 3 and the verification system 9 It is possible to prevent a result mismatch due to an operation difference and a response message being erroneously transmitted due to a mismatch with the state of the state holding function unit 9c.

  In addition, in the hardware system renewal, the performance of the hardware used in the verification system 9 may be higher than that of the hardware used in the operation system 3. In this case, for example, a cross sequence may occur due to a difference in hardware specifications or the like, and there is a case where the electronic message cannot be distributed at an expected timing. Therefore, the data generation unit 7 in the present embodiment creates various sequence files so that the sequence order is not changed in order to ensure the sequence order. For example, the data generation unit 7 replaces every single sequence, and creates a sequence file by adjusting the sequence order so that another sequence can be operated when the processing of one sequence is completed. As a result, the data comparison / determination unit 11 can compare and confirm the operational data 21 and the verification data 25 in pairs.

(Initial state setting step in communication system test method)
Next, an initial state setting process of the verification system 9 that is mainly executed in the initial state setting unit 2 will be described with reference to FIGS. 2, 3 and 6 and FIG. FIG. 9 schematically shows the state of the verification system 9 set to the initial state. As shown in FIG. 9, the application function unit 9b of the verification system 9 includes a plurality of function groups to be verified (in FIG. 6, N function groups # 1 to #N) (N is a natural number of 2 or more). It has a plurality of function groups (not shown) that are not subject to verification, and exhibits a plurality of functions. The function groups # 1 to #N of the application function unit 9b exhibit the same functions as the function groups # 1 to #N provided in the application function unit 3b of the operation system 3.

  The adapter function unit 9a of the verification system 9 includes the same number of counter adapters # 1 to #N (N is a natural number of 2 or more) as many as the function groups # 1 to #N to be verified provided in the application function unit 9b. And a plurality of opposing adapters (not shown) that are not subject to verification. The opposing adapters # 1 to #N of the adapter function unit 9a are configured to perform the same functions as the opposing adapters # 1 to #N provided in the adapter function unit 3a of the operation system 3. The opposing adapter # 1 is provided corresponding to the function group # 1, and the opposing adapter #N is provided corresponding to the function group #N. The inter-system IN data file and the action definition file output from the function group # 1 are input to the pseudo counter system 15 via the counter adapter # 1, and the inter-system OUT data file that is a response message of the pseudo counter system 15 is the counter adapter. Input to the function group # 1 via # 1. Similarly, the function group #N executes data transmission / reception with the pseudo-opposite system 15 via the counter adapter #N.

The initial state setting unit 2 stores the first operational recording data 21a in the state holding function unit 9c. The first operation recording data 21a is data indicating the states of the function groups # 1 to #N and the counter adapters # 1 to #N of the operation system 3 immediately before the capture of the operation message data 21c. For this reason, when the first operation recording data 21a is stored in the state holding function unit 9c, the state holding function unit 9c is in the same state as the state holding function unit 3c immediately before the capture of the operation message data 21c.
The initial state setting unit 2 stores the operation definition file 23a including the dummy message 27 and the inter-system IN data file 23b in the data storage unit 3f of the application function unit 9b. Further, the initial state setting unit 2 interrupts the time synchronization by the NTP server and sets the time of the management server to the time immediately before the start of capture in the operation system 3.

  In this way, the initial states of the adapter function unit 9a, the application function unit 9b, and the state holding function unit 9c provided in the verification system 9 are the adapter function unit 3a and the application function unit 3b at the start of message capture in the operation system 3. And the state of the state holding function unit 3c. At this time, the time state of the management server of the verification system 9 is also set to be the same as the time at which the message capture start time in the operation system 3. As a result, the entire state of the verification system 9 is matched with the state of the operation system 3 immediately before the start of the message capture, and the environment of the operation system 3 immediately before the operation message data 21c is captured is reproduced in the verification system 9 and initialized. Is done. By initial setting of the verification system 9, the acquired operation message data 21 c and operation operation log data 21 e can be used for comparison with the verification data 25 as they are.

(Operation test and data comparison / judgment process in communication system test method)
Next, the operation test and data comparison / determination process of the verification system 9 will be described with reference to FIGS. 3, 5 and 9 and FIG. FIG. 10 is a block diagram schematically showing an operation test and a data comparison / determination step of the verification system 9. When the initial setting of the verification system 9 is completed, the function groups # 1 to #N to be verified provided in the application function group 9b of the verification system 9 are operated and the test data 23 flows. As shown in FIG. 10, the application function unit 9b executes a scenario based on the operation definition file 23a stored in the data storage unit 9f, and, for example, the operation definition file 23a and the inter-system IN data file via the adapter unit 9a. 23b is sent to the pseudo-opposite system 15. The pseudo-opposite system 15 that has received these files operates based on the operation definition file 23a, and sends output message data 25c (inter-system OUT data) as a response message to the verification system 9. The operation definition file 23a includes an instruction to store the verification recording data in the state holding function unit 9c and store the output message data 25c and the verification operation log file 25d in the log file storage unit 9d when the scenario is executed. . Therefore, as the operation test of the verification system 9 is executed, the first verification recorded data 25a recorded immediately after the initial state is set and the second verification recorded in the state holding function unit 9c immediately after the operation test is completed. The verification recorded data 25b is held. The log file storage unit 9d stores the output message data 25c at the same timing as when the operation message data 21c was captured in the operation system 3, and the verification operation log data 25d is stored after the operation test is completed. .

  When the operation test of the verification system 9 is completed, the data comparison / determination unit 11 acquires the first and second verification recorded data 25a and 25b from the state holding function unit 9c of the verification system 9 and outputs them from the log file storage unit 9d. The message data 25c and the verification operation log data 25d are acquired. Further, the data comparison / determination unit 11 acquires the first and second operation recording data 21a and 21b, the operation message data 21c, and the operation operation log data 21d from the data collection unit 5. Next, the data comparison / determination unit 11 compares the first operation recording data 21a with the first verification recording data 25a, compares the second operation recording data 21b with the second verification recording data 25b, and operates the telegram data. 21d is compared with the output message data 25c, and the operation operation log data 21e is compared with the verification operation log data 25d. In the communication system testing method according to the present embodiment, the data distributed in the operation system 3 after the verification system 9 is set to the initial state so as to be the same environment as the operation system 3 immediately before the start of the message capture. The same data as that described above flows into the verification system 9. Therefore, if the function to be tested of the verification system 9 exhibits a function equivalent to that of the operation system 3, the data compared by the data comparison / determination unit 11 are the same. Therefore, the data comparison / determination unit 11 determines that the data to be compared is normal if there is no difference between the data to be compared, and determines that the data is to be abnormal if there is a difference between the data to be compared.

  As described above, in the communication system test method according to the present embodiment, by using the tool that automatically compares the output message data that flows out in the verification system 9 and the operation message data captured from the operation system 3, The test can be performed automatically. Also, the first and second verification recorded data 25a and 25b held in the state holding function unit 9c are also stored in the state holding function unit 3c of the operation system 3 after the operation test of the verification system 9 is completed. It is possible to confirm whether or not the database recording function of the state holding function unit 9c of the verification system 9 is correctly realized by comparing with the second operation recording data 21a and 21b using a tool that automatically compares. . Further, in the communication system testing method according to the present embodiment, operation log data is used for comparison in addition to telegram data and verification recorded data before and after verification. For this reason, according to the test method for the communication system according to the present embodiment, the normality of the system operation after the system renewal can be surely confirmed. Furthermore, the operation test of the verification system 9 is executed using the data in the operation system 3 as it is. For this reason, if it is determined that there is no abnormality in the communication system test method according to the present embodiment, the existing function can operate normally even if the verification system 9 is replaced with a system after the system update.

  As described above, according to the communication system test apparatus 1 and the communication system test method according to the present embodiment, it is possible to improve the efficiency of the test in system renewal and degradation confirmation. In particular, according to the present embodiment, it is possible to carry out from the comprehensive test item creation to the normality confirmation using the operational data, that is, the test range of the comprehensive test, test data creation, test item implementation, and test result normality judgment. This makes it possible to improve the efficiency of the test.

  With regard to the test range of the comprehensive test, it is possible to comprehensively test the main route operated in the operation system. Regarding test data creation, it is possible to create a comprehensive test message by data capture in the operation system. Regarding the test item implementation, since the telegram data distributed in the operation system is distributed as it is to the verification system, the operation test can be performed without requiring manpower. Thus, according to the present embodiment, an error in creating a message that occurs when manually creating message data is prevented, and time required for manually creating a message is not required.

  As for the test result normality determination, the normality of the verification system is determined by comparing the verification data acquired by the operation test with the telegram data operated in the operation system. For this reason, it is possible to accurately determine whether the verification system 9 is normal or abnormal. Furthermore, it is possible to prevent errors in the results of manual test confirmation. Therefore, according to the test apparatus and test method of the communication system according to the present embodiment, the test item creation, test data creation, verification test execution, and evaluation of the test execution result in the comprehensive test are not dependent on the items, and the operation is constant. It becomes possible in.

In the communication system test apparatus and test method according to the present embodiment, if the telegram data (input / output information) can be acquired from the operation system 3 side, the test of the verification system 9 can be performed even in a black box situation where the inter-system sequence is not known. Can be carried out.
Further, in the present embodiment, the operation of the verification system 9 is verified with the application (AP) layer instead of the communication layer (TCP). In this embodiment, since the connection destination is determined as an application rather than system connection, it is possible to test the verification system 9 without adjusting the interface between a plurality of systems. As described above, the test apparatus and test method for the communication system according to the present embodiment do not require setting of an interface and can perform a black box-like test, so that manual adjustment is unnecessary.

  In the test apparatus and test method for the communication system according to the present embodiment, since the connection (TCP layer) and the session (AP layer) are separated, the sequence can be managed in the session layer. As a result, according to the present embodiment, it is possible to reproduce and verify each session even in a multi-session or a cross sequence, so that it is possible to perform an operation test of the verification system 9 without having to be aware of the interface. .

  Further, the test apparatus 1 according to the present embodiment has a function of acquiring the state of the operation system 3 (Export of the old system) and reflecting the state to the verification system 9 (Import to the new system). Thereby, according to this Embodiment, even if it is the message | telegram data acquired from the operation system 3 in the middle of a session, the function to match the state of the operation system 3 and the verification system 9 can be exhibited, Even if the test data is created from the operation data, the operation test of the verification system 9 can be performed without any problem.

  Conventionally, since the operation system cannot be stopped once the system operation is started, it is difficult to perform the verification system operation test in the same operation state as the operation system during the operation system session. is there. On the other hand, the test apparatus and the test method according to the present embodiment have an advantage that the operation test of the verification system can be performed in the operation state equivalent to that of the operation system even during the session of the operation system.

  In addition, the test method described in Patent Document 1 is applied when the start and end of a session between the telephone and the exchange are unclear or when the period from the start to the end of the session is relatively long. Is difficult. In addition, in this test method, when there are a plurality of systems, it is difficult to manually set an interface because it is unclear which sequence is returned to which system when a certain sequence is received. That is, in this test method, when the sequence of the entire session of the operation system is unknown, it is extremely difficult to use the data obtained by the operation system for the operation test of the verification system. On the other hand, the test apparatus and test method according to the present embodiment can use data obtained by the operation system for the operation test of the verification system even when the sequence of the entire session of the operation system is unknown. Has the advantage of being.

Next, an efficiency improvement effect when the communication system test apparatus and the communication system test method according to the present embodiment are implemented in the waterfall model will be described. In the waterfall model, work contents related to test execution are generally classified into the following (1) to (7).
(1) Create test items from the document of the corresponding design process.
(2) Create test procedure manuals for the created test items.
(3) Create data in the device and simulator data of the peripheral device for test execution.
(4) Prepare the test environment.
(5) The test is actually performed.
(6) Check the test results.
(7) Total test results.

  When the communication system test apparatus and test method according to the present embodiment are used, the work of the next process is different from the general work content. First, the operations shown in the above (2) and (3) can be performed by creating a data capture procedure manual and acquiring test execution data each time. In addition, once the data capture procedure manual is created, the subsequent corrections require only minor changes such as date and acquisition destination, and do not depend on the number of test items. On the other hand, in the case of the conventional test method, the procedure manual increases in proportion to the number of test items.

In the present embodiment, the work shown in (5) above only involves inputting data into the system, and therefore, compared with the conventional test method in which items are performed one by one, the time (and before and after) Preparation and post-work time) only. In the case of the conventional test method, the number of tests to be performed increases in proportion to the number of test items.
When this embodiment is applied to the work shown in (6) above, it is only necessary to compare the operation message data, which is the correct answer data, with the verification data, so it is confirmed in a uniform time using a data comparison tool or the like. On the other hand, in the conventional test method, the confirmation time of the test result increases in proportion to the number of test items.

  When using the test apparatus and test method according to the present embodiment, the efficiency of each work time with respect to the conventional test method is 77% in the work shown in the above (2) (the work time is the work of the conventional test method). It was reduced to 23% of the time), 75% in the work shown in (5) above (reduced to 25% of the work time in the conventional test method), and 67% in the work shown in (6) (the work time was the conventional time). The work time of the test method is reduced to 33%), and the work shown in the above (1), (4) and (7) is 0% (the work time is the same as the conventional test method). The work shown is -233% (working time is extended to 333% of the working time of the conventional test method). The efficiency improvement effect is about 33% as a whole (the working time is reduced to about 67% of the working time of the conventional test method). The efficiency improvement effect is calculated by sharing the contents and number of test items between the present embodiment and the conventional method. For this reason, according to the test apparatus and the test method according to the present embodiment, it is possible to achieve the efficiency of the comprehensive test time without increasing the test man-hours and to maintain the system quality.

  The present invention can be embodied as a computer program. For example, the function of each part of the test apparatus 1 can be realized as a test program. Thus, some or all of the present invention can be incorporated into hardware or software (including firmware, resident software, microcode, state machines, gate arrays, etc.). Furthermore, the present invention may take the form of a computer program product on a computer-usable or computer-readable storage medium, which incorporates computer-usable or computer-readable program code. It is done. In the context of this specification, a computer usable or computer readable medium includes, stores, communicates, propagates, programs that are used by or in conjunction with an instruction execution system, apparatus or device, Alternatively, any medium that can be transported can be used.

The present invention is not limited to the above embodiment, and various modifications can be made.
In the above embodiment, the data generation unit 7 generates the dummy message 27 according to the specification using the dummy message generation tool based on the definition specification regarding the data distribution between the operation system 3 and the opposing system 13. However, the present invention is not limited to this. For example, for an operational system in which the definition specifications are not clearly defined, add a message that starts a session directly by hand, or create a new tool to create a dummy message, You may make it add a dummy message | telegram to the header of the sequence produced based on it. Alternatively, for example, the test apparatus 1 determines a sequence in which a definition specification exists and a sequence in which a definition specification exists for the captured operation message data, creates a dummy message for the sequence in which the definition specification exists, and the definition specification The non-existing sequence may be configured not to be evaluated and deleted.

  The scope of the present invention is not limited to the illustrated and described exemplary embodiments, but includes all embodiments that provide the same effects as those intended by the present invention. Furthermore, the scope of the invention is not limited to the combinations of features of the invention defined by the claims, but can be defined by any desired combination of particular features among all the disclosed features.

DESCRIPTION OF SYMBOLS 1 Test apparatus 2 Initial state setting part 3 Operation system 3a, 9a Adapter function part 3b, 9b Application function part 3c, 9c State maintenance function part 3d, 9d Log file storage part 3e, 9e Application execution part 3f, 9f Data storage part 5 Data collection unit 5a Capture function unit 5b Data output function unit 7 Data generation unit 7a Screen interface function unit 7b File acquisition function unit 7c Test data creation function unit 7d Data aggregation function unit 7e Sequence creation function unit 9 Verification system 11 Data comparison / determination Unit 13 Opposite system 15 Pseudo-opposite system 17 Router network 21 Operation data 21a First operation recording data 21b Second operation recording data 21c Operation message data 21d Time information data 21e Operation log data 23 Test data 23a Operation definition file 23b System IN data file 25 verification data 25a first verification From data 25b second verification From data 25c output data telegram 25d verifying operation log data 27 dummy telegram 29 Operation terminal 31 aggregate file 33 Sequence File

Claims (9)

Capturing operational telegram data that circulates between the operational system in which the session is actually established and the opposing system for a predetermined period, start time data indicating the time when the capture of the operational telegram data is started, and the operational system A data collection unit for collecting operational recording data recorded in the operational data storage unit provided in
The operation in which the data collection unit captures test data to be distributed between a verification system having the same function as at least a part of the operation system and a pseudo counter system that realizes an operation equivalent to the counter system. A data generation unit that generates data based on message data;
Based on the test data, the start time data, and the operation recording data, the environment similar to the operation system before the data collection unit captures the operation message data is reproduced in the verification system. An initial state setting unit for setting the verification system to an initial state;
Verification data acquired when the test data generated by the data generation unit is input to the verification system set to the initial state by the initial state setting unit, the operation message data, and the operation recording A test apparatus for a communication system, comprising: a data comparison / determination unit that compares data to determine normality / abnormality of the function of the verification system. The test apparatus for a communication system according to claim 1, wherein the test data includes a dummy telegram for artificially generating the start of the session. The operational recording data is
Including first operation recording data recorded in the operation data storage unit before the start of capture, and second operation recording data stored in the operation data storage unit after the capture ends,
The verification data is
First, which is recorded in a verification data storage unit provided in the verification system after the initial state is set and before the operation test of the function of the verification system is started and compared with the first operational recording data. Verification recording data,
3. The communication system test according to claim 1, further comprising: second verification recording data recorded in the verification data storage unit and compared with the second operation recording data after the operation test is completed. apparatus. The verification data includes output message data that is output when the test data is distributed between the verification system and the pseudo-opposite system and is compared with the operation message data. 4. The test apparatus for a communication system according to any one of items 3 to 3. The data comparison / determination unit is an operation operation log data indicating an operation history in the operation system collected by the data collection unit, and output after completion of an operation test of the function of the verification system. 5. The communication system test apparatus according to claim 1, wherein the operation log is compared with verification operation log data indicating an operation history of the communication system according to claim 1. The said data collection part has a capture function part which captures the said operation message data, and a data output function part which outputs the captured said operation message data to a predetermined file, Any one of Claim 1-5 characterized by the above-mentioned. A communication system test apparatus according to claim 1. The data generator is
A file acquisition function unit for acquiring the predetermined file;
The test apparatus for a communication system according to claim 6, further comprising: a test data creation function unit that creates the test data from the operation message data stored in the acquired predetermined file. Computer
Capturing operational telegram data that circulates between the operational system in which the session is actually established and the opposing system for a predetermined period, start time data indicating the time when the capture of the operational telegram data is started, and the operational system A data collection unit for collecting operational recording data recorded in the operational data storage unit provided in
The operation in which the data collection unit captures test data to be distributed between a verification system having the same function as at least a part of the operation system and a pseudo counter system that realizes an operation equivalent to the counter system. A data generator that generates data based on the telegram data;
Based on the test data, the start time data, and the operation recording data, the environment similar to the operation system before the data collection unit captures the operation message data is reproduced in the verification system. An initial state setting unit for setting the verification system to an initial state; and
Verification data acquired when the test data generated by the data generation unit is input to the verification system set to the initial state by the initial state setting unit, the operation message data, and the operation recording A test program for a communication system, characterized in that it functions as a data comparison / determination unit that compares data and determines whether the function of the verification system is normal or abnormal. Capturing operational telegram data that circulates between the operational system in which the session is actually established and the opposing system for a predetermined period, start time data indicating the time when the capture of the operational telegram data is started, and the operational system Collecting the operation recording data recorded in the operation data storage section provided in
The operation in which the data collection unit captures test data to be distributed between a verification system having the same function as at least a part of the operation system and a pseudo counter system that realizes an operation equivalent to the counter system. Based on the telegram data,
Based on the test data, the start time data, and the operation recording data, the same environment as the operation system before the data collection unit captures the operation message data is reproduced in the verification system. Set the verification system to the initial state,
For the verification system set in the initial state, the verification data acquired when the test data generated by the data generation unit is input, and the operation message data and the operation recording data are compared. A test method for a communication system, comprising: determining whether the function of the verification system is normal or abnormal.