JP2004021523A - Load test system to network server - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a load test system in which a plurality of specified virtual clients is generated in one tested terminal, one test manager terminal managing a plurality of the tested terminals is provided, and various load tests are performed to a plurality of virtual clients generated by the test manager terminals. <P>SOLUTION: The tested terminal is provided with a means for distributing load test operation information set/instructed by a set means, a test client module, and test data to each tested terminal; and a load test operation execution means for operating a client module distributed according to the distributed load test operation information and generating a load access to a tested server instructed by using the distributed test data. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention simulates various access patterns on various network server systems existing on a network to generate a load, thereby performing a server load test. One co-test manager terminal that is generated in the test execution terminal and manages the plurality of test execution terminals is prepared, and various load test operations are instructed to the plurality of virtual clients generated from the test manager terminal, The present invention relates to a load test system capable of recording a performance result of a target network server and performing a performance test.
[0002]
[Prior art]
In recent years, with the development of network technology, applications realized as server / client systems on a network have been increasing. Testing such applications is very difficult, unlike traditional stand-alone applications, because measuring their performance and detecting bottlenecks requires testing the entire system, including the network. Has become.
[0003]
For the purpose described above, in the conventional load test system for a network server, a plurality of client terminals are actually prepared, individual network addresses are set for each of them, and a client application is executed. Access to the server to generate a load.
[0004]
Also, in some load test tools, a plurality of client processes are generated on one terminal to generate access to the server. However, each client process uses the same network address for the server. Since access was made, it was not possible to faithfully simulate access from a plurality of client terminals having actual different network addresses.
[0005]
[Problems to be solved by the invention]
In a conventional load test system for a network server, a plurality of client terminals are actually prepared, individual network addresses are set for each of them, and a client application is executed to actually access the server and perform load. However, such a system requires many costs, such as the number of terminals to be used, a space for installing the terminals, and a person who operates the terminals.
[0006]
In addition, it is very difficult to maintain the reproducibility of the test when the operation is performed manually.
[0007]
In order to solve these problems, the present invention creates a plurality of designated virtual clients in one test execution terminal, prepares one co-test manager terminal that manages the plurality of test execution terminals, and provides the test manager An object is to provide a system that can perform various load test operations on a plurality of virtual clients generated from a terminal.
[0008]
Conversely, if a plurality of client processes are performed on one test execution terminal, a load is also applied to the test execution terminal. Naturally, an access from one terminal can reproduce an environment closer to a more accurate use environment than a plurality of accesses to the load test terminal from within one test execution terminal.
[0009]
In order to solve such a problem, the present invention performs a load test on a reference server in place of the load test target server on the load test terminal as a pre-process for performing a load test, thereby executing the load test system itself. A system capable of acquiring load data and subtracting the acquired data from the load test result data performed on the actual load test target server to obtain the load test result information of only the load test target server. The purpose is to provide.
[0010]
[Means for Solving the Problems]
In order to achieve the above object, the present invention comprises one test manager terminal for giving a load test execution instruction, a plurality of test execution terminals for actually executing a load test, and a load test terminal to which a load can be applied. Then, on the test execution terminal, virtual clients for the number of load test servers specified by the test manager terminal are generated, each virtual client has a unique IP address, and various load test operations are performed by the test manager terminal. In the load test system for the load test target server which generates a load access to the load test target server on the load test terminal by instructing from the load test operation information, the load test operation information set / instructed by the setting means, the test client Means for distributing module and test data to each test execution terminal A load test operation that can operate the client module that is also distributed according to the distributed load test operation information and generate a load access to the specified test target server using the distributed test data. The means is provided on the test execution terminal.
[0011]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, an embodiment of a load test system for a network server according to the present invention will be described with reference to the accompanying drawings. However, the following description also serves as an explanation of an embodiment of a test medium terminal and a test execution terminal according to the present invention and a recording medium on which an application for realizing those functions is recorded.
[0012]
FIG. 1 is a system block diagram of a test manager terminal and a test execution terminal according to the present invention. Reference numeral 1 denotes a system bus, and each component block to be described below is connected to the system bus.
[0013]
Reference numeral 2 denotes a CPU (Central Processing Unit). Reference numeral 3 denotes a program memory (referred to as PMEM), which selects / reads a program for this processing from the external storage device 18 as appropriate, and executes the program by the second CPU.
[0014]
The data input from the input device 10 is stored as code information in PMEM which is also a text memory.
Reference numeral 4 denotes a network control unit which controls input / output data in a network port 5.
[0015]
A signal output from the network port 5 is transmitted to a network port of another device on the network via the network. Data exchange with other terminals on the network is performed via the network control unit 4.
[0016]
Reference numeral 6 controls the connected modem 7 and the like when performing communication using the public network. That is, the load test system can also be applied to a load test in a network connection using a public network other than the LAN.
[0017]
An input control unit 8 is connected to an input / output device 9. The operator issues an operation command of the system by operating the input / output device 9.
[0018]
Reference numeral 10 denotes a display control unit, and reference numeral 11 denotes a display device connected to the display control unit. The result of the input instruction by the input device 9, the test setting information, and the information of the test execution result are displayed here.
[0019]
An output device control unit 12 controls output of data to the connected printer 13.
An external storage device control unit 14 controls the external storage device 15 connected thereto. A program for controlling the functions of the present apparatus, image data, and the like are stored in the external storage device 15.
[0020]
FIG. 2 shows a relationship diagram on a network of terminals forming the load test system of the present embodiment.
[0021]
A load test system is composed of a plurality of terminals connected to the same network.
[0022]
21 indicates a network. Reference numeral 22 denotes a test manager terminal that performs load test operation scenario creation, test instruction, and test control processing. The test operator performs all operations impressed by the test via this terminal.
[0023]
Reference numeral 23 denotes a test execution terminal for performing a load access by a load test. A plurality of clients are generated in each terminal, and one client has an individual IP address in 21 networks and performs load access to the test target server. Reference numeral 24 denotes a load test terminal that is accessed by the test execution terminal. There is a load test target server in this terminal.
[0024]
FIG. 3 shows what kind of data is exchanged between the test manager terminal and each test execution terminal which constitute the load test system of the present embodiment, and how the test target server device on the network is connected. The following describes the processing procedure for generating a load access and acquiring the performance result.
[0025]
{Circle around (1)} The test manager terminal 31 on the network selects a test execution terminal 32 for executing a test from the list of test execution terminals displayed on the screen of the terminal, and performs the test operation and the operation on the terminal. The client module to be specified and the test data used by the client module are specified.
[0026]
(2): Using the distribution means of the test manager terminal 31, the data set in (1) is used to transmit only necessary data to each test execution terminal.
[0027]
The distribution method 36 uses the http protocol, which is TCP / IP, in order to reliably distribute even large data.
[0028]
{Circle around (3)} According to the test preparation instruction instruction 37 from the test manager terminal 31, each test execution terminal performs initialization processing of the distributed client module.
[0029]
{Circle around (4)} Each test execution terminal / load test terminal obtains an internal set time via the network by the test manager terminal 38. Each terminal sets the acquired set time in its own terminal.
[0030]
{Circle around (5)} The test manager terminal starts a load test according to each test start instruction command.
[0031]
{Circle around (6)} Each test terminal activates the client module according to the specified test operation information, and generates a load access to the test target server 34 using the distributed test data. At this time, the operation state of the client module is notified to the test manager terminal as status information.
[0032]
The test operation instruction 37 from the test manager terminal 31 to the test execution terminal 32 uses an UDP event (broadcast) in consideration of test accuracy and communication speed.
[0033]
During the test, the test manager terminal saves the test support status, each test terminal saves the load access processing status, and the loaded test terminal saves the resource usage status within the own terminal when the test execution terminal accesses the test result file, respectively. .
[0034]
{Circle around (7)} When the load access processing of the above (6) is completed, each test execution terminal uploads the test result information 35 to the test manager. At this time, the delivery method 35 uses the http protocol which is TCP / IP in order to reliably deliver even large data. The test manager totalizes the received test result information and notifies the user.
[0035]
FIG. 4 shows an example of a module configuration diagram of the load test system according to the present invention.
[0036]
The figure shows an example in which a load test is performed on one load test network server using a load test system including one test manager terminal and “n” test execution terminals. Has been explained. Here, for the sake of simplicity, an example in which a load is applied to one network server is shown. However, the present invention is not limited to this limitation, and a load test is performed on a plurality of load test terminals. It is possible to perform
[0037]
The test execution terminal management module 41 checks the execution status of the test in each test execution terminal. This module acquires status change event information from the test execution terminal by using the event processing module. Alternatively, an instruction to start / stop the execution of a test can be issued by issuing an event to a desired test execution terminal.
[0038]
The test operation information setting module 42 sets a test operation actually performed by each test execution terminal. At the same time, the client module for test executed to generate a load on each load test execution terminal and the test data used by the client module are specified using this module. With the data set here, appropriate data is distributed to each test execution terminal using the data distribution module 44.
[0039]
The event processing module 43 is a module for issuing an event on the network, acquiring event information on the network, and analyzing the contents. The test manager terminal is used to instruct processing to each test execution terminal, and the test execution terminal is used to notify a change in its own test execution status.
[0040]
The data distribution module is a module for distributing designated data to a desired terminal on the network. The test manager terminal is used to distribute test-related data to each test execution terminal, and the test execution terminal is used to transmit test result information to the test manager terminal.
The functions of the test manager terminal are provided by the modules 41 to 44 described above.
[0041]
The load generation processing management module 45 activates the client module according to the test operation information received by the data distribution module 44, and generates a load access to the test target server using the distributed test data.
[0042]
Here, the load generation processing is realized by starting up the client module distributed from the test manager under the designated conditions. A plurality of client modules can be started at the same time ("N" in the case of the example in the figure), and a plurality of different client modules can be started simultaneously on one test execution terminal. Is also possible.
[0043]
The operation status of each client module is notified to the test manager terminal using the event place processing module 43.
[0044]
The load generation processes 1 to N of 46 are threads or processes generated / managed by the load generation process management module, and generate a load access to the target server according to the test operation information specified by the test manager.
[0045]
The network access control module performs a network control process in a load test such as mapping a plurality of network addresses in a pseudo manner with respect to a plurality of load generation processes.
[0046]
Numeral 48 acquires the status / used resources of the load test terminal or the status / used resources of the load test target server 49 and notifies the test manager terminal.
[0047]
FIG. 5 shows a module configuration diagram when the load target server is replaced with a reference server provided by the load test system based on the module configuration diagram of the load test system according to the present invention described in FIG. Since 51 to 58 are the same as the modules of 41 to 48, the description is omitted.
[0048]
Reference numeral 60 denotes a reference server provided by the load test system. The server records a response process from the load test system and a result such as a time at that time as a log. The difference between the recording result and the test result of the 59 load test target servers to be performed thereafter can be calculated by the resources actually consumed by the load test target server excluding the resources used by the load test system.
[0049]
FIG. 6 shows an example of the data structure of the test operation information in the present invention.
[0050]
In the data identification information 601, an identification ID indicating that the data is test operation information data of the present load test system, version management information of information, and the like are set.
[0051]
In the test manager terminal information 602, identification information on the network such as the network address of the test manager terminal that created the setting information is set.
[0052]
In the number 603 of test execution terminals, the number of test execution terminals managed by this setting information is set. Subsequent test terminal setting information exists as many as this number.
[0053]
As described above, 601 to 603 are management information of the entire test operation information.
[0054]
In the test execution terminal information 604, identification information on the network such as a network address of a terminal that executes a test according to the following setting information is set.
[0055]
In the number 605 of load processes to be generated, the number of client processes (threads or processes) for generating a load generated in the test execution terminal is set. For this few minutes, there is subsequent “load processing setting” information.
[0056]
In the process simultaneous execution information 606, when a plurality of load processes are generated by the execution terminal, the number of processes to be simultaneously performed and the generation intervals are set.
[0057]
In the timeout information 607, a threshold for determining a timeout in each process in the terminal is set.
[0058]
As described above, 604 to 607 are management information regarding all processes in the test execution terminal.
[0059]
The client module name 608 is the name of a client module used to generate the load processing. This module is distributed from the test manager terminal to each test execution terminal (or, in the case of a module that originally exists in the test execution terminal, performs a duplication check and does not perform useless distribution processing).
[0060]
The test data file name 609 is data used when the client module 608 accesses the target server in the load processing.
In the client module operation setting information 610, a list of functions executed by the client module 608 for a load test and information for synchronizing processing timing are set.
[0061]
The former is, for example, a module in which a client module performs a series of printing processes,
1. Function to initialize the printer
2. Ability to enumerate printers
3. Function to register a job in the printer
4. Function to enumerate the status of the indicated printer
In this case, information for designating the function to be performed in the load processing, for example, 1, 2, or 3 is set.
[0062]
In the latter case, for example, when a plurality of load processes are performed on one test execution terminal, for example, when the processes 1, 2, and 3 are performed as described above, one process varies among the load processes. This is setting information for synchronizing the start point of the second process even if the process ends.
[0063]
The virtual network address information 611 indicates whether, when a single test execution terminal performs a plurality of load processes, the load-tested terminal is accessed from an individual network terminal for each load process. For realizing a function for pretending to be fake.
[0064]
By providing this information to the above-described network access control module, the above function can be realized.
[0065]
In the information 612 of the load test terminal, identification information on the network such as a network address of a test target server accessed by the load processing is set.
[0066]
In the other control information 613, detailed setting information necessary for performing this load processing is set.
[0067]
As described above, 608 to 613 are setting information for each load process.
[0068]
FIG. 7 is a flowchart showing a procedure of an operation instruction process from the test manager terminal to the test execution terminal during the load test execution process in the test execution terminal of the present invention.
[0069]
As a premise, in order to simplify the description of this flowchart, it is assumed that the settings regarding the test setting information, the test client module, and the test data have been completed by the test manager prior to the processing of this flowchart.
[0070]
First, in step S701, a test start instruction event is issued to the test execution terminal. In step S702, it is checked whether the transmission of the test operation information file was successful. If each test execution terminal has successfully transmitted the test operation information file, the process proceeds to step S703; otherwise, the process proceeds to step S712 to stop the test.
[0071]
In step S703, using the data necessary for the test distributed to the test execution terminal, the load processing is started according to the instructed setting, and the test preparation processing is executed.
[0072]
In step S704, it is checked whether each test execution terminal has succeeded in test preparation. If not successful, an error notification of step S705 is received from each test execution terminal.
[0073]
The test execution terminal checks whether or not the data (test setting information, test client module, and test data) necessary for executing the test exists. If it is determined that the data exists, the process proceeds to step S707, and the process proceeds to step S707. If it is determined not to do so, the process moves to step S707.
[0074]
In step S705, the test execution terminal notifies the test manager of error information indicating that the specified test cannot be executed because the data exists.
[0075]
In step S706, the process waits until a test preparation completion event is issued from the test execution terminal.
[0076]
In step S707, the test manager issues a test start instruction event to the test execution terminal.
[0077]
If it is determined in step S708 that the event is a test start instruction event from the test manager, the process proceeds to step S709. If not, the event waiting process is performed again in step S708.
[0078]
In step S709, the test execution terminal issues an execution instruction to all the virtual clients, and performs load generation processing on the target server.
[0079]
In step S710, the test manager monitors the test end of all test execution terminals. Upon receiving the end notifications of all the test execution terminals, the process proceeds to step S711. In step S711, all the load test processing notifications are received from the test execution terminal, and the test processing ends.
[0080]
FIG. 8 is a flowchart illustrating an example of a procedure of an operation instruction processing of the reference server at the time of the load test using the reference server.
[0081]
However, here, the test operation settings have already been made in the test manager terminal, and in each test execution terminal, the test operation settings, test data, etc. have already been transmitted to each test execution terminal. It is assumed that the preparation for performing the load test based on the set test operation is completed.
[0082]
In S801, a test start instruction is issued from the test manager terminal to each test execution terminal.
[0083]
In S802, each client on each test execution terminal simultaneously performs load access to the reference server.
[0084]
In S803, the reference server on the loaded test terminal that has received the access records the access time / access partner information and records the used resource information.
[0085]
In S804, a reply is sent to the client on the test execution terminal.
[0086]
In step S805, upon receiving an end event from the test manager terminal, the flow shifts to step S806 to end the test. If there is no end event, the process returns to S802. In S806, the test result information is transmitted to the test manager terminal.
[0087]
FIG. 9 shows a table of items of result information recorded by the reference server at the time of a load test using the reference server.
[0088]
Reference numerals 902 and 903 denote client name / IP address information on the test execution terminal accessing the server.
[0089]
Reference numeral 901 denotes a time required for the client on the test execution terminal to access the reference server from the load test system, and reference numeral 906 denotes a time required for the client on the test execution terminal to access the load test target server from the load test system. It is the time it took. The difference between the time 906 and the time 901 is the time (909) required for the load test target server to respond to the client access.
[0090]
Reference numeral 904 denotes a CPU usage rate used by the entire load test system (including the reference server) when a client on the test execution terminal accesses the reference server from the load test system. Reference numeral 907 denotes a CPU usage rate used by the load test target server and the load test system when a client on the test execution terminal accesses the load test target server from the load test system.
[0091]
The difference between the CPU usage rate 907 and the CPU usage rate 904 is the CPU usage rate (910) used by the load test target server for client access.
[0092]
Reference numeral 905 denotes a memory usage used by the entire load test system (including the reference server) when a client on the test execution terminal accesses the reference server from the load test system. Reference numeral 908 denotes the memory usage used by the load test target server and the load test system when the client on the test execution terminal accesses the load test target server from the load test system.
[0093]
The difference between the memory usage at 908 and the memory usage at 905 is the memory usage (911) used by the load test target server for client access.
[0094]
[Other application examples of the present invention]
An application example of the present invention will be described below.
[0095]
<Application Example 1>
In the present invention, data (test setting information, test client module, and test data) necessary for each test execution terminal to perform a load test is distributed from the test manager to each test execution terminal. Was.
[0096]
However, in order to reduce the load on the test manager, a process in which each test terminal acquires data necessary for a test may be performed in the following process.
[0097]
Step 1: The test manager has completed preparation of data necessary for the test.
[0098]
Step 2: Notify each test execution terminal of an event instructing acquisition of data necessary for the test.
[0099]
Step 3: Each test execution terminal requests the test manager for test data required by itself, and downloads the data.
[0100]
Step 4: When the test execution terminal finishes downloading necessary data, the test execution terminal prepares for execution of the test, and notifies the test manager of the result by issuing an event.
[0101]
Step 5: The test manager issues a test execution instruction event to all test execution terminals when it is confirmed that a test execution preparation completion event has been received from all test execution terminals.
[0102]
【The invention's effect】
As is clear from the above description, according to the present invention, various client access patterns are realized for server systems on various networks, and efficient inspection of the entire system including the network is realized. Becomes possible. Further, it is possible to provide a server performance information acquisition function in a state closer to the actual environment by measuring the load amount of the load test system applied at the time of the load test and comparing the measured values.
[Brief description of the drawings]
FIG. 1 is a system block diagram of a test manager terminal and a test execution terminal according to the present invention.
FIG. 2 is a diagram showing an example of a network configuration in a load test system according to the present invention.
FIG. 3 is a diagram showing a data exchange procedure in the load test system according to the present invention.
FIG. 4 is a module configuration diagram of a load test system according to the present invention.
FIG. 5 is a module configuration diagram of a load test system when a reference server according to the present invention is used.
FIG. 6 is a diagram illustrating an example of a data structure of test operation information according to the present invention.
FIG. 7 is a flowchart illustrating a load test execution processing procedure in the test execution terminal of the present invention.
FIG. 8 is a flowchart showing a load test execution processing procedure in the test execution terminal of the present invention.
FIG. 9 is a diagram showing test result information items obtained during a test of the reference server of the present invention.
[Explanation of symbols]
1 System bus
2 CPU
3 PMEM
4 Network control unit
5 Network port
6 Communication control unit
7 Modem
8 Input control unit
9 Input device
10 Display control unit
11 Display device
12 Output device controller
13 Printer
14 External storage device controller
15 External storage device

Claims (4)

The test execution terminal includes one test manager terminal for instructing execution of a load test, a plurality of test execution terminals for actually executing a load test, and a load test terminal to which a load can be applied. Virtual clients for the number of load test servers specified by the manager terminal are generated, each of the virtual clients has a unique IP address, and various load test operations are instructed from the test manager terminal on the load test terminal. A load test system for a load test target server that generates a load access to the load test target server,
Means for distributing the load test operation information, the test client module and the test data set / instructed by the setting means to each test execution terminal, and operating the client modules which are also distributed according to the distributed load test operation information Load test on a network server, comprising load test operation executing means capable of generating a load access to a test target server instructed using distributed test data on a test execution terminal. system. 2. The test manager terminal according to claim 1, wherein when the test client module has a plurality of test execution functions, test function selecting means for instructing which function to execute as a load test is provided on the test manager terminal. Load test system to the network server described. In the load test operation information, information of the test client module and the test data used by each test execution terminal can be individually described, and different client modules and different test data are used on each test execution terminal. 3. The load test system for a network server according to claim 1, wherein a load test can be performed simultaneously. A load test is performed on a reference server as a substitute for the load test target server on the load test terminal to obtain load data of the load test system itself, and the obtained data is transmitted to the actual load test target server. 4. A load test for a network server according to claim 1, 2, or 3, further comprising means for obtaining load test result information of only the load test target server by taking a difference from the performed load test result data information. system.

Images