JP2923874B2 - High load emulation performance evaluation method and device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and an apparatus for evaluating the performance of high-load emulation, and more particularly to a method for evaluating the performance and designing the performance of a server computer system used in a back-office information system in a company or public organization. The present invention relates to a high load emulation performance evaluation method and an apparatus for use.

[0002]

2. Description of the Related Art Conventionally, a computer system performance evaluation method of this type includes a simulation performance evaluation method based on measurement of individual performance of each element constituting a server computer system and modeling using a queuing network, and an element constituting a server computer system. There are three methods: a performance table based on individual performance measurements for each device and a stacking performance evaluation method using the same, and an emulation performance evaluation method that configures a pseudo environment and evaluates the results based on actual measurement results.

As examples of the simulation performance evaluation method which is the first method, "Computer performance evaluation device and control device used for this device" described in Japanese Patent Application Laid-Open No. 4-112338 and 49th Information Processing Society of Japan Horikawa et al.'S "Performance Evaluation Tool for Open Systems" published in National Conference Papers, pp. 6-185 to 6-186 (late 1994).

Next, an outline of the simulation performance evaluation method will be described with reference to FIG.

FIG. 7 is a block diagram of a simulation performance evaluation method showing a first example of a computer performance evaluation method, which is the same as that described in the above publication.

Referring to FIG. 7, the simulation performance evaluation method includes a simulator section 71 and a performance evaluation section 7.
2 and a control device 73. The control device 73 includes a simulator interface 74 and a performance evaluation interface 7.
5 and a user interface 76.

[0007] The simulator unit 71 is a CPU (register, TL) of a virtual computer system to be evaluated for performance.
B, execution unit, paging unit, etc.) and cache memory, etc., and their functions are substituted.

[0008] When given the probability information on the use of each component of the virtual computer system, the performance evaluation unit 72
Get the performance of the number of instructions executed per unit time.

When the user gives the hardware specifications of the virtual computer system to be evaluated for performance through the control device 73 and issues a program execution instruction, the simulator unit 7
A control device 7 simulates program execution and stores data relating to the use of each component of the virtual computer system.
Return to 3.

When the user creates probability information from the data from the control device 73 and passes it to the performance evaluation section 72, the performance evaluation section 7
2 performs performance evaluation using a matrix method or the like.

The simulator interface 74 and the performance evaluation interface 75 are respectively
1 and the user interface 76, and between the performance evaluation unit 73 and the user interface 76, the processing proceeds by mediating the exchange of the information.

The user interface 76 analyzes a command input from the user and controls the simulator unit 71 through the simulator interface 74.
The performance evaluation unit 72 is controlled through the performance evaluation interface 75, and the simulation result and the performance evaluation result are presented to the user.

As described above, according to the first conventional example, the configuration of the virtual computer system and the simulator unit for realizing the internal state of the virtual computer system whose performance is to be evaluated at the time of execution of the instruction are represented by a queue model. Performs a computer system performance evaluation by including a performance evaluation unit that performs performance evaluation from information on use of each component of the virtual computer system, and a control device that controls the simulator unit and the performance evaluation unit as one device At this time, a series of processes for simulating a program on a computer system to be evaluated and performing performance evaluation using probability information or the like based on the simulation can be performed in a consistent operation environment.

Further, the control device can evaluate the performance of the computer system by providing a simulator interface, a performance evaluation interface, and a user interface.

Next, as an example of the stacking calculation performance evaluation method as the second method, a "computer performance evaluation system" described in JP-A-61-294559 and a research report of Information Processing Society of Japan (96. 3.19) Information system 58-
8, pp55-60, a paper by Usufuchi et al., "Study on Performance Prediction Method of Distributed System".

An outline of the stacking calculation performance evaluation method will be described with reference to FIG.

FIG. 8 is a block diagram of a stacked computer performance evaluation system showing a second example of the computer performance evaluation system, which is the same as that described in the above publication.

Referring to FIG. 8, the computer performance evaluation system includes a CPU time measuring device 81 for measuring a CPU time used by a job in an operating computer system 80.
A storage device 82 for holding the measurement result, an input / output frequency measurement device 83 for measuring the number of input / output operations for each file used by the job in the operation computer system 80, and a storage device 84 for storing the measurement result. A storage device 85 for holding performance values for each CPU model, an instruction device 86 for specifying the CPU model, a processing device 87 for calculating the CPU time of a job in the specified CPU model, and an access time for each disk device model Storage device 88 to hold
And an instruction device 89 for designating the model of the disk device and a file stored in each disk device, and a processing device 810 for calculating the number of times of input / output and the input / output time for each specified disk device.

The CPU time used by the job from the operation computer system 80 is measured by the CPU time measuring device 81, and the measured value is stored in the storage device 82. The number of physical I / Os for each file used by the job is measured by the I / O count measuring device 83, and the measured value is stored in the storage device 84.

Next, the operating CPU model and the CPU model whose load amount is to be predicted are designated to the instruction device 86. The processing device 87 reads the specified two types of CPU performance values from the storage device 85 and calculates the performance ratio between the two types.

Next, the CPU time measured is read from the storage device 82, and the performance ratio is calculated based on the measured value to obtain the CPU time of the model to be predicted.

For example, if the operating CPU model is model A,
Assuming that the upper model whose CPU time is to be predicted is model B and its performance ratio is 1: N, the measured CP of model A is
Multiply U time by 1 / N.

Next, a description will be given of the estimation of the load when changing the file stored in the disk device or when changing the disk model.

The file stored in each disk device and the disk model are designated to the instruction device 89. Processing unit 8
Reference numeral 10 reads the number of times of input / output for each file measured from the storage device 84, and calculates the number of times of input / output for each disk device from this and the storage information of the designated file.

Next, the average access time of the designated disk model is read from the storage device 88, and the access time is multiplied by the number of times of input / output to obtain the input / output time.

For example, when it is specified that three files X, Y, and Z are stored in a certain disk device, the number of times of input / output of each file read from the storage device 84 is x times and y times, respectively. , Z times, the number of times of input / output to / from the disk device is calculated as (x + y + z) times. If the average access time of this disk model read from the storage device 88 is T, the input / output time is (T
× (x + y + z)). In this conventional example, the performance of the computer is evaluated in this way.

Next, as an example of the emulation performance evaluation method which is the third method, Japanese Patent Application Laid-Open No. Hei.
Japanese Patent Application Laid-Open Publication No. H10-260, entitled "Online System Performance Evaluation Control Method Using Terminal Simulation".

Next, an outline of the emulation performance evaluation method will be described with reference to FIG.

FIG. 9 is a block diagram of an emulation performance evaluation method showing a third example of the computer performance evaluation method, which is equivalent to that described in the above-mentioned publication.

Referring to FIG. 9, this on-line system performance evaluation method uses a client computer system 90.
A server computer system 100, an input transaction file unit 93, a performance evaluation data storage unit 94,
It is composed of a console 96 and a performance evaluation list 97.

The client computer system 90 includes a terminal simulation control unit 91, a performance evaluation control unit 92, and a performance evaluation data output unit 93. The server computer system 100 includes a communication control unit 98, a transaction processing unit 9
9 is provided.

Then, the terminal simulation control section 91 which is a client program reads the input transaction from the input transaction file section 93 and passes the transaction to the performance evaluation control section 92 for transmission to the server computer system 100. Next, the terminal simulation control unit 91 receives the response data from the server computer system 100 and writes it into the performance evaluation data storage unit 94.

The performance evaluation control unit 92 obtains the transmission time t1 of the transaction for the server computer system from the terminal simulation control unit 91 which is a client program, and executes the transaction on the server computer system 100.
Send to Further, it receives response data from the server computer system 100 and obtains a reception time t6.

The input transaction file unit 93 stores a transaction to be transmitted to the server computer system 100. The performance evaluation data storage unit 94 stores response time data from the server computer system 100.

Further, the performance evaluation data output unit 95 receives the data in the performance evaluation data storage unit 94, and outputs a response time, an average response time, a minimum response time,
Outputs performance evaluation analysis data such as maximum response time.

The console 96 is used to input information such as the position of the input transaction file unit 3, the transmission transaction interval, and the start / end of emulation to the terminal simulation control unit 91 which is a client program.

Next, the operation of the online system performance evaluation method will be described.

From the console 96, the position of the performance evaluation data storage unit 94 and the input transaction file unit 93
Enter the position of. Further, the number of terminals having the client program to be emulated and the transaction interval for transmission to the server computer system are input.

The terminal simulation control section 91, which is a client program, analyzes the above data, issues a connection request to the server computer system 100, reads a transaction from the input transaction file section 93 at each transmission interval, and reads the transaction. Send to

The performance evaluation control unit 92 obtains the transmission time t1 before transmitting a transaction to the server computer system 100, sets the transmission time t1 in a management table for each terminal which is a client program, and
Send the transaction to 00.

The transaction is processed by the communication control unit 98 and the transaction processing unit 69 in the server computer system 100, and response data is returned to the communication processing device.

The performance evaluation controller 92 receives the response data from the server computer system 100 and
Is obtained, and the response time is obtained by subtracting the transmission time t1 from the management table for each terminal.

The terminal simulation control section 91 as a client program stores performance evaluation data in the performance evaluation data storage section 94. Then, an emulation end command is input from the console 96 to end the emulation.

The performance evaluation data output section 95 receives data from the performance evaluation data storage section 94 and outputs performance evaluation analysis data. The performance evaluation analysis data includes data such as response time, average response time, minimum response time, and maximum response time for each transaction.

By incorporating a performance evaluation function in the server computer system 100, t1, t2 to t in FIG.
It is also possible to obtain performance analysis data at each time point of 5, t6. According to the above procedure, the performance of the computer system is measured in the method of this example.

[0046]

These three conventional methods have the following problems.

First, in the simulation performance evaluation method of the first method, the simulator unit 7 corresponding to a dedicated tool is used.
1 carefully measures the performance data of the basic components alone when there is no competition for acquiring resources such as disks and memories caused by the activation of a plurality of processes in the computer system, and based on the performance data, Various capability values such as response time are evaluated on a computer using a queuing network model built in the evaluation unit 72. Here, components such as a disk and a memory correspond to the components constituting the queuing network model.

In this simulation performance evaluation method,
When the estimated and calculated response time cannot satisfy the desired value in the required specification, it is in principle easy to evaluate how to change the system configuration to ensure this.

However, for that purpose, the simulator unit 7
The measurement tool which is 1 requires a high-performance tool. In particular, a measurement tool depending on hardware is required to grasp the event information and the like of the CPU, and there is a problem that the method cannot be used sufficiently in evaluating the performance of a server in an open information system.

Also, in the case of server performance evaluation in an open information system, the workload for the evaluation is large, and the queuing network model formed in the performance evaluation unit 72 has a huge amount of description. In addition, there is a problem that it takes time and takes much time for evaluation.

Next, in the stacking calculation performance evaluation method of the second method, similarly to the above-described simulation performance evaluation method, the performance evaluation is performed by measuring the performance data of the basic element alone and evaluating based on the measurement.

Therefore, the various problems related to the measurement of the performance data of the simulation performance evaluation method also apply to the stacked calculation performance evaluation method. In this case, the CPU time measuring device 81 for measuring the CPU time used by the process in the computer system and the input / output count measuring device 83 for measuring the physical input / output count for each file used by each process can perform highly accurate measurement. There is a problem that it is difficult to obtain the information in an open information system by evaluating the capability of the server.

Next, the emulation performance evaluation method of the third method is a method of actually giving an equivalent transaction to a computer system to be measured and measuring its response time, and a measurement tool is also commercially available.

However, according to the document: "Nikkei Computer (1991.6.3), pp. 101-121: Kazufumi Sugiura", a conventional measurement tool using this method is:
The client computer system for setting up a transaction is composed of a plurality of screen operations, and has a drawback that it is impossible to accurately emulate the appearance rate in consideration of a cache hit for a completed transaction.

The number of client computer systems is
It must be prepared for the actual operation (actual operation), and the terminal simulation control unit 91, which is a client program, cannot follow the response of the server, or the response time measured by the terminal simulation control unit 91 It may not represent the actual situation. Therefore, there is a problem in that the evaluation accuracy is lowered, so that it may not be used for a large-scale open information system.

In other words, the problems of the conventional computer performance evaluation method are that the tool capability of performance data measurement must be able to cope with detail or on a large scale and that sufficient accuracy can be ensured. And it is difficult to design an evaluation model with a small amount of description ".

Therefore, the object of the present invention is as follows:
By providing a means to realize the point, that is, "to be able to describe the evaluation model with concrete and easily measurable data" and "to adopt a method that does not depend on special equipment for performance data measurement", An object of the present invention is to provide a high-load emulation performance evaluation method and an apparatus for solving the problem.

[0058]

According to the present invention, a typical processing procedure including a series of operations on a GUI screen of each business function constituting the system when a user proceeds with the processing in an interactive manner with the system. SQ attached to the entire screen displayed by
A workload is defined and created so that L statements can be linked and continuously issued, and then a plurality of business functions received by the server computer system are accepted according to the ratio of the average arrival rate to the server computer system. The workloads are connected in series to form a measurement transaction, and then the workloads are rearranged in order to prepare a plurality of the measurement transactions. And the emulation evaluation of the relationship between the number of simultaneous processes and the throughput and the response time under the condition where the waiting time does not occur, and the performance including the change of the throughput and the response time with respect to the increase of the number of the simultaneous processes by a black box measurement method. Understand the evaluation data necessary for the evaluation, and The user estimates the number of simultaneous processes to be processed simultaneously with a simple probability calculation different from the model based on the queuing network from the desired throughput and response time, and obtains the performance evaluation obtained by the black box measurement method by a simulation evaluation method. A high-load emulation performance evaluation method is provided, in which an experiment to be compared with the evaluation data is performed a finite number of times to determine the capability of the server computer system to be evaluated.

Further, CASE (Computer Ai)
deed Software Engineering)
SQL (Structured) of the access program to the database managed by the repository in the tool
Query Language (DDL) statement and DDL (Data Define) defining each resource of the database.
and a second means for automatically generating a measurement environment in the server computer system based on the procedure of the server construction method.
Means, a third means for semi-automatically generating a measurement transaction based on a procedure of a workload creation method for performing a load test, and a third means for generating the measurement transaction based on the procedure of the workload creation method. A fourth means for continuously giving the measured server computer system and measuring a change in the performance evaluation value by a black box measurement method, and estimating the number of simultaneous processes to be simultaneously processed by a simple probability calculation different from the model based on the queuing network And a fifth means for performing a finite number of experiments for comparing the evaluation data necessary for the performance evaluation obtained by the black box measurement method with the simulation evaluation method and performing a capability determination of the server computer system to be evaluated. A high-load emulation performance evaluation apparatus characterized by comprising:

Then, the condition phrase of the SQL statement retrieved by the first means and the D phrase retrieved by the first means
A temporary tuning unit that determines whether or not a secondary index is added when accessing the database by comparing with a DL statement, and automatically generates a corresponding program description when the secondary index is not added; A high-load emulation performance evaluation device is obtained.

The procedure of the server construction method employed by the second means provides a procedure for automatically generating a measurement environment in the measurement server computer system and a pre-process for remote login to the server computer system to be measured. A physical structure construction step of reading a specified value relating to disk allocation and the like from a dedicated repository and generating the program description based on a skeleton, and then constructing an evaluation database with the program description; and the dedicated repository for storing data. A logical structure construction step of extracting the DDL statement defining each resource of the database from the database, generating the resource on the evaluation database, and executing the program description generated by the temporary tuning means; The specified value for the data is An evaluation data generating step of generating data after generating a default program description for data generation by reading from the memory, and the SQL statement for randomly operating data so that the evaluation database is in a state close to a steady state. A high-load emulation performance evaluation device is provided, comprising: an evaluation data adjustment step issued in a small scale; a replica generation step of generating a replica of the evaluation database in another area; and a post-processing step.

[0062]

DESCRIPTION OF THE PREFERRED EMBODIMENTS First, the configuration of the present invention will be supplementarily described.

As described above, the present invention employs two realizing means to solve the problems of the prior art. That is, the first is a means for realizing "the evaluation model can be described with specific and easily measurable data", and the second is "a method that does not depend on special equipment for performance data measurement". It is a means of realization.

The first realization means corresponds to the configuration of the “workload creation method” and the “simulation evaluation method”, and the second realization means corresponds to the “black box measurement method” and the “server performance evaluation device”. The configuration of this applies.

The "server performance evaluation device" has a function for automatically executing a series of procedures.
In particular, the “server construction method” and the “temporary tuning function” are added as new functions.

Here, each of the above components will be described.

(1) Workload Creation Method In the evaluation model adopted in the simulation performance evaluation method, each element constituting the server computer system needs to be represented by an equivalent “queue network model”. For that purpose, it was necessary to grasp the performance data of each element constituting the server computer system.

On the other hand, the object of the present invention is to “be able to describe an evaluation model with specific and easily measurable data”, so that information on each element constituting the server computer system is hidden and handled in the model. The ability is grasped from the external behavior of the entire server computer system. Therefore, the evaluation is performed by grasping the change in the performance data of the entire server computer system with respect to the change in the load received by the server computer system.

Here, the load received by the server computer system is called a workload (Work Load), and the number of workloads to be processed simultaneously is made equal to the actual operation, and a new workload is set in order to solve the problem of the emulation performance evaluation method. Devise a creation method.

This workload creation method does not give a user thinking time, but uses a typical processing procedure (GUI
(Graphical User Interfac
e) A series of SQL statements issued in a single operation of the screen) are combined into one workload so as to be able to be continuously issued by concatenating the SQL statements. , One measurement transaction is configured by serially connecting a plurality of measurement transactions. Then, several measurement transactions in which the order of the internal workload is rearranged are prepared.

(2) Black Box Measurement Method The black box measurement method is a method in which measurement transactions created by the above-described workload creation method are sequentially run, and the number of measurement transactions during simultaneous processing in a state where no processing wait time occurs. And emulates the relationship between the throughput of each workload and the response time of each workload, emulating the queuing network model without using any special equipment to accurately analyze the data required for performance evaluation. You can understand well.

The black box measuring method includes a pre-processing step for performing a preparatory operation, a pre-measuring step for pre-evaluating a measurement overhead such as a network connection time, and the like.
SQL to calculate the response time ratio of each screen in each workload
An allocation evaluation step, a measurement execution step of actually performing measurement in a measurement transaction including a workload,
A next measurement input step for inputting the next measurement transaction, a measurement maintaining step for maintaining the measurement for a certain period of time to ensure the reliability of the measurement, a forced termination step for forcibly terminating the measurement to start the next measurement work, It comprises a post-processing step of returning the server computer system to the state before the start of measurement and an end step of performing end processing.

(3) Simulation Evaluation Method In the simulation evaluation method, the number of processes to be simultaneously processed from the client computer system is determined from the operating environment, the throughput of the user request, and the response time without using the queuing network theory model. By performing finite number of trial experiments to compare with the performance data on the throughput and response time obtained by the above black box measurement method, it is possible to successfully grasp stochastic events, and easily evaluate the server computer system to be evaluated. Perform ability judgment.

The simulation evaluation method includes a preprocessing step for setting various counters, a vector generation step for clarifying data for each workload, a simultaneous connection number estimation step for estimating the number of simultaneous connections of the client computer system, A step of estimating the number of simultaneous processes of the client computer system; a step of evaluating a response performance for performing a response performance evaluation; a step of evaluating a throughput performance for performing a throughput performance evaluation; And an overall evaluation step for determining the processing capacity of the system.

(4) High-load emulation performance evaluation method The high-load emulation performance evaluation apparatus of the present invention uses the data generated by the CASE tool so that the performance evaluation work can be performed without depending on special hardware. A loader process that takes in data, a dedicated repository that centrally manages various parameters and workloads specified when performing measurements, and a condition setting screen that specifies various parameters specified when performing measurements , Environment creation screen for creating a measurement environment, client-side environment creation tool process, server-side environment creation tool process and client generator process, and evaluation processing control for actually starting a performance data measurement method called a black box measurement method Screen and black box measurement methods An emulation controller process that controls the performance data measurement method called, a result retrieval screen that summarizes the results of the performance data measurement, a performance data store that is defined inside a dedicated repository, and an aggregation process that actually performs the collection work, and an actual simulation An evaluation screen for activating a performance evaluation method called an evaluation judgment method and an evaluation process for actually performing a performance evaluation method called a simulation evaluation judgment method are provided.

(5) Temporary Tuning Function The conditional phrase of the SQL statement extracted by the extraction loader process in the high-load emulation performance evaluation apparatus of the present invention, and the DDL statement extracted by the function of extracting the DDL statement extracted by the extraction loader process This is a function for determining whether or not a secondary index is added when accessing the database, and automatically generating a corresponding Script when the secondary index is not added. It has a temporary tuning function.

(6) Server Construction System The server construction system employed by the high-load emulation performance evaluation apparatus of the present invention is a measurement server computer system equivalent to an actual server computer system and a measurement environment equivalent to an operation environment. The pre-processing step of remotely logging in to the measured server computer system, and reading out a specified value related to disk allocation and the like from the dedicated repository and setting a default Sc
physical structure construction step of constructing an evaluation database using the script file after generating the script file, extracting a DDL statement defining a database table or the like from the dedicated repository, and creating a table on the evaluation database.
Script generated by index and temporary tuning function
A logical structure construction step of performing the following, a data value generation step of reading out specified values related to the evaluation data from the dedicated repository and generating data after generating a default Script file for data generation, and setting the evaluation database in a steady state. An evaluation data adjustment step for issuing a small-scale SQL statement for randomly performing data manipulation so as to be close to a situation, a replica generation step for generating a replica of the evaluation database in another area, and a post-processing step I have.

Next, the operation of the present invention will be described.

The server computer system performance evaluation device according to the present invention is realized by a principle based on an emulation performance evaluation method which is a performance evaluation method using an actual machine.

However, the conventional emulation performance evaluation method has the above-mentioned problems, and in order to solve these problems, in the present invention, a high load is applied to the server computer system to be measured, and the emulation performance depends on the client computer system. The measurement data that eliminates the problem described above is collected, and then the measurement data is subjected to estimation correction to realize the evaluation. Therefore, a method based on this principle is defined as a “high load simulation performance evaluation method”.

This high load simulation performance evaluation method has the following novelty.

That is, the first novelty is that the server computer system to be evaluated is not represented by a queuing network model equivalent to its components, but the entire server computer system is made a black box. Becomes easier. In addition, in order to obtain the characteristics of the entire server computer system in the form of a black box, continuous transaction processing is continuously provided so as to cause contention for resources such as a disk and a memory between processes started internally. Drive to the extreme. Then, a characteristic of the server computer system is obtained by measuring a change in a response time and a throughput with respect to an increase in the number of processes started from a client computer system running simultaneously on the server computer system. Thereby, the components in the computer system conventionally modeled by the queuing network are completely hidden, so that the later evaluation becomes extremely easy.

Next, the second novelty is to estimate the number of processes started from the client computer system that are simultaneously processed by probability estimation from the number of processed user requests per unit time and the desired response time. The number of processes is defined as the number of simultaneous processes. Thereafter, performance evaluation is performed by comparing the number of simultaneous processes with the characteristics of the server computer system obtained in the above-described extreme operation. When performing the performance evaluation, the related parameter also considers the variation.

When the performance evaluation is performed by the server computer system performance evaluation apparatus according to the present invention, the following three items are mainly used.
Phase work needs to be performed.

That is, the first phase is an evaluation environment construction phase.

The second phase is a measurement by the black box measurement method for obtaining the characteristics of the computer system under the overload condition. Here, a process equivalent to the actual operation is performed, a plurality of processes started from the client computer system are operated, and competition for obtaining resources such as disks and memories occurs between the plurality of processes as in the actual operation. As a result, the response time and the change in the throughput with respect to the increase in the number of processes running simultaneously on the server computer system are measured, and the performance characteristics of the entire server computer system are obtained.

In the third phase, the expected value of the number of processes to be processed simultaneously (this is started from the client computer system) and the fluctuation thereof are determined from the average arrival rate of requests and the average connection time determined by the user requirements. Presumed,
This is a simulation evaluation method for determining whether or not the desired response time and throughput are satisfied by comparing with the characteristics of the server computer system obtained earlier.

In the above black box measurement method, some input items including parameters must be set in advance. The specification information required by this black box measurement method includes SQL statements used in various tasks that are also workloads, the maximum data amount to be retained when transitioning to regular operation, the frequency of processing of various tasks, and the desired response of various tasks. Time and maximum allowable response time.

When implementing the black box measurement method, it is necessary to perform the following procedure. Procedure 1: Prepare a work SQL statement (for all work) that is actually operated and roughly tuned as a user characteristic load (workload).・ Procedure 2: Dummy data equivalent to the number of cases taking into account regular operation after two to three years is prepared. Step 3: The environment of the computer system to be evaluated is set to be the same as the actual operation environment or to be on the side of insufficient capacity from the viewpoint of performance. Step 4: Prepare a plurality of client computer systems that can start the measurement transaction, which can be multi-processed and have higher performance than the server computer system to be evaluated. In addition, the overhead such as process startup and the like and the network load that occur at the time of measurement on the client computer system and the network are measured in advance. Procedure 5: The measurement transaction is continuously provided from the client computer system, the response time is measured for each workload for each measurement transaction, and the measurement overhead and the network load are compensated later. Step 6: Change the number of simultaneously processed measurement transactions and measure changes in response time and throughput.

Next, the workload used when performing the black box measurement method will be described.

FIG. 1 is a diagram showing the concept of the relationship between the business function evaluated by the server computer system performance evaluation apparatus according to the present invention, the workload, and the measurement transaction.

The workload used when performing the black box measurement method follows the concept shown in FIG. That is, the specific procedure is as follows. Procedure 1: A series of steps issued in a typical processing procedure (one operation on a GUI screen) without giving a "thinking time" which is a period until the user receives a response and inputs a new request. One workload is created so that the SQL statements can be linked and continuously issued. Therefore, “between” each SQL statement essentially includes the processing on the client computer system and the thinking time. Procedure 2: A plurality of types of workloads are connected in series according to the ratio of the average arrival rates to form one measurement transaction. Then, prepare several measurement transactions in which the order of the internal workload has been changed. Step 3: When the workload uses a buffer or the like in the memory (for example, Select processing of an SQL statement), if the same thing is continuously processed, the hit ratio of the buffer is improved. Will also be improved. Therefore, the data sequence processed by the SQL statement is set to 10
Prepare more than the series so that the buffer hit rate does not increase carelessly. Procedure 4: Considering the neighboring arrangement of the data arranged on the disk, select so that the search data is uniformly distributed.

According to the above-described procedure, the workload “End Time” and “Start T”
The response time of the computer system to be measured is obtained by extracting the time difference of “im” and the overhead time separately measured.

Next, the principle of the simulation evaluation method is as follows. Procedure 1: Estimate the expected value of the number of client processes (or the number of client computer systems) that are being processed simultaneously and the variation thereof from the desired response time determined by the user requirements, the average arrival rate of various services, and the average connection time. Procedure 2: The expected value of the estimated number of client processes is introduced into the relationship between the number of simultaneous processes and the response time obtained by the black box measurement method, and the magnitude of the actual response time is compared. -Procedure 3: Not only the response time but also the throughput are similarly compared. Procedure 4: The number of client processes to be estimated (the number of client computer systems under simultaneous processing) is changed by giving the variation of the request response time as a random number, and in each case, the evaluation of procedures 2 and 3 is performed. afterwards,
It is determined whether all the measurement results are satisfied.

Next, an embodiment of the present invention will be described with reference to the drawings.

FIG. 2 is a block diagram showing an embodiment of a server computer system performance evaluation device used in the high load emulation performance evaluation device of the present invention, and FIG. 3 is a processing procedure of the entire high load emulation performance evaluation device of the present invention. 4 is a flowchart of a black box measurement method adopted by the server computer system performance evaluation device in FIG. 2, FIG. 5 is a flowchart of a simulation evaluation method adopted by the server computer system performance evaluation device in FIG. 2, and FIG. 3 is a flowchart of a server construction method adopted by the server computer system performance evaluation device in FIG.

Referring to FIG. 2, when a program of the core information system is developed by the CASE tool 2, the result is usually stored in the CASE tool repository 3 including the one under design.

This server computer system performance evaluation device 1
When the performance evaluation of the server computer system is performed, the extraction loader process 4 is first started. Then, when the extraction loader process 4 starts up, the SQL statement a1 that can become a workload from the CASE tool repository 3
And a DDL sentence a2 defining a database table and the like, and are copied to a dedicated repository 5 existing inside the server computer system performance evaluation apparatus 1.

Next, the condition setting screen 6 is started, and various parameters including the following parameters a3 to a14 are registered in the dedicated repository 5.・ A3: Maximum data amount retained when the system is shifted to the regular operation ・ a4: Frequency of transaction occurrence of various tasks ・ a5: Definition area of various definition table items ・ a6: Definition range and setting standard of inquiry SQL statement ・ a7: Definition range and setting standard of data operation SQL statement ・ a8: Maximum value of measurement transaction ・ a9: Machine name, network address, login name and base directory of measurement client computer system 7 ・ a10: Machine name of measured server computer system 8 ,
Network address and login name ・ a11: Detailed specification of the data storage method of the measured server computer system 8 ・ a12: Minimum measurement maintenance time ・ a13: Measurement transaction input interval time ・ a14: Evaluation database replica 21 destination of evaluation database 17 Machine name, network address,
Emulation environment is generated by giving the above various parameter data.

Next, an environment creation drawing 9 is started to generate an environment of the black box measurement system. When the environment creation drawing 9 is started, construction is performed in the order of construction of the environment of the measurement client computer system 7 and construction of the environment of the measured server computer system 8.

When constructing the environment of the measurement client computer system 7, the measurement client environment creation tool process 10 is started. When constructing the environment of the measured server computer system 8, the server environment creation tool process 11 is started.

When the measurement client environment creation tool process 10 starts up, all the SQL statements a15 that can be a workload are taken out of the dedicated repository 5, and the transaction occurrence frequency a4 of various tasks is selected from the various parameters registered on the condition setting screen 6. And the definition range and setting criterion a6 of the inquiry SQL statement and the definition range and setting criterion a7 of the data manipulation SQL statement.

Thereafter, as shown in FIG.
By arranging the L statements so as to form a series of processes that do not cause regression, a script of the workload of each business is created.

The workload Scri is distributed by the distribution corresponding to the transaction occurrence frequency a4 of the various tasks.
The script of the measurement transaction is created by arranging the pts sequentially and randomly. To further facilitate this order and randomly, the S
A plurality of scripts are created, and the S
It is stored as a script file group 12.

As shown in FIG. 1, a series of SQL statements issued according to a typical processing procedure (one operation of a GUI screen) can be continuously issued without giving a thinking time. After one workload is created, several types of workloads are connected in series according to the ratio of average arrival rates to form one measurement transaction, and several measurements are rearranged in the order of internal workload. The method of creating a transaction is defined as "workload creation method".

Note that the measurement transaction Script
When creating t, the condition setting of the query SQL statement or the value range setting of the data manipulation SQL statement are performed according to the definition range and setting criteria a6 of the query SQL statement and the definition range and setting criteria a7 of the data manipulation SQL statement. Is set.

When the measurement client environment creation tool process 10 collects the SQL statement a15 that can be a workload from the dedicated repository 5, whether or not the DDL statement a16 of the secondary index is added to the conditional phrase is determined. Is determined. Here, when the DDL sentence is not added, “Script” for automatically creating an index as a tuning index is generated, and “Sc” is stored in the dedicated repository 5.
Rip "a17 is stored. This function is defined as a" temporary tuning function ".

In addition, the script of the measurement transaction
A clear name is added to the individual files a53 in the t-file group 12. Thereafter, the measurement client environment creation tool process 10 issues a command a56 for activating the measurement client generator process 13.

Measurement client generator process 1
When the server 3 starts, the machine name a18 of the measurement client computer system, the network address a19 of the measurement client computer system, the login name a20 of the measurement client computer system,
The base directory a21, the machine name a22 of the measured server computer system, the network address a23 of the measured server computer system, and the login name a24 of the measured server computer system are extracted.

Thereafter, the user remotely logs in to the measurement client computer system 7 and makes preparations for creating a measurement client environment.

Then, the measurement client generator process 13 sequentially executes the following steps to create a measurement environment in the measurement client computer system 7. Step 1: The maximum value a8 of the measurement transaction is grasped and stored with reference to the dedicated repository 5. After the construction counter is initialized, the process proceeds to step S2. Step 2: Name the test order and specify the storage directory for environment variables and various files. Step 3: Script of the previous measurement transaction
The number of files a53 corresponding to the value of the construction counter is taken out from the t-file group 12, transferred to the measurement client computer system 7, stored in the storage directory specified in step 2, and then the construction counter is counted up. Step 4: File a53 of the Script file group 12 of the measurement transaction preceding the skeleton held in the measurement client generator process 13
Issue the command a54 for generating the measurement process description file 16 so that the measurement transactions described in the above can be sequentially activated, and receive the measurement evaluation result log file 1 for receiving the evaluation result of each measurement transaction.
The file path a55 of 4A, 14B is designated. In addition,
When generating the measurement process description file 16, the machine name a2 of the server computer system to be measured previously extracted
2, the network address a23 of the measured server computer system, and the login name a24 of the measured server computer system, the login destination is described in the measurement process description file 16. Also, the measurement evaluation result log file 14
When the file path a55 of A, 14B is designated, it is designated to be created under the storage directory of step 2. Step 5: The file a53 transferred in Step 3 is transferred to the storage directory specified in Step 2 under the measurement client computer system, and the measurement process description file 16 is actually created. Step 6: The entries of the measurement evaluation result log files 14A and 14B are created in advance in the file path a55 of the measurement evaluation result log files 14A and 14B specified in Step 4. Step 7: Compare the stored maximum value a8 of the measurement transaction with the count value of the construction counter,
If a8 ≧ count value, the process returns to step 3. Also a8
If the count value is <the count value, the work related to the measurement transaction has been completed, so the measurement client generator process 13 reports a work end report a45 to the measurement client environment creation tool process 10.

Next, the server environment creation tool process 11
Rises, the following steps, ie, FIG.
The “server construction method” is implemented according to the flow shown in (1).

Referring also to FIG. 6, when the server environment creation tool process 11 is started, the machine name a22 of the measured server computer system 8 is sent from the dedicated repository 5,
The network address a23 of the measured server computer system 8, the login name a24 of the measured server computer system 8, and the detailed designation information a11 of the data storage method of the measured server computer system 8 are obtained. Log in remotely to This step is referred to as a “pre-processing step” (step S61).

After that, regarding the disk allocation and the like, the parameters specified in the detailed specification information a11 of the data storage method of the measured server computer system 8 are written into the default Script file, and then the Scr is written.
The evaluation database 17 is created with the ip file a46. This step is called a “physical structure construction step” (step S62).

The server environment creation tool process 11 constructs the physical structure of the evaluation database 17 (S62).
After completing the above, the DDL sentence a16 defining the table and the like of the database is taken out from the dedicated repository 5, and the table, the index and the like are defined on the previously created evaluation database 17.
“Script” a17 for creating a tuning index stored in the dedicated repository 5 is also extracted from the dedicated repository 5 and additionally defined on the evaluation database 17 created earlier. This step is called a “logical structure construction step” (step S63).

The server environment creation tool process 11 constructs the logical structure of the evaluation database 17 (S6).
After completing step 3), in order to generate actual evaluation data, the maximum data amount a3 to be retained when shifting from the dedicated repository 5 to the regular operation, and the definition area a of various definition table items
5 and definition range and setting standard a of the inquiry SQL statement
6 and the definition range and setting standard a of the data manipulation SQL statement
7 is obtained. Then, in order to generate evaluation data, after referring to the definition range and setting criterion a6 of the query SQL statement and the definition range and setting criterion a7 of the data manipulation SQL statement, data matching the definition area a5 of the various definition table items is referred to. Data amount a3 retained when the system is shifted to regular operation
The “Script” a57 to be created is automatically assembled and issued to the evaluation database 17 to generate data. This step is referred to as “evaluation data generation step” (step S64).

Next, the server environment creation tool process 11
Issues a small SQL statement for performing data manipulation at random so that the evaluation database 17 is close to a steady state. Therefore, the SQL that extracts the data manipulation SQL statement a15 that becomes the workload from the dedicated repository 5 and performs the data manipulation randomly by adding the related modification.
The sentence a58 is automatically generated and issued to the evaluation database 17. This step is referred to as “evaluation data adjustment step” (step S65).

In the case of the black box measurement method, the state of the database changes each time the measurement is performed. Therefore, the evaluation database replica 21 of the evaluation database 17 is generated in another area. The generation destination of the evaluation database replica 21 is specified in the dedicated repository 5 and is obtained therefrom. This step is referred to as a “replica generation step” (step S66).

When the server environment creation tool process 11 completes the steps up to the replica generation step (S66), all the environments have been created, and a creation completion notification a59 is displayed on the environment creation screen 9. This step is called a "post-processing step" (step S67).

The above steps S61 to S67
The server environment creation tool process 11 terminates the server construction method by executing the processing up to this point.

Next, when actually performing the black box measurement method, the evaluation processing control screen 18 is started.
Then, after giving an argument a60 such as the number of tests on the evaluation processing control screen 18, when a start command a61 is input,
The emulation controller process 19 starts with the argument a60.

Emulation controller process 1
9 obtains the following information from the dedicated repository 5. It is the machine name a18 of the measurement client computer system 7,
A network address a19, a login name a20, and a base directory a21, and a machine name a22, a network address a23, and a login name a24 of the measured server computer system 8.

After that, the user logs in to the measurement client computer system 7 and implements the black box measurement method according to the steps shown in FIG.

Referring to FIG. 4 as well, a list under the base directory is obtained, and a list of directories in which the previously created measurement process description file 16 is stored and which differs for each test order is created. This step is referred to as a "pre-processing step" (step S41).

A start command a62 of the measurement process description file 16 by sequentially specifying directories for each test order
Is issued, measurement processes 15A and 15B are generated for each measurement transaction. Measurement process 15A,
15B, before the transaction a63 is sequentially applied to the evaluation database 17 on the measured server computer system 8, only logs in to the measured server computer system 8 and measures the network connection time a52 in advance to log in the measurement evaluation result. Write to files 14A and 14B. This step is referred to as a “prior measurement step” (step S42).

Next, SQL distribution evaluation is performed. This is the number of individual Ss in each workload that makes up the measurement transaction.
This is to calculate the time consumption rate for the entire workload for the QL statement. For this purpose, the measurement process description file 16 describing the measurement transaction is copied to generate an SQL distribution evaluation measurement process description file 20, and a comment for counting the elapsed time for each SQL statement corresponding to one screen transition is included. Let

Further, the information of the base directory a9 used as the path of the log file 25 storing the result of counting the elapsed time is obtained from the dedicated repository 5. Thereafter, by reading the SQL distribution evaluation measurement process description file 20 and issuing a command a65 for starting the SQL distribution measurement process 29 a plurality of times, the time distribution ratio of each SQL statement in each workload to the entire workload is determined. The data a66 for calculation is measured and stored in the log file 25. This step is referred to as “SQL distribution evaluation step” (step S43).

After step 43, the measurement processes 15A and 15B for each measurement transaction write the “Start Time” a47 of the first workload to the measurement evaluation result log files 14A and 14B, and the evaluation on the server computer system 8 to be measured. Multiply the data base 17 by the measurement transaction a63.

"Start Time" of Workload
The a47 and the “EndTime” a48 of the workload are recorded for all the workloads included in the measurement transaction. The measurement transaction repeats this step forever unless there is a forced termination command a49 from the emulation controller process 19. This step is referred to as a “measurement execution step” (step S44).

When the plurality of measurement processes 15A and 15B start simultaneously, the emulation controller process 19 extracts the measurement transaction input interval time a13 from the dedicated repository 5, and one measurement process (for example, 15A) starts the processing of step S44. After waiting for the measured transaction input interval time a13 from “Start Time” a47,
Start command a6 of another measurement process description file 16
Issue 2. By doing so, a measurement process (for example, 15B) corresponding to another measurement transaction is generated. The processing of steps S43 and S44 is also performed in the measurement process 15B. This step is referred to as a “next measurement input step” (step S45).

Next, it is checked whether all the measurement processes 15A and 15B under the directory in which the measurement process description file 16 is stored have been started (step S46), and if all have been started (Yes in step S46). . The emulation controller process 19 extracts the minimum measurement and maintenance time information a12 from the dedicated repository 5, and maintains the operation for the designated time. This step is referred to as a “measurement maintenance step” (step S47).

If No in step S46, that is, if all the measurement processes have not started, the process proceeds to step S46.
Returning to step 43, the procedure of steps S44 and S45 is repeated until the result of step S46 becomes Yes.

In step S47, when the operation has been performed for the specified time of the minimum measurement maintenance time information a12, the emulation controller process 19 forcibly terminates all the measurement processes 15A and 15B in the measurement client computer system 7 at once. Issues the command a49. This step is referred to as a “forced end step” (step S48).

At the stage when step S48 is completed, the emulation controller process 19 sends the machine name of the evaluation database replica 21 from the dedicated repository 5,
The network address, login name, and base directory designation a14 are obtained, and are replaced with the evaluation database replica 21 on the server computer system 8 to be measured.
This step is called a "post-processing step" (step S49).

Emulation controller process 1
9 is when the entire test order is not completed (step S410)
No), the process returns to step S43 to execute the next test order. Then, when all the test orders are completed (Yes in step S410), an end notification a50 is displayed on the evaluation processing control screen 18, and the processing ends. This step is called an “end step” (step S411).

The above steps S41 to S41
The processing up to 1 is executed, and the black box measurement method ends.

Next, when totaling the results of the black box measurement method for evaluation, the result totaling screen 22 is activated. Then, when the result tabulation screen 22 is launched, all SQs that can become workloads from the dedicated repository 5 are displayed.
The L sentence a51 is obtained and displayed in a list format.

Thus, the user selects and inputs an SQL sentence on the screen in order to add a mark to the SQL sentence corresponding to the screen for which the performance is to be ensured.

Thereafter, the tallying process 23 is started using the marked SQL statement a25 as an argument. The tallying process 23 sends the machine name a18 of the measurement client computer system 7 and the network address a
19 and the login name a20, and logs in to the measurement client computer system 7.

Thereafter, each data is collected and totalized in the following steps and stored in the capacity data store 24 in the dedicated repository 5. Step 1: Search under the base directory and create a list of all directories in which the previously created measurement process description file 16 is stored. Step 2: Next, the measurement evaluation result log files 14A and 14 existing under different directories for each test order
See all B. Step 3: All measurement evaluation result log files 14A, 1
In 4B, data of a portion that does not reach the designated number of measurement processes is discarded, and only a necessary portion is left. Step 4: All measurement evaluation result log files 14A, 1
Measured transaction number a2 from necessary part of 4B
8 and “Start Time” a4 of the workload
7. The information of the “End Time” a48 and the network connection time a52 of the workload are extracted and stored in the capacity data store 24 in the dedicated repository 5. Step 5: Next, an SQL distribution strengthening calculation is performed. From the dedicated repository 5, a base directory a9 to be used as the path of the log file 25 storing the result of counting the elapsed time is obtained. Thereafter, a time distribution ratio a26 to the workload is calculated for each of the SQL statements in each word load from the lapse of time for each SQL statement stored in the log file 25, and the workload name and the SQL are calculated.
It is stored in the capability data store 24 in the dedicated repository 5 in association with the statement. Step 6: Next, for the SQL sentence a25 selected and input on the result totaling screen 22, to indicate that the SQL statement a25 has been selected, the capability data store 24 in the dedicated repository 5
To that effect. Step 7: “Start Time” a47 of the workload obtained in step 4 above and “E” of the workload
nd Time "a48 and network connection time a
The workload response time a27 is calculated from each of the information 52, and the workload response time a27 is stored in the capacity data store 24 in the dedicated repository 5 together with the measured transaction number a28. Step 8: Thereafter, the number of measured transactions a28
The data of the workload response time a27 of the same type of workload is collected every time, and the number of measured transactions / the average response time a29 and the response time variance a30 for each workload are obtained.
An average throughput value a31 and a throughput variance value a32 for each workload are also obtained and stored in the capacity data store 24 in the dedicated repository 5. Step 9: By the above steps 1 to 8,
Since all the work of the tallying process 23 has been completed, the work completion message a33 is returned to the result tallying screen 22 and the processing is ended. Thereafter, a work completion message a33 is displayed on the result totaling screen 22.

Next, when executing the simulation evaluation method, the evaluation screen 26 is started. When the evaluation screen 26 is activated, a request for inputting auxiliary parameters for evaluation is displayed, and the following parameters a34 to a41 are input. · A34: average connection time (T) · a35: connection time dispersion value · a36: service duration · a37: desired response time of various tasks corresponding to workload · a38: maximum allowable response of various tasks corresponding to workload Time or variation value a39: Random walk safety factor (n) for each workload a40: Number of simulation evaluations performed a41: Upper limit of allowable error occurrence rate After inputting the above parameters, start evaluation a42 on evaluation screen 26 When input, the evaluation screen 26 starts the evaluation process 27 using the parameters a34 to a41 as arguments.

After that, the evaluation process 27 obtains the following information from the capability data store 24 in the dedicated repository 5.・ A4: Frequency of transaction occurrence of various tasks ・ a29: Number of measured transactions / average response time for each workload ・ a30: Number of measured transactions / variance of response time for each workload ・ a31: Number of measured transactions / each workload Average throughput value a32: number of measured transactions / throughput variance value for each workload a26: time distribution ratio of individual SQL statements in each workload to the workload Next, the evaluation process 27 is based on the flowchart shown in FIG. Implement a simulation evaluation method.

Referring to FIG. 5, a value is set for each parameter. Further, the simulation counter and the error occurrence counter are reset to zero. This step is referred to as a “preprocessing step” (step S5).
1).

The number-of-simulations counter is compared with the number of times of simulation evaluation a40 input on the evaluation screen 26 (step S53). If the number of times of simulation evaluation is less than the specified number, the following processing is performed. If the number of simulation evaluations is equal to or greater than the specified number, the process proceeds to step S58.

If Yes in step S53, that is, if the number of times of performing the simulation evaluation is less than the specified number (Yes in step S53), a random number-generated variation in the desired response time in various tasks corresponding to each workload. The value obtained by adding or subtracting the value is obtained and expressed as a vector. And
The probability phenomenon is expressed by changing the variation value.

[0146]

At this time, the SQL sentence a25 with the mark added is taken out from the dedicated repository 5, and the time distribution ratio of the SQL sentence is obtained for each workload and expressed in a vector. It is expressed as in equation (2). (Ρ ₁ , ρ ₂ ,..., Ρ _m ) (2) Here, by dividing each element of the vector expression of Expression (1) by each element of the vector expression of Expression (2), the allowable time of the entire workload is obtained. Can be requested. The result is expressed as in equation (3). (Et ₁ , Et ₂ ,..., Et _M ) (3) This step is referred to as a “vector generation step” (step S 52).

Next, using the probability calculation, the expected number of client computer systems that are simultaneously connected to the server computer system at a certain point in time and are being processed is estimated. Here, the number of simultaneous connections means the number (the number of processes) of the client computer system that is simultaneously connected to the server computer system even during access. In this step, the number of simultaneous connections is estimated.

On the other hand, in the next step, the number of simultaneous processes, which means the client computer system that is simultaneously processing, is estimated from the number of simultaneous connections.

Here, the transaction occurrence frequency a4 of various tasks and the average connection time (T) a34 input on the evaluation screen are obtained from the dedicated repository 5. The transaction occurrence frequency a4 of various tasks means the occurrence frequency of the workload.

[0151] Here workload m: If you mean the arrival rate λ _m of (1 ≦ m ≦ M), average connection time (T) a34
, The number of simultaneous connections is represented by a Poisson distribution, so that the probability of the number of simultaneous connections N _same is derived as in equation (4).

[0152]

From equation (4), the workload m: (1 ≦ m ≦
The probability that M) is “connected” by j is derived by equation (5). This is where the workloads consist of mutually exclusive events.

[0154]

This step is called “simultaneous connection number estimation step” (step S54).

Next, the number of simultaneous processes of the client computer system is estimated. Since the response time of the workload is expressed by a general function of the number of concurrent connections N _same,, the random walk safety factor of each workload obtained from only repository 5 (n) a39 and average connection time (T) a34, a connection Is derived in equation (6). Here, the random walk safety coefficient (n) a39 is added in consideration of the return of the screen selection and the like.

[0157]

If the above equation (6) is used, the expected value of the number of processes (the number of processes) which are further processed under the condition that j workloads m: (1 ≦ m ≦ M) are connected is calculated by the equation (7). ).

[0159]

From the above, the workload m: (1 ≦ m
(≦ M) is the conditional probability, and the expected value during the processing is calculated by the product of Expressions (5) and (7). Therefore, the expected value N during the simultaneous processing of the workload m: (1 ≦ m ≦ M)
(M) is calculated by equation (8).

When the number of simultaneous connections N _same is a sufficiently large value, the arrival of the workload can be regarded as an independent event. Therefore, if the sum of the workload m (1 ≦ m ≦ M) is obtained, The expected value of the number of simultaneous processes at any time is calculated by Expression (9).

[0162]

From the above equations (1) to (9), the number of simultaneous processes is estimated from the allowable time of the entire workload obtained in step S52. This step is referred to as a “simultaneous processing number estimation step” (step S55).

Next, the evaluation process 27 determines the response time of each workload with respect to the number of measured transactions a28 indicating the number of simultaneous processes obtained from the performance data store 24 in the dedicated repository 5, and the tolerance of the entire workload of the formula (3). Compare with time.

Here, each parameter is represented by the following equation (10).
It is defined as in equation (12). Average response time for each workload m under the number of measured transactions (N): μ _m (N) (10) Variance of response time per workload m under the number of measured transactions (N): σ _m (N) (11) Variation of the estimated number of simultaneous connections N: δ _N (12) Under the above definition, it is determined for each workload m whether or not the following two equations hold.

[0166]

The equation (13) is used when evaluating with an average value. In contrast, equation (14) is 95% of the specified screen
Is used under conditions that satisfy the desired response time.

In this step, equation (13) or equation (1)
If there is even one workload that does not satisfy 4), both the error occurrence counter and the simulation counter are incremented, the detailed evaluation information a43 is written in the evaluation log file 28, and the process returns to step S52. In addition, the total workload is calculated by the equation (13) or the equation (1).
When 4) is satisfied, the detailed information a43 of the evaluation is written in the evaluation log file 28, and then the process proceeds to step S57. This step is referred to as a “response performance evaluation step” (step S56).

Next, performance evaluation of throughput is performed. For this purpose, the transaction occurrence frequency a4 of various tasks of the parameter obtained from the dedicated repository 5 is obtained,
The evaluation process 27 obtains the following necessary information from the capability data store 24 in the dedicated repository 5.

[0170]

Under the above definition, it is determined whether or not the following equation (17) holds for each workload m.

[0172]

Equation (17) targets only the average value. In this step, when even one workload does not satisfy the expression (17), both the error occurrence counter and the simulation number counter are incremented, and the detailed evaluation information a43 is written in the evaluation log file 28. Return. Also, the total workload is given by the formula (1)
When 7) is satisfied, only the simulation counter is incremented, the detailed evaluation information a43 is written in the evaluation log file 28, and the process returns to step S52. This step is the “throughput performance evaluation step”
(Step S57).

The evaluation process 27 compares both the error occurrence counter and the number-of-simulations counter. If the error occurrence rate exceeds the allowable error occurrence rate upper limit value a41, a return value a44 of insufficient capacity is returned and the process ends. Otherwise, a return value a44 without any problem is returned and the processing is terminated. This step is called “Comprehensive evaluation step”
(Step S58).

When the simulation evaluation performed by the evaluation process 27 is completed, the return value a44 is returned to the evaluation screen 26 and displayed on the evaluation screen 26. Evaluation screen 2
6 shows detailed evaluation information a4 in the evaluation log file 28.
3 is also displayed in a list.

The above steps S51 to S58
The processing up to is executed, and the simulation evaluation ends.

[0177]

As described above, the high-load emulation performance evaluation method according to the present invention performs a series of operations on the GUI screen of each business function constituting the system when the user proceeds with the system in an interactive manner. Workload is defined and created so that SQL statements can be linked and attached continuously to all screens displayed in a typical processing procedure, including the ratio of the average arrival rate to the server computer system. A workload is created by combining multiple workloads of all business functions received by the server computer system in series to form a measurement transaction, and then rearranging the workload order to prepare multiple measurement transactions Run multiple measurement transactions in sequence according to the method of Related emulation evaluate the response time Putto, grasp the evaluation data necessary to evaluate the performance, including changes in throughput and response time to increase the concurrency by the black box measurement method,
Estimate the number of simultaneous processes to be performed simultaneously with simple probability calculation different from that of the queuing network model from the operating environment and user-desired throughput and response time, and evaluate the performance obtained by the black box measurement method using the simulation evaluation method. By performing a finite number of experiments to compare the evaluation data with the evaluation data and determining the capability of the server computer system to be evaluated, the high-load emulation performance evaluation device of the present invention is managed by the repository in the CASE tool. First means for extracting SQL statements of a database access program and DDL statements defining each resource of a database from a repository, and second means for automatically generating a measurement environment in a server computer system based on a procedure of a server construction method Means and work load testing Third means for semi-automatically generating a measurement transaction based on the procedure of the load creation method, and continuously providing the measurement transaction created based on the procedure of the workload creation method to the measured server computer system, and performing black box measurement Fourth means for measuring the change of the performance evaluation value by the method, and the number of simultaneous processes to be simultaneously processed by the simple probability calculation different from the model by the queuing network, and obtained by the simulation evaluation method by the black box measurement method A finite number of trials of an experiment comparing with the evaluation data necessary for performance evaluation to determine the capability of the server computer system to be evaluated.
Means, the condition index of the SQL statement retrieved by the first means is compared with the DDL statement retrieved by the first means, and a secondary index for accessing the database is added. The provision of a temporary tuning means for judging whether or not the program description has not been added and automatically generating a corresponding program description when the program description has not been added. Provision of a procedure for automatically generating a measurement environment in the computer system, pre-processing step for remote login to the server computer system to be measured, and reading out specified values related to disk allocation etc. from the dedicated repository and generating a program description based on the skeleton The physical structure construction step of constructing an evaluation database using this program description and saving the data A logical structure construction step of extracting a DDL statement defining each resource of the database from the dedicated repository, generating a resource on the evaluation database, and executing the program description generated by the temporary tuning means, and specifying a value for the evaluation data. An evaluation data generation step of generating data after generating a default program description for data generation by reading from the dedicated repository, and a small SQL statement for randomly operating data so that the evaluation database is in a state close to a steady state. By providing an evaluation data adjustment step to be issued to a scale, a replica generation step to generate a replica of the evaluation database in another area, and a post-processing step, the performance of the conventional simulation performance evaluation method and the stacked calculation performance evaluation method is reduced. Measure data A dedicated high-performance tool that is needed in order has the effect that it becomes unnecessary.

In addition, the problem that a large amount of description of the queuing network model is required at the time of performance evaluation in the conventional simulation performance evaluation method is solved. In the workload creation method, the SQL statement used in actual operation is used. , A workload and a measurement transaction are easily created. In the simulation evaluation method, the number of simultaneous processes is estimated by probability calculation without performing modeling using a queuing network.

Further, in the conventional emulation performance evaluation method, it is necessary to prepare a client computer system for actual operation, and it is composed of a plurality of screen operations.
Although there has been a problem that the appearance rate in consideration of a cache hit cannot be accurately emulated for a completed transaction, the black box measurement method of the present invention essentially solves this problem. It has the effect of.

[Brief description of the drawings]

FIG. 1 is a diagram showing a concept of a relationship between a business function evaluated by a server computer system performance evaluation device according to the present invention, a workload, and a measurement transaction.

FIG. 2 is a block diagram showing an embodiment of a server computer system performance evaluation device used in the high load emulation performance evaluation device of the present invention.

FIG. 3 is a flowchart showing a processing procedure of the entire high-load emulation performance evaluation device of the present invention.

FIG. 4 is a flowchart of a black box measurement method adopted by the server computer system performance evaluation device in FIG. 2;

FIG. 5 is a flowchart of a simulation evaluation method adopted by the server computer system performance evaluation device in FIG. 2;

FIG. 6 is a flowchart of a server construction method adopted by the server computer system performance evaluation device in FIG. 2;

FIG. 7 is a block diagram of a simulation performance evaluation method showing a first example of a computer performance evaluation method.

FIG. 8 is a block diagram of a stacking performance evaluation method showing a second example of the computer performance evaluation method.

FIG. 9 is a block diagram of an emulation performance evaluation method showing a third example of a computer performance evaluation method.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 Server computer system performance evaluation apparatus 2 CASE tool 3 CASE tool repository 4 Extraction loader process 5 Dedicated repository 6 Condition setting screen 7 Measurement client computer system 8 Server server system to be measured 9 Environment creation screen 10 Measurement client environment creation tool process 11 Server environment Creation tool process 12 Script file group 13 Measurement client generator process 14A, 14B Measurement evaluation result log file 15A, 15B Measurement process 16 Measurement process description file 17 Evaluation database 18 Evaluation processing control screen 19 Emulation controller process 20 SQL distribution evaluation measurement process description File 21 Evaluation database replica 22 Result tabulation screen 23 Tabulation process 24 Performance data Tor 25 Log file 26 Evaluation screen 27 Evaluation process 28 Evaluation log file 29 SQL allocation measurement process 71 Simulator unit 72 Performance evaluation unit 73 Controller 74 Simulator interface 75 Performance evaluation interface 76 User interface 80 Operating computer system 81 CPU time measurement device 82, 84, 85, 88 storage device 83 input / output count measuring device 86, 89 pointing device 87, 810 processing device 90 client computer system 91 terminal simulation control unit 92 performance evaluation control unit 93 input transaction file unit 94 performance evaluation data storage unit 95 Performance evaluation data output unit 96 Console 97 Performance evaluation list 98 Communication control unit 99 Transaction processing unit 100 Server computer system a1, a15, a25, a51, a58 SQL statement a2, a16 DDL statement a3 Maximum amount of data retained when transition to regular operation a4 Transaction occurrence frequency of various tasks a5 Definition area of various definition table items a6 Definition range and setting standard of inquiry SQL statement a7 Definition range and setting standard of data operation SQL statement a8 Maximum value of measurement transaction a9 Machine name, network address, login name and base directory of measurement client computer system 7 a10 Machine name of server computer system 8 to be measured,
Network address and login name a11 Detailed specification information of the data storage method of the measured server computer system 8 a12 Measurement maintenance minimum time information a13 Measurement transaction input interval time a14 Machine name and network address of the evaluation database replica 21 of the evaluation database 17 ,
Login name and base directory specification a17, a57 Script a18 Machine name of measurement client computer system 7 a19 Network address of measurement client computer system 7 a20 Login name of measurement client computer system 7 a21 Base directory a22 Machine name of measurement target server computer system 8 a23 Network address of server computer system 8 to be measured a24 Login name of server computer system 8 to be measured a26 Time distribution ratio of individual SQL statements in each workload to workload a27 Workload response time a28 Number of measured transactions a29 Measurement transaction Number / Response time average value for each workload a30 Measured transactions / Response time variance value for each workload a31 Measured Number of transactions / average throughput value for each workload a32 Measured transaction number / dispersion value of throughput for each workload a33 Work completion message a34 Average connection time (T) a35 Connection time dispersion value a36 Service duration a37 Various types of workloads Desired response time of work a38 Maximum allowable response time or variation value of various work corresponding to workload a39 Random walk safety factor (n) a40 Number of simulation evaluations performed a41 Permissible error occurrence upper limit a42 Evaluation start a43 Detailed information of evaluation a44 Return value a45 Work end report a46 Script file a47 Start Time a48 End Time a49 Forced end command a50, a59 End notification a52 Network connection time a53 File a54, a56, a65 Command a55 File path a60 Argument a61 Start command a62 Start command a63 Transaction a66 Data for calculating time distribution ratio

Claims (4)

(57) [Claims] 1. When a user proceeds with a process in a conversational manner with a system, it is attached to a full screen displayed in a typical processing procedure including a series of operations of a GUI screen of each business function constituting the system. A workload is defined and created so that SQL statements can be linked and continuously issued, and then a plurality of business functions received by the server computer system are received in accordance with a ratio of an average arrival rate to the server computer system. The workloads are connected in series to form a measurement transaction, and then the workloads are rearranged in order to prepare a plurality of the measurement transactions. Emulates the relationship between the number of concurrent processes, throughput, and response time in a situation where no waiting time occurs Evaluate, grasp the evaluation data necessary for performance evaluation including the change of the throughput and the response time with respect to the increase in the number of simultaneous processes by the black box measurement method, and wait from the operation environment and the user-desired throughput and the response time. The number of simultaneous processes to be processed simultaneously by a simple probability calculation different from the model based on the matrix network is estimated, and an experiment for comparing with the evaluation data necessary for the performance evaluation obtained by the black box measurement method by the simulation evaluation method is performed finitely. A high-load emulation performance evaluation method, comprising performing a trial to determine the capability of the server computer system to be evaluated. A first means for extracting, from the repository, an SQL statement of an access program to a database managed by a repository in the CASE tool and a DDL statement defining each resource of the database; A second means for automatically generating a measurement environment in the server computer system based on a procedure, a third means for semi-automatically generating a measurement transaction based on a procedure of a workload creation method for performing a load test, Fourth means for continuously giving the measurement transaction created based on the procedure of the workload creation method to the server computer system to be measured and measuring a change in the performance evaluation value by a black box measurement method, and a model based on a queuing network The number of simultaneous processes that should be processed simultaneously with a simple probability calculation different from A fifth means for performing a finite number of trials of an experiment for comparing with the evaluation data necessary for the performance evaluation obtained by the black box measurement method by the evaluation method, and determining the capability of the server computer system to be evaluated. High load emulation performance evaluation device characterized by the following. 3. The SQL retrieved by the first means
By comparing a conditional phrase of a sentence with the DDL sentence extracted by the first means, it is determined whether or not a secondary index has been added when accessing the database. 3. The high-load emulation performance evaluation device according to claim 2, further comprising a temporary tuning means for automatically generating a program description. 4. The procedure of the server construction method adopted by the second means provides a procedure for automatically generating a measurement environment in a server computer system for measurement and a pre-process for remote login to the server computer system to be measured. A physical structure construction step of reading a specified value relating to disk allocation and the like from a dedicated repository and generating the program description based on a skeleton, and then constructing an evaluation database with the program description; and the dedicated repository for storing data. A logical structure construction step of extracting the DDL statement defining each resource of the database from the database, generating the resource on the evaluation database, and executing the program description generated by the temporary tuning means; Is the specified value for data in the dedicated repository? An evaluation data generation step of generating data after generating a default program description for data generation by reading data from the database, and reducing the SQL statement for randomly performing data manipulation so that the evaluation database is close to a steady state. 4. The high-load emulation performance according to claim 2, further comprising: an evaluation data adjustment step for issuing a scale, a replica generation step for generating a replica of the evaluation database in another area, and a post-processing step. Evaluation device.