JPH08202668A - Device and method for evaluating performance - Google Patents

Abstract

PURPOSE: To provide a device and method for performance evaluation which can evaluate the performance of the upstream process (initial process) of a large-scale decentralized system. CONSTITUTION: This device consists of an input/output means 2 for inputting parameters regarding the processability and response of a system on a keyboard or with a mouse, etc., in advance, a reliability parameter storage means 5 which stores the-inputted parameters, e.g. parameters characteristic of a processor, an I/O, a package, etc., in a reliability parameter table 51, a reliability parameter management means 3 which registers and retrieves parameters at need, and a performance evaluating means 4 which evaluates the processability from CPU load rates at stationary time by the processors on the basis of the parameters and evaluates the response of a response request item requested for the system.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a performance evaluation device and evaluation method in the design stage of a computer system, and more particularly to a performance evaluation device and evaluation method applicable to the development of a control system for a large-scale distributed system.

[0002]

2. Description of the Related Art A conventional centralized computer system has a relatively small number of types of hard and soft packages that make it up, the direction of information flow is simple, and the processability and response during system design are improved. The evaluation of the sex was possible by a system designer by hand, and the tuning at the final stage of the software production process was relatively easy due to the simplicity of the software structure like the hardware configuration.

However, in recent years, as computer systems have become larger and more distributed, the types of hardware and software that make up the systems have increased, the flow of information has become complicated, and performance evaluations at the time of system design have been gradual. It's getting harder.

In addition, the scale of the software system is increased,
With the increase in complexity, the review of the performance at the final stage of the software manufacturing process has become a cause not only of the difficulty of identifying countermeasure items but also of a large amount of backtracking work of the design.

This is a serious problem in system design / manufacturing work since not only software revision but also hardware revision may occur.

However, at present, an effective performance evaluation method in the upstream process (initial process) has not been provided.

[0007]

The basic problems of the above-mentioned conventional methods are that the management of the performance is insufficient due to the increase of reliability parameters, and that the response request items due to the complicated information flow are effective. That is, there is no proposal of a method for evaluating excellent processability and responsiveness.

An object of the present invention is to provide a performance evaluation apparatus and an evaluation method capable of performing performance evaluation in an upstream process (initial process) of a large-scale distributed system.

[0009]

In order to achieve the above object, the present invention provides a performance evaluation apparatus for evaluating the performance of the computer system having a plurality of processors in terms of processability and responsiveness. And input / output means for inputting parameters relating to responsiveness and responsiveness, and a reliability parameter for storing the parameters in a reliability parameter table.
Data storage means, reliability parameter management means for registering and retrieving the parameters as needed, and evaluation of the processability from the steady-state CPU load factor of each processor based on the parameters. And a performance evaluation unit that evaluates the responsiveness.

Another feature of the present invention is a performance evaluation method for evaluating the performance and responsiveness of a computer system having a plurality of processors, wherein parameters relating to the operability and responsiveness of the computer system are input in advance. Input / output step and reliability parameter
A reliability parameter storing step for storing each parameter, a reliability parameter managing step for registering / retrieving the parameter as required, and a CPU in a steady state for each processor based on the parameter. It is provided with a performance evaluation step of evaluating the processability from the load factor and evaluating the responsiveness.

[0011]

According to the present invention, the input / output means inputs parameters relating to the processability and responsiveness of the system with a keyboard, a mouse or the like in advance. The reliability parameter storage means stores parameters that are input, for example, parameters that are unique to each processor, I / O, and package. Reliability parameter
The parameter management means registers / searches the parameters stored in the reliability parameter storage means as needed. The performance evaluation means is based on the parameters and is a C at steady state for each processor.
The processability is evaluated from the PU load factor and the responsiveness of the response request items required by the system is evaluated. This makes it possible to evaluate the CPU load factor for each processor, which is the processability of a large-scale, distributed system, and the response time required for the system, which is responsiveness, in the upstream process (initial process) during system design. Becomes

[0012]

EXAMPLE A performance evaluation apparatus and an evaluation method according to an example of the present invention will be described below.

FIG. 1 shows the functional arrangement of a performance evaluation apparatus for a computer system according to an embodiment of the present invention.

The performance evaluation apparatus 1 has an input / output means 2 for inputting parameters relating to system processability and responsiveness in advance with a keyboard, a mouse and the like, and input parameters such as processor, I / O and package. A reliability parameter storage means 5 for storing unique parameters in the reliability parameter table 51, and a reliability parameter management means 3 for registering and retrieving parameters as necessary. It is composed of a performance evaluation means 4 which evaluates the processability from the steady-state CPU load factor of each processor based on the parameters and evaluates the response of the response request items required by the system.

FIG. 2 shows a table structure of the reliability parameter table 51 of FIG.

The reliability parameter table 51 is for each processor, I / O, and package that constitute the distributed system, and the parameter name and parameter relating to the corresponding processability and responsiveness.
For example, in the processor 1P ₁ , the processing time α ₁ per instruction, the processing time β ₁ per step, the number of instructions γ ₁ per step, and the parameter value δ _{1 of the} OS macro processing time are stored. Is stored.

These can be registered / retrieved from the input / output means 2 as needed, and may be automatically referred to by the responsiveness evaluation means 4.

FIG. 3 shows a general definition of the CPU load factor relating to the processability at the time of system design. The CPU load factor F of the processes 1 to 5 is expressed by the following formula.

[0019] CPU load ratio _{F = {(H L -H i} ) / H L} × 100 H L: measurement time period H _i: if sum where H _L the unit time of the idle time of the set time period H _i is the sum of the idle time in the corresponding unit time zone, and the idle time indicates the time when the CPU is not executing the OS and the user program. The CPU load factor value is used to evaluate the processability of each processor.

FIG. 4 shows the setting of the target value of the CPU load factor for each processing group of the steady processing in each processor. The CPU load factor is divided into steady processing and non-steady processing for each of the processors P ₁ to P _S , and the processing group of steady processing 1
The CPU load factor target values M ¹ ₁ , M ¹ ₂ and M ¹ ₃ are set for ^every ₂ and ₃ and the total sum M ^l _t of the target values of the entire steady process is set.

FIG. 5 shows the setting of the target value T of the response time for the response request items of the distributed system. The response request items required for the distributed system are not identified for each processor, the target value T of the response times is set, and the response times are evaluated by comparing with the calculated response time K. is there. For example, when the target value T of the response time is larger than or equal to the calculated value K of the response time in item 1, the CPU load factor becomes 0%. The response time for the response request item is the time until the screen is displayed in response to the input by the user request, the request time unique to the equipment and the device, and the like.

By setting the target value of the CPU load factor shown in FIG. 4 and the target value of the response time shown in FIG. 5, the performance can be evaluated in the upstream process (initial process) of system design. .

FIG. 6 shows the CP for each processing group of the steady processing.
The relationship between the U load factor target value and the CPU load factor calculated value is shown.
Further, FIG. 7 shows a CPU of each process in the process group of FIG.
The calculated load factor is shown. That is, with respect to the CPU load factor target value M for each processing group of the steady processing for each first level processor in FIG. 6, activation for each processing included in the corresponding processing group for the second level in FIG. Frequency n, number of execution steps s,
The CPU load factor f is calculated according to the number of times the OS macro is issued m, and the CPU load factor f is summed up.
This shows that the CPU load factor calculation value F is calculated for each group and compared with the CPU load factor target value M to be automatically evaluated. Also, other than the boot frequency n, execution step s, and OS macro issue count m, other CPU load consumption items are added to the CP.
There is also a case where the U load factor f is calculated and they are summed.
It is also possible to evaluate each case by dividing the starting frequency n, the execution step s, etc. at this time into peaks of steady processing and cases other than peaks.

Here, as shown in FIG. 6, when the comparison result of the target value M of the processing group 2 and the calculated value F is M ^l ₂ ≧ F ^l ₂ , the evaluation result is that the calculated value F is within the target value M. And the result is OK.

Further, processing 1 to processing u of processing group 2
CPU load factor f of FIG. 7 is as shown in FIG. 7, and at this time, the processor 1P ₁ of the reliability parameter table 51 of FIG.
The processing time β ₁ per step and the OS macro processing time δ ₁ are automatically referred to and calculated. The sum of them becomes the calculated value F ^{12 of} the processing group ₂ and is compared with the target value for each processing group.

Further, the total of the processing groups 1 to q in FIG. 6 becomes the total F ¹ _t of the CPU load factor calculation values of the steady processing,
By comparing with the total M ¹ _t of the target values, it becomes possible to evaluate the entire CPU load factor of the steady process. Here, t indicates total, and 1 indicates the processor 1P ₁ .

Further, as shown in FIGS. 6 and 7, the processing of the processors constituting the distributed system is divided into the second level from top to bottom, and the CPU load factor of the CPU load factor is set according to the target value for each processing group. For example, if the processing group 1 in FIG. 1 is an OS, the processing group 2 is a common platform or communication processing, and the processing group 3 is an application in FIG. For example, regarding the processing group 3,
It is also possible to group and evaluate up to the level.

In addition, when the processing inputs and outputs with I / O,
Even if a soft package is used, it can be dealt with by changing the calculation formula of the load factor for the processing.

It is also possible to determine the CPU load ratio for one process of a certain process group from the target value of the process group and calculate the execution step backwards. It is also possible to provide a design guide such as how many steps the execution steps of the corresponding process must be designed to achieve the rate target value.

FIG. 8 shows a flowchart of the processing of the performance evaluation means.

The processing procedure will be described below.

Step S005: Processor P = 1 to s
Repeat the process for.

Step S010; processing group j = 1 to 1
Repeat the process for q.

Step S015: Processing i within processing group
The process is repeated for = 1 to u.

Step S020: cpu for process i
Calculate the load factor. (F _i ) f _i = n _i * (s _i * β ₁ + m _i * δ ₁ ) where n _i ; activation frequency of process i s _i ; execution step of process i m _i ; OS macro issue of process i Number of times β ₁ ; processing time per step δ ₁ ; OS macro processing time Step S025; cpu load factor of the corresponding processing i is added.

F ^P _j = F ^P _j + f _i (where i = u) Step S030: The cpu load factor (F ^P _j ) of the corresponding processing group j is added.

F ^P _t = F ^P _t + F ^P _j (where j = 1 ... q) Step S035; cpu load factor target value (M ^P _j ) and cpu load factor calculated value (M ^p _j ) of the processing group j for the corresponding processor ( Compare F ^P _j ).

Input and output cpu load factor calculated value of the corresponding processing group j of the corresponding processor evaluation NG; [0038] _{^{(1) M P j ≧ F}} P j ⇒ rated _{^{OK (2) M P j <}} F P j ⇒ evaluation NG step S040 Output to the device.

Display contents; evaluation NG result, processor N
o, processing group number, cpu load factor target value, cpu load factor calculated value Step S045: The cpu load factor target value (total) of the corresponding processor is compared with the cpu load factor calculated value (total).

(1) M ^P _t ≧ F ^P _t ⇒ evaluation OK (2) M ^P _t <F ^P _t ⇒ evaluation NG Step S050; cpu load factor calculation value (total) evaluation NG of the corresponding processor to the input / output device Output.

Display contents; evaluation NG result, processor N
o, cpu load factor target value (total),
cpu load factor calculated value (total) Step S055; The process is repeated for the response request items k = 1 to r.

Step S060: Processor P = 1 to s
Repeat the process.

Step S065: The corresponding response time t ^P _k in the corresponding processor is _obtained .

T ^P _k = s * β ₁ + m * δ ₁ where s: total number of execution steps for the response request item m; OS macro issue count for the response request item β ₁ ; processing time per step δ ₁ ; OS macro processing time k: Corresponding response request item No Step S070; Add response time for processors P = 1 to s.

K _k = K _k + t ^P _k / {1- (F ^P _t / 100)} Step S075; the response time target value (T _k ) and the response time calculation value (K _k ) for the corresponding response request item Nok are calculated. Compare.

(1) T _k ≧ K _k ⇒ evaluation OK (2) T _k <K _k ⇒ evaluation NG Step S080; The response time calculated value evaluation NG of the corresponding response request item Nok is output to the input / output device.

Display contents; evaluation NG result, response request item N
o, response time target value, response time calculation value CPU from step S005 to step S050
This is a logic for evaluating each load rate processing group and each processor. When the evaluation is NG in step S040, which processing group and which processing group has become NG are output to the screen. It is like this.

If the evaluation is NG in step S050, the steady processing target value is overwritten by which processor.
It will be displayed on the screen whether or not it became NG.

From step S055 to step S080, the response time calculation value for each processor is calculated for each response request item, and the response time calculation value for the response request item is obtained by summing the response time calculation values for each relevant processor. It is a logic to evaluate by comparing with the response time target value, and when the evaluation is NG in step S080,
Which response request item is NG is output on the screen, and it can be notified that the related process or the flow of information needs to be reviewed.

FIGS. 9 and 10 show a calculation method for obtaining the calculated response time K ₁ with respect to the target value T ₁ of the response time of the response request item 1 in FIG.

FIG. 9 shows the target processing for the response request item 1 from the processor 1P ₁ to the processor SP _S.

The terminator of P ₁ indicates an input device TN ¹ ₁ from the outside and an output device TN ¹ ₂ to the outside. A circle indicates a process, an arrow indicates a data flow between the processes, and a double line. ○
A mark indicates a process related to the response request item 1.

The same applies to P _S. However, the connection of related processes and the connection of processors differ depending on the response request item.

FIG. 10 shows only the processing related to the response request item 1, the total number of execution steps s for the response request item 1 of P _{1 and} the number of OS macro issuances m.
Is calculated as follows.

Total number of execution steps s = number of execution steps of processing 1 s ₁ + number of execution steps of processing 2 s ₂ + number of execution steps of processing 3 s ₃ Total of number of times OS macro issuance m = issuance of os macro of processing 1 Number of times m ₁ + os macro of processing 2 Number of times m ₂ + o of processing 3
s Number of macro issuing times m ₃ Number of execution steps for response request item 1 of P _S s
And the total number m of times the OS macro is issued are calculated by the following.

[0056] Total = process 1 of OS macro total = P _S of the processing 1 executed step number + P _S of the processing 2 executed step number + P number of execution steps through 3 _S OS macro issue count m execution step number s in OS macro issue count of OS macro issuing times + processing 3 of issuing times + treatment 2 where, when the response time t ¹ ₁ at the cpu load factor 0% P _1, cpu load factor F response time at ¹ _t% Is as follows.

K ¹ ₁ = t ¹ ₁ / {1- (F ¹ _t / 100)} Further, when the response time t ^s ₁ at the cpu load rate of P _S is 0%,
cpu load factor F ^s response time in the _t% is the following.

[0058] ^{_{k s 1 = t s 1 /}} {1- (F s t / 100)} Further, the target value of the response time to the response request item 1 in cpu load of 0% for each processor, T ₁ = t ¹ ₁ ＋… ＋
This is as indicated by t ^s ₁ .

The calculated response time for the response request item 1 at the cpu load factor F ^P _t % of each processor is K ₁ = k ¹ ₁
It is as shown by +… + k ^s ₁ .

[0060]

According to the present invention, the performance can be evaluated particularly in the upstream process (initial process) of a large-scale distributed system, so that the efficiency of system design / manufacturing work can be greatly improved.

[Brief description of drawings]

FIG. 1 is a diagram showing a functional configuration of a performance evaluation apparatus according to an embodiment of the present invention.

FIG. 2 is a diagram showing a table configuration of a reliability parameter table 51 of FIG.

FIG. 3 is a diagram illustrating a general definition of a CPU load rate related to processability during system design.

FIG. 4 is an explanatory diagram of setting a target value of a CPU load factor for each processing group of steady processing.

FIG. 5 is an explanatory diagram of setting target values of response times of response request items of the distributed system.

FIG. 6 is a diagram showing a relationship between a CPU load factor target value and a CPU load factor calculated value for each processing group of steady processing.

7 is a diagram showing CPU load factor calculation values of respective processes in the process group of FIG.

FIG. 8 is a flowchart of the processing of the responsiveness evaluation means of FIG.

FIG. 9 is a diagram showing a target process relating to a response request item 1 from the processor 1 to the processor S.

FIG. 10 is a diagram showing a calculation method in the case of obtaining a calculated value K ₁ of a response time with respect to a target value T _{1 of} a response time of a process related to the response request item 1.

[Explanation of symbols]

1 ... Performance evaluation device, 2 ... Input / output means, 3 ... Reliability parameter management means, 4 ... Performance evaluation means, 5 ... Reliability parameter
Data storage means, 51 ... reliability parameter table

Claims (6)

[Claims] 1. A performance evaluation apparatus for evaluating the performance and responsiveness of a computer system having a plurality of processors, comprising: input / output means for inputting parameters relating to the operability and responsiveness of the computer system in advance; To a reliability parameter table, reliability parameter storage means for storing and searching the parameters as necessary, and the above-mentioned parameters based on the parameters. A performance evaluation apparatus comprising: a performance evaluation unit that evaluates the processability from the CPU load factor in a steady state of each processor and evaluates the responsiveness. 2. The performance evaluation apparatus according to claim 1, wherein the parameter is a parameter unique to each processor, each I / O, and each package. 3. The performance evaluation apparatus according to claim 1, wherein the performance evaluation of the processability and responsiveness is performed in an upstream process of the computer system. 4. A performance evaluation method for evaluating the performance and responsiveness of a computer system having a plurality of processors, the input / output step of previously inputting parameters relating to the operability and responsiveness of the computer system, and the parameter. To a reliability parameter table, a reliability parameter storing step, a reliability parameter managing step for registering / retrieving the parameter as needed, and the above-mentioned parameters based on the parameters. A performance evaluation method comprising: a performance evaluation step of evaluating the processability and the responsiveness from a CPU load factor in a steady state for each processor. 5. The performance evaluation method according to claim 4, wherein the parameter is a parameter unique to each processor, each I / O, and each package. 6. The performance evaluation method according to claim 4, wherein the performance evaluation of the processability and responsiveness is performed in an upstream process of the computer system.