JP5636922B2 - Performance prediction apparatus, performance prediction method, and program - Google Patents

Description

  The present invention relates to a technology for predicting the performance of an information processing system to be measured, and particularly relates to a performance prediction apparatus, a performance prediction method, and a program for accurately predicting a specific region of a performance index.

  In constructing an information processing system, in order to determine whether the system to be constructed satisfies the required performance, it is predicted in advance how much the system has. Further, the performance prediction of the information processing system to be measured is predicted based on a model prepared in advance.

  Performance prediction based on a model is predicted, for example, in Patent Document 1 by converting performance indexes such as throughput, response, and resource usage of a parallel computer in a multitask environment by converting the system configuration of the parallel computer into a queuing model. Means have been proposed.

  In the prediction based on a model such as a queuing model as described above, it is assumed that a parameter related to the IT system to be constructed, such as a processing time (Demand) per request, is given in advance. Predict.

  However, when the performance of the IT system to be built is predicted using parameters given in advance, the parameter values given in advance may differ from the parameter values given to the IT system actually built. Therefore, there is a problem that the performance index predicted using the model is different from the performance index of the IT system actually constructed.

  Therefore, for example, Non-Patent Document 1 corrects the above parameters by feeding back an actually obtained performance index to a model using a Kalman filter for an operating IT system. A method for improving the prediction accuracy has been proposed.

  Patent Document 2 proposes a method for optimizing measurement points for measuring a performance index in order to predict performance. Specifically, the performance index predicted from the model to be controlled and the predicted performance index are sampled at the set measurement points, and the error between the sampled performance index is minimized. The arrangement of measurement points is optimized.

JP 2000-298593 A JP 2009-258068 A

Tao Zheng et al., “Performance Model Estimation and Tracking Using Optimal Filters”, IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. 34, NO. 3, pp 391-406, 2008

In order to judge whether the built information processing system meets the required performance, it is important to accurately predict how much performance the system has for each request and situation. It is.
For example, in an IT system, when a contract is made so that the performance index falls within a certain range, such as SLA (Service Level Agreement), it is necessary to determine the condition that the performance index does not satisfy the SLA. is there.

However, the method described in Patent Document 1 is a method performed in advance for the information processing system to be constructed, and the methods described in Patent Document 2 and Non-Patent Document 1 are intended to improve the accuracy of the entire model. For example, it does not contribute to the purpose of accurately predicting a pinpoint area such as a condition that does not satisfy the SLA as described above.
That is, with the methods described in Patent Document 1, Patent Document 2, and Non-Patent Document 1, it is possible to accurately predict how much performance the system has by paying attention to a specific performance index region. Can not.

  Accordingly, the present invention provides a performance prediction apparatus, method, and program capable of accurately predicting a specific region of a performance index for an information processing system to be measured in order to solve the above-described problem. The purpose is to do.

  The performance prediction apparatus according to the present invention is a model storage unit that stores a model of an input-output relationship related to an information processing system to be measured, and is a target for prediction when predicting the performance of the information processing system. Using a specific output acquisition means for acquiring a specific output and a model of the information processing system stored in the model storage means, an input corresponding to the specific output acquired by the specific output acquisition means is calculated, Based on the input corresponding to the specific output, the input content determining means for determining the input content, and the measurement target system for executing the information processing system so that the input content determined by the input content determining means is input. An information collecting means for collecting an input of the information processing system executed by the execution means and the measurement target system executing means, and an output corresponding to the input; A model fitting means for fitting a model acquired from the model storage means based on the input of the information processing system collected by the information collecting means and an output corresponding to the input, and fitting by the model fitting means And a performance predicting means for predicting the performance of a specific output as a prediction target for the information processing system based on the new model.

  ADVANTAGE OF THE INVENTION According to this invention, the performance prediction apparatus, the performance prediction method, and program which can predict the specific area | region of a performance parameter | index with sufficient precision with respect to the information processing system made into measurement object can be provided.

It is a figure which shows the structural example of this invention. It is a block diagram which shows the function structural example of the performance prediction apparatus by this invention. It is a flowchart which shows the example of process progress which concerns on the 1st Embodiment of this invention. It is explanatory drawing which shows an example of the input with respect to the model of the relationship between the input and output of a system, and a specific output. FIG. 5 is an explanatory diagram illustrating an example in which the vicinity of an input u_1 with respect to a specific output is represented on the uy plane of FIG. 4. It is explanatory drawing which shows an example which represented the model h '(u) fitted to the input of the measurement object system 2, and the output corresponding to an input with the black point, and fitted according to the input / output on the ui plane. It is a block diagram which shows the function structural example of 2nd Embodiment of the performance prediction apparatus by this invention. It is a flowchart which shows the example of the process progress which concerns on the 2nd Embodiment of this invention. Calculated based on the model h (u) before fitting, the input u_1 for the specific output y_1 calculated based on h (u), the model h (u) ′ fitted, and h (u) ′ It is explanatory drawing which shows an example which represented input u_2 with respect to specific output y_1 on the ui plane. The input / output of the measurement target system 2 is represented by black dots, and u_1 + ε_A is input from the input u_1 to the specific output y_1 calculated based on the models h (u) and h (u) and u_1−ε_B in the vicinity of u_1. It is explanatory drawing which shows an example represented to the plane. The input / output of the measurement target system 2 is represented by black dots, and u_1 + ε_A is input from the input u_1 to the specific output y_1 calculated based on the models h (u) and h (u) and u_1−ε_B in the vicinity of u_1. It is explanatory drawing which shows an example represented to the plane. It is a block diagram which shows the function structural example of 3rd Embodiment of the performance prediction apparatus by this invention. It is a flowchart which shows the example of process progress which concerns on the 3rd Embodiment of this invention. The input / output of the measurement target system 2 is represented by black dots, the model h (u) before fitting, the input u_1 for the specific output y_1 calculated based on h (u), the fitted model h (u) ′, and , H (u) ′ is an explanatory diagram showing an example in which the input u_2 for the specific output y_1 calculated based on h (u) ′ is represented on the ui plane.

  Embodiments of the present invention will be described below with reference to the drawings.

[Basic configuration]
FIG. 1 is a diagram showing a configuration example of the present invention. In FIG. 1, a performance prediction apparatus 1 is connected to a measurement target system 2 that is a measurement target via a communication network 3 so as to be communicable. The performance prediction apparatus 1 is an information processing apparatus that operates according to a program such as a server or a personal computer. The measurement target system 2 is an information processing system configured by an information processing apparatus that operates according to a program such as one or a plurality of servers or personal computers. In addition, when comprised by several information processing apparatus, the communication between them may be performed via the communication network 3, and may be performed with the communication cable etc. which were directly connected. . The communication network 3 may be the Internet, for example, or may be a LAN (Local Area Network), but is limited to these as long as it enables communication between the performance prediction apparatus 1 and the measurement target system 2. It is not a thing.

[First Embodiment]
FIG. 2 is a block diagram showing an example of a functional configuration in the first embodiment of the performance prediction apparatus according to the present invention. As shown in FIG. 2, the performance prediction apparatus 1 according to the present embodiment includes a model accumulation unit 101, a specific output acquisition unit 102, a performance prediction unit 103, an input content determination unit 104, and a measurement target system execution unit 105. , Information collecting means 106 and model fitting means 107 are included.

  The model storage unit 101 has a function of storing, for example, a model of a relationship between input and output of a performance index of an information processing system to be measured in a storage medium such as an HDD (HARD DISK DRIVE) provided in the computer. ing. In this embodiment, a model related to the input / output relationship of the measurement target system 2 is stored in advance. Here, the input is, for example, the number of requests that the system must process within a unit time, and the output is, for example, the system throughput or response time, but is not limited thereto. Instead, if the model can describe the relationship between an independent variable and a dependent variable, the independent variable may be input and the dependent variable may be output.

  The specific output acquisition unit 102 has a function of acquiring a specific output to be predicted from an input device such as a keyboard or a storage medium such as an HDD provided in a computer or the like in accordance with a user operation. ing. The specific output is input to the performance prediction apparatus 1 as a reference for performing performance prediction via the specific output acquisition unit 102, and is handled as a specific performance index region of interest of the measurement target system 2. Note that the specific output acquisition unit 102 may acquire, for example, from a server via a communication network such as the Internet.

The performance prediction unit 103 predicts the performance of a specific output as a prediction target for the output to be predicted obtained from the specific output acquisition unit 102 based on the model stored in the model storage unit 101, and It has a function to output the result. Further, as will be described later, the performance predicting unit 103 predicts the performance of a specific output targeted for prediction for the measurement target system 2 based on the new model fitted by the model fitting unit 107, and the result thereof. Has a function of outputting. A new model may be obtained directly from the model fitting unit 107 or may be acquired via the model storage unit 101.
Note that when the model is fitted by performing the processing described later by the performance predicting unit 103, the prediction accuracy is improved as compared with the case where the performance is predicted based on a model that is held in advance.

  Specifically, the input content determination unit 104 is information that operates in accordance with a performance prediction program (hereinafter referred to as a program) that is read from a storage unit such as a ROM (READ ONLY MEMORY) and expanded in a RAM (RANDOM ACCESS MEMORY). This is realized by a CPU (CENTRAL PROCESSING UNIT) which is a control unit of the processing apparatus. The input content determination unit 104 uses the model of the relationship between the input and output of the measurement target system 2 stored in the model storage unit 101 and uses the measurement target system 2 corresponding to the specific output acquired by the specific output acquisition unit 102. And a function for determining the input content for causing the measurement target system execution means 105 to output the input based on the calculated input. The input content is used to increase the accuracy of prediction of an input corresponding to a specific output, and the determination method will be described later.

Specifically, the measurement target system execution unit 105 is realized by a CPU of an information processing apparatus that operates according to a program. The measurement target system execution unit 105 has a function of accessing the measurement target system 2 via the communication network 3 and operating the measurement target system 2 so that the input content determined by the input content determination unit 104 is input. .
The output from the measurement target system execution unit 105 at this time is an input corresponding to the specific output calculated by the input content determination unit 104 so as to be input to the measurement target system 2.

  Specifically, the information collecting unit 106 is realized by a CPU of an information processing apparatus that operates according to a program. The information collection unit 106 accesses the measurement target system 2 via the communication network 3 and collects the input to the measurement target system 2 executed by the measurement target system execution unit 105 and the output corresponding to the input. It has. Note that the input to the measurement target system 2 may be acquired from the measurement target system execution unit 105.

  Specifically, the model fitting unit 107 is realized by a CPU of an information processing apparatus that operates according to a program. The model fitting unit 107 has a function of fitting the model acquired from the model storage unit 101 based on the input of the measurement target system 2 collected by the information collecting unit 106 and the output corresponding to the input. Here, fitting means correcting a model of the measurement target system 2 prepared in advance so as to match the actual measurement value. That is, it means that a new model (corrected model) can be provided to the performance prediction means 103 by correcting the parameters of the model so that the model is matched with the actually measured input and output.

  The new fitted model may be sent directly from the model fitting unit 107 to the performance prediction unit 103 or may be sent to the performance prediction unit 103 via the model storage unit 101. Further, a parameter relating to correction may be acquired from the model fitting unit 107 from the performance prediction unit 103 side, and the original model may be acquired from the model storage unit 101 and used as a new fitted model.

  In the description of the embodiment, various means generate data such as input contents, but it is obvious that the generated data is appropriately recorded in a storage medium such as a RAM or an HDD and is readable. Will.

  In addition to the various means described above, it goes without saying that a storage means for storing and holding the created data and a display means for displaying the created data on the screen are provided as necessary.

  Next, the operation of the performance prediction apparatus 1 will be described. FIG. 3 is a flowchart illustrating a processing example of the performance prediction apparatus 1.

  The specific output acquisition unit 102 acquires a specific output to be predicted in accordance with, for example, a user input operation (step S11). The specific output acquisition unit 102 may acquire a specific output in accordance with a command from a server or the like via a communication network such as the Internet.

  Next, the input content determination unit 104 uses the model of the relationship between the input and output of the measurement target system 2 stored in the model storage unit 101 and inputs corresponding to the specific output acquired by the specific output acquisition unit 102. Is calculated (step S12).

Here, FIG. 4 shows an example of a model of the relationship between the input and output of the measurement target system 2 to be used and an input for a specific output. In the example shown in FIG. 4, a curve obtained by plotting a model y = h (u) of the relationship between the input u and the output y on the uy plane, a specific output y_1, and an input u_1 corresponding thereto are shown.
Note that the model h associates the output y with the input u. For example, h (u) = a * exp (b * u) (where a and b are parameters given in advance). It may be expressed by a mathematical expression related to u. Alternatively, it may be a queuing network and is not limited to these.
In the following description, the input and output are each assumed to be one variable, but it goes without saying that the present invention can be easily applied even if they are multivariable.

  Based on the model h, the method for calculating the input u_1 corresponding to the specific output y_1 is, for example, if the model h is expressed by a mathematical expression related to u as described above and its inverse function can be obtained. You may obtain | require from an inverse function. Or in the case of more general h, you may obtain | require by numerical calculation.

  Next, the input content determination unit 104 determines the input content used to increase the prediction accuracy of the input u_1 corresponding to the specific output y_1 based on the input u_1 corresponding to the specific output y_1 calculated in step S12. Determine (step S13). The determination of input contents will be described in detail below.

  The input u_1 calculated in step S12 is based on the model, and the actual system does not always output the output y_1 to the input u_1. However, it can be expected that the actual system input for output y_1 will be approximately in the vicinity of u_1. Therefore, if the vicinity of u_1 is determined as the input content, it is expected to meet the purpose of predicting the input to the output y_1 more accurately.

  FIG. 5 shows an example in which the vicinity of u_1 is represented on the ui plane of FIG. In the example shown in FIG. 5, the range from u_1-ε_B to u_1 + ε_A represents the vicinity of u_1. The input content determination unit 104 may determine ε_A and ε_B as, for example, several percent of u_1. Alternatively, an input corresponding to plus or minus several% of y_1 may be calculated from the model, and each may be determined corresponding to ε_A and ε_B. The determination of the range is not limited to these methods, and other methods may be used. Further, the range determination method may be automatically selected for each type of specific output acquired.

  The input content determination means 104 selects some input values in the vicinity of u_1 and determines the actual input content. Here, the method of selecting the input value may be a value obtained by equally dividing the vicinity of u_1 by a predetermined number. Alternatively, the input value intervals may be biased according to a predetermined algorithm such that the input value is biased toward the smaller or larger u value. At this time, u_1 itself may or may not be included as an input value.

  Next, the measurement target system execution unit 105 accesses the measurement target system 2 via the communication network 3 and operates the measurement target system 2 so that the input content determined by the input content determination unit 104 is input (step). S14). Specifically, the measurement target system execution unit 105 automatically generates a request that has the determined input content, and causes the measurement target system 2 to execute the request process. In addition, when the measurement target system 2 is a plurality of servers, etc., a request corresponding to each role is sent to the required server in response to the operation requested to the measurement target system 2 and the process is executed. Let

  Next, the information collection unit 106 accesses the measurement target system 2 via the communication network 3 and collects the input of the measurement target system 2 executed by the measurement target system execution unit 105 and the output corresponding to the input. (Step S15).

Next, the model fitting unit 107 fits the model h acquired from the model storage unit 101 based on the input of the measurement target system 2 collected by the information collecting unit 106 and the output corresponding to the input, and creates a new one. A model h ′ is generated (step S16).
For example, when the model acquired from the model storage unit 101 is h (u) = a * exp (b * u), the fitting method is the measurement collected by the information collection unit 106 for this model h (u). The parameters a and b are corrected so as to minimize the error with the input / output of the target system 2.
Examples of the method for minimizing the error include, but are not limited to, a method using a least square method and a genetic algorithm.
FIG. 6 shows an example in which the input of the measurement target system 2 and the output corresponding to the input are represented by black dots, and the model h ′ (u) fitted according to the input / output is represented on the uy plane.

  Next, the performance predicting unit 103 acquires a new model h ′ fitted by the model fitting unit 107, and predicts the performance of a specific output as a prediction target for the measurement target system 2 based on the model h ′. The result is output (step S21).

  Next, the effect of this embodiment will be described.

As described above, in the present embodiment, the model fitting unit 107 is acquired from the model storage unit 101 based on the input of the measurement target system 2 collected by the information collection unit 106 and the output corresponding to the input. Fit the model.
Therefore, according to the present embodiment, a model closer to the actual situation can be obtained by fitting parameters such as a and b included in the model h using the correspondence between the input and output of the measurement target system 2. From the model, an input corresponding to a specific output with higher accuracy than the input u_1 corresponding to the specific output obtained earlier can be obtained.

  Further, according to the present embodiment, only the input / output in the vicinity of the input u_1 corresponding to the specific output to be predicted is measured by the selection of the input content determination unit 104, so that the effect of reducing the number of times of input and output measurement is obtained. I can expect. In particular, the effect is significant when either input, output, or both become multivariable. That is, according to the present embodiment, in comparison with the technique for measuring input / output over the entire range when fitting a model, only the input / output in the vicinity of the input u_1 is actually measured, thereby specifying the number of measurements. The input corresponding to the output of can be predicted with the same accuracy.

[Second Embodiment]
FIG. 7 is a block diagram showing a functional configuration example of the second embodiment of the performance prediction apparatus according to the present invention. As shown in FIG. 7, the performance prediction apparatus 1 according to the present embodiment includes a model storage unit 101 ′, a specific output acquisition unit 102, a performance prediction unit 103, an input content determination unit 104 ′, and a measurement target system execution unit. 105, an information collecting unit 106, a model fitting unit 107, and a repetition control unit 108.

Description of the same parts as those in the first embodiment is simplified or omitted, and differences from the first embodiment will be described.
The model storage unit 101 ′ has a function of storing a model of the relationship between the input and output of the performance index of the system to be measured, for example, in a storage medium such as an HDD provided in a computer or the like. Further, the model storage unit 101 ′ has a function of storing the model fitted by the model fitting unit 107 as a new model.

  Specifically, the input content determination unit 104 ′ is realized by a CPU of an information processing apparatus that operates according to a program. The input content determination unit 104 ′ uses the model of the relationship between the input and output of the system stored in the model storage unit 101 ′ to input the input to the system corresponding to the specific output acquired by the specific output acquisition unit 102. A function of calculating and determining input contents for causing the measurement target system execution means 105 to output the input based on the input corresponding to the input of the measurement target system 2 already collected by the information collection means 106 in addition to the calculated input. It has.

Specifically, the repetition control means 108 is realized by a CPU of an information processing apparatus that operates according to a program.
The iterative control unit 108 receives the fact that the model storage unit 101 ′ has stored the model fitted by the model fitting unit 107 as a new model, and thereafter the input content determination unit 104 ′ based on the new model. The input corresponding to the specific output (the specific output already acquired by the specific output acquisition unit 102) is calculated again to determine the input content, the measurement target system execution unit 105 is operated according to the determined input content, and the information The collection unit 106 collects the input from the measurement target system 2 and the output corresponding to the input, and repeats a series of processes of re-fitting the new model used based on the input / output collected by the model fitting unit 107. Or whether to repeat the process It has a function that.
In this embodiment, the case where the repetition control unit 108 has a function of determining whether to repeat a series of processes based on the accuracy of the model fitted by the model fitting unit 107 will be described below. However, the reference for the repetition control means 108 to determine repetition may be, for example, whether or not the repetition control unit 108 has been repeated a predetermined number of times, and is not limited to this example.

  Next, the operation of the performance prediction apparatus in this embodiment will be described. FIG. 8 is a flowchart illustrating a processing example of the performance prediction apparatus. The same steps as those in the first embodiment described with reference to FIG. 3 are denoted by the same step numbers, and description thereof is omitted.

  The model storage unit 101 'stores the model fitted by the model fitting unit 107 as a new model (step S17).

  If the accuracy of the new model fitted by the model fitting unit 107 is sufficient, the iterative control unit 108 proceeds to the performance prediction process, and if not, moves the process to step S12 (step S18).

The accuracy of the new model fitted here is the value of the input for the specific output calculated based on the new model fitted and the input for the specific output calculated based on the original model before fitting. May be based on the difference.
FIG. 9 shows an input u_1 for a specific output y_1 calculated based on the original models h (u) and h (u) before fitting, a new model h (u) ′ fitted, and h (u) An example is shown in which the input u_2 for the specific output y_1 calculated based on 'is represented on the uy plane. If the difference between u_1 and u_2 is smaller than a value given in advance, the accuracy of the new model h (u) ′ is sufficient. If the difference is not small, the accuracy of the new model h (u) ′ is not sufficient. And Alternatively, in addition to the difference between u_1 and u_2, it may be based on an error between the input and output of the measurement target system 2 collected by the information collecting unit 106 for the new fitted model h (u) ′. In this way, it is possible to consider whether the new model correctly reflects the input / output relationship not only at one point of input to the specific output y_1 but also in the vicinity thereof. In addition, when the fitting is repeated two or more times, specific differences between a plurality of models, such as the difference between the first value and the first value, and the difference between the first value and the second value are used. The value for the output may be used to determine whether the accuracy of the last fitted model (new model) is sufficient based on the threshold value and the degree of convergence. Any of the determination methods is particularly suitable when the model takes a complicated shape.

  The input content determination unit 104 ′ specifies the input based on the input corresponding to the specific output calculated in step S12, the input of the measurement target system 2 collected by the information collection unit 106, and the output corresponding to the input. The input content used for improving the accuracy of the prediction of the input corresponding to the output is determined (step S13 ′). Note that when the process of step S13 ′ is performed for the first time, since the model has not yet been fitted, input is performed based on the input corresponding to the specific output calculated in step S12, as in step S13 of the first embodiment. The contents shall be determined. The following is a case where the process of step S18 is performed at least once and the process returns to step S12.

Here, in FIG. 10, the input / output of the measurement target system 2 is represented by black dots, and the model h (u), the input u_1 calculated based on h (u) for a specific output y_1, and the vicinity of u_1. An example in which u_1 + ε_A from −ε_B to the uy plane is shown.
When ε_A and ε_B are determined by the method described in the first embodiment, for example, in the example of FIG. Included in the range. In view of the fact that the model h (u) is fitted using the input / output of the measurement target system 2 in step S17, in the example of FIG. If the range is intensively measured, the accuracy of the next fitting is expected to increase. Another example is shown in FIG. In the example illustrated in FIG. 11, the input / output of the measurement target system 2 is not included in the range from u_1−ε_B to u_1 + ε_A. However, even in this case, the model h (u) is fitted at a large value of u. Therefore, if the range from u_1-ε_B to u_1 is measured more than the range from u_1 to u_1 + ε_A, u_1 is Since the value for fitting is also acquired on the side of the sandwiched model h (u) where the value of u is small, it is expected that the accuracy of the next fitting will be further increased. As described above, an input to be intensively measured may be determined as the input content.

  Further, in the example of FIG. 10, since u_1 is inside the input / output range of the measurement target system 2, u_1-ε_B and u_1 + ε_A are respectively set to the two input / output closest to u_1 obtained from the measurement target system 2. It may be a value. By doing so, the vicinity of u_1 can be narrowed. However, since input / output generally includes an error, the next proximity or the next proximity may be selected based on the accuracy of the model.

  Next, effects of the second embodiment will be described.

In the second embodiment, the repetition control means 108 determines whether or not to repeat a series of processes.
Therefore, unlike the first embodiment, it is expected that the accuracy of predicting an input corresponding to a specific output is increased by repeating the process. Further, it is expected that an input corresponding to a specific output can be predicted to a desired accuracy by setting the accuracy of the model fitted by the model fitting means 107 as a reference for repeating the process.

[Third Embodiment]
FIG. 12 is a block diagram showing a functional configuration example of the third embodiment of the performance prediction apparatus according to the present invention. As shown in FIG. 12, the performance prediction apparatus 1 according to the present embodiment includes a model storage unit 101 ′, a specific output acquisition unit 102, a performance prediction unit 103, an input content determination unit 104, and a measurement target system execution unit 105. And an information collecting means 106 ′ and a model fitting means 107.

Description of the same parts as those in the first embodiment is simplified or omitted, and differences from the first embodiment will be described.
The model storage unit 101 ′ has a function of storing a model of the relationship between the input and output of the performance index of the system to be measured, for example, in a storage medium such as an HDD provided in a computer or the like. Further, the model storage unit 101 ′ has a function of storing the model fitted by the model fitting unit 107 as a new model.

  Specifically, the information collecting unit 106 'is realized by a CPU of an information processing apparatus that operates according to a program. The information collection unit 106 ′ has a function of accessing the measurement target system 2 via the communication network 3 and collecting the input of the measurement target system 2 executed by the measurement target system execution unit 105 and the output corresponding to the input. I have. The information collecting unit 106 ′ has a function of accessing the measurement target system 2 via the communication network 3 and collecting an appropriate input of the measurement target system 2 and an output corresponding to the appropriate input. Further, the information collecting unit 106 ′ collects an appropriate input and an output corresponding to the input in a range (for example, a predetermined load region or a specific time zone) designated by the specific output acquiring unit 102 or the like. Also good.

  Here, the appropriate input and the output corresponding to the appropriate input may be, for example, input / output obtained when the measurement target system 2 is tested, or the measurement target system 2 is operated at a low load. Input / output etc. may be used. Appropriate inputs and outputs corresponding to appropriate inputs are not limited to these examples. Generally, there is no relationship with a specific output to be predicted. It should be obvious that there may be a relationship.

  Next, the operation of the performance prediction apparatus in this embodiment will be described. FIG. 13 is a flowchart illustrating a processing example of the performance prediction apparatus. The same steps as those in the first embodiment described with reference to FIG. 3 are denoted by the same step numbers, and description thereof is omitted.

  The information collection unit 106 'accesses the measurement target system 2 via the communication network 3, and collects an appropriate input of the measurement target system 2 and an output corresponding to the appropriate input (step S31).

The model fitting unit 107 fits the model acquired from the model storage unit 101 ′ based on the input of the measurement target system 2 collected by the information collecting unit 106 ′ and the output corresponding to the input, and generates a new model. (Step S32).
The fitting method may be performed by the method described in the first embodiment by appropriately using a least square method, a genetic algorithm, or the like.

  Here, in FIG. 14, input / output of the measurement target system 2 is represented by black dots, a model h (u) before fitting, an input u_1 calculated based on h (u) for a specific output y_1, a fitted model h An example is shown in which the input u_2 calculated based on (u) ′ and h (u) ′ for a specific output y_1 is represented on the uy plane. In the example illustrated in FIG. 14, the input / output acquired from the measurement target system 2 is a region away from the specific output y_1. However, as described above, when the parameters given in advance are different from the parameters indicated by the actually constructed IT system, the input / output is not related to the specific output y_1. However, it is useful to correct the parameters of the model, and can be expected to improve the accuracy of the input prediction for the specific output y_1 in the subsequent processing steps.

  Further, the model fitting unit 107 estimates the model h (u) acquired from the model storage unit 101 ′ based on the model h (u), the collected appropriate input and its output, and makes it a prediction target. It is good also as fitting to the model of the vicinity of a specific output.

  The model storage unit 101 'stores the model fitted by the model fitting unit 107 as a new model (step S33).

Next, effects of the third embodiment will be described.
In the third embodiment, the model fitting means 107 is acquired from the model storage means 101 ′ based on the appropriate input of the measurement target system 2 collected by the information collection means 106 ′ and the output corresponding to the input. Fit the model.
Therefore, unlike the first embodiment, since a model fitted in advance with actual input / output of the system can be used in step S12, it can be expected to improve the accuracy of input prediction for a specific output.

  Further, the plurality of embodiments can be appropriately combined. When used in combination, it is possible to perform fitting based on appropriate input / output acquired from the measurement target system 2 and to repeat the fitting appropriately.

  In this way, by operating the performance prediction device, for example, when a specific output to be predicted is in a high-load area, measurement of a high-load performance index requires a lot of time and resource consumption. Can solve easy problems.

  In the third embodiment, it is possible to improve the accuracy of prediction of a specific output in a high load region by collecting low load input / output as appropriate input / output in step S31. Can be.

Further, the input content determination means 104 determines whether or not the input specific output corresponds to the high load operation region of the evaluation target system 2, and when the input content corresponds to the high load operation region, the input / output of the low load operation region is measured. As described above, the input content to the measurement target system execution unit 105 may be determined.
At this time, instead of fitting only from the input / output at the time of low load, the measurement target system execution means and the information are acquired so as to obtain one or several points of input / output at the time of the high load under the instruction of the input content determining means. The fitting accuracy of the model can be increased by operating the collecting means and fitting the input / output at the low load and the input / output at the high load together. Note that the number of inputs / outputs acquired at high load may be determined in view of the cost required for high load measurement.

  In these operations, especially when the fitting described in the second embodiment is repeated, the number of times of repeating the high load input / output is reduced by increasing the accuracy of the model in advance by the low load input / output. Can bring about the effect of lowering the overall measurement cost.

  As described in the above embodiment, according to the present invention, an input corresponding to a specific output to be predicted from a system model is calculated for a system to be measured and determined based on the input. It is possible to provide a performance prediction apparatus, a performance prediction method, and a program for accurately predicting a specific region of a performance index by causing the system to execute the input contents and fitting a model by input / output of the system.

  It should be noted that the specific configuration of the present invention is not limited to the above-described embodiment, and changes within a range not departing from the gist of the present invention are included in the present invention.

In addition, a part or all of the above-described embodiments can be described as follows. Note that the following supplementary notes do not limit the present invention.
[Appendix 1]
Model storage means for storing a model of the relationship between input and output related to the information processing system to be measured;
When predicting the performance of the information processing system, specific output acquisition means for acquiring a specific output to be predicted;
The input corresponding to the specific output acquired by the specific output acquisition unit is calculated using the model of the information processing system stored in the model storage unit, and the input corresponding to the specific output is input. Input content determining means for determining the content;
Measurement target system execution means for executing the information processing system so as to input the input content determined by the input content determination means;
Information collecting means for collecting the input of the information processing system executed by the measurement target system executing means and the output corresponding to the input;
Model fitting means for fitting the model acquired from the model storage means based on the input of the information processing system collected by the information collection means and the output corresponding to the input;
Based on a new model fitted by the model fitting means, performance prediction means for predicting the performance of a specific output targeted for prediction for the information processing system;
A performance prediction apparatus comprising:

[Appendix 2]
The performance predicting apparatus as set forth in the above supplementary note, wherein the model storing means stores the model fitted by the model fitting means as a new model.

[Appendix 3]
The information collecting means collects an appropriate input of the information processing system and an output corresponding to the appropriate input;
The model fitting means estimates a model acquired from the model storage means based on the model, the collected appropriate input, and an output corresponding to the appropriate input, and outputs a specific output to be predicted. The performance prediction apparatus according to the above supplementary note, wherein fitting is performed to a nearby model.

[Appendix 4]
The appropriate input is an input when the information processing system has a low load,
The said specific output is an output in case the said information processing system is a heavy load, The performance prediction apparatus of the said remarks characterized by the above-mentioned.

[Appendix 5]
The input content determination means, when the specific output to be predicted is an output at the time of a high load operation, the measurement target system execution means, so that the information processing system becomes a low load operation Calculate the input as the input content,
The model fitting means, based on the input for performing the low load operation of the information processing system collected by the information collecting means, and the output at the time of the low load operation output according to the input, from the model storage means Fitting the acquired model,
The performance prediction means predicts the output performance at the time of high load operation targeted for prediction for the information processing system based on a new model that has been fitted based on the value of low load operation. The performance prediction device described in the appendix.

[Appendix 6]
The model accumulating unit accumulates the fitted model as a new model, and the input content determining unit calculates an input corresponding to a specific output acquired by the specific output unit based on the new model. The input content is determined based on the input corresponding to the output of the input, the measurement target system execution means executes the information processing system with the input content as input, and the information collection means causes the measurement target system execution means to execute. And collecting the model based on the input of the information processing system collected by the information collecting unit and the output corresponding to the input. Whether to repeat the process of fitting the new model obtained from the means Or the performance predicting apparatus according to the above supplementary note, further comprising a repetition control means for determining whether or not to repeat a series of processes.

[Appendix 7]
The performance prediction apparatus according to the above supplementary note, wherein the iterative control means determines whether to repeat or not repeat the series of processes based on the accuracy of the model fitted by the model fitting means.

[Appendix 8]
The performance prediction apparatus according to the above supplementary note, wherein the input content determination unit determines the input content based on an input of the information processing system collected by the information collection unit and an output corresponding to the input. .

[Appendix 9]
Accumulate in advance a model of the relationship between input and output for the information processing system to be measured,
Obtaining a specific output that is a target of prediction, which is a reference when predicting the performance of the information processing system;
Using the model, calculating the input corresponding to the specific output, based on the input corresponding to the calculated specific output, determining the input content to the information processing system,
Causing the information processing system to execute the input content determined as an input,
Collecting the input of the information processing system to be executed and the output corresponding to the input;
Fitting the model based on the collected input of the information processing system and the output corresponding to the input,
A performance prediction method characterized by predicting the performance of a specific output targeted for prediction for the information processing system based on a new fitted model.

[Appendix 10]
The model is accumulated by sequentially accumulating the fitted model as a new model.

[Appendix 11]
The information collection of the input and output of the information processing system collects the appropriate input of the information processing system and the output for the appropriate input,
The obtained original model is estimated based on the model, the collected appropriate input, and an output corresponding to the appropriate input, and fitted to a model in the vicinity of a specific output targeted for prediction. The performance prediction method according to the above supplementary notes.

[Appendix 12]
The appropriate input is an input when the information processing system has a low load,
The said specific output is an output in case the said information processing system is a heavy load, The performance prediction method of the said remarks characterized by the above-mentioned.

[Appendix 13]
The input content to the information processing system is determined so that the information processing system is in a low load operation when a specific output to be predicted is an output at the time of a high load operation. And calculate the input
At the time of model fitting, fitting the acquired model based on the collected input for causing the information processing system to perform low-load operation and the output at the time of low-load operation output according to the input,
The performance prediction method according to the above supplementary note, wherein the performance of the output at the time of high load operation targeted for prediction for the information processing system is predicted based on a new model that is fitted based on the value of low load operation .

[Appendix 14]
Accumulating the fitted model as a new model, calculating an input corresponding to a specific output acquired based on the new model, determining an input content based on the input corresponding to the specific output, The information processing system is executed using the input content as input, the input of the information processing system executed and the output corresponding to the input are collected, and the collected input of the information processing system and the output corresponding to the input are collected. In addition, the performance prediction method according to the above supplementary note, wherein a repetitive control process for determining whether to repeat the series of processes of fitting the acquired new model or not to repeat the series of processes is performed.

[Appendix 15]
The performance prediction method according to the above supplementary note, wherein the iterative control process determines whether or not to repeat the series of processes based on the accuracy of the fitted model.

[Appendix 16]
In the input content determination process, the input content is determined based on the collected input of the information processing system and the output corresponding to the input.

[Appendix 17]
The control unit of the information processing device
Model storage means for storing a model of the relationship between input and output related to the information processing system to be measured;
When predicting the performance of the information processing system, specific output acquisition means for acquiring a specific output to be predicted;
The input corresponding to the specific output acquired by the specific output acquisition unit is calculated using the model of the information processing system stored in the model storage unit, and the input corresponding to the specific output is input. Input content determining means for determining the content;
Measurement target system execution means for executing the information processing system so as to input the input content determined by the input content determination means;
Information collecting means for collecting the input of the information processing system executed by the measurement target system executing means and the output corresponding to the input;
Model fitting means for fitting the model acquired from the model storage means based on the input of the information processing system collected by the information collection means and the output corresponding to the input;
Based on a new model fitted by the model fitting means, performance prediction means for predicting the performance of a specific output targeted for prediction for the information processing system;
A program characterized by functioning as

[Appendix 18]
The program according to the above supplementary note, wherein the model storage unit is operated so as to store the model fitted by the model fitting unit as a new model.

[Appendix 19]
The information collecting means collects an appropriate input of the information processing system and an output corresponding to the appropriate input;
The model fitting means estimates a model acquired from the model storage means based on the model, the collected appropriate input, and an output corresponding to the appropriate input, and outputs a specific output to be predicted. The program according to the above supplementary note, which is operated so as to fit to a nearby model.

[Appendix 20]
The appropriate input is an input when the information processing system has a low load,
The program according to the above supplementary note, wherein the specific output is operated so as to be an output when the information processing system is under a heavy load.

[Appendix 21]
The input content determination means, when the specific output to be predicted is an output at the time of a high load operation, the measurement target system execution means, so that the information processing system becomes a low load operation Calculate the input as the input content,
The model fitting means, based on the input for performing the low load operation of the information processing system collected by the information collecting means, and the output at the time of the low load operation output according to the input, from the model storage means Fitting the acquired model,
Based on the new model that has been fitted based on the value of the low load operation from the performance predicting means, the information processing system is operated so as to predict the performance of the output at the time of high load operation targeted for prediction. The program according to the above supplementary note, which is characterized.

[Appendix 22]
The model accumulating unit accumulates the fitted model as a new model, and the input content determining unit calculates an input corresponding to a specific output acquired by the specific output unit based on the new model. The input content is determined based on the input corresponding to the output of the input, the measurement target system execution means executes the information processing system with the input content as input, and the information collection means causes the measurement target system execution means to execute. And collecting the model based on the input of the information processing system collected by the information collecting unit and the output corresponding to the input. Whether to repeat the process of fitting the new model obtained from the means Or the program according to the above supplementary note, wherein the control unit is further operated as a repetition control means for determining whether or not to repeat a series of processes.

[Appendix 23]
The program according to the above supplementary note, wherein the repetition control unit is operated so as to determine whether or not to repeat the series of processes based on the accuracy of the model fitted by the model fitting unit.

[Appendix 24]
The additional description is characterized in that the input content determination unit is operated to determine the input content based on the input of the information processing system collected by the information collection unit and the output corresponding to the input. Program.

  The present invention can be applied to an application for accurately predicting a specific region of a performance index for an information processing system to be measured. For example, in an IT system, it is important to determine conditions under which the performance index does not satisfy the SLA, but this can be used for accurate estimation of such conditions.

DESCRIPTION OF SYMBOLS 1 Performance prediction apparatus 2 Measurement object system 3 Communication network 101, 101 'Model storage means 102 Specific output acquisition means 103 Performance prediction means 104, 104' Input content determination means 105 Measurement object system execution means 106, 106 'Information collection means 107 Model Fitting means 108 Repeat control means

Claims (10)

Model storage means for storing a model of the relationship between input and output related to the information processing system to be measured;
When predicting the performance of the information processing system, specific output acquisition means for acquiring a specific output to be predicted;
The input corresponding to the specific output acquired by the specific output acquisition unit is calculated using the model of the information processing system stored in the model storage unit, and the input corresponding to the specific output is input. Input content determining means for determining the content;
Measurement target system execution means for executing the information processing system so as to input the input content determined by the input content determination means;
Information collecting means for collecting the input of the information processing system executed by the measurement target system executing means and the output corresponding to the input;
Model fitting means for fitting the model acquired from the model storage means based on the input of the information processing system collected by the information collection means and the output corresponding to the input;
Based on a new model fitted by the model fitting means, performance prediction means for predicting the performance of a specific output targeted for prediction for the information processing system;
A performance prediction apparatus comprising:   The performance prediction apparatus according to claim 1, wherein the model storage unit stores the model fitted by the model fitting unit as a new model. The information collecting means collects an appropriate input of the information processing system and an output corresponding to the appropriate input;
The model fitting means estimates a model acquired from the model storage means based on the model, the collected appropriate input, and an output corresponding to the appropriate input, and outputs a specific output to be predicted. 3. The performance prediction apparatus according to claim 2, wherein fitting is performed to a nearby model. The appropriate input is an input when the information processing system has a low load,
4. The performance prediction apparatus according to claim 3, wherein the specific output is an output when the information processing system is heavily loaded. The input content determination means, when the specific output to be predicted is an output at the time of a high load operation, the measurement target system execution means, so that the information processing system becomes a low load operation Calculate the input as the input content,
The model fitting means, based on the input for performing the low load operation of the information processing system collected by the information collecting means, and the output at the time of the low load operation output according to the input, from the model storage means Fitting the acquired model,
The performance prediction means predicts the output performance at the time of high load operation, which is a target of prediction for the information processing system, based on a new model subjected to fitting based on a value of low load operation. Item 3. The performance prediction apparatus according to Item 2. The model storage means stores the fitted model as a new model, and the input content determination means calculates an input corresponding to the specific output acquired by the specific output acquisition means based on the new model, and The input content is determined based on an input corresponding to a specific output, the measurement target system execution unit executes the information processing system with the input content as an input, and the information collection unit is executed by the measurement target system execution unit. The information processing system input and the output corresponding to the input are collected, and the model fitting unit is configured to collect the model based on the input of the information processing system collected by the information collection unit and the output corresponding to the input. Repeat a series of processes to fit the new model acquired from the storage means The performance prediction apparatus according to claim 2, further comprising a repetition control unit that determines whether or not to repeat a series of processes.   7. The performance prediction apparatus according to claim 6, wherein the iterative control unit determines whether or not to repeat the series of processes based on the accuracy of the model fitted by the model fitting unit.   7. The input content determination unit determines the input content based on an input of the information processing system collected by the information collection unit and an output corresponding to the input. The performance prediction apparatus according to claim 1. Accumulate in advance a model of the relationship between input and output for the information processing system to be measured,
Obtaining a specific output that is a target of prediction, which is a reference when predicting the performance of the information processing system;
Using the model, calculating the input corresponding to the specific output, based on the input corresponding to the calculated specific output, determining the input content to the information processing system,
Causing the information processing system to execute the input content determined as an input,
Collecting the input of the information processing system to be executed and the output corresponding to the input;
Fitting the model based on the collected input of the information processing system and the output corresponding to the input,
A performance prediction method characterized by predicting the performance of a specific output targeted for prediction for the information processing system based on a new fitted model. The control unit of the information processing device
Model storage means for storing a model of the relationship between input and output related to the information processing system to be measured;
When predicting the performance of the information processing system, specific output acquisition means for acquiring a specific output to be predicted;
The input corresponding to the specific output acquired by the specific output acquisition unit is calculated using the model of the information processing system stored in the model storage unit, and the input corresponding to the specific output is input. Input content determining means for determining the content;
Measurement target system execution means for executing the information processing system so as to input the input content determined by the input content determination means;
Information collecting means for collecting the input of the information processing system executed by the measurement target system executing means and the output corresponding to the input;
Model fitting means for fitting the model acquired from the model storage means based on the input of the information processing system collected by the information collection means and the output corresponding to the input;
Based on a new model fitted by the model fitting means, performance prediction means for predicting the performance of a specific output targeted for prediction for the information processing system;
A program characterized by functioning as

Images