WO2015083333A1 - Performance estimation device, performance estimation method, and storage medium on which computer program is stored - Google Patents

Abstract

Provided is a performance estimation device, and the like, that can adjust a system performance estimation model so that the performance estimation model appropriately illustrates the actual system, and can evaluate the validity of the adjusted performance estimation model. A performance estimation device (100) comprises the following: a performance estimation unit (101) that uses a performance estimation model, in which a system which is the target of performance estimation is illustrated, to calculate estimated output with respect to input to the system; a model adjustment unit (102) that adjusts the parameters of the performance estimation model on the basis of first statistics with respect to the estimated output which was calculated, and first statistics with respect to the actual output of the system; and a validity estimation unit (103) that estimates the validity of the performance estimation model on the basis of second statistics with respect to the estimated output calculated using the performance estimation model the parameters of which were adjusted, and second statistics with respect to the actual output of the system.

Description

Performance prediction apparatus, performance prediction method, and storage medium storing computer program

The present invention relates to a technique for modeling a system including information processing apparatuses and predicting the performance of the system using the model. The present invention also relates to a technique for evaluating the validity of the model based on the actual behavior (behavior) of a system constituted by information processing apparatuses.

In recent years, with the spread of a system composed of information processing devices such as computers (hereinafter sometimes simply referred to as “system”), a technique for accurately predicting the performance of such a system is required. In order to predict the performance of such a system, a technique for predicting a specific performance index related to the system using a model that schematically illustrates the behavior of the actual system has been proposed. In the following, the above model may be referred to as a “performance prediction model”.

Here, as related technologies existing prior to the present application, for example, there are the following patent documents.

Patent Document 1 discloses a system performance prediction method and the like proposed by the present applicant as a technique related to system performance evaluation. The technique disclosed in Patent Document 1 replaces a part of a system model obtained by modeling a system that is a performance prediction target with a black box. Moreover, the technique disclosed in Patent Document 1 sets the parameters of the black box so that the difference between the output from the system model after replacement for a specific input value and the output from the actual system becomes small. adjust. According to the technique disclosed in Patent Document 1, it is possible to predict the performance of a system by using a system model that reflects the actual behavior of the system.

Patent Document 2 discloses a technique for predicting performance indexes such as throughput, response, and resource usage rate of a parallel computer in a multitasking environment by converting the parallel computer system into a queuing model. The technique disclosed in Patent Document 2 predicts the performance of a system using parameters given in advance with respect to the system to be constructed (for example, processing time (Demand) per request). Further, the technique disclosed in Patent Document 2 reduces the number of parameters by using a parameter having a small influence on the model as a fixed value, and adjusts the remaining appropriate parameters to be adjusted. According to the technique disclosed in Patent Document 2, by analyzing the queuing model, a formula for predicting the improvement degree of processing performance and a performance index is obtained, and a predicted value can be obtained based on the formula. It is.

Patent Document 3 discloses a technique for adjusting a model for an application using a log embedded in the application (software). The technique disclosed in Patent Literature 3 embeds a model adjustment log in an application source code, and compares the log output of the application with the log output of a model that schematically represents the application. The technique disclosed in Patent Document 3 adjusts the parameters of the application model based on the log output and reflects the parameters in the model. According to the technique disclosed in Patent Document 3, it is possible to create a model reflecting the behavior of an application according to the execution environment of the application.

Patent Document 4 discloses a technique related to a system performance prediction apparatus. The technique disclosed in Patent Literature 4 inputs data related to hardware conditions, software conditions, and workload conditions in order to create a simulation model for a system that is a target of performance prediction. Furthermore, the technique disclosed in Patent Document 4 calculates a performance index (for example, calculation capability per unit time) based on a specific calculation formula based on the condition data. According to the technique disclosed in Patent Document 4, it is possible to provide a technique for objectively predicting the system performance regardless of the human qualities of the evaluator.

International Publication No. 2012/173283 JP 2000-298593 A JP 2002-215423 A JP 2000-172537 A

For the performance index as described above, the value predicted by the model may be different from the value actually measured from the target system. For this reason, for example, by adjusting the parameters included in the model, it is required to perform performance prediction in accordance with the actual condition of the target system. In this case, for example, it is necessary to evaluate whether the model in which such parameters are adjusted is appropriate (that is, whether the actual system behavior can be appropriately modeled).

Here, the technique disclosed in Patent Document 1 described above determines the validity of the model based on the structure of the black box described above. Thus, in the technique disclosed in Patent Document 1, knowledge for evaluating the black box and its structure is required.

The technique disclosed in Patent Document 2 does not adjust parameters. Thus, the parameter value given in advance may be different from the parameter value obtained from the actually constructed target system. For this reason, there is a problem that there is a difference between the performance index predicted using the model and the performance index of the actually constructed target system.

The technique disclosed in Patent Document 3 reduces the number of parameters with a parameter having a small influence on the model as a fixed value, and adjusts the remaining appropriate parameters to be adjusted. However, the technique disclosed in Patent Document 3 has a problem that the number of parameters is merely reduced, and the validity of the result of adjusting the remaining parameters is not sufficiently evaluated. Furthermore, as described above, the technique disclosed in Patent Document 3 compares the log output of an application and its application model, and adjusts parameters and the like. However, for example, in the case of a commercial application, a log output command may not be embedded, and there is a problem that the scope of application of the technique disclosed in Patent Document 3 is limited.

The technology disclosed in Patent Document 4 only calculates a performance predicted value based on an input parameter and a performance index calculated from the parameter by a specific arithmetic expression. For this reason, the technique disclosed in Patent Document 4 cannot sufficiently evaluate whether or not the model appropriately models an actual system.

Therefore, with the related technology described above, it is not possible to sufficiently evaluate whether or not the system performance prediction model appropriately models the actual system.

Therefore, the present invention has been made in view of the above-described problems. That is, the present invention provides a performance prediction device and the like that can adjust a model for predicting the performance of the system based on the actual behavior of the system and can evaluate the validity of the model. Main purpose.

In order to achieve the above object, a performance prediction apparatus according to an aspect of the present invention has the following configuration. That is, a performance prediction apparatus according to an aspect of the present invention uses a performance prediction model that schematically illustrates a system that is a target of performance prediction, a performance prediction unit that calculates a predicted output for an input to the system, and the calculation A model adjustment unit that adjusts parameters of the performance prediction model based on the first statistic for the predicted output and the first statistic for the actual output of the system, and the performance prediction after the parameter adjustment. A validity evaluation unit that evaluates the validity of the performance prediction model based on the second statistic for the predicted output calculated using the model and the second statistic for the actual output of the system; Prepare.

The performance prediction method according to one aspect of the present invention has the following configuration. That is, in the performance prediction method according to one aspect of the present invention, the information processing apparatus calculates a predicted output with respect to an input to the system using a performance prediction model that schematically illustrates a system that is a target of performance prediction. Based on the first statistic for the calculated predicted output and the first statistic for the actual output of the system, the parameters of the performance prediction model are adjusted, and the performance prediction model after adjusting the parameters is used. The validity of the performance prediction model is evaluated based on the second statistic for the predicted output calculated in the above and the second statistic for the actual output of the system.

The performance prediction program according to one aspect of the present invention has the following configuration. That is, the performance prediction program according to an aspect of the present invention uses a performance prediction model that schematically illustrates a system that is a target of performance prediction, and calculates a prediction output for an input to the system, and the calculated prediction. Based on the first statistic for the output and the first statistic for the actual output of the system, using the process for adjusting the parameters of the performance prediction model, and using the performance prediction model after adjusting the parameters Based on the calculated second statistic for the predicted output and the second statistic for the actual output of the system, the computer is caused to perform processing for evaluating the validity of the performance prediction model.

Note that the object of the present invention can also be realized by a computer-readable storage medium in which the performance prediction program is stored.

According to the present invention, it is possible to provide a performance prediction device and the like that can evaluate the validity of a model for performing system performance prediction based on the actual behavior of the system.

FIG. 1 is a block diagram illustrating a functional configuration of a performance prediction apparatus according to the first embodiment of the present invention. FIG. 2 is a flowchart illustrating an example of processing performed by the performance prediction apparatus according to the first embodiment of the present invention. FIG. 3 is a block diagram illustrating a functional configuration of the performance prediction apparatus according to the second embodiment of the present invention. FIG. 4 is a diagram illustrating an example of a performance prediction model according to the second embodiment of the present invention. FIG. 5A is a diagram illustrating an example of a performance prediction model and data output from an actual system according to the second embodiment of the present invention. FIG. 5B is a diagram illustrating an example of a performance prediction model and data output from an actual system according to the second embodiment of the present invention. FIG. 6 is a flowchart illustrating an example of processing performed by the performance prediction apparatus according to the second embodiment of the present invention. FIG. 7 is a block diagram illustrating a functional configuration of a performance prediction apparatus according to the third embodiment of the present invention. FIG. 8 is a block diagram illustrating a hardware configuration of an information processing apparatus capable of realizing the performance prediction apparatus according to each embodiment of the present invention.

Next, embodiments of the present invention will be described in detail with reference to the drawings. The configurations described in the following embodiments are merely examples, and the technical scope of the present invention is not limited thereto.

<First Embodiment>
A first embodiment of the present invention will be described with reference to FIGS. 1 and 2. FIG. 1 is a block diagram illustrating a functional configuration of the information processing apparatus 100 according to the first embodiment of the invention.

The information processing apparatus 100 illustrated in FIG. 1 is an information processing apparatus that functions as a performance prediction apparatus in the present embodiment. The information processing apparatus 100 in the present embodiment may be an apparatus that operates according to a specific program (computer program, software program, etc.) such as an electronic computer (computer). The hardware configuration of the information processing apparatus in the present embodiment will be described later.

The information processing apparatus 100 in the present embodiment includes a performance prediction unit 101, a model adjustment unit 102, and a validity evaluation unit 103. These components are connected so as to be communicable with each other via an arbitrary communication means by a software configuration or a hardware configuration.

The performance prediction unit 101 uses a model that schematically illustrates the relationship between the input and output of a performance index related to a system that is a performance prediction target (not shown) (hereinafter also referred to as a performance prediction target system). Predict output against system input. Hereinafter, the output predicted by the model may be referred to as “predicted output”. Hereinafter, the model may be referred to as a “performance prediction model” or a “prediction model”.

Here, the input may be, for example, the number of requests that the system must process within a unit time. The output may be a performance index such as the throughput, response time, CPU (Central Processing Unit) usage rate, and memory usage rate of the system. However, the input and output are not limited to these. For example, in the performance prediction model, when the input and output can be described as a relationship between an independent variable and a dependent variable, the independent variable may be the input, and the dependent variable may be the output.

In the present embodiment, the performance prediction model may be represented by an arbitrary expression form imitating (schematic) the behavior of the system in order to predict the performance of the system. For example, the performance prediction model may be configured to predict an output from the system by performing calculation or simulation according to a predetermined procedure with respect to an input to a certain system.

Specifically, for example, for each component of a computer such as a CPU, a queue imitating (scheming) the operation of the component may be employed as the performance prediction model. In addition, even if a mechanism (for example, a neural network, a hidden Markov model, a polynomial function approximation, etc.) that can determine an appropriate output for an input by means such as learning or regression is adopted as the performance prediction model. Good. Note that these performance prediction models are merely examples. The performance prediction model in the present embodiment is not limited to these examples.

In the present embodiment, information (data) representing the performance prediction model is input to the information processing apparatus 100 by an arbitrary method. The information processing apparatus 100 may store information representing the performance prediction model in a storage unit or the like (not shown). Note that the information representing the performance prediction model may be stored in any external device that exists outside the information processing apparatus 100. In this case, the information processing apparatus 100 may be configured to refer to information representing the performance prediction model stored in the external apparatus as necessary.

The model adjustment unit 102 determines the performance based on the first statistic for the predicted output of the performance prediction model calculated by the performance prediction unit 101 and the first statistic for the actual output of the performance prediction target system. Adjust the parameters of the prediction model. Here, the parameter of the performance prediction model may be, for example, a processing time per request in the case of a queue model, and may be a synaptic load in the case of a neural network. In the present embodiment, any adjustment item that can change the configuration and output of the performance prediction model may be adopted as the adjustment parameter of the performance prediction model. For this reason, in the present embodiment, an appropriate adjustment parameter may be appropriately selected depending on the specific configuration of the performance prediction model. Hereinafter, adjusting a parameter included in the performance prediction model may be expressed as “adjusting the performance prediction model”.

Here, the first statistic is an arbitrary statistic that can be statistically obtained from the predicted output of the performance prediction model or the output of the performance prediction target system. As such a first statistic, for example, an average value, a variance, a frequency distribution, or the like may be employed, but is not limited thereto. The first statistic in this embodiment may be any value or distribution that is statistically calculated.

The model adjustment unit 102 may obtain the actual output value of the performance prediction target system by an arbitrary method according to the configuration of the system. For example, the model adjustment unit 102 may acquire the actual output value of the performance prediction target system from a storage unit (such as reference numeral 802 in FIG. 8 described later) included in the information processing apparatus 100. The model adjustment unit 102 may acquire the value from an external device of the information processing apparatus 100 (a reference numeral 804 in FIG. 8 to be described later, an apparatus other than the information processing apparatus 100, or the like). The present embodiment is not limited to this. For example, the performance prediction unit 101 described above may acquire the actual output value of the performance prediction target system and provide it to the model adjustment unit 102.

Also, the model adjustment unit 102 may provide the adjusted parameter to the performance prediction unit 101, and the performance prediction unit 101 may hold the parameter. Note that the present embodiment is not limited to this, and for example, other elements (for example, the model adjustment unit 102, the validity evaluation unit 103, and the like) constituting the information processing apparatus 100 may hold the parameters. . Note that the performance prediction unit 101 may update the performance prediction model using the adjusted parameters and calculate a prediction output using the updated performance prediction model.

The algorithm for adjusting the performance prediction model, for example, makes the first statistic calculated from the predicted output of the performance prediction model closer to the first statistic calculated from the actual output of the performance prediction target system. In addition, the performance prediction model may be adjusted. More specifically, the algorithm may adjust the performance prediction model using an appropriate method such as a learning or optimization method. Hereinafter, an algorithm for adjusting the performance prediction model may be referred to as a “model adjustment algorithm”.

For example, when the performance prediction model is expressed by a neural network, the model adjustment algorithm may correct the synaptic load by a learning algorithm such as backpropagation. When the performance prediction model is a polynomial approximation, the model adjustment algorithm may correct the coefficient of each term by a least square method or the like. When the performance prediction model is a queue, the model adjustment algorithm may correct the request processing time using a Kalman filter or the like.

In addition to this, for example, as a method for evaluating the difference between the first statistic for the actual output from the performance prediction target system and the first statistic for the output from the performance prediction model, reinforcement learning, A genetic algorithm, a Monte Carlo method, a linear programming method, or the like may be employed. The model adjustment algorithm may correct the parameters of the performance prediction model by appropriately using these methods. The method for adjusting the performance prediction model is not limited to these examples. As a method for adjusting the performance prediction model, an appropriate method may be adopted as appropriate according to the configuration of the performance prediction model.

The validity evaluation unit 103 evaluates whether or not the performance prediction model after the parameter adjustment is valid as a performance prediction model related to the performance prediction target system.

Hereinafter, the validity evaluation for the performance prediction model after parameter adjustment by the validity evaluation unit 103 will be described. In the following, the performance prediction model after parameter adjustment may be simply referred to as “adjusted performance prediction model”.

First, in the present embodiment, one arbitrary statistic other than the first statistic is selected for the output of the performance prediction model and the output of the performance evaluation target system. Then, the selected statistic is adopted as the second statistic. Note that what kind of statistic is selected as the second statistic may be set in the information processing apparatus 100 in advance.

As described above, for example, the model adjustment unit 102 converts the first statistical amount of the prediction output calculated by the performance prediction unit 101 and the first statistical amount calculated from the actual output of the performance prediction target system. Based on this, the performance prediction model is adjusted. Specifically, the model adjustment unit 102 determines, for example, that the difference between the first statistic for the predicted output calculated by the performance prediction unit 101 and the first statistic for the actual output of the performance prediction target system is The performance prediction model is adjusted to be smaller.

Here, when the adjusted performance prediction model is valid, between the first statistic for the predicted output calculated by the performance prediction unit 101 and the first statistic for the actual output of the performance prediction target system. It is expected that the difference will be small. At the same time, the difference between the second statistic for the predicted output calculated by the performance prediction unit 101 and the second statistic for the actual output of the performance prediction target system is also expected to be small. This is because if the adjusted performance prediction model is appropriate, it is expected that the actual system behavior is correctly modeled by the performance prediction model.

On the other hand, when the adjusted performance prediction model is not valid, it is between the second statistic for the predicted output calculated by the performance prediction unit 101 and the second statistic for the actual output of the performance prediction target system. The difference is expected to be large. If the adjusted performance prediction model is not valid, the performance prediction model adjusted based on the first statistic cannot actually imitate (scheme) the behavior of the actual system correctly. It is considered that there is a difference between the output of the performance prediction model that is not valid and the actual output of the performance prediction target system. For this reason, it is considered that wrinkles due to such differences appear in other statistics (for example, the second statistics) that are not considered in the adjustment of the performance prediction model.

Therefore, the validity evaluation unit 103 decreases the difference between the second statistic for the predicted output calculated by the performance prediction unit 101 and the second statistic for the actual output of the performance prediction target system. It can be evaluated that the performance prediction model after adjustment is appropriate.

Here, the difference in the second statistic may be obtained, for example, by subtraction or division if the second statistic is an average value, and based on the distance between distributions in the case of a frequency distribution. It may be sought. The method for obtaining the difference between the second statistics is not limited to these. The validity evaluation unit 103 may appropriately select a method for obtaining a difference regarding the second statistic as appropriate according to the property of the second statistic.

Note that the evaluation criteria for the validity of the adjusted performance prediction model is not limited to the above. In this embodiment, an arbitrary evaluation criterion using the second statistic for the predicted output calculated by the performance prediction unit 101 and the second statistic for the actual output of the performance prediction target system is adopted. It's okay. The validity evaluation unit 103, for example, statistics such as a correlation between the second statistic for the predicted output calculated by the performance prediction unit 101 and the second statistic for the actual output of the performance prediction target system. A standard criterion may be adopted as the evaluation criterion.

Note that the information processing apparatus 100 according to this embodiment described above may include an arithmetic processing unit, a storage unit, an input / output unit, and the like as a hardware configuration. The functions provided in the information processing apparatus 100 described above may be realized by cooperation of these hardware and various programs stored in the storage unit. An example of a specific hardware configuration of the information processing apparatus 100 in the present embodiment will be described later.

Next, the operation of the information processing apparatus 100 in this embodiment will be described with reference to FIG. FIG. 2 is a flowchart illustrating an example of processing performed by the information processing apparatus 100 according to the present embodiment.

First, in step S201, the performance prediction unit 101 calculates a predicted output for an input to the performance prediction target system, using the performance prediction model. Here, the input is given to the information processing apparatus 100 (particularly, the performance prediction unit 101) by an appropriate method as appropriate. For example, an actual input to the performance prediction target system may be separately recorded by an arbitrary method, and the recorded input may be given to the information processing apparatus 100. Further, for example, the performance prediction target system may be configured so that an actual input to the performance prediction target system is distributed to an input to the information processing apparatus 100. The input may be given to the information processing apparatus 100 via an input / output device (not shown) in the information processing apparatus 100 or a communication apparatus.

Next, in step S202, the model adjustment unit 102 calculates a first statistic for the predicted output of the performance prediction model and a first statistic for the actual output of the performance prediction target system. Then, the model adjustment unit 102 determines whether or not the adjustment of the performance prediction model has been completed based on the calculated first statistic.

Here, the model adjustment unit 102 compares, for example, the first statistic for the actual output of the performance prediction target system with the first statistic for the predicted output predicted by the model, and determines the difference between them. You may evaluate based on a reference | standard. Then, the model adjustment unit 102 may determine whether or not the adjustment of the performance prediction model has been completed based on the evaluation.

More specifically, for example, the model adjustment unit 102 may complete the adjustment of the performance prediction model when the difference becomes a value equal to or less than a predetermined accuracy. In this case, the performance prediction unit 101 can predict the behavior of the real system within a predetermined accuracy by the performance prediction model that has been adjusted.

In addition, the model adjustment unit 102 can employ the following criteria as a criterion for determining whether or not the adjustment of the performance prediction model is completed, for example. That is, the model adjusting unit 102 adjusts the difference between the first statistic for the actual output of the performance prediction target system and the first statistic for the predicted output predicted by the performance prediction model after a certain number of adjustments. May not be changed (that is, when the adjustment can be regarded as being in an equilibrium state), it may be determined that the adjustment is completed. Alternatively, the model adjustment unit 102 may determine that the adjustment is complete when the adjustment is performed a predetermined number of times.

As a result of the determination in step S202, when the model adjustment unit 102 determines that the adjustment of the performance prediction model has not been completed (NO in step S203), the model adjustment unit 102 executes the process of step S204.

In step S204, the model adjustment unit 102 performs the performance based on the first statistic for the predicted output of the performance prediction model calculated in step S201 and the first statistic for the actual output of the performance prediction target system. Adjust the prediction model.

In step S204, after the model adjustment unit 102 adjusts the performance prediction model, the process returns to step S201 to continue the process, and the performance prediction unit 101 recalculates the prediction output predicted by the adjusted performance prediction model.

Next, as a result of the determination in step S202, if the model adjustment unit 102 determines that the adjustment of the performance prediction model has been completed (YES in step S203), the validity evaluation unit 103 executes the process of step S205.

In step S205, the validity evaluation unit 103 evaluates the validity of the performance prediction model after being adjusted by the model adjustment unit 102. In this case, the validity evaluation unit 103, for example, as described above, the second statistic for the predicted output from the adjusted performance prediction model, and the second statistic for the actual output from the performance prediction target system, Based on the above, the validity of the performance prediction model may be evaluated.

As described above, the information processing apparatus 100 according to the present embodiment calculates the predicted output predicted by the performance prediction model with respect to the input of the performance prediction target system. Then, the information processing apparatus 100 according to the present embodiment enables the performance prediction model so that the difference between the first statistic for the actual output of the performance prediction target system and the first statistic for the predicted output is small. Adjust. Then, the information processing apparatus 100 according to the present embodiment validates the adjusted performance prediction model based on the second statistic for the actual output of the performance prediction target system and the second statistic for the predicted output. Assess sex.

For this reason, according to the information processing apparatus 100 in the present embodiment, the validity of the performance prediction model adjusted using the actual behavior of the performance prediction target system is evaluated by using the actual behavior of the performance prediction target system. can do.

This is because the output of the performance prediction target system represents the actual behavior of the performance prediction target system, and the information processing apparatus 100 in the present embodiment uses the first statistic with respect to the output of the performance prediction target system. This is because the performance prediction model is adjusted. In addition, the information processing apparatus 100 according to the present embodiment evaluates the validity of the performance prediction model using the second statistic with respect to the output of the performance prediction target system.

<Second Embodiment>
Next, a second embodiment of the present invention will be described. In the following description, characteristic portions according to the present embodiment will be mainly described, and a duplicate description of the same configuration as that of the first embodiment will be omitted.

The performance prediction apparatus 300 according to the present embodiment can be applied to the performance prediction model having a plurality of adjustment parameter candidates (hereinafter also referred to as “parameter adjustment candidates”). The performance prediction apparatus 300 according to the present embodiment evaluates the validity of the performance prediction model with respect to a plurality of parameter adjustment candidates, and presents the parameter adjustment candidates that are evaluated to have higher validity to the user. This is different from the first embodiment. Hereinafter, the difference will be mainly described.

FIG. 3 is a block diagram illustrating a functional configuration of the information processing apparatus 300 according to the second embodiment of the present invention. Note that the specific hardware configuration of the information processing apparatus 300 may be the same as that of the first embodiment.

The validity evaluation unit 303 shown in FIG. 3 evaluates the validity of the performance prediction model for a plurality of parameter adjustment candidates.

Here, the plurality of parameter adjustment candidates will be described using the specific example illustrated in FIG. FIG. 4 is a specific example of a performance prediction model in which a performance prediction target system is modeled using a queue. D1 and D2 in FIG. 4 are parameters representing the processing time (Demand) of the queue.

For example, the model adjustment unit 102 in the present embodiment has a small difference between the first statistic for the predicted output from the performance prediction model illustrated in FIG. 4 and the first statistic for the actual output of the performance prediction target system. The parameters D1 and D2 are adjusted so that In this case, there are (1) a method for adjusting D1 without changing D2, (2) a method for adjusting D2 without changing D1, and (3) a method for adjusting both D1 and D2. Can be a candidate for adjustment method.

The validity evaluation unit 303 evaluates the validity of the adjusted performance prediction model for each of these candidates.

The validity evaluation unit 303 may evaluate the validity of the performance prediction model with respect to the plurality of parameter adjustment candidates using an appropriate method as appropriate. For example, the validity evaluation unit 303 may set a specific criterion for validity evaluation and evaluate the validity of the performance prediction model based on the criterion.

More specifically, the validity evaluation unit 303 may evaluate the validity of the performance prediction model based on the following criteria, for example. That is, the validity evaluation unit 303 adjusts a specific parameter adjustment candidate (hereinafter referred to as a first parameter candidate), a second statistic for the predicted output calculated by the performance prediction unit 101, and A second statistic for the actual output of the performance prediction target system is obtained. The validity evaluation unit 303 further adjusts another parameter adjustment candidate (hereinafter referred to as a second parameter candidate), the second statistic for the predicted output calculated by the performance prediction unit 101, and the performance. A second statistic for the actual output of the prediction target system is obtained.

The validity evaluation unit 303 calculates the difference between the second statistic for the predicted output calculated for the first parameter candidate and the second statistic for the actual output of the performance prediction target system (hereinafter referred to as the first difference). Calculated). Further, the validity evaluation unit 303 calculates the difference between the second statistic calculated for the second parameter candidate for the predicted output and the second statistic for the actual output of the performance prediction target system (hereinafter referred to as the second statistic). (Referred to as the difference).

The validity evaluation unit 303 may compare the first difference with the second difference, and may evaluate the smaller difference as a more appropriate parameter adjustment candidate. Note that the validity evaluation unit 303 may appropriately calculate the difference according to the second statistic. For example, if the second statistic is an average value of the output, the validity evaluation unit 303 may calculate a difference between the average values. If the second statistic is any distribution (frequency distribution or the like) with respect to the output, the validity evaluation unit 303 may calculate the distance between the distributions as a difference.

Note that the validity evaluation method is not limited to the above. The validity evaluation unit 303 is an arbitrary criterion that can be determined based on the second statistic for the predicted output calculated by the performance prediction unit 101 and the second statistic for the actual output of the performance prediction target system. May be adopted.

Further, the validity evaluation unit 303 may notify the presentation unit 304 of the validity evaluation result for the parameter adjustment candidate.

The presenting unit 304 presents the parameter adjustment candidates evaluated by the validity evaluation unit 303 as having higher validity to a user or the like (not shown). In the present embodiment, the presentation unit 304 may present the parameter adjustment candidates to the user by an appropriate method as appropriate. The presentation unit 304 may present the parameter adjustment candidates using a display device such as a screen (not shown). The presenting unit 304 transmits the parameter adjustment candidate to any external information processing apparatus or the like that is communicably connected to the information processing apparatus 300, thereby performing the related via the external information processing apparatus. Parameter adjustment candidates may be presented.

Hereinafter, with reference to FIG. 5A and FIG. 5B, presentation of the evaluation result by the validity evaluation unit 303 will be described using a specific example.

5A and 5B are diagrams illustrating the relationship between the output and the second statistic when the queuing model illustrated in FIG. 4 is adopted as the performance prediction model. In the specific examples illustrated in FIGS. 5A and 5B, the input to the performance prediction model is “the number of requests to be processed within a unit time”, and the output is “TAT (Turn Around Time) for each request”.

In the specific examples illustrated in FIGS. 5A and 5B, an average value is employed as the first statistic for the output. Further, a frequency distribution is adopted as the second statistic. That is, the first statistic with respect to the output from the performance prediction model is the average value of the TAT, and the second statistic is the frequency distribution of the TAT.

5A and 5B, the horizontal axis represents the output (TAT for each request). Each plot in each graph is a representative point of a specific TAT interval (for example, 0 seconds or more and less than 0.002 seconds). How to select such a representative point is arbitrary, and may be, for example, the median value of the TAT interval, or an upper limit value or a lower limit value. Also, how to divide the TAT section may be arbitrarily determined.

5A and 5B, the vertical axis represents the second statistic (TAT frequency distribution) with respect to the output. More specifically, the plot on the graph is a numerical value obtained by normalizing the number of requests included in the specific TAT section by the total number of requests.

5A and 5B, the second statistic is used for the actual output from the performance prediction target system and the predicted output from the performance prediction model (the queue model illustrated in FIG. 4) after parameter adjustment. Each frequency distribution which is a quantity is shown to be comparable.

In the specific examples illustrated in FIGS. 5A and 5B, the parameters of the performance prediction model include the first statistic for the output of the performance prediction model and the first statistic for the actual output of the performance prediction target system. It is adjusted so that the difference between and becomes smaller. The first statistic with respect to the output of the performance prediction model and the first statistic with respect to the actual output of the prediction target system are average values of TAT, respectively.

More specifically, the model adjustment unit 102 according to the present embodiment, for each of the specific examples illustrated in FIGS. 5A and 5B, the first statistic for the output of the performance prediction model is the performance prediction target system. The parameters of the performance prediction model are adjusted so as to approach the first statistic for the actual output. In the specific examples illustrated in FIGS. 5A and 5B, the average value of the TAT is 0.005 seconds.

Here, FIG. 5A is a result of comparing the second statistics when the method of adjusting D1 without changing (1) D2 is adopted. FIG. 5B shows the result of comparing the second statistics when (2) the method of adjusting D2 without changing D1 is employed.

As is clear from these figures, in the graph illustrated in FIG. 5A, the second statistic (the TAT frequency distribution) with respect to the actual output of the performance prediction target system and the second with respect to the predicted output of the performance prediction model. Are very similar distributions. However, in the graph illustrated in FIG. 5B, a relatively large error is present between the second statistic for the actual output of the performance prediction target system and the second statistic for the predicted output of the performance prediction model. There is.

Accordingly, the validity evaluation unit 303 is more appropriate for the performance prediction model adjusted by adopting the method (1) than for the performance prediction model adjusted by adopting the method (2). It can be determined that the property is high.

As described above, the presenting unit 304 can present the adjustment parameter in the case of FIG. 5A with higher validity to a user (not shown). The presenting unit 304 may present the difference between the second statistics when the adjustment candidates for each parameter are adjusted together with the adjustment candidates for the plurality of parameters. Moreover, the presentation unit 304 may present a graph as illustrated in FIGS. 5A and 5B.

Next, the operation of the information processing apparatus 300 in the present embodiment configured as described above will be described with reference to the flowchart illustrated in FIG. In the flowchart illustrated in FIG. 6, steps similar to those in FIG. 2 of the first embodiment are denoted by the same reference numerals, and for these steps, processing similar to that in the first embodiment is performed. Therefore, explanation is omitted.

First, the processes in steps S201 to S205 may be the same as those in the first embodiment. In step S201 to step S205, one or more parameter candidates may be adjusted.

Next, in step S601, the model adjustment unit 102 determines whether there is a parameter adjustment candidate that has not been adjusted yet. If there is an unadjusted parameter adjustment candidate (YES in step S602), the performance prediction unit 101 continues processing from step S201. In this case, the model adjustment unit 102 adjusts the parameter adjustment candidates as necessary, and the validity evaluation unit 303 evaluates the validity of the adjusted performance prediction model (steps S201 to S204).

When adjustment is performed for all parameter adjustment candidates (NO in step S602), the presentation unit 304 executes the process of step S603.

In step S603, the presentation unit 304 presents the parameter adjustment candidates evaluated by the validity evaluation unit 303 as having higher validity based on a specific standard to the user.

As in the first embodiment, the information processing apparatus 300 according to the present embodiment described above includes the first statistic for the actual output of the performance prediction target system and the first statistic for the predicted output of the performance prediction model. Adjust the performance prediction model based on the quantity. Further, the information processing apparatus 300 according to the present embodiment described above is adjusted using the second statistic for the actual output of the performance prediction target system and the second statistic for the prediction output of the performance prediction model. The validity of the performance prediction model is evaluated. Therefore, the information processing apparatus 300 according to the present embodiment has the same effects as those of the first embodiment.

Further, as described above, the information processing apparatus 300 according to the present embodiment evaluates the validity of the performance prediction model with respect to a plurality of parameter adjustment candidates, and the parameter adjustment candidates evaluated as having higher validity. Can be presented to the user.

The performance prediction model that models the performance prediction target system may be composed of a plurality of modules corresponding to each part of the system, and in this case, it is necessary to determine which module parameter is appropriate to be corrected. . According to the present embodiment, it is possible to compare the validity of each module when the parameters are adjusted, and to determine which module's parameter is most appropriate to be modified.

Note that the model adjustment unit 102 in the above-described embodiment may acquire the parameter adjustment candidates from a storage unit (such as reference numeral 802 in FIG. 8 described later) included in the information processing apparatus 300. The model adjustment unit 102 may acquire the parameter adjustment candidates from an external device of the information processing apparatus 300. In addition, the model adjustment unit 102 or the validity evaluation unit 303 in this embodiment may sequentially generate adjustment candidates for the parameters based on the adjustment parameters of the performance prediction model.

<Third Embodiment>
Next, a configuration common to the above-described embodiments will be described as a third embodiment of the present invention with reference to FIG. FIG. 7 is a block diagram illustrating a functional configuration of a performance prediction apparatus according to the third embodiment of the present invention.

As illustrated in FIG. 7, the information processing apparatus 700 as a performance prediction apparatus in the present embodiment includes a performance prediction unit 701, a model adjustment unit 702, and a validity evaluation unit 703.

In the present embodiment, the performance prediction unit 701, the model adjustment unit 702, and the validity evaluation unit 703 may be connected so as to be able to communicate with each other.

The performance prediction unit 701 calculates a predicted output with respect to an input to the system, using a performance prediction model that schematically illustrates a system that is a target of performance prediction.

The model adjustment unit 702 is based on the first statistic for the predicted output from the performance prediction model calculated by the performance prediction unit 701 and the first statistic for the actual output of the system. Adjust the parameters of the system performance prediction model.

The validity evaluation unit 703 includes a second statistic for the predicted output calculated using the performance prediction model after parameter adjustment by the model adjustment unit 702 and a second statistic for the actual output of the system. Based on this, the validity of the performance prediction model is evaluated.

The information processing apparatus 700 according to the present embodiment described above calculates a predicted output predicted by the performance prediction model with respect to an input of the performance prediction target system. Then, the information processing apparatus 700 according to the present embodiment uses the performance prediction model so that the difference between the first statistic for the actual output of the performance prediction target system and the first statistic for the predicted output becomes small. adjust. Then, the information processing apparatus 700 according to the present embodiment uses the second statistic for the actual output of the performance prediction target system and the second statistic for the predicted output to determine the validity of the adjusted performance prediction model. To evaluate.

That is, according to the present embodiment, the validity of the performance prediction model adjusted using the actual behavior of the performance prediction target system can be evaluated by using the actual behavior of the performance prediction target system.

<Configuration of hardware and software program (computer program)>
Hereinafter, a hardware configuration capable of realizing each of the above-described embodiments will be described.

The information processing apparatus (reference numeral 100 in FIG. 1, reference numeral 300 in FIG. 3, reference numeral 700 in FIG. 7), which is the performance prediction apparatus described in each of the above embodiments, is configured by a dedicated hardware device that realizes each function. May be. In that case, each of the units shown in FIGS. 1, 3, and 7 may be realized as hardware (an integrated circuit or the like on which processing logic is mounted) that is partially or fully integrated. Hereinafter, the reference numeral 100 in FIG. 1, the reference numeral 300 in FIG. 3, and the reference numeral 700 in FIG. 7 may be collectively referred to simply as a performance prediction apparatus.

Further, the above-described performance prediction apparatus may be configured by hardware as illustrated in FIG. 8 and various software programs (computer programs) executed by the hardware.

The arithmetic device 801 in FIG. 8 is an arithmetic processing device such as a general-purpose CPU (Central Processing Unit) or a microprocessor. The arithmetic device 801 may read various software programs stored in a non-volatile storage device 803, which will be described later, into the storage device 802, and execute processing according to the software programs.

The storage device 802 is a memory device such as a RAM (Random Access Memory) that can be referred to from the arithmetic device 801, and stores software programs and various data. Note that the storage device 802 may be a volatile memory device.

The non-volatile storage device 803 is a non-volatile storage device such as a magnetic disk drive or a semiconductor memory device using a flash memory, and may record various software programs, data, and the like.

The network interface 806 is an interface device connected to a communication network, and for example, a wired and wireless LAN (Local Area Network) connection interface device or the like may be employed.

For example, when providing an independent input device or the like for inputting (transmitting) various input data to the performance prediction device in each of the above embodiments, an arbitrary communication network is used between the input device and the performance prediction device. You may connect so that communication is possible. The input data may be, for example, information on the performance prediction model, input to the performance prediction target system, or information on the first and second statistics for the output from the performance prediction target system. In this case, the performance prediction apparatus may be connected to the communication network via the network interface 806.

In addition, when the performance prediction device is configured to directly acquire input / output data from the performance prediction target system, the performance prediction device and the performance prediction target system are communicatively connected via an arbitrary communication network. Also good. In this case, the performance prediction apparatus may be connected to the communication network via the network interface 806.

The external storage device 804 is a device that processes reading and writing of data with respect to a storage medium 805 described later, for example.

The storage medium 805 is an arbitrary recording medium capable of recording data, such as an optical disk, a magneto-optical disk, and a semiconductor flash memory.

The input / output interface 807 is a device that controls input / output between an external input device (such as a keyboard and a mouse) and an external output device (such as a display device and a printer).

In each of the above embodiments, the storage medium 805 on which various input data for the performance prediction devices are recorded is read into the external storage device 804, whereby various input data is input to the performance prediction devices. Also good. A user (not shown) may input the various input data using an input / output device connected to the input / output interface 807.

The present invention described using the above-described embodiments as an example may be realized as follows using, for example, the hardware illustrated in FIG. That is, a software program capable of realizing the function of the flowchart referred to in the description of each embodiment is supplied to the performance prediction apparatus configured using the hardware illustrated in FIG. Thereafter, the arithmetic device 801 executes the software program.

In each of the above-described embodiments, each unit illustrated in FIGS. 1, 3, and 7 can be realized as a software module that is a function (processing) unit of a software program executed by the above-described hardware. it can. However, the division of each software module shown in these drawings is a configuration for convenience of explanation, and various configurations can be assumed for implementation.

For example, when the units illustrated in FIGS. 1, 3, and 7 are realized as software modules, these software modules may be stored in the nonvolatile storage device 803. And when the arithmetic unit 801 performs each process, you may comprise so that these software modules may be read to the memory | storage device 802. FIG.

Further, these software modules may be configured to be able to transmit various data to each other by an appropriate method such as shared memory or inter-process communication. With such a configuration, these software modules can be connected so as to communicate with each other.

Further, each software program may be recorded in the storage medium 805. The software program recorded in the storage medium 805 is appropriately read out through the external storage device 804 and stored in the non-volatile memory 803 at the shipping stage or operation stage of the communication device or the like. Also good.

For example, referring to FIG. 1, FIG. 3, and FIG. 7, a case where the performance prediction unit (reference numeral 101, reference numeral 301, reference numeral 701) is realized by a software program can be considered. In this case, a software program that realizes the function of the performance prediction unit may be read into the storage device 802, and the arithmetic device 801 may be controlled to execute the software program.

Also, for example, a case where a software program that realizes the function of the performance prediction unit receives the above-described various input data from an arbitrary external device is conceivable. In this case, the arithmetic device 801 may receive the various input data by controlling the network interface 806 each time reception of various input data occurs with the external device, for example.

Similar to the performance prediction unit, the model adjustment unit (reference numeral 102, reference numeral 702), validity evaluation part (reference numeral 103, reference numeral 303, reference numeral 703), and presentation part (reference numeral 304) can also be realized by a software program. .

Also, for example, a case where a software program that implements the function of the presentation unit 304 presents parameter adjustment candidates is conceivable. In this case, for example, the arithmetic device 801 may control the input / output interface 807 and output the parameter adjustment candidates to an external device (screen device or the like) connected to the input / output interface 807.

Further, when the constituent elements of the performance prediction apparatus are realized as a software program, the performance prediction model, input / output data, first and second statistics, etc. described in each of the above embodiments have an appropriate data structure as appropriate. Or the like, may be stored in the storage device 802 or the nonvolatile storage device 803.

In the above case, the supply method of various software programs to the performance prediction apparatus is installed in the apparatus using an appropriate jig in the manufacturing stage before shipment or the maintenance stage after shipment. Can be adopted. In addition, as the supply method, a general procedure can be adopted at present, such as a method of downloading from the outside via a communication line such as the Internet. In such a case, the present invention can be understood to be constituted by a code constituting the software program or a computer-readable storage medium in which the code is recorded.

The present invention has been described above as an example applied to the exemplary embodiment described above. However, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various modifications and improvements can be made to such embodiments. In such a case, new embodiments to which such changes or improvements are added can also be included in the technical scope of the present invention. This is clear from the matters described in the claims.

Note that a part or all of the above-described embodiment and its modifications can be described as the following supplementary notes. However, the present invention described by way of example with the above-described embodiment and its modifications is not limited to the following.

(Appendix 1)
A performance prediction means for calculating a predicted output for an input to the system, using a performance prediction model schematically representing a system that is a target of performance prediction;
Model adjustment means for adjusting parameters of the performance prediction model based on a first statistic for the calculated predicted output and a first statistic for the actual output of the system;
The validity of the performance prediction model is evaluated based on the second statistic for the predicted output calculated using the performance prediction model after adjusting the parameters and the second statistic for the actual output of the system. And a validity evaluation means for performing the performance prediction.

(Appendix 2)
Further comprising a presentation means for presenting the parameter adjustment candidates of the performance prediction model to the user,
The validity evaluation means evaluates the validity of the performance prediction model based on a specific criterion for a plurality of parameter adjustment candidates,
The performance predicting apparatus according to appendix 1, wherein the presenting means presents to the user a parameter adjustment candidate that has been evaluated by the validity evaluating means to be highly valid based on the specific criterion. .

(Appendix 3)
The performance prediction apparatus according to appendix 1 or appendix 2, wherein the first statistic is an average value, a variance, or a frequency distribution.

(Appendix 4)
The performance prediction apparatus according to any one of appendix 1 to appendix 3, wherein the second statistic is an average value, a variance, or a frequency distribution.

(Appendix 5)
The performance prediction apparatus according to any one of appendix 1 to appendix 4, wherein at least one of the first statistic and the second statistic includes a plurality of types of statistic.

(Appendix 6)
The performance prediction device according to any one of appendix 1 to appendix 5, wherein the second statistic is a statistic of a different type from the first statistic.

(Appendix 7)
The model adjusting means includes
Adjusting the parameter of the performance prediction model so that the difference between the first statistic for the predicted output calculated by the performance prediction means and the first statistic for the actual output of the system is small. The performance prediction apparatus according to any one of appendix 1 to appendix 6, which is characterized.

(Appendix 8)
The validity evaluation means includes:
Assessing the validity of the performance prediction model by determining the magnitude of the difference between the second statistic for the predicted output and the second statistic for the actual output of the system based on the specific criteria The performance predicting apparatus according to appendix 2, wherein:

(Appendix 9)
Information processing device
Using a performance prediction model that models the system that is the target of performance prediction, calculate the predicted output for the input to the system,
Adjusting the parameters of the performance prediction model based on the first statistic for the calculated predicted output and the first statistic for the actual output of the system;
The validity of the performance prediction model is evaluated based on the second statistic for the predicted output calculated using the performance prediction model after adjusting the parameters and the second statistic for the actual output of the system. A performance prediction method characterized by:

(Appendix 10)
A process for calculating a predicted output with respect to an input to the system using a performance prediction model that models a system that is a target of performance prediction;
Adjusting the parameters of the performance prediction model based on a first statistic for the calculated predicted output and a first statistic for the actual output of the system;
The validity of the performance prediction model is evaluated based on the second statistic for the predicted output calculated using the performance prediction model after adjusting the parameters and the second statistic for the actual output of the system. A computer program that causes a computer to execute the processing.

(Appendix 11)
The validity evaluation means includes:
Of the plurality of parameter adjustment candidates, the difference between the second statistic for the output of the performance evaluation model adjusted for the first parameter adjustment candidate and the second statistic for the actual output of the system A first difference,
The difference between the second statistic for the output of the performance evaluation model adjusted for the second parameter adjustment candidate and the second statistic for the actual output of the system, which is different from the first parameter adjustment candidate Is compared with the second difference,
3. The performance prediction apparatus according to appendix 2, wherein the parameter adjustment candidate with the smaller difference between the first difference and the second difference is evaluated as being highly valid.

(Appendix 12)
The model adjusting means includes
The difference between the first statistic for the predicted output calculated by the performance predicting means and the first statistic for the actual output of the system is reduced based on a determination criterion representing a predetermined accuracy. The performance prediction apparatus according to any one of appendix 1 to appendix 6, wherein a parameter of the performance prediction model is adjusted.

The present invention has been described above using the above-described embodiment as an exemplary example. However, the present invention is not limited to the above-described embodiment. That is, the present invention can apply various modes that can be understood by those skilled in the art within the scope of the present invention.

This application claims priority based on Japanese Patent Application No. 2013-251406 filed on Dec. 4, 2013, the entire disclosure of which is incorporated herein.

DESCRIPTION OF SYMBOLS 100 Information processing apparatus 101 Performance prediction part 102 Model adjustment part 103 Validity evaluation part 300 Information processing apparatus 303 Validity evaluation part 304 Presentation part 700 Information processing apparatus 701 Performance prediction part 702 Model adjustment part 703 Validity evaluation part 801 Arithmetic apparatus 802 Storage device 803 Non-volatile storage device 804 External storage device 805 Storage medium 806 Network interface 807 Input / output interface

Claims (10)

1. A performance prediction means for calculating a predicted output for an input to the system, using a performance prediction model schematically representing a system that is a target of performance prediction;
   Model adjustment means for adjusting parameters of the performance prediction model based on a first statistic for the calculated predicted output and a first statistic for the actual output of the system;
   The validity of the performance prediction model is evaluated based on the second statistic for the predicted output calculated using the performance prediction model after adjusting the parameters and the second statistic for the actual output of the system. And a validity evaluation means for performing the performance prediction.
2. Further comprising a presentation means for presenting the parameter adjustment candidates of the performance prediction model to the user,
   The validity evaluation means evaluates the validity of the performance prediction model based on a specific criterion for a plurality of parameter adjustment candidates,
   2. The performance prediction according to claim 1, wherein the presenting means presents to the user an adjustment candidate of a parameter evaluated by the validity evaluating means as being highly valid based on the specific criterion. apparatus.
3. 3. The performance prediction apparatus according to claim 1, wherein the first statistic is an average value, a variance, or a frequency distribution.
4. 4. The performance prediction apparatus according to claim 1, wherein the second statistic is an average value, a variance, or a frequency distribution.
5. 5. The performance prediction apparatus according to claim 1, wherein at least one of the first statistic and the second statistic includes a plurality of types of statistic.
6. 6. The performance prediction apparatus according to claim 1, wherein the second statistic is a statistic of a different type from the first statistic.
7. The model adjusting means includes
   Adjusting the parameter of the performance prediction model so that the difference between the first statistic for the predicted output calculated by the performance prediction means and the first statistic for the actual output of the system is small. The performance prediction apparatus according to claim 1, wherein the performance prediction apparatus is characterized.
8. The validity evaluation means includes:
   Assessing the validity of the performance prediction model by determining the magnitude of the difference between the second statistic for the predicted output and the second statistic for the actual output of the system based on the specific criteria The performance prediction apparatus according to claim 2, wherein:
9. Information processing device
   Using a performance prediction model that models the system that is the target of performance prediction, calculate the predicted output for the input to the system,
   Adjusting the parameters of the performance prediction model based on the first statistic for the calculated predicted output and the first statistic for the actual output of the system;
   The validity of the performance prediction model is evaluated based on the second statistic for the predicted output calculated using the performance prediction model after adjusting the parameters and the second statistic for the actual output of the system. A performance prediction method characterized by:
10. A process for calculating a predicted output with respect to an input to the system using a performance prediction model that models a system that is a target of performance prediction;
    Adjusting the parameters of the performance prediction model based on a first statistic for the calculated predicted output and a first statistic for the actual output of the system;
    The validity of the performance prediction model is evaluated based on the second statistic for the predicted output calculated using the performance prediction model after adjusting the parameters and the second statistic for the actual output of the system. A storage medium storing a computer program, characterized by causing a computer to execute

Images