JP5967091B2 - System parameter setting support system, data processing method of system parameter setting support device, and program - Google Patents

Description

  The present invention relates to a system parameter setting support system that supports system parameter setting, a data processing method of a system parameter setting support apparatus, and a program.

  An IT (Information Technology) system such as a Web system or an online system is required to maintain a certain performance during actual operation. Therefore, performance tests are performed in a test environment before actual operation, and the specifications and number of servers are adjusted and OS (Operating System), middleware, or application parameters are tuned to meet the desired performance. It is common. An example of such a Web system performance evaluation system is described in Patent Document 1.

  Further, in Patent Document 2, for each combination of a plurality of different parameters, the execution result of the program is accepted as statistical information, and based on the statistical information, the parameter group with the largest guaranteed value is detected, thereby increasing the value. It is described that the guaranteed value level is realized.

  Generally, a person who performs a performance test determines the system configuration of the test target system and the test specifications such as the number of simultaneous access clients, the number of simultaneous requests, the request interval, or the request method, and places a load on the test target system. Measure resource consumption, turnaround time, throughput, etc. to find the bottleneck. In order to avoid bottlenecks, the performance evaluator edits the parameter setting file of the OS, middleware, and application, and performs performance tuning of the OS, middleware, and application. Then, in order to confirm the effect, a load test is performed again. By repeating this cycle several times, an appropriate parameter setting value that satisfies the desired performance is determined.

JP 2003-131907 A JP 2006-31605 A

  In the technique described in the above-described document, parameter tuning is manually changed by trial and error, so that there is a problem that parameters that do not need to be changed may be changed. For example, if a load test was performed with a parameter changed to eliminate a bottleneck, but there was no performance improvement effect, the next parameter could be changed without undoing the parameter change. is there.

  Depending on the value of the parameter, a problem may occur only under special conditions. If a parameter that has not been restored is a parameter with such characteristics, problems such as performance problems may occur during actual operation. there is a possibility. When a failure occurs, all parameters are inspected and a setting error is manually detected. However, the number of parameters constituting the IT system is large, and the number of parameters is also proportional to the scale of the system. Therefore, there is a problem that a lot of man-hours and time are required for the response.

  An object of the present invention is to provide a system parameter setting support system, a data processing method for a system parameter setting support apparatus, and a program that solve the difficulty of extracting parameter settings that are estimated to be the cause of performance failure in actual operation, which is the above-described problem Is to provide.

The system parameter setting support system of the present invention includes:
A system parameter setting support system that performs tests sequentially in time series and tunes parameters in consideration of the test results of each test,
At each time of the test performed sequentially in time series, the test result was compared with the previous time to calculate the change, and based on the change in the calculated test result, did the result exceed the predetermined index? A test result evaluation means for evaluating whether or not,
A change parameter extraction means for extracting a test corresponding to a test result evaluated by the test result evaluation means that an effect equal to or greater than the predetermined index cannot be obtained, and extracting a parameter that has been changed during the extracted test; ,
Is provided.

The data processing method of the system parameter setting support device of the present invention includes:
A system parameter setting support device that performs tests sequentially in time series and tunes parameters in consideration of the test results of each test,
In each of the tests performed sequentially in time series, compare the test results with the previous time to calculate the change,
Based on the change in the test result, extract a test that does not achieve an effect that exceeds a predetermined index,
It is a data processing method for extracting the extracted parameters that have changed during the test.

The computer program of the present invention is:
A computer that implements a system parameter setting device that performs tests sequentially in time series and tunes parameters in consideration of the test results of each test,
A procedure for calculating the change by comparing the test result with the previous time in each time of the test sequentially performed in time series,
A procedure for extracting a test based on the change in the test result that does not achieve an effect that exceeds a predetermined index;
It is a program for executing a procedure for extracting extracted parameters that have changed during the test.

  It should be noted that any combination of the above-described constituent elements and a conversion of the expression of the present invention between a method, an apparatus, a system, a recording medium, a computer program, etc. are also effective as an aspect of the present invention.

  The various components of the present invention do not necessarily have to be independent of each other. A plurality of components are formed as a single member, and a single component is formed of a plurality of members. It may be that a certain component is a part of another component, a part of a certain component overlaps with a part of another component, or the like.

  In addition, although a plurality of procedures are described in order in the data processing method and the computer program of the present invention, the described order does not limit the order in which the plurality of procedures are executed. For this reason, when implementing the data processing method and computer program of this invention, the order of the several procedure can be changed in the range which does not have trouble in content.

  Furthermore, the plurality of procedures of the data processing method and the computer program of the present invention are not limited to being executed at different timings. For this reason, another procedure may occur during the execution of a certain procedure, or some or all of the execution timing of a certain procedure and the execution timing of another procedure may overlap.

  According to the present invention, there are provided a system parameter setting support system, a data processing method for a system parameter setting support apparatus, and a program capable of extracting a parameter setting estimated to be a cause of a performance failure in actual operation.

  The above-described object and other objects, features, and advantages will become more apparent from the preferred embodiments described below and the accompanying drawings.

It is the schematic which shows the network structure of the system parameter setting assistance system which concerns on embodiment of this invention. It is a functional block diagram which shows the structure of the system parameter setting assistance system which concerns on embodiment of this invention. It is a flowchart which shows operation | movement of the system parameter setting assistance system which concerns on embodiment of this invention. It is a functional block diagram which shows the structure of the system parameter setting assistance system which concerns on embodiment of this invention. It is a flowchart which shows operation | movement of the system parameter setting assistance system which concerns on embodiment of this invention. It is a functional block diagram which shows the structure of the system parameter setting assistance system which concerns on embodiment of this invention. It is a flowchart which shows the operation | movement at the time of the load test of the system parameter setting assistance system which concerns on embodiment of this invention. It is a flowchart which shows the operation | movement at the time of the setting mistake extraction of the system setting assistance system which concerns on embodiment of this invention. It is a figure which shows the test case list of the system setting assistance system which concerns on embodiment of this invention. It is a figure which shows an example of a structure of the load test result memory | storage part of the system setting assistance system of the Example of this invention. It is a figure which shows an example of a structure of the parameter setting memory | storage part of the system setting assistance system of the Example of this invention. It is a figure which shows an example of the test result in the system setting assistance system of the Example of this invention. It is a figure which shows an example of the comparison result of the performance parameter in the system setting assistance system of the Example of this invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, the same reference numerals are given to the same components, and the description will be omitted as appropriate.

(First embodiment)
FIG. 1 is a schematic diagram showing a network configuration of a system parameter setting support system according to an embodiment of the present invention.
As shown in the figure, a system parameter setting support system according to an embodiment of the present invention includes a test device 1, a test target system 2, a test history storage device 3, a parameter setting device 4, an evaluation device 5, A setting confirmation device 6. The test device 1, the test target system 2, the test history storage device 3, the parameter setting device 4, the evaluation device 5, and the setting confirmation device 6 are connected to each other via a network 7.

Each device includes, for example, a CPU (Central Processing Unit), a memory, a hard disk, and a communication device (not shown), a server computer or a personal computer connected to an input device such as a keyboard and a mouse, and an output device such as a display and a printer, Or it is realizable with the apparatus equivalent to them. Each function of each unit can be realized by the CPU reading the program stored in the hard disk into the memory and executing it. It will be understood by those skilled in the art that there are various modifications to the implementation method and apparatus. Each drawing described below shows functional unit blocks realized by an arbitrary combination of hardware and software of an arbitrary computer, not a configuration of hardware units.
Further, in the following drawings, the configuration of parts not related to the essence of the present invention is omitted and is not shown.

  Further, in the present embodiment of FIG. 1, each device is configured from a different computer, but is not limited to this. The same computer can be configured to realize the functions of at least two devices. The computer may be configured by a virtual server or the like. Alternatively, each device may be composed of a plurality of computers.

  The test apparatus 1 can be a client terminal that transmits a request to the test target system 2 in order to perform a test. The test includes, for example, a performance test that verifies the operation and performance of the system while applying various loads to the test target system 2 and changing various parameter settings. The test can be performed, for example, before the test target system 2 is actually operated, or when the test target system 2 is updated or maintained.

  The test target system 2 includes an IT system to be tested, and further performs a test for measuring system performance in response to a request from the test apparatus 1. The test target system 2 has a plurality of parameters whose settings can be changed. Tuning is performed while changing these parameters during a performance test, and parameters for actual operation are determined.

  The test history storage device 3 can be a server computer that stores and accumulates test results measured by the test target system 2 and parameters set in the test target system 2.

  The parameter setting device 4 can be a tool for receiving parameter settings of the test target system 2 and transferring the parameters to the test target system 2. A user such as an operation manager can perform parameter tuning using the parameter setting device 4 before entering the actual operation stage. The parameter whose setting has been changed is stored in a memory (not shown) of the test target system 2 and used in the actual operation stage.

In the present embodiment, the parameter tuning is performed manually while measuring the effect of the change based on the manual and past experience. In a large scale system, the number of parameters increases, and moreover, a plurality of operators may be involved in tuning work. Therefore, changing parameters during tuning is complicated and difficult to manage accurately.
In addition, the user of the test target system 2 or the operation manager of the test target system 2 can change the parameters by using the parameter setting device 4 in the actual use stage.

  The parameter setting device 4 can also be included in the test target system 2. Various configurations of the tool for realizing the parameter setting device 4 can be considered. For example, a tool for realizing the parameter setting device 4 may be installed in a client terminal connected to the test target system 2 via the network 7. Alternatively, it is also possible to access a server that accepts parameters using a browser from a client terminal and input parameters by operating from the client terminal. In addition, a plurality of users can change parameters of the test target system 2 using the parameter setting device 4.

  The evaluation device 5 evaluates the effect of the parameter change by the parameter setting device 4, extracts the parameter change for which the effect has not been improved, transmits it to the setting check device 6, and the user such as the operation manager of the test target system 2 Can be a server computer to be presented.

  The setting confirmation device 6 can be a client terminal that presents the parameters extracted by the evaluation device 5 to the user. The presentation method to the setting confirmation apparatus 6 is not particularly limited. For example, it can be presented to the user by displaying on a display device (not shown) of the setting confirmation device 6. Alternatively, it can be transmitted as electronic information to the setting confirmation device 6 via the network 7. Alternatively, it can be printed out on an output device (not shown) such as a printer of the setting confirmation device 6. Information may be provided by accessing the setting confirmation device 6 from the evaluation device 5 or may be obtained by requesting information by accessing the evaluation device 5 from the setting confirmation device 6.

FIG. 2 is a functional block diagram showing the configuration of the system parameter setting support system according to the embodiment of the present invention.
The system parameter setting support system of the present invention performs performance parameter changes that have only contributed to performance improvement from the history of performance test results, the performance parameter settings used in each test, and the performance parameter settings used in actual operation. Extract.

  A system parameter setting support system according to an embodiment of the present invention is a system parameter setting support system that performs tests sequentially in time series and tunes parameters in consideration of the test results of each test. At each test that is performed sequentially, the test result is compared with the previous test, and the change is calculated. Based on the change in the calculated test result, it is evaluated whether or not an effect exceeding a predetermined index is obtained. The test result evaluation unit 51 and the test result evaluation unit 51 extract a test corresponding to the test result evaluated that the effect equal to or higher than the predetermined index cannot be obtained, and extract the parameter changed during the extracted test. A change parameter extraction unit 53.

The test history storage device 3 includes a test result storage unit 31 and a parameter setting storage unit 32.
The test result storage unit 31 stores a history of load test results executed in the test target system 2.
The parameter setting storage unit 32 stores a history of parameter names and setting values extracted by the test target system 2 for each test execution.

The evaluation device 5 includes a test result evaluation unit 51 and a change parameter extraction unit 53.
The test result evaluation unit 51 sequentially compares the test results stored in the test result storage unit 31 of the test history storage device 3 in time series, and calculates the degree of change from the previous time. Then, based on the change in the calculated test result, it is evaluated whether or not an effect exceeding a predetermined index is obtained. In this embodiment, the calculated value is compared with a predetermined value, and if there is no difference beyond the predetermined value, or if the performance changes in the direction of worsening, the parameter is changed. However, it is evaluated that a good effect on performance was not obtained.

  The changed parameter extraction unit 53 refers to the parameter setting storage unit 32 of the test history storage device 3, compares the changed parameter settings sequentially in time series, and evaluates that the test has no effect on performance improvement. Extract changes.

In the evaluation apparatus 5 according to the present embodiment, the CPU executes various processing operations corresponding to the computer program, whereby the various units as described above are realized as various functions.
The computer program according to the present embodiment is a computer program for realizing the evaluation apparatus 5. In each test of the tests sequentially performed in time series, the test result is compared with the previous test result and the change is calculated. It is described to execute a procedure for extracting a test that does not provide an effect that exceeds a predetermined index based on the change, and a procedure for extracting a parameter that has changed during the extracted test.

  Note that the computer program of the present embodiment may be recorded on a computer-readable recording medium. The recording medium is not particularly limited, and various forms can be considered. The program may be loaded from a recording medium into a computer memory, or downloaded to a computer through a network and loaded into the memory.

  Moreover, the data processing method by the evaluation apparatus 5 of this Embodiment in the above structures is demonstrated below. FIG. 3 is a flowchart showing the operation of the system parameter setting support system of this embodiment.

  In the data processing method of the system parameter setting support device (evaluation device 5) of the present embodiment, the evaluation device 5 compares the test results with the previous time at each time of the tests sequentially performed in time series (step S11). The change is calculated (step S13), and based on the change in the test result, a test that does not produce an effect that exceeds the predetermined index is extracted (YES in step S15), and the extracted parameter that has changed during the test is extracted. Is extracted (step S17).

The operation of the system parameter setting support system of the present embodiment configured as described above will be described below.
Here, an operation for extracting a parameter setting that may cause a setting error will be described with reference to FIGS. 1 to 3.
This flow repeats the loop processing from i = 1 to (parameter change count −1) (step S11).

  First, the test result evaluation unit 51 of the evaluation device 5 calculates the difference between the first and second test results, for example, the degree of change (step S13). Furthermore, when the test result evaluation unit 51 compares the calculated value with a predetermined value and no difference is found beyond the predetermined value, or when the performance changes in a direction where the performance deteriorates. (YES in step S15), it is considered that the parameter change performed for the second time is not effective, and the process proceeds to step S17. Then, the changed parameter extraction unit 53 extracts the changed parameter name and parameter value by comparing the parameter settings in the first and second tests (step S17).

  Thereafter, the process returns to step S11, and evaluation of the second and third test results is started. Thereafter, step S13 to step S15 or step S17 is repeated as many times as the number of times the test is executed by changing the parameter, that is, {total test number -1} times.

In addition, when a difference is seen more than a predetermined value in step S15 (NO in step S15), it is considered that the parameter change performed for the second time has shown an effect, and the processing after step S17 is omitted. Returning to step S11, the difference between the next second and third test results is calculated (step S13).
With the above operation, the evaluation device 5 changes the parameter when the parameter is changed but the test result is not different from a predetermined value or when the performance changes. The parameter name and parameter setting value are extracted.

  The parameter name and parameter setting value of the changed parameter extracted in this embodiment can be output to the setting confirmation device 6 and presented to the user.

  As described above, according to the system parameter setting support system according to the embodiment of the present invention, no performance improvement was found based on the performance test results and performance parameter settings stored in the history during the test. Parameter changes can be extracted. That is, it is possible to extract a parameter setting that is estimated to be a cause of performance failure in actual operation.

  The reason is that the effect of the parameter change can be measured by extracting the parameter change from the parameter setting history and comparing the performance test results before and after the parameter change to see the degree of the change.

  In addition, according to the system parameter setting support system according to the embodiment of the present invention, it is possible to search for parameter changes that have not been found effective in the performance test. Also, unnecessary parameter changes made in the performance test can be extracted.

(Second Embodiment)
FIG. 4 is a functional block diagram showing the configuration of the system parameter setting support system according to the embodiment of the present invention.
The system parameter setting support system according to the present embodiment is different from the above-described embodiment in that, among the parameters set as the final setting values, parameters that do not require improvement in performance improvement are extracted. .
The system parameter setting support system of this embodiment includes a test history storage device 3 similar to that of the above-described embodiment of FIG. 2, and further includes an evaluation device 50 instead of the evaluation device 5 of FIG. 2.

  The evaluation apparatus 50 according to the embodiment of the present invention includes a test result evaluation unit 51 and a change parameter extraction unit 53 similar to those of the above-described embodiment of FIG. It further includes a parameter setting error candidate extraction unit 54 that compares the parameter setting value extracted by the parameter extraction unit 53 with the changed parameter value and determines whether or not they are the same value.

  Further, the parameter setting error candidate extraction unit 54 of the evaluation apparatus 50 according to the embodiment of the present invention extracts a parameter determined to be the same value as the final parameter setting value as a parameter setting error candidate.

  The parameter setting error candidate extraction unit 54 extracts a test whose degree of change calculated by the test result evaluation unit 51 is equal to or less than a predetermined index, extracts a parameter change at that time, and enters the actual operation environment. The applied parameter setting, that is, the parameter setting having the same value as the final setting value of the parameter stored in the parameter setting storage unit 32 is extracted. Here, the final setting value of the parameter is a setting value used in actual operation of the test target system 2 after the test is completed.

The operation of the system parameter setting support system of the present embodiment configured as described above will be described below.
FIG. 5 is a flowchart showing the operation of the system parameter setting support system of this embodiment. Here, an operation for extracting a parameter setting that may cause a setting error will be described with reference to FIGS. 4 and 5. The flowchart of this embodiment includes steps S11 to S17 similar to the operation of the system parameter setting support system of the above embodiment described with reference to FIG. 3, and further includes steps S21 to S25.

In this flow, the loop processing is repeated from i = 1 to (parameter change count−1).
The description of the same steps as in the above embodiment is omitted.
After the changed parameter is extracted in step S17, the parameter setting error candidate extraction unit 54 of the evaluation device 50 extracts the parameter final setting value from the parameter setting storage unit 32 and compares it with the parameter setting extracted in step S17. (Step S21). If the parameter setting is the same value (YES in step S23), the parameter setting error candidate extraction unit 54 extracts the parameter as a setting error candidate (step S25).

  Thereafter, the process returns to step S11, and evaluation of the second and third test results is started (step S13). Thereafter, Steps S13 to S17 and Steps S21 to S25 are repeated as many times as the number of times the test is executed by changing the parameter, that is, {total number of tests -1}.

  Further, if the parameter setting extracted in step S23 is different from the parameter final setting value (NO in step S23), it is considered that the parameter change having no effect has been restored, and the processing in step S25 and subsequent steps is performed. The process returns to step S11, and the difference between the next and third test results is calculated (step S13).

  With the above operation, the evaluation apparatus 50 has no effect on performance improvement, but can extract the value of the parameter setting of the system that has been released to the actual operation environment or is scheduled to be released.

  The parameter name and parameter setting value of the changed parameter extracted in this embodiment can be output to the setting confirmation device 6 and presented to the user. Further, the parameter name and parameter setting value of the setting error candidate parameter extracted in the present embodiment can be output to the setting confirmation device 6 and presented to the user.

As described above, according to the system parameter setting support system according to the embodiment of the present invention, the performance test result and performance that have the same effects as those of the above-described embodiment and that further store the history at the time of the test. Based on the parameter settings and the performance parameter settings released to the production environment, perform performance parameter settings that are likely to cause performance problems in the production environment or that are likely to cause performance problems that have already occurred. Can be extracted.
The reason for this is that it is not necessary to compare the parameter setting values extracted in the above embodiment that did not improve performance with the performance parameter setting values released to the actual operating environment, and confirm whether they are the same value. This is because it can be confirmed whether or not the parameter change has been applied to the production environment.

(Third embodiment)
FIG. 6 is a functional block diagram showing the configuration of the system parameter setting support system according to the embodiment of the present invention.
The system parameter setting support system of the present embodiment is different from the above-described embodiment in that it extracts parameter changes that are not necessary to improve performance and presents them to the user. Furthermore, the system parameter setting support system of the present embodiment is different from the above-described embodiment in that it further includes a configuration for performing a test while changing parameters and storing the result.

The system parameter setting support system of this embodiment includes a test history storage device 3 and an evaluation device 50 similar to those of the above embodiment of FIG.
Details of the test apparatus 1, the test target system 2, the parameter setting apparatus 4, and the setting confirmation apparatus 6 described with reference to FIG.

As shown in FIG. 6, the test apparatus 1 includes a test execution unit 11 and a test result display unit 12.
In the present embodiment, the test execution unit 11 transmits a request to the test target system 2 in order to perform a load test in which a load is applied to the test target system 2 and a response time is measured. Here, the request is, for example, a request for a communication response to the test target system 2, a request for execution of a process (program) in a CPU (not shown), a memory (not shown) of the test target system 2. ), Etc. that require access to.

  As shown in FIG. 9, for example, there is a method of applying a load of different test cases. In the present embodiment, the load test is performed for three types of load cases, for example, when the load is quiet, normal, and peak. For example, the test execution unit 11 transmits a request once per second to the test target system 2 when the load is light. During normal times, 10 requests are transmitted per second, and during peak times, 100 requests are transmitted per second.

  Further, in the present embodiment, as shown in FIG. 9, the load test is performed by dividing the case where different loads are applied, but the present invention is not limited to this. For example, while changing the load, the degree of change in the load can be changed, or a test can be performed for a transition period of the load change.

  Returning to FIG. 6, the test result display unit 12 displays the result of the load test on the test target system 2. A user such as an operation manager of the test target system 2 can proceed with the tuning operation while referring to the test result displayed on the test result display unit 12 and further changing the parameter setting using the parameter setting device 4. .

The test target system 2 includes a test processing unit 21, a test result measurement unit 22, and a parameter setting extraction unit 23.
The test processing unit 21 executes a process to be a test target of the test target system 2. For example, a process of reading a plurality of data from a database and converting it into a format for display on a Web browser, a process of updating data in the database, and the like are executed.

  The test result measuring unit 22 measures the performance information of the test processing unit 21 during the load test, and calculates statistical data as the load test result. The performance information includes the response speed to the request, the number of processes per unit time, and the like. In the statistical processing, for example, average processing, percentile processing, and the like are performed.

  The parameter setting extraction unit 23 extracts parameter settings when the test processing unit 21 performs a test, specifically, parameter names and setting values. The parameters include various performance parameters related to the performance of applications, middleware, and OS.

  For example, the parameters include performance parameters such as heap memory size, maximum number of threads, maximum number of database connections, or maximum number of simultaneous sessions, processing timeout period, maximum number of server processes. The parameter setting extraction unit 23 can extract parameters described in, for example, a predetermined parameter setting file. Alternatively, a setting value for a predetermined parameter name can be extracted.

The parameter setting device 4 includes a parameter change receiving unit 41.
The parameter change accepting unit 41 accepts a parameter setting value input by a person who sets parameters, such as an operation manager, using an operation unit (not shown) or the like, and passes it to the test processing unit 21 of the test target system 2.

  The setting confirmation device 6 includes a presentation unit 61. The presenting unit 61 displays the parameter setting that has been extracted by the parameter setting error candidate extracting unit 54 or the parameter setting of the performance parameter setting error candidate, specifically, at least the parameter name and setting value.

The operation of the system parameter setting support system of the present embodiment configured as described above will be described below.
First, an operation until a parameter setting satisfying a desired performance after performing a load test will be described with reference to FIGS.
FIG. 7 is a flowchart showing a procedure of operations performed until a parameter setting that satisfies a desired performance is found by performing a load test.

  The test execution unit 11 of the test apparatus 1 transmits a load request for performing a load test to the test processing unit 21 (step S101). At this time, a plurality of pattern requests are transmitted. When transmitting a load request to the test processing unit 21, in the test target system 2, the test processing unit 21 of the test target system 2 processes the request received from the test apparatus 1 (step S201).

  Then, the test result measuring unit 22 of the test target system 2 measures performance data, and calculates performance statistical data, for example, an average value of each test pattern from the measured data (step S203). Then, the calculated performance statistical data is stored in the test result storage unit 31 of the test history storage device 3 (step S301).

  Next, the parameter setting extraction unit 23 of the test target system 2 extracts the parameter setting from the test processing unit 21 (step S205) and stores it in the parameter setting storage unit 32 of the test history storage device 3 (step S303). Here, the parameter setting extraction unit 23 of the test target system 2 extracts the parameter setting value corresponding to the parameter name of the parameter whose setting is changed in advance from the setting file.

  Next, the test result measurement unit 22 of the test target system 2 displays the performance statistical data calculated in step S203 on the test result display unit 12 of the test apparatus 1 (step S103). The user confirms by referring to the test result displayed on the test result display unit 12. If the test result does not achieve the desired performance, that is, if there is a problem with the test result (YES in step S105), the user changes the performance parameter using the operation unit (not shown) of the parameter setting device 4 Then, the parameter change receiving unit 41 receives the change (step S401). The parameter change accepted by the parameter setting device 4 is transmitted to the test target system 2, and the parameter is changed in the test target system 2 (step S207). Then, the load test is performed again (step S101).

On the other hand, when the desired performance is achieved (NO in step S105), the parameter setting extraction unit 23 of the test target system 2 extracts the set parameter (step S209), and sets it as the parameter final setting value. It memorize | stores in the parameter setting memory | storage part 32 of the test history memory | storage device 3 (step S305).
As described above, the test result storage unit 31 stores a history of load test results, and the parameter setting storage unit 32 stores parameter settings at the time of performing each test.

Next, an operation for extracting parameter settings that may cause a setting error will be described with reference to FIGS.
FIG. 8 is a flowchart showing the operation at the time of setting error extraction.
Note that the process of the setting device 50 of this flow repeats the loop process from i = 1 to (parameter change count−1).
First, the test result evaluation unit 51 of the evaluation device 50 refers to the test result storage unit 31 of the test history storage device 3 (step S311) and calculates the difference between the first and second test results, for example, the degree of change. (Step S503). Furthermore, when the test result evaluation unit 51 compares the calculated value with a predetermined value and no difference is found beyond the predetermined value, or when the performance changes in a direction where the performance deteriorates. (YES in step S505), it is considered that the parameter change performed for the second time is not effective, and the process proceeds to step S507. Then, the changed parameter extraction unit 53 refers to the parameter setting storage unit 32 of the test history storage device 3 (step S313), compares the parameter settings in the first and second tests, and changes the parameter name and parameter. A value is extracted (step S507).

  Further, the parameter setting error candidate extraction unit 54 extracts the parameter final setting value from the parameter setting storage unit 32 and compares it with the parameter setting extracted in step S507 (step S509). If the parameter setting is the same value (YES in step S511), the parameter setting is transmitted to the presentation unit 61 of the setting confirmation device 6. Then, the presentation unit 61 of the setting confirmation device 6 displays the parameter setting received from the evaluation device 50 (step S601). Thereafter, the process returns to step S501, and evaluation of the second and third test results is started (step S503). Thereafter, Steps S501 to S511 are repeated as many times as the number of times the test is executed by changing the parameters, that is, {total number of tests-1} times.

  In addition, when a difference is seen more than a predetermined value in step S505, that is, when there is a performance improvement (NO in step S505), it is considered that the second parameter change has shown an effect, The processing after step S507 is omitted, and the process returns to step S501 to calculate the difference between the second and third test results (step S503).

  Further, if the parameter setting extracted in step S511 is different from the parameter final setting value (NO in step S511), it is assumed that the parameter change that has not been effective has been restored, and the processes in and after step S511 are performed. The process returns to step S501, and the difference between the next and third test results is calculated (step S503).

  As described above, performing the processing from step S503 to step S511 {total number of tests -1} times did not improve the performance, but the parameter setting that was released to the actual operating environment as it is was set. Values can be extracted and displayed.

As described above, according to the system parameter setting support system according to the embodiment of the present invention, the same effect as the above embodiment is obtained, and the performance parameter setting having a low effect is extracted by comparing the load test results. Can be presented to the user.
When a performance problem occurs in an actual operating environment, it is usually necessary to check all parameters of the system and confirm the relationship with the generated performance problem. Performance parameters can be narrowed down in advance. For this reason, it becomes possible to expedite the cause and the countermeasure of the performance problem.

  In addition, by using this embodiment before starting operation in the actual operating environment, it is possible to detect performance parameter setting errors before moving to the actual operating environment, thus improving the reliability of the IT system in the actual operating environment. be able to.

As mentioned above, although embodiment of this invention was described with reference to drawings, these are the illustrations of this invention, Various structures other than the above are also employable.
For example, the present invention can also be applied to uses such as a program that realizes a load test and parameter confirmation, and a program that realizes operation monitoring of an IT system and a cause search when a performance failure occurs.
Alternatively, when a performance problem occurs in an IT system such as a Web system or an online system, the present invention narrows down parameter setting values that may cause a performance failure from the parameter settings of the IT system, It can also be used for investigation.

Further, in the above embodiment, the configuration is such that the changed parameter setting is extracted when an effect greater than the predetermined index is not obtained. However, the parameter setting changed when the effect greater than the predetermined index is obtained. It can also be set as the structure which extracts.
In other words, the change parameter extraction unit of the evaluation device extracts a test corresponding to the test result that the test result evaluation unit has evaluated that an effect equal to or greater than a predetermined index has been obtained, and the parameter that has been changed during the extracted test is extracted. Can be extracted.

  According to this configuration, for example, when a proper operation is performed in a system already in operation and a good performance is obtained, the parameter setting can be applied to another similar system.

Example 1
Embodiments of the system parameter setting support system of the present invention will be described below.
First, in this embodiment, a specific operation until a parameter setting satisfying a desired performance is found by performing a load test will be described with reference to FIGS. 6, 7, 9 to 11.
FIG. 10 is a diagram illustrating an example of the configuration of the load test result storage unit of the system setting support system according to the embodiment of this invention. FIG. 11 is a diagram illustrating an example of the configuration of the parameter setting storage unit of the system setting support system according to the embodiment of this invention.

  First, as shown in FIG. 9, the test execution unit 11 of the test apparatus 1 executes three test cases in a quiet period (1 request per second), a normal period (10 requests per second), and a peak time (100 requests per second). (Step S101). The test processing unit 21 of the test target system 2 processes the received request (step S201). The test result measuring unit 22 of the test target system 2 measures the response time for the request (step S203). Here, as shown in FIG. 10, it is assumed that the measurement result of the first test shows 100 msec in the off-season case, 130 msec in the normal case, and 800 msec in the peak case. The test result measurement unit 22 stores these values in the test result storage unit 31 of the test history storage device 3 (step S301).

  Further, the parameter setting extraction unit 23 of the test target system 2 extracts the parameter setting when the test processing unit 21 performs the test, and as shown in the first time in FIG. 11, the heap memory size 64M, the maximum number of threads 40, A value of 10 DB maximum connections is acquired (step S205). And it memorize | stores in the parameter setting memory | storage part 32 of the test history memory | storage device 3 (step S303).

  Next, the test result display unit 12 of the test apparatus 1 displays the first test result shown in FIG. 10 performed by the test processing unit 21 (step S103). Here, if the response time is 400 msec or less as the performance request value, the peak case 800 msec does not satisfy the request in this test result (YES in step S105), so the user uses the operation unit to set the maximum number of threads for the parameter. Is changed to 20 and is received by the parameter change receiving unit 41 of the parameter setting device 4 (step S401).

  In order to evaluate the effect of the changed parameter, the test execution unit 11 of the test apparatus 1 makes a request (step S101), and the test processing unit 21 of the test target system 2 processes the request and executes a load test ( Step S201). The test result measuring unit 22 of the test target system 2 measures the response time for the request (step S203). As a result, as shown in FIG. 10, the measurement result of the second test is 103 msec in the off-season case, 120 msec in the normal case, and 780 msec in the peak case. The test result measurement unit 22 stores these values in the test result storage unit 31 of the test history storage device 3 (step S301).

  Further, the parameter setting extraction unit 23 of the test target system 2 extracts the parameter setting when the test processing unit 21 performs the test for the second time (step S205). The parameter settings extracted at this time are as shown in the second time in FIG. Then, the parameter setting extraction unit 23 stores the parameter setting at this time in the parameter setting storage unit 32 of the test history storage device 3 (step S303).

  However, since the peak case 780 msec is not less than the required performance value of 400 msec even in the second test result, there is a problem with the result (YES in step S105), and the user sets the parameter DB maximum connection count to 50 using the operation unit. It is assumed that the parameter change receiving unit 41 of the parameter setting device 4 receives the change (step S401). Here, the maximum number of threads is kept at 20. The parameter change accepted by the parameter setting device 4 is transmitted to the test target system 2, and the parameter is changed in the test target system 2 (step S207).

  Again, in order to evaluate the effect of the changed parameter, the test execution unit 11 of the test apparatus 1 makes a request (step S101), and the test processing unit 21 of the test target system 2 processes the request and executes a load test. (Step S201). The test result measuring unit 22 of the test target system 2 measures the response time for the request (step S203). As a result, as shown in FIG. 10, it is assumed that the measurement result of the third test shows 98 msec in the off-season case, 123 msec in the normal case, and 304 msec in the peak case. The test result measurement unit 22 stores these values in the test result storage unit 31 of the test history storage device 3 (step S301).

  Further, the parameter setting extraction unit 23 extracts the parameter setting when the test processing unit 21 performs the test for the third time (step S205). The parameter settings extracted at this time are as shown in the third time in FIG. Then, the parameter setting extraction unit 23 stores it in the parameter setting storage unit 32 of the test history storage device 3 (step S303).

  Here, since the third test result satisfies the performance requirement value of 400 msec or less in all test cases, it is determined that there is no problem in the result (NO in step S105), and the load test is terminated here. The parameter setting extraction unit 23 of the test target system 2 at this time extracts the parameter setting at this time (step S209), and uses the extracted parameter as a parameter final setting value in the parameter setting storage unit 32 of the test history storage device 3. Store (step S305).

Next, specific examples for extracting parameter settings that may cause a change error will be described with reference to FIGS. 6 and 8 and FIGS. 11 to 13.
FIG. 12 is a diagram illustrating an example of a test result in the system setting support system according to the embodiment of this invention. FIG. 13 is a diagram illustrating an example of performance parameter comparison results in the system setting support system according to the embodiment of this invention.

  First, the test result evaluation unit 51 of the evaluation device 50 calculates a change rate as a difference between the first test result and the second test result (steps S501 and S503). As shown in FIG. 12A, the result was + 3.0% in the off-season case, -7.7% in the normal case, and -2.5% in the peak case. In the present embodiment, it is assumed that there is an effect of changing parameters when improvement of −20% or less is observed. Since the change rates of the test results for the first and second tests are all -20% or more, it is determined that the parameter setting changed for the second evaluation has no particular effect (YES in step S505).

  Next, the changed parameter extraction unit 53 extracts the changed parameter name and parameter value by comparing the parameter settings in the first and second tests (step S507). As shown in FIG. 13A, it can be confirmed that the maximum number of threads has been changed from 40 to 20 in the second test. In order to confirm whether or not this parameter setting is also used in the actual operation environment, the parameter setting error candidate extraction unit 54 extracts the parameter final setting value from the parameter setting storage unit 32 and extracts the parameter setting extracted in step S507. (Step S509).

As shown in FIG. 11, when compared with the parameter final set value, it can be confirmed that the second parameter and the parameter final set value are the same as shown in FIG. 13B (YES in step S511). That is, since the parameter change that had no effect is released to the actual operation environment as it is, there is a high possibility that a problem will occur due to this parameter. Therefore, the presentation unit 61 of the setting confirmation apparatus 6 uses the parameter name “maximum thread”. Number ”and parameter value“ 20 ”are displayed (step S601). For example, as shown in FIG. 13B, the parameter name of the parameter of the setting error candidate, the parameter value before the setting is changed, the parameter value after the setting is changed, and the final setting value are included. A list may be displayed on the screen. Accordingly, it is possible to notify the user which parameter is a setting error candidate parameter and which value should be returned to the setting.
Then, the second and third comparisons are made (steps S513 and S501).

  Returning to step S501, the rate of change between the second and third test results is calculated (step S503). As shown in FIG. 12B, the result was -4.9% in the off-season case, -2.5% in the normal case, and -61.0% in the peak case. Here, since an improvement of −20% or less is observed in the peak case, it is determined that there is an effect of changing the parameters (NO in step S505). Therefore, without confirming the parameter change after step S507, the process returns to step S501 and proceeds to the next process. In this embodiment, since the comparison of all the test results from the first time to the third time is completed, the parameter extraction process is completed here.

  As a result, the parameter name “maximum number of threads” and the parameter value “20” are displayed as parameter setting error candidates on the presentation unit 61 of the setting confirmation apparatus 6. If other parameter setting error candidates are detected, for example, a record can be added to the list as shown in FIG. 13B and displayed. The user can confirm a parameter setting error candidate on the displayed screen, and can perform an operation to restore the parameter setting with the parameter setting device 4 as necessary.

  While the present invention has been described with reference to the embodiments and examples, the present invention is not limited to the above embodiments and examples. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

A part or all of the above-described embodiment can be described as in the following supplementary notes, but is not limited thereto.
(Appendix 1)
A system parameter setting support device that performs tests sequentially in time series and tunes parameters in consideration of the test results of each test,
In each of the tests performed sequentially in time series, compare the test results with the previous time to calculate the change,
Based on the change in the test result, extract a test that does not achieve an effect that exceeds a predetermined index,
A data processing method for a system parameter setting support apparatus that extracts extracted parameters that have changed during the test.
(Appendix 2)
In the data processing method of the system parameter setting support device according to attachment 1,
The system parameter setting support device further includes:
Data of the system parameter setting support device that, after the test, compares the set value of the parameter where the change was detected and the final set value of the parameter to determine whether or not they are the same value Processing method.
(Appendix 3)
In the data processing method of the system parameter setting support device according to appendix 1 or appendix 2,
A data processing method for a system parameter setting support apparatus, wherein the system parameter setting support apparatus further presents a parameter name and a setting value of the extracted parameter with the change to the user.
(Appendix 4)
In the data processing method of the system parameter setting support device according to attachment 2,
A data processing method for a system parameter setting support device, wherein the system parameter setting support device further extracts, as a parameter setting error candidate, a parameter determined to be the same value as the final setting value of the parameter.
(Appendix 5)
In the data processing method of the system parameter setting support device according to attachment 4,
The system parameter setting support device further provides a data processing method of the system parameter setting support device for presenting the parameter name and setting value of the extracted parameter setting error candidate to the user.

(Appendix 6)
A computer that implements a system parameter setting support device that performs tests sequentially in time series and tunes parameters in consideration of the test results of each test,
A procedure for calculating the change by comparing the test result with the previous time in each time of the test sequentially performed in time series,
A procedure for extracting a test based on the change in the test result that does not achieve an effect that exceeds a predetermined index;
The program for performing the procedure which extracts the parameter which was changed at the time of the said extracted test.
(Appendix 7)
In the program described in Appendix 6,
After the test, the computer further executes a procedure for comparing the setting value of the parameter where the change is detected and the final setting value of the parameter to determine whether the change is the same or not. Program for.
(Appendix 8)
In the program described in appendix 6 or appendix 7,
A program for causing a computer to further execute a procedure of presenting a parameter name and a set value of a parameter that has been extracted to the user.
(Appendix 9)
In the program described in Appendix 7,
A program for causing a computer to further execute a procedure for extracting a parameter determined to be the same as the final setting value of the parameter as a parameter setting error candidate.
(Appendix 10)
In the program described in Appendix 9,
A program for causing a computer to further execute a procedure for presenting a parameter name and a set value to a user for the extracted parameter setting error candidate.

  This application claims the priority on the basis of Japanese application Japanese Patent Application No. 2011-170701 for which it applied on August 4, 2011, and takes in those the indications of all here.

Claims (10)

A system parameter setting support system that performs tests sequentially in time series and tunes parameters in consideration of the test results of each test,
At each time of the test performed sequentially in time series, the test result was compared with the previous time to calculate the change, and based on the change in the calculated test result, did the result exceed the predetermined index? A test result evaluation means for evaluating whether or not,
A change parameter extraction means for extracting a test corresponding to a test result evaluated by the test result evaluation means that an effect equal to or greater than the predetermined index cannot be obtained, and extracting a parameter that has been changed during the extracted test; ,
A system parameter setting support system comprising: In the system parameter setting support system according to claim 1,
After the test, the system parameter further includes a determination unit that compares the final setting value of the parameter with the setting value of the parameter that has been changed extracted by the changed parameter extraction unit, and determines whether or not they are the same value Setting support system. In the system parameter setting support system according to claim 1 or 2,
A system parameter setting support system further comprising a changed parameter presenting means for presenting a parameter name and a set value of the parameter that has been extracted by the changed parameter extracting means to the user. In the system parameter setting support system according to claim 2,
A system parameter setting support system further comprising parameter setting error candidate extraction means for extracting, as the parameter setting error candidate, the parameter determined by the determination means to be the same value as the final setting value of the parameter. In the system parameter setting support system according to claim 4,
A system parameter setting support system further comprising parameter setting error candidate presenting means for presenting a parameter name and a setting value to the user for the parameter setting error candidate extracted by the parameter setting error candidate extracting means. A system parameter setting support device that performs tests sequentially in time series and tunes parameters in consideration of the test results of each test,
In each of the tests performed sequentially in time series, compare the test results with the previous time to calculate the change,
Based on the change in the test result, extract a test that does not achieve an effect that exceeds a predetermined index,
A data processing method for a system parameter setting support apparatus that extracts extracted parameters that have changed during the test. In the data processing method of the system parameter setting support device according to claim 6,
The system parameter setting support device further includes:
Data of the system parameter setting support device that, after the test, compares the set value of the parameter where the change was detected and the final set value of the parameter to determine whether or not they are the same value Processing method. In the data processing method of the system parameter setting support device according to claim 7,
A data processing method for a system parameter setting support device, wherein the system parameter setting support device further extracts, as a parameter setting error candidate, a parameter determined to be the same value as the final setting value of the parameter. A computer that implements a system parameter setting support device that performs tests sequentially in time series and tunes parameters in consideration of the test results of each test,
A procedure for calculating the change by comparing the test result with the previous time in each time of the test sequentially performed in time series,
A procedure for extracting a test based on the change in the test result that does not achieve an effect that exceeds a predetermined index;
The program for performing the procedure which extracts the parameter which was changed at the time of the said extracted test. The program according to claim 9,
After the test, the computer further executes a procedure for comparing the setting value of the parameter where the change is detected and the final setting value of the parameter to determine whether the change is the same or not. Program for.

Images