WO2013015292A1 - Quality evaluation apparatus, quality evaluation method, and storage medium having program for same stored therein - Google Patents

Abstract

The present invention provides a quality evaluation apparatus capable of showing the state of SLA compliance at a specific point of time during provision of service. The quality evaluation apparatus is provided with: an acceptable range calculating means for calculating the acceptable range for the quantity of instances of monitoring data not satisfying a monitoring standard value, which is the standard value for the monitoring data, said acceptable range being calculated on the basis of both a violating condition defining a condition of operation violation by the object to be monitored, and a total monitoring count for monitoring for the violating condition, said total monitoring count being calculated from the monitoring data acquisition interval and the monitoring period; and a compliance rate calculating means for calculating a compliance rate with respect to the violating condition at an arbitrary point of time during the monitoring period, on the basis of the monitoring data and the acceptable range.

Description

Quality evaluation apparatus, quality evaluation method, and recording medium recording program therefor

The present invention relates to a quality evaluation apparatus, a quality evaluation method, and a program for evaluating the quality of service.

In the IT (Information Technology) field, contracts centered on services, such as SaaS (Software as a Service), are increasing.
In such a contract, a service level agreement (SLA) is used as a means for guaranteeing the quality of service. An SLA is a contract between a service provider and a user for a quantitative and explicit agreement on service levels (eg, service content, scope and quality) for each of one or more service level items. is there.
The service level of the service level item in the SLA is, for example, “the service operation rate is 99.9% or more” and “the average recovery time when a failure occurs is within 12 hours”.
Non-Patent Document 1 shows service level items and reference information for determining the service level when exchanging SLA related to SaaS.
Further, the service level of each service level item exchanged in the SLA may be used for evaluating the quality of the service.
Patent Document 1 discloses an example of a service evaluation system.
The service evaluation system disclosed in Patent Document 1 first converts previously provided SLA definition information into monitoring items. Here, the SLA definition information is a service level item and definition information of the service level. The monitoring items are a set value and a monitoring method.
Next, the service evaluation system monitors individual items (for example, operation status, response status, and number of connected users) in the software service of the software service provider, and records a history.
Next, the service evaluation system analyzes the recorded history, and records the result of analysis based on the history of a predetermined period (for example, from 0:00 to 24:00 on the previous day) as a usage record.
Next, the service evaluation system compares the set value with the usage record to determine whether there is a possibility of SLA violation.

JP 2010-257272 A

The service evaluation system described in Patent Document 1 described above has a problem that it cannot indicate the SLA compliance status of the monitoring target system. Here, the SLA compliance status indicates how close the monitored system is to the occurrence of SLA violation at a specific time point during service provision, and indicates how much the SLA is violated. Information.
The reason for this is that the service evaluation system of Patent Document 1 compares the result of a predetermined period (monitoring period) with the set value corresponding to the SLA definition to determine whether there has been an SLA violation. .
Specifically, in the service evaluation system disclosed in Patent Document 1, the SLA definition information includes, for example, “one of the service level item reference values in 98% of the total response time in a monitoring period of 24 hours”. The response time is within 3 seconds. ” Here, the response time is, for example, the time of one operation from when a certain service using terminal transmits a certain request to a certain service provider until obtaining a response to the request. That is, “98% of the total response time” indicates “98 response times out of 100 response times corresponding to 100 requests (responses)”.
In this case, for example, at 11 hours from the start of monitoring, the number of occurrences of violation of the reference value (response time exceeding 3 seconds) is 1.5% of the total number of responses, and 0.5% The former is closer to the occurrence of SLA violation. That is, the number of occurrences of the reference value violation during monitoring or the occurrence rate thereof is information indicating the SLA compliance status. However, the service evaluation system of Patent Document 1 cannot show such information.
[Object of the invention]
An object of the present invention is to provide a quality evaluation apparatus, a quality evaluation method, and a program therefor that solve the above-described problems.

The quality evaluation apparatus according to the present invention includes a violation condition that defines a condition for a violation operation to be monitored, a monitoring data acquisition interval corresponding to the violation condition, and a total monitoring number calculated based on a monitoring period corresponding to the violation condition. And an allowable range calculating means for calculating and outputting an allowable range indicating an allowable range of the amount of monitoring data that does not satisfy the monitoring reference value that is the reference value of the monitoring data, and
Compliance rate calculation means for calculating and outputting a compliance rate for the violation condition at an arbitrary time point during the monitoring period based on the monitoring data and the allowable range.
In the quality evaluation method of the present invention, the computer
The monitoring based on the violation condition that defines the condition of the violation operation to be monitored, and the total number of monitoring calculated based on the monitoring data acquisition interval corresponding to the violation condition and the monitoring period corresponding to the violation condition Calculate and output an allowable range indicating the allowable range of the amount of monitoring data that does not meet the monitoring reference value, which is the data reference value,
Based on the monitoring data and the allowable range, a compliance rate with respect to the violation condition at an arbitrary point in the monitoring period is calculated and output.
The program recorded in the nonvolatile storage medium of the present invention is stored in a computer.
The monitoring based on the violation condition that defines the condition of the violation operation to be monitored, and the total number of monitoring calculated based on the monitoring data acquisition interval corresponding to the violation condition and the monitoring period corresponding to the violation condition Processing for calculating and outputting an allowable range indicating an allowable range of the amount of monitoring data that does not satisfy the monitoring reference value that is the data reference value;
Based on the monitoring data and the allowable range, a compliance rate with respect to the violation condition at an arbitrary point in the monitoring period is calculated and output.

The present invention has an effect of making it possible to indicate the SLA compliance status at a specific time point during service provision.

FIG. 1 is a block diagram showing the configuration of the quality evaluation apparatus according to the first embodiment of the present invention. FIG. 2 is a diagram illustrating an example of the SLA definition in the first embodiment of the present invention. FIG. 3 is a diagram illustrating an example of a monitoring log according to the first embodiment of this invention. FIG. 4 is a diagram illustrating an example of monitoring log total recording according to the first embodiment of this invention. FIG. 5 is a flowchart showing an operation of recording the SLA definition 2100 in the SLA definition storage unit 1021 according to the first embodiment of this invention. FIG. 6 is a flowchart showing the overall operation of the quality evaluation apparatus in the first embodiment of the present invention. FIG. 7 is a flowchart showing the operation of the monitoring unit in the first embodiment of the present invention. FIG. 8 is a flowchart showing the operation of the SLA allowable range calculation unit in the first embodiment of the present invention. FIG. 9 is a flowchart showing the operation of the SLA compliance rate calculation unit in the first embodiment of the present invention. FIG. 10 is a block diagram showing the configuration of the apparatus according to the second embodiment of the present invention. FIG. 11 is a block diagram showing the configuration of the apparatus according to the third embodiment of the present invention. FIG. 12 is a diagram illustrating an example of the SLA definition in the third embodiment of the present invention. FIG. 13 is a diagram illustrating an example of monitoring log total recording according to the third embodiment of the present invention. FIG. 14 is a flowchart illustrating the operation of the total monitoring number estimation unit according to the third embodiment of this invention. FIG. 15 is a block diagram showing a configuration of a quality evaluation apparatus according to the fourth embodiment of the present invention. FIG. 16 is a flowchart illustrating the operation of the SLA violation predicting unit according to the fourth embodiment of this invention. FIG. 17 is a block diagram showing a configuration of a quality evaluation apparatus according to the fifth embodiment of the present invention. FIG. 18 is a flowchart illustrating the operation of the SLA compliance rate aggregation unit according to the fifth embodiment of this invention. FIG. 19 is a block diagram showing the configuration of a quality evaluation apparatus that causes a computer to execute predetermined processing by the program of the present invention. FIG. 20 is a block diagram illustrating an example of a non-volatile storage medium on which a program is recorded.

[First Embodiment]
FIG. 1 is a block diagram showing a configuration of a quality evaluation apparatus 1000 according to the first embodiment of the present invention.
As shown in FIG. 1, the quality evaluation apparatus 1000 according to the first embodiment includes a monitoring unit 1011, an SLA definition storage unit 1021, a log storage unit 1022, an SLA violation range calculation unit 1031 and an SLA compliance rate calculation. Part 1032.
Based on the SLA definition, the monitoring unit 1011 performs monitoring on a monitoring target system (not shown) and acquires monitoring data that is a result of the monitoring.
FIG. 2 is a diagram illustrating an example of the SLA definition 2100. Referring to FIG. 2, the SLA definition 2100 is composed of one or more SLA definition records 2107. The SLA definition record 2107 includes a service level item 2101, a monitoring reference value 2102, an SLA violation condition (also called violation condition) 2103, a monitoring method 2104, a monitoring interval (also called an acquisition interval) 2105, and a monitoring period 2106.
A service level item 2101 indicates an item monitored as an SLA. The monitoring reference value 2102 indicates a value that the corresponding service level item 2101 should satisfy. The SLA violation condition 2103 indicates a condition for determining that the corresponding service level item 2101 is an SLA violation. The monitoring method 2104 indicates a method (means) for monitoring the corresponding service level item 2101. A monitoring interval 2105 indicates an interval at which the corresponding service level item 2101 is monitored. The monitoring period 2106 indicates a monitoring period for determining whether there is an SLA violation.
The monitoring target system is, for example, a WEB three-layer system including a single server or a plurality of servers, and a data center.
The monitoring items (service level item 2101) to be executed for the monitored system include, for example, availability such as service time and operation rate, reliability such as average recovery time and failure notification time, response time and CPU usage rate. And performance information such as batch processing time.
That is, the monitoring unit 1011 performs a monitoring operation on the service level item 2101 based on the information in the SLA definition record 2107. Specifically, the monitoring unit 1011 acquires monitoring data by executing a monitoring operation at a time interval corresponding to the monitoring interval 2105 using the monitoring method indicated by the monitoring method 2104 corresponding to a certain service level item 2101. To do. For example, the monitoring unit 1011 performs the monitoring operation at “every 5 minutes” indicated by the monitoring interval 2105 using “ping” indicated by the monitoring method 2104 in which the service level item 2101 corresponds to “service availability”. Execute and get monitoring data.
Further, each time the monitoring unit 1011 executes the monitoring operation, the monitoring unit 1011 generates a monitoring log related to the service level item 2101 based on the acquired monitoring data, and outputs the monitoring log to the log storage unit 1022. The monitoring data is monitoring data acquired by executing the monitoring operation once.
In addition, the monitoring unit 1011 generates a monitoring log total record for the service level item 2101 based on the acquired monitoring data for each segment of the monitoring period 2106, and outputs it to the log storage unit 1022. Note that the monitoring data is all the monitoring data acquired from the previous segment of the monitoring period 2106 to the present segment.
The monitoring unit 1011 may output the monitoring log record 2206 to the SLA allowable range calculation unit 1031 and the SLA compliance rate calculation unit 1032. Similarly, the monitoring unit 1011 may output the monitoring log aggregation record 2307 to the SLA allowable range calculation unit 1031 and the SLA compliance rate calculation unit 1032. In this case, the SLA allowable range calculation unit 1031 and the SLA compliance rate calculation unit 1032 may store the monitoring log 2200 and the monitoring log total recording 2300 in a storage unit (not shown).
FIG. 3 is a diagram illustrating an example of the monitoring log 2200. Referring to FIG. 3, the monitoring log 2200 includes a monitoring log record 2206 including at least a monitoring date / time 2201, a monitoring item 2202, a monitoring result 2203, and a reference value violation presence / absence 2204.
The monitoring date / time 2201 indicates the date / time when the monitoring unit 1011 executed the monitoring operation.
A monitoring item 2202 indicates an item of monitoring operation executed by the monitoring unit 1011. When the content of the monitoring item 2202 matches the content of the service level item 2101, the monitoring log record 2206 including the monitoring item 2202 corresponds to the SLA definition record 2107 including the service level item 2101. That is, the monitoring log record 2206 is a monitoring log record 2206 for the monitoring operation executed by the monitoring unit 1011 based on the SLA definition record 2107.
The monitoring result 2203 is information indicating the result of the monitoring operation generated by the monitoring unit 1011 based on the monitoring data acquired by executing the monitoring operation.
For example, referring to FIG. 3, the monitoring result 2203 included in the monitoring log record 2206 in which the monitoring item 2202 is “service time” is “◯”. This indicates that the result of the monitoring operation by the monitoring unit 1011 is normal. That is, when the monitoring result 2203 is “◯”, the monitoring is performed during the time interval indicated by the monitoring interval 2105 (for example, from 2010/2/15, 14:59 to 2010/2/15, 15:00). Indicates that the service of the target system is operating normally.
On the other hand, when there is a time when the operation of the service of the monitoring target system is not normal, the monitoring result 2203 is, for example, a value less than 100% corresponding to the time when the operation is not confirmed.
In addition, the monitoring result 2203 included in the monitoring log record 2206 in which the monitoring item 2202 is “service operating rate” is “100%”. This is the service operation rate calculated by the monitoring unit 1011 based on the unscheduled stop time that occurs from the monitoring start time based on the monitoring period 2106 to the time when the service operation rate is calculated. For example, when the monitoring start time based on the monitoring period 2106 is “2010/2/15, 14:00” and the service operation rate calculation time is “2010/2/15, 15:20”, the monitoring period 2106 The monitoring time is 100 minutes from the monitoring start time based on the time when the service availability is calculated. For example, when the monitoring time from the monitoring start time to the time when the service operation rate is calculated is 100 minutes, and the unscheduled stop time is 10 minutes, the monitoring result 2203 is “90%”.
The monitoring result 2203 included in the monitoring log record 2206 in which the monitoring item 2202 is “response time” is “5 seconds”. Here, the monitoring log record 2206 whose monitoring item 2202 is “response time” corresponds to the SLA definition record 2107 whose service level item 2101 is “response time”. That is, “5 seconds” of the monitoring result 2203 is a response time measured by the monitoring unit 1011 using ping based on “ping” of the monitoring method 2104 included in the corresponding SLA definition record 2107.
The monitoring result 2203 included in the monitoring log record 2206 in which the monitoring item 2202 is “number of transactions” is “300”. This is the number of transactions that occurred during the monitoring time of the service availability, for example, counted by the monitoring unit 1011.
Reference value violation presence / absence 2204 indicates whether the corresponding monitoring result 2203 violates the corresponding monitoring reference value 2102 (in this case, “present”) or not (in this case, “none”).
The monitoring log record 2206 in which the presence / absence of the reference value violation 2204 is “Yes” corresponds to the monitoring data that does not satisfy the monitoring reference. For example, “5 seconds” in which the monitoring item 2202 shown in FIG. 3 is the “response time” monitoring result 2203 is “less than 3 seconds” in which the service level item 2101 shown in FIG. 2 is the monitoring reference value 2102 of “response time”. Is not satisfied. Accordingly, the presence or absence of a reference value violation 2204 for the monitoring item 2202 “response time” shown in FIG. 3 is “present”.
FIG. 4 is a diagram illustrating an example of the monitoring log tabulation record 2300. Referring to FIG. 4, the monitoring log total record 2300 includes a monitoring log total record 2307 including at least a monitoring period 2302, a report date and time 2303, and an SLA maintenance status 2306.
When the SLA maintenance status 2306 is “◯”, there is no service level item 2101 corresponding to the SLA violation condition 2103 in all the SLA definition records 2107 defined in the SLA definition 2100 as shown in FIG. Indicates. When the SLA maintenance status 2306 is “x”, it indicates that any one or more of the service level items 2101 corresponds to the SLA violation condition 2103.
The SLA definition storage unit 1021 stores an SLA definition 2100.
The log storage unit 1022 stores a monitoring log 2200 and a monitoring log total record 2300.
The SLA allowable range calculation unit 1031 calculates the total number of monitoring based on the monitoring interval 2105 and the monitoring period 2106 of the SLA definition 2100. Subsequently, the SLA allowable range calculation unit 1031 generates an SLA allowable range (also referred to as an allowable range) based on the calculated total number of monitoring and the corresponding SLA violation condition 2103, and outputs it to the SLA compliance rate calculation unit 1032. . The SLA allowable range is information indicating a range of an amount of violation (amount of reference value violation) with respect to the monitoring reference value 2102 that is allowable in the SLA, that is, does not violate the SLA even if it occurs. The amount of violation with respect to the monitoring reference value 2102 corresponds to the amount of monitoring data that does not satisfy the monitoring reference value 2102, and is indicated by the number of violations or time. Note that the amount of violation with respect to the monitoring reference value 2102 may be indicated by the ratio of the number of violations or time to the monitoring reference value 2102. For example, when the monitoring data is an information processing amount or a communication amount, the amount of violation with respect to the monitoring reference value 2102 may be indicated by an insufficient information processing amount or a communication amount. Alternatively, the amount of violation with respect to the monitoring reference value 2102 may be indicated by the ratio of the information processing amount or communication amount that is insufficient or exceeded to the monitoring reference value 2102.
The SLA compliance rate calculation unit 1032 calculates the number of occurrences of the reference value violation based on the presence or absence of the reference value violation 2204 included in the monitoring log record 2206 shown in FIG.
For example, the SLA compliance rate calculation unit 1032 calculates the number of occurrences of reference value violations in the range from the start time of the monitoring period 2106 (for example, every day at midnight) to the time for calculating the SLA compliance rate. Further, the SLA compliance rate calculation unit 1032 may calculate the number of occurrences of violation of the reference value in a specific range input from an input unit (not shown) connected to the quality evaluation apparatus 1000.
Next, the SLA compliance rate calculation unit 1032 calculates and outputs the SLA compliance rate based on the calculated number of occurrences of violation of the reference value and the SLA allowable range acquired from the SLA allowable range calculation unit 1031.
The SLA compliance rate is a ratio of the amount of violation of the reference value that has already occurred to the range of the amount of violation of the reference value indicated by the SLA tolerance. Here, the amount of violation of the reference value that has already occurred is, for example, a reference value that has occurred from a certain specific time (for example, monitoring start time) to another specific time (for example, the current time during service provision). The amount of violations. That is, the SLA compliance rate indicates how close to the SLA violation or how much the SLA is violated at a specific point in time during service provision.
Next, the operation of the present embodiment will be described in detail with reference to FIGS.
First, an operation for recording the SLA definition 2100 in the SLA definition storage unit 1021 will be described. FIG. 5 is a flowchart showing an operation of recording the SLA definition 2100 in the SLA definition storage unit 1021 in the present embodiment.
An input unit (not shown) of the quality evaluation apparatus 1000 receives an input of the SLA definition 2100 (step A101).
Next, the input unit records the received SLA definition 2100 in the SLA definition storage unit 1021. (Step A102)
This completes the description of the operation of recording the SLA definition 2100 in the SLA definition storage unit 1021.
Next, the overall operation of the quality evaluation apparatus 1000 that executes monitoring and calculates the SLA compliance rate will be described. FIG. 6 is a flowchart showing the overall operation of the quality evaluation apparatus 1000 that executes monitoring and calculates the SLA compliance rate.
First, the monitoring unit 1011 executes monitoring of the monitoring target system based on the SLA definition 2100 (step B101).
Next, the monitoring unit 1011 generates a monitoring log record 2206 and a monitoring log total record 2307 based on the executed monitoring. (Step B102).
Specific examples of monitoring execution and the generated monitoring log record 2206 will be described in the following (1) to (3).
(1) For example, the monitoring unit 1011 performs monitoring as follows in response to the SLA definition record 2107 whose service level item 2101 is “service time”.
First, the monitoring unit 1011 transmits a ping (pin) to the monitoring target system at, for example, 00 seconds per minute. Note that 00 seconds per minute is based on “every minute” indicated by a monitoring interval 2105. The transmission of the ping is based on “ping” indicated by the monitoring method 2104.
Next, when the monitoring unit 1011 receives a response to the ping from the monitoring target system within a predetermined time, the monitoring unit 1011 determines that the service is operating in the corresponding monitoring section. Note that the corresponding monitoring section is a section of a time that is back by the time (for example, one minute) indicated by the monitoring interval 2105 from the reception time of the response to ping.
In other words, the monitoring unit 1011 considers that the service is not stopped in the monitoring target system, for example, for one minute in the monitoring section, and is in an operating state. Then, the monitoring unit 1011 generates a monitoring log record 2206 including “◯” as the monitoring result 2203 and “None” as the reference value violation presence or absence 2204.
On the other hand, if the monitoring unit 1011 does not receive a response to the ping from the monitoring target system within a predetermined time, the monitoring unit 1011 is normal such that the service is stopped or an abnormality has occurred in the monitoring target system. Judge that it is not in operation. In other words, the monitoring unit 1011 considers that the monitoring target system is in a stopped state for, for example, one minute in the monitoring section. Then, the monitoring unit 1011 generates a monitoring log record 2206 including “×” as the monitoring result 2203 and “Yes” as the presence or absence of reference value violation 2204.
(2) For example, the monitoring unit 1011 performs monitoring as follows in response to the SLA definition record 2107 in which the service level item 2101 is “service availability”.
First, the monitoring unit 1011 transmits a ping (pin) to the monitoring target system at 00 seconds every 5 minutes, for example. Note that 00 seconds every 5 minutes is based on “every 5 minutes” indicated by a monitoring interval 2105. The transmission of the ping is based on “ping” indicated by the monitoring method 2104.
Next, when the monitoring unit 1011 receives a response to the ping from the monitoring target system within a predetermined time, the monitoring unit 1011 determines that the service is operating in the corresponding monitoring section. Note that the corresponding monitoring section is a section of time that is back by the time (for example, 5 minutes) indicated by the monitoring interval 2105 from the reception time of the response to ping.
That is, the monitoring unit 1011 considers that the service in the monitoring target system has not been stopped and is in an operating state during the monitoring section (for example, for 5 minutes). Then, the monitoring unit 1011 records in a memory (not shown) in the monitoring unit 1011 that the monitoring target system is operating normally during the monitoring section.
On the other hand, if the monitoring unit 1011 does not receive a response to the ping from the monitoring target system within a predetermined time, the monitoring unit 1011 is normal such that the service is stopped or an abnormality has occurred in the monitoring target system. Judge that it is not in operation. That is, the monitoring unit 1011 considers that the monitoring target system is in a stopped state during the monitoring section. Then, the monitoring unit 1011 records in the memory in the monitoring unit 1011 that the monitoring target system is not operating normally during the monitoring section.
Next, the monitoring unit 1011 generates a monitoring log record 2206 including the section operation rate and the presence / absence of a reference value violation 2204 determined based on the monitoring reference value 2102 as the monitoring result 2203. The section operation rate is a ratio of time during which the operation is normally performed to the time from the monitoring start time to the calculation time. The monitoring unit 1011 calculates based on, for example, a record of the operation of the monitoring target system recorded in a memory in the monitoring unit 1011.
Further, the monitoring unit 1011 compares the calculated section monitoring rate with the monitoring reference value 2102 to determine the presence / absence 2204 of the reference value violation. That is, when the calculated section monitoring rate is less than “99.9%” of the monitoring reference value 2102, the monitoring unit 1011 determines whether the reference value violation 2204 is “present”.
(3) For example, the monitoring unit 1011 performs monitoring as follows in response to the SLA definition record 2107 whose service level item 2101 is “response time”.
First, the monitoring unit 1011 transmits a ping to the monitoring target system at 00 seconds every 5 minutes, for example. Note that 00 seconds every 5 minutes is based on “every 5 minutes” indicated by a monitoring interval 2105. The transmission of the ping is based on “ping” indicated by the monitoring method 2104.
Next, the monitoring unit 1011 generates a monitoring log record 2206 including a response time to ping from the monitoring target system as the monitoring result 2203 and a reference value violation presence / absence 2204 determined based on the monitoring reference value 2102.
When there is no response to the ping from the monitoring target system within a predetermined time, the monitoring unit 1011 sets the monitoring result 2203 as “unmeasurable” or an exceptional value separately defined in the SLA definition 2100.
In addition, the monitoring unit 1011 compares the response time with the monitoring reference value 2102 to determine whether or not there is a reference value violation 2204. That is, if the response time is not “within 3 seconds” of the monitoring reference value 2102, the monitoring unit 1011 determines that the reference value violation 2204 is “present”.
The above is a specific example of the execution of monitoring and the generated monitoring log record 2206.
Next, the monitoring unit 1011 records the generated monitoring log record 2206 in the monitoring log 2200 and the generated monitoring log total record 2307 in the monitoring log total recording 2300 (step B103).
Next, the SLA allowable range calculation unit 1031 calculates the SLA allowable range based on the SLA definition 2100 recorded in the SLA definition storage unit 1021, and outputs it to the SLA compliance rate calculation unit 1032 (step B104).
Next, the SLA compliance rate calculation unit 1032 calculates and outputs the SLA compliance rate based on the monitoring log 2200 and the SLA allowable range acquired from the SLA allowable range calculation unit 1031 (step B105).
Next, the communication means (not shown) acquires the SLA compliance rate from the SLA compliance rate calculation unit 1032 and transmits the SLA compliance rate to the terminal device (not shown) via the network (not shown) (step) B106). Further, for example, the SLA compliance rate display means (not shown) may acquire the SLA compliance rate from the SLA compliance rate calculation unit 1032 and display it on the display (not shown).
The above is description of operation | movement of the quality evaluation apparatus 1000 whole which performs monitoring and calculates a SLA compliance rate.
Next, with reference to FIG. 7, the monitoring operation corresponding to Step B101, Step B102, and Step B103 of FIG. 6 will be described in detail. FIG. 7 is a flowchart showing the operation of the monitoring unit 1011.
First, the monitoring unit 1011 acquires the SLA definition 2100 from the SLA definition storage unit 1021 (step C101).
Next, the monitoring unit 1011 performs monitoring of the monitoring target system for the service level item 2101 included in each SLA definition record 2107 of the acquired SLA definition storage unit 1021 based on the corresponding monitoring interval 2105 and monitoring method 2104. To do. Subsequently, the monitoring unit 1011 generates a corresponding monitoring result 2203 based on the monitoring result. (Step C102).
Next, the monitoring unit 1011 generates a monitoring log record 2206 including the generated monitoring result 2203 and the presence or absence of a reference value violation 2204 determined based on the monitoring result 2203 and the monitoring reference value 2102 (step C103).
Next, the monitoring unit 1011 records the generated monitoring log record 2206 in the log storage unit 1022 (step C104).
Next, the monitoring unit 1011 generates a monitoring log total record 2307 (Step C105).
Next, the monitoring unit 1011 outputs the generated monitoring log total record 2307 to the log storage unit 1022 (C106).
In step C105, the monitoring unit 1011 determines whether each monitoring period 2106 is completed. When the monitoring period 2106 is completed, the monitoring unit 1011 generates a monitoring log total record 2307. Furthermore, when the log total record 2307 is generated, the monitoring unit 1011 outputs this monitor log total record 2307.
Further, the monitoring unit 1011 may always generate the monitoring log total record 2307 in step C105. In this case, when the generated SLA maintenance status 2306 is “x”, the monitoring unit 1011 may output the generated monitoring log total record 2307.
The above is the description of the operation of the monitoring unit 1011.
Next, the operation of the SLA allowable range calculation unit 1031 corresponding to step B104 of FIG. 6 will be described in detail with reference to FIG. FIG. 8 is a flowchart showing the operation of the SLA allowable range calculation unit 1031.
First, the SLA allowable range calculation unit 1031 acquires the monitoring interval 2105 of the SLA definition record 2107 corresponding to the service level item 2101 for calculating the SLA allowable range from the SLA definition storage unit 1021 (step D101).
Next, the SLA allowable range calculation unit 1031 acquires the monitoring period 2106 from the corresponding SLA definition record 2107 (step D102).
Next, the SLA allowable range calculation unit 1031 calculates the total number of monitoring within the monitoring period 2106 based on the acquired monitoring period 2106 and the monitoring interval 2105 (step D103). For example, if the monitoring period 2106 is 1 day and the monitoring interval 2105 is 1 minute, the SLA allowable range calculation unit 1031 calculates the total number of monitoring calls “1440” ((24 × 60) minutes ÷ 1 minute).
Next, the SLA allowable range calculation unit 1031 acquires the SLA violation condition 2103 from the corresponding SLA definition record 2107 (step D104).
Next, the SLA allowable range calculation unit 1031 calculates and outputs an SLA allowable range based on the acquired SLA violation condition 2103 and the calculated total number of monitoring calls Call (step D105).
For example, if the monitoring reference value 2102 is 98%, violation of the reference value is allowed up to 2%. Therefore, for example, when the total number of calls Call is “1440” times, the SLA allowable range calculation unit 1031 calculates the SLA allowable range as “28.8” (1440 × 0.02) times. When time is used as the SLA allowable range, if the total number of monitoring Call is “1” days, that is, “1440”, “28.8” is the SLA allowable range.
The operation of the SLA allowable range calculation unit 1031 has been described above.
Next, the operation of the SLA compliance rate calculation unit 1032 corresponding to step B105 of FIG. 6 will be described in detail with reference to FIG. FIG. 9 is a flowchart showing the operation of the SLA compliance rate calculation unit 1032.
First, the SLA compliance rate calculation unit 1032 determines the reference value violation amount (the number of reference value violations or the number of reference value violations) based on the presence or absence of reference value violation 2204 in the monitoring log 2200 as shown in FIG. Time) is calculated (step E101).
Next, the SLA compliance rate calculation unit 1032 acquires the SLA allowable range output by the SLA allowable range calculation unit 1031. (Step E102)
Next, the SLA compliance rate calculation unit 1032 calculates and outputs the SLA compliance rate based on the calculated reference value violation amount and the acquired SLA allowable range (step E103). The SLA compliance rate is calculated as a ratio of the reference value violation amount to the SLA allowable range.
For example, it is assumed that the number of violations of the reference value from the start of the monitoring period to the time of calculating the SLA compliance rate is 10 times, and the SLA allowable range for violation of the reference value is “28.8≈29 times”. In this case, the SLA compliance rate calculation unit 1032 calculates the SLA compliance rate as “66% ((29-10) / 29 * 100)”.
Further, it is assumed that the number of violations of the reference value from the start of the monitoring period to the time of calculating the SLA compliance rate is 2, and the SLA allowable range for violation of the reference value is “28.8≈29 minutes”.
Here, as described above, the monitoring unit 1011 executes the monitoring operation corresponding to the corresponding service level item 2101 based on the monitoring interval 2105 defined in the SLA definition 2100. When the reference value violation occurs, the monitoring unit 1011 considers that the violation has continued during the monitoring interval 2105 (for example, 1 minute). Therefore, the occurrence time of the reference value violation can be obtained as the product of the number of occurrences of the reference value violation and the monitoring interval 2105. For example, when the reference value violation occurs twice and the monitoring interval 2105 is “5 minutes”, the reference value violation time is 10 minutes.
In this case, the SLA compliance rate calculation unit 1032 calculates the SLA compliance rate as “66% ((29-10) / 29 * 100)”.
When the number of violations of the reference value exceeds the SLA allowable range, the SLA compliance rate calculation unit 1032 calculates the SLA compliance rate as follows. For example, it is assumed that the number of violations of the reference value from the start of the monitoring period to the time of calculating the SLA compliance rate is 35, and the SLA allowable range for violation of the reference value is “28.8≈29”.
In this case, the SLA compliance rate calculation unit 1032 calculates the SLA compliance rate as “−17% ((29-35) / 29 * 100.” That is, the SLA compliance rate calculation unit 1032 indicates that the number of violations of the reference value is When the SLA allowable range is exceeded, the SLA compliance rate is calculated as a negative value, and the degree of SLA violation is calculated as the absolute value.
The same applies to the case where the SLA allowable range is indicated by time, and the reference value violation time exceeds the SLA allowable range.
The above is the description of the operation of the SLA compliance rate calculation unit 1032.
In the present embodiment, as shown in the flowchart of FIG. 6, the monitoring unit 1011 performs monitoring, the SLA allowable range calculation unit 1031 calculates the SLA allowable range, and the SLA compliance rate calculation unit 1032 calculates the SLA compliance rate. Operate as a series of operations synchronously. This is an example and may be as follows, for example.
When the content of the SLA definition 2100 of the quality evaluation apparatus 1000 is updated, the SLA allowable range calculation unit 1031 calculates the SLA allowable range, and adds storage unit (not shown, for example, an SLA allowable range item to the SLA definition structure) You may make it do). Each of the monitoring unit 1011 and the SLA compliance rate calculating unit 1032 sets a timer (not shown) so that an interrupt is notified at a necessary timing based on the SLA definition 2100, and operates using the notified interrupt as a trigger.
The SLA compliance rate display means (not shown) may display the SLA compliance rate acquired from the SLA compliance rate calculation unit 1032 on a display (not shown) in time series. Thereby, it is possible to monitor in detail the state change of the service level item 2101 until the SLA violation.
Further, the SLA compliance rate display means (not shown) displays a warning when the SLA compliance rate acquired from the SLA compliance rate calculation unit 1032 becomes 0%, assuming that an SLA violation has occurred provisionally. It may be. In this way, even in the middle of the monitoring period 2106, it is possible to detect at an early stage the possibility of being determined as an SLA violation with a high probability at the end of the monitoring period, and to issue a warning.
In this embodiment, the monitoring unit 1011 outputs a monitoring log record 2206 including the monitoring result 2203 generated based on the result of executing the monitoring operation. This is an example of the present embodiment, and may be as follows, for example.
First, the monitoring unit 1011 directly outputs the result of executing the monitoring operation to at least one of the SLA allowable range calculation unit 1031 and the SLA compliance rate calculation unit 1032.
Next, the SLA allowable range calculation unit 1031 and the SLA compliance rate calculation unit 1032 generate a monitoring result 2203 based on the result of executing the received monitoring operation. Next, the SLA allowable range calculation unit 1031 and the SLA compliance rate calculation unit 1032 determine whether there is a violation of the reference value based on the generated monitoring result 2203 and the monitoring reference value 2102. Next, the SLA compliance rate calculation unit 1032 calculates the value of the reference value violation based on the determined presence or absence of the reference value violation.
Based on the monitoring result 2203 and the reference value violation value calculated in this way, the SLA allowable range calculation unit 1031 and the SLA compliance rate calculation unit 1032 operate substantially the same as the operations shown in the flowcharts of FIGS. I do.
The first effect of the present embodiment is that it is possible to indicate the SLA compliance status at a specific time point during service provision.
The reason is that it is configured to include the following elements. First, the SLA allowable range calculation unit 1031 calculates the total number of monitoring within the period indicated by the monitoring period 2106, and calculates the SLA allowable range based on the total number of monitoring and the SLA violation condition 2103. Next, the SLA compliance rate calculation unit 1032 calculates the SLA compliance rate based on the SLA allowable range and the reference value violation amount from the start of the monitoring period to a specific time.
The second effect of the present embodiment is that it becomes possible to monitor in detail the state change of the service level item 2101 until the SLA violation.
The reason is that the SLA compliance rate calculation unit 1032 calculates the SLA compliance rate every time the monitoring unit 1011 executes monitoring, that is, every time operation information and performance information such as availability and reliability of the monitored system are acquired. This is because the output is made.
The third effect of the present embodiment is that when the SLA compliance rate becomes 0%, it is considered that an SLA violation has occurred tentatively, and even during the monitoring period 2106, the SLA violation has a high probability at the end of the monitoring period. It is possible to quickly detect the case determined to be and issue a warning.
The reason is that even when no SLA violation has occurred (when the SLA compliance rate is 0%), the SLA compliance rate calculation unit 1032 calculates and outputs the SLA compliance rate.
The fourth effect of the present embodiment is that the SLA compliance status can be continuously indicated even when the allowable range changes, for example, when the violation standard is changed depending on the time zone due to power consumption limitation.
The reason is that the execution of monitoring by the monitoring unit 1011, the calculation of the SLA allowable range by the SLA allowable range calculating unit 1031, and the calculation of the SLA compliance rate by the SLA compliance rate calculating unit 1032 operate as a series of operations in synchronization. It is because it tried to do.
[Second Embodiment]
Next, a second embodiment of the present invention will be described in detail with reference to the drawings. Hereinafter, the description overlapping with the above description is omitted as long as the description of the present embodiment is not obscured.
FIG. 10 is a block diagram showing a configuration of a quality evaluation apparatus 3000 according to the second embodiment of the present invention.
As shown in FIG. 10, the quality evaluation apparatus 3000 according to the second embodiment includes an SLA allowable range calculation unit 1031 and an SLA compliance rate calculation unit 1032.
The SLA allowable range calculation unit 1031 calculates the total number of monitoring based on the corresponding monitoring interval 2105 and monitoring period 2106. Subsequently, the SLA allowable range calculation unit 1031 determines the allowable amount of monitoring data that does not satisfy the monitoring reference value 2102 that is the reference value of monitoring data based on the calculated total number of monitoring and the corresponding SLA violation condition 2103. An SLA allowable range indicating a possible range is calculated and output.
The SLA compliance rate calculation unit 1032 calculates the compliance rate with respect to the SLA violation condition 2103 at any point in the middle of the monitoring period 2106 based on the recording of monitoring data and the SLA allowable range output by the SLA allowable range calculation unit 1031. Calculate and output the indicated SLA compliance rate. Specifically, the SLA compliance rate is a ratio of the amount of monitoring data that does not satisfy the monitoring reference value 2102 to the SLA allowable range. Note that the SLA compliance rate calculation unit 1032 calculates the amount of monitoring data that does not satisfy the monitoring reference value 2102 based on, for example, the recording of monitoring data during a period in which the SLA compliance rate is to be calculated. Further, the period for which the SLA compliance rate is to be calculated is, for example, a period from the time when monitoring starts to the current time.
The effect of this embodiment is that it is possible to indicate the SLA compliance status at a specific time point during service provision.
The reason is that it is configured to include the following elements. First, the SLA allowable range calculation unit 1031 calculates the total number of monitoring within the period indicated by the monitoring period 2106, and calculates the SLA allowable range based on the total number of monitoring and the SLA violation condition 2103. Next, the SLA compliance rate calculation unit 1032 calculates the SLA compliance rate based on the record of the monitoring data and the SLA allowable range.
[Third Embodiment]
Next, a third embodiment of the present invention will be described in detail with reference to the drawings. Hereinafter, the description overlapping with the above description is omitted as long as the description of the present embodiment is not obscured.
FIG. 11 is a block diagram showing a configuration of a quality evaluation apparatus 4000 according to the third embodiment of the present invention.
As illustrated in FIG. 11, the quality evaluation apparatus 4000 according to the third embodiment further includes a total monitoring number estimation unit 4011 as compared with the quality evaluation apparatus 3000 according to the second embodiment.
FIG. 12 is a diagram illustrating an example of the SLA definition 2400. The SLA definition 2400 includes at least one SLA definition record 2401 in which the monitoring interval 2105 indicates “irregular”. The SLA definition 2400 may include an SLA definition record 2107 similar to the SLA definition 2100 in FIG.
The total monitoring number estimation unit 4011 calculates and outputs the total number of monitoring for the service level item 2101 whose monitoring interval 2105 is irregular as shown in the monitoring interval 2105 of the SLA definition record 2401 shown in FIG. Specifically, the total monitoring number estimation unit 4011 calculates and outputs the total number of monitoring in the monitoring period 2106 shown in FIG. 2 based on, for example, the number of transactions 4305 in the monitoring log total 4300 shown in FIG.
FIG. 13 is a diagram showing an example of a monitoring log total recording (also called operation history) 4300. Referring to FIG. 13, the monitoring log total record 4300 includes a monitoring log total record 4307 including at least a monitoring period 2302, a report date and time 2303, an SLA maintenance status 2306, and the number of transactions 4305.
The transaction number 4305 indicates the total number of transactions that have occurred in the monitoring target system during the period indicated by the corresponding monitoring period 2302.
Note that the monitoring period 2302, the reporting date 2303, and the SLA maintenance status 2306 are the same as those shown in FIG.
Next, the operation of this embodiment will be described with reference to FIG. FIG. 14 is a flowchart showing the operation of the total monitoring number estimation unit 4011 of this embodiment.
The total monitoring number estimation unit 4011 acquires the monitoring interval 2105 included in each SLA definition record 2401 of the SLA definition 2400 (step F101).
Next, the total monitoring number estimation unit 4011 determines whether or not the monitoring interval 2105 indicates irregularity.
(Step F102). If no irregularity is indicated (NO in step F102), the process proceeds to D102 of FIG.
If it indicates irregular (YES in step F102), the total monitoring number estimation unit 4011 acquires the total number of monitoring for each past monitoring period 2302 shown in FIG. 13 (step F103). For example, the total monitoring number estimation unit 4011 acquires the number of transactions 4305 from the monitoring log total recording 4300 as shown in FIG. 13 recorded in the log storage unit 1022 as the total number of monitoring for each past monitoring period 2302. .
Note that the total number of monitoring for each past monitoring period 2302 that is acquired by the total monitoring number estimation unit 4011 in step F103 is, for example, one of the SLA violation condition 2103 and the monitoring interval 2105 of the SLA definition record 2401 shown in FIG. Defined. In the present embodiment, based on the fact that the monitoring interval 2105 of the SLA monitoring record 2401 shown in FIG. 12 is “per transaction (irregular)”, the total monitoring number estimation unit 4011 determines all the monitoring intervals 2302 in the past. The transaction number 4305 is acquired as the monitoring number.
Specifically, when the monitoring interval 2105 is “per transaction (irregular)”, for example, the total monitoring number estimation unit 4011 refers to the monitoring log total recording 4300 illustrated in FIG. 13 and acquires the transaction number 4305. . In this case, the number of transactions 4305 indicates the number of times of monitoring in the past monitoring period 2302.
Note that the total monitoring number estimation unit 4011 may directly calculate the number of monitoring times in the past monitoring period 2302 based on the monitoring log (also referred to as operation history) 2200 shown in FIG. In this case, the monitoring unit 1011 outputs a monitoring log record 2206 including the number of transactions processed in the monitoring target system to the total monitoring number estimation unit 4011.
Moreover, the quality evaluation apparatus 4000 may calculate the number of times of monitoring in the past monitoring period 2302 as follows. First, the monitoring unit 1011 outputs a monitoring log record (also referred to as an operation history) 2206 including the number of transactions in the monitoring target system to the monitoring log 2200. Next, the total monitoring number estimation unit 4011 calculates the total number of transactions in a certain period (past monitoring period 2302) based on the monitoring log record 2206 recorded in the monitoring log 2200 and whose monitoring item 2202 is “number of transactions”. It may be calculated directly.
In FIG. 14 again, the total monitoring number estimation unit 4011 calculates an estimated value of the total number of monitoring in the target period based on the acquired total number of monitoring (transaction number 4305) for each past monitoring period 2302 (step F104). . Here, the target period is a period from the start of the monitoring period to the end of the monitoring period in the monitoring period 2106 shown in FIG. 12, including the target time for calculating the SLA compliance rate.
Specifically, the total monitoring number estimation unit 4011 calculates an estimated value of the total monitoring number as follows, for example.
First, the total monitoring number estimation unit 4011 assumes that the data C = {C1, C2,..., Ck} of all the monitoring numbers in a plurality of past monitoring periods 2302 has a normal distribution, and sets a confidence interval. . Here, C1, C2, and Ck indicate the total number of monitoring for each past monitoring period 2302. k is the total number of monitoring in the past monitoring period 2302.
When the estimated value of the total number of monitoring is Call and the confidence interval is expressed as a range of the total number of monitoring Call using the t distribution, it is as shown in Equation 1.

Here, Cave is an average value of the data C of the total number of monitoring in a plurality of past monitoring periods 2302, as shown in Equation 2.

Further, “square of σ” is the variance of the data C of the total number of monitoring in a plurality of past monitoring periods 2302, as shown in Equation 3.

Further, t is a value obtained from a t distribution having an average of 0 and a standard deviation of 1. The value of t varies depending on the confidence interval to be set. For example, for a 95% confidence interval, t = 2.262, and for a 99% confidence interval, t = 3.250. The 95% confidence interval is an interval including 95% of the population estimated from the sample. Also, the 99% confidence interval has higher accuracy (reliability) than the 95% confidence interval. Note that the value of the t distribution is recorded in a storage unit (not shown) in the total monitoring number estimation unit 4011 as a table, for example.
For example, if there is a past distribution with an average value Cave = 1400, a variance “square of σ” = 19544, and the number of samples k = 10, using the 95% confidence interval, from t = 2.262, the upper limit of Call Values and lower limits are as shown in Equation 4.

Therefore, 1300 <Call <1500.
As described above, the total monitoring number estimation unit 4011 calculates the range of the total monitoring number Call.
Next, as described above, the allowable range CP and SLA compliance rate calculation unit 1032 calculated by the SLA allowable range calculation unit 1031 when the total monitoring number estimation unit 4011 calculates the range of the total monitoring number Call is calculated. An example of the SLA compliance rate α to be performed will be specifically described.
For example, in the case of the SLA definition record 2401 in FIG. 12, the SLA violation condition 2103 is “the rate when the reference value cannot be observed is 2% or more”. In this case, the amount of violation of the reference value is allowed to be less than 2%. Therefore, in the case of 1300 ≦ Call ≦ 1500, the allowable range CP of the number of violations of the reference value is 24 <CP <30 from 1300 × 0.02 = 26 and 1500 × 0.02 = 30. The same applies when the violation of the reference value is described in terms of time.
At this time, if the number of violations of the reference value from the start of the monitoring period to the calculation time is 10, the SLA compliance rate calculation unit 1032 determines that the SLA compliance rate 1032 from the allowable range of reference value violation 24 <CP <30. α is calculated as follows.
(25-10) ÷ 25 × 100 = 60%, (29-10) ÷ 29 × 100 = 66%, and therefore 61% ≦ α ≦ 66%.
This also applies to the case where the reference value violation value is expressed in time.
The total monitoring number estimation unit 4011 of this embodiment may be applied to the first embodiment.
The first effect of the present embodiment is that it is possible to indicate the SLA compliance status at a specific time point during service provision even for the service level item 2101 for which the number of monitoring is not fixed.
The reason is that the total monitoring number estimation unit 4011 estimates the total monitoring number based on the monitoring log total record 4300 for the service level item 2101 for which the total monitoring number in the monitoring period 2106 is not fixed (the range of the total monitoring number). ) Is calculated.
The second effect of the present embodiment is that the SLA compliance status of the first effect is high even when the number of monitoring log total records 4307 in the monitoring log total record 4300 is relatively small (10 or less). It is possible to show that.
The reason is that it is configured to include the following elements. First, the total monitoring number estimation unit 4011 sets a confidence interval using the t distribution, and calculates an estimated value of the total number of monitoring corresponding to this confidence interval. Next, the SLA allowable range calculation unit 1031 calculates the allowable range based on the estimated value of the total number of monitoring. Next, the SLA compliance rate calculation unit 1032 calculates the SLA compliance rate based on the allowable range.
[Fourth Embodiment]
Next, a fourth embodiment of the present invention will be described in detail with reference to the drawings. Hereinafter, the description overlapping with the above description is omitted as long as the description of the present embodiment is not obscured.
FIG. 15 is a block diagram showing a configuration of a quality evaluation apparatus 5000 according to the fourth embodiment of the present invention.
As illustrated in FIG. 15, the quality evaluation apparatus 5000 according to the fourth embodiment further includes an SLA violation prediction unit 5011 as compared to the quality evaluation apparatus 3000 according to the second embodiment.
The SLA violation predicting unit 5011 calculates and outputs the time when the occurrence of the SLA violation is predicted based on the SLA compliance rate output from the SLA compliance rate calculating unit 1032. In addition, the SLA violation prediction unit 5011 outputs information indicating that there is a risk of an SLA violation when the predicted time is within the time before the end of the current monitoring period.
When the elapsed time t from the start of monitoring, the predicted occurrence time Te of SLA violation, and the SLA compliance rate α, the SLA violation prediction unit 5011 calculates Te = 100 × t ÷ (100−α). Subsequently, the SLA violation prediction unit 5011 outputs the calculated prediction occurrence time Te.
For example, when the compliance rate α at the time when one hour has elapsed from the start of monitoring is 95%, the SLA violation prediction unit 5011 calculates Te = 100 × 1 ÷ (100−95) = 20. That is, the expected occurrence time of the SLA violation in this case is 20 hours after the start of monitoring.
Furthermore, for example, when the monitoring period 2106 of a certain SLA definition record 2107 is 24 hours, there is a possibility that an SLA violation may occur within the monitoring period 2106, so information (warning) indicating that there is a risk is output. To do. Here, “SLA violation occurs within the monitoring period 2106” means that “the time when the SLA violation occurs is within the time before the end of the current monitoring period 2106, 21 hours after the start of monitoring. ".
The SLA violation prediction unit 5011 outputs, for example, an expected occurrence time of an SLA violation or an expected occurrence time and a warning to a display (not shown) via an SLA violation display unit (not shown). Also, the SLA violation prediction unit 5011 sends, to the terminal device (not shown) via the communication means (not shown) and the network (not shown), the expected occurrence time of the SLA violation, or the expected occurrence time and warning, May be transmitted.
Next, the operation of the SLA violation prediction unit 5011 will be described with reference to FIG. FIG. 16 is a flowchart showing the operation of the SLA violation prediction unit 5011.
The SLA violation prediction unit 5011 acquires the SLA compliance rate from the SLA compliance rate calculation unit 1032 (step I101).
Next, the SLA violation prediction unit 5011 calculates a predicted SLA violation occurrence time based on the acquired SLA compliance rate (step I102). The SLA violation prediction unit 5011 calculates the expected occurrence time Te of SLA violation using the following formula, for example. Te = 100 × t ÷ (100−α). Here, t is the time at the time of monitoring, and α is the SLA compliance rate.
Further, the SLA violation prediction unit 5011 determines whether or not the calculated predicted occurrence time Te of SLA violation is included in the current monitoring period 2106 (step I103). If it is not included in the current monitoring period 2106 (NO in step I103), the process proceeds to step I105.
When included in the current monitoring period 2106 (YES in step I103), the SLA violation predicting unit 5011 outputs a warning indicating that there is a risk of occurrence of an SLA violation to an output unit (not shown) (step I104).
In step I105, the SLA violation prediction unit 5011 outputs the calculated predicted occurrence time of the SLA violation to an output unit (not shown) (step I105).
The above is the description of the operation of the SLA violation prediction unit 5011.
Note that the SLA violation prediction unit 5011 of this embodiment may be applied to the first embodiment and the third embodiment.
The effect of this embodiment is that it is possible to output the expected occurrence time of an SLA violation.
The reason is that the SLA violation prediction unit 5011 calculates the SLA violation predicted occurrence time based on the SLA compliance rate.
[Fifth Embodiment]
Next, a fifth embodiment of the present invention will be described in detail with reference to the drawings. Hereinafter, the description overlapping with the above description is omitted as long as the description of the present embodiment is not obscured.
FIG. 17 is a block diagram showing a configuration of a quality evaluation apparatus 6000 according to the fifth embodiment of the present invention.
As illustrated in FIG. 17, the quality evaluation apparatus 6000 according to the fifth embodiment further includes an SLA compliance rate aggregation unit 6011 as compared to the quality evaluation apparatus 3000 according to the second embodiment.
The SLA compliance rate aggregation unit 6011 is based on the SLA compliance rate of each service level item 2101 output from the SLA compliance rate calculation unit 1032, and the overall SLA compliance status of the monitored system and the service level that is the bottleneck The information of item 2101 is output.
For example, when the monitoring target system includes server A, server B, and server C, the SLA compliance rate aggregation unit 6011 calculates the overall SLA compliance status of the monitoring target system as follows.
For example, it is assumed that the service level item 2101 of each of the server A, the server B, and the server C outputs values of 70%, 30%, and 50% as the SLA compliance rate of “response time”, respectively. In this case, the SLA compliance rate aggregation unit 6011 calculates the maximum value as 70%, the minimum value as 30%, and the average value as 50% ((70 + 30 + 50) ÷ 3).
That is, when it is predetermined to output an average value as the overall SLA compliance status, the SLA compliance rate aggregation unit 6011 has an overall SLA compliance status of 50%. Further, when it is predetermined to output the minimum value, the overall SLA compliance status is 30%. Further, an item having a minimum SLA compliance rate is an item that is a bottleneck in SLA maintenance.
In addition, the SLA compliance rate aggregation unit 6011 indicates that the SLA compliance rate of the response time of the server A is 30% and the response time of the server A is a bottleneck as information on the service level item 2101 that is a bottleneck. Is output.
The SLA compliance rate aggregating unit 6011 displays, via a SLA violation display unit (not shown), information on the overall SLA compliance status of the monitored system and information on the service level item 2101 that is the bottleneck. (Shown). In addition, the SLA compliance rate aggregation unit 6011 sends the overall SLA compliance status of the monitoring target system and the bottle to the terminal device (not shown) via, for example, a communication unit (not shown) and a network (not shown). Information on the service level item 2101 which is a bottleneck may be transmitted.
Next, the operation of the SLA compliance rate aggregation unit 6011 will be described. FIG. 18 is a flowchart showing the operation of the SLA compliance rate aggregation unit 6011.
First, the SLA compliance rate aggregation unit 6011 acquires a plurality of SLA compliance rates output from the SLA compliance rate calculation unit 1032 (step K101).
Next, the SLA compliance rate aggregation unit 6011 aggregates the acquired plurality of SLA compliance rates, calculates the maximum value, the minimum value, and the average value and outputs them (step K102).
The minimum value of the SLA compliance rate indicates how close the SLA compliance status of the service level item 2101 taking the minimum value among the plurality of service level items 2101 is close to the SLA violation. In addition, the average value of the SLA compliance rate is an index indicating how much a plurality of service level items 2101 are observed in the monitoring target system.
Next, the SLA compliance rate aggregation unit 6011 outputs information on the service level item 2101 that is a bottleneck in addition to the overall SLA compliance status such as an average value, a minimum value, and a maximum value (step K103). A service level item 2101 that is a bottleneck is a service level item 2101 that takes the lowest SLA compliance rate among a plurality of SLA compliance rates.
The above is the description of the operation of the SLA compliance rate aggregation unit 6011.
The first effect of the present embodiment is that it becomes easy to collectively monitor the SLA compliance status for each of the plurality of service level items 2101.
The reason is that the SLA compliance rate aggregation unit 6011 outputs a result of aggregating a plurality of SLA compliance rates.
The second effect of the present embodiment is that it is possible to take measures to prevent the service level item 2101 that is a bottleneck from being an SLA violation.
The reason is that the SLA compliance rate aggregation unit 6011 outputs the information of the service level item 2101 that is a bottleneck.
The SLA compliance rate aggregation unit 6011 of this embodiment may be applied to the first embodiment, the third embodiment, and the fourth embodiment.
Each component described in each of the above embodiments may cause a computer to execute a predetermined process by a program as shown in FIG. 19, for example.
FIG. 19 is a block diagram illustrating a configuration of a quality evaluation apparatus 7000 that causes a computer to execute predetermined processing using a program. Referring to FIG. 19, the quality evaluation device 7000 includes a CPU (Central Processing Unit) 7100, a disk device 7200, a storage unit 7300, a communication unit 7500, an input unit 7600, and an output unit 7700.
The CPU 7100 expands the program stored in the disk device 7200, for example, in the storage unit 7300, and executes predetermined processing based on the expanded program.
The disk device 7200 is, for example, an optical disk, a flexible disk, a magnetic optical disk, an external hard disk, and a semiconductor memory, and includes a nonvolatile storage medium. The disk device 7200 stores a program that causes a computer to execute part of the processing of the communication control device in each of the above-described embodiments. The disk device 7200 may store an SLA definition, a monitoring log, and a monitoring log total record as the SLA definition storage unit 1021 and the log storage unit 1022.
The storage unit 7300 stores the program and data generated and referred to in each of the above-described embodiments.
The CPU 7100, the disk device 7200, and the storage unit 7300 correspond to the SLA allowable range calculation unit 1031, the SLA compliance rate calculation unit 1032, the total monitoring number estimation unit 4011, the SLA violation prediction unit 5011, and the SLA compliance rate aggregation unit 6011.
The communication unit 7500 implements an interface with a network, for example. The communication unit 7500 may be included as part of the SLA allowable range calculation unit 1031 and the SLA compliance rate calculation unit 1032.
The input unit 7600 is realized by, for example, a mouse, a keyboard, a built-in key button, and the like, and is used for input operations. The input unit 3073 is not limited to a mouse, a keyboard, and a built-in key button, but may be a touch panel, an accelerometer, a gyro sensor, a camera, or the like.
The output unit 7700 is realized by a display, for example, and is used for confirming the output.
FIG. 20 is a block diagram illustrating an example of a non-volatile storage medium on which a program is recorded. The recording medium 7400 may be a non-volatile recording medium that stores information non-temporarily.
The quality evaluation device 7000 may include a recording medium 7400 supplied from the outside. The CPU 7100 may read and execute the program code stored in the recording medium 7400. Alternatively, the CPU 7100 may store the code of the program stored in the recording medium 7400 in the storage unit 7300, the disk device 7200, or both. That is, this embodiment includes an embodiment of a recording medium 7400 that stores a program (software) executed by the quality evaluation apparatus 7000 (CPU 7100) temporarily or non-temporarily.
As mentioned above, although this invention was demonstrated with reference to each embodiment and an Example, this invention is not limited to the said embodiment and Example. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.
Each component described in each of the above embodiments does not necessarily have to be individually independent. For example, in each component, a plurality of components may be realized as one module, or one component may be realized as a plurality of modules. Each component is configured such that a component is a part of another component, or a part of a component overlaps a part of another component. Also good.
In the embodiments described above, each component and a module that realizes each component may be realized by hardware as long as necessary, or may be realized by a computer and a program. It may be realized by mixing hardware modules, computers, and programs. The program is provided by being recorded on a non-volatile computer-readable recording medium such as a magnetic disk or a semiconductor memory, and read by the computer when the computer is started up. The read program causes the computer to function as a component in each of the above-described embodiments by controlling the operation of the computer.
Further, in each of the embodiments described above, a plurality of operations are described in order in the form of a flowchart, but the described order does not limit the order in which the plurality of operations are executed. For this reason, when each embodiment is implemented, the order of the plurality of operations can be changed within a range that does not hinder the contents.
Furthermore, in each embodiment described above, a plurality of operations are not limited to being executed at different timings. For example, another operation may occur during the execution of a certain operation, or the execution timing of a certain operation and another operation may partially or entirely overlap.
Furthermore, in each of the embodiments described above, a certain operation is described as a trigger for another operation, but the description does not limit all relationships between the certain operation and the other operations. For this reason, when each embodiment is implemented, the relationship between the plurality of operations can be changed within a range that does not hinder the contents. The specific description of each operation of each component does not limit each operation of each component. For this reason, each specific operation | movement of each component may be changed in the range which does not cause trouble with respect to a functional, performance, and other characteristic in implementing each embodiment.
While the present invention has been described with reference to the embodiments, the present invention is not limited to the above embodiments. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.
This application claims the priority on the basis of Japanese application Japanese Patent Application No. 2011-162312 for which it applied on July 25, 2011, and takes in those the indications of all here.

According to the present invention, the present invention can be applied to usage monitoring and prediction of service level violations in operation management.

1000 Quality Evaluation Apparatus 1011 Monitoring Unit 1021 SLA Definition Storage Unit 1022 Log Storage Unit 1031 SLA Allowable Range Calculation Unit 1032 SLA Compliance Rate Calculation Unit 2100 SLA Definition 2101 Service Level Item 2102 Monitoring Reference Value 2103 SLA Violation Condition 2104 Monitoring Method 2105 Monitoring Interval 2106 Monitoring period 2107 SLA definition record 2200 Monitoring log 2201 Monitoring date 2202 Monitoring item 2203 Monitoring result 2204 Reference value violation 2206 Monitoring log record 2300 Monitoring log total record 2302 Monitoring period 2303 Reporting date 2306 SLA maintenance status 2307 Monitoring log total record 2400 SLA Definition 2401 SLA definition record 3000 Quality evaluation device 4000 Quality evaluation device 4011 Total monitoring number estimation unit 5000 Quality evaluation Price device 5011 SLA violation prediction unit 6000 Quality evaluation device 6011 SLA compliance rate aggregation unit

Claims (10)

1. The monitoring based on the violation condition that defines the condition of the violation operation to be monitored, and the total number of monitoring calculated based on the monitoring data acquisition interval corresponding to the violation condition and the monitoring period corresponding to the violation condition An allowable range calculating means for calculating and outputting an allowable range indicating an allowable range of the amount of monitoring data that does not satisfy the monitoring reference value that is a data reference value;
   Compliant rate calculating means for calculating and outputting a compliance rate for the violation condition at an arbitrary point in the monitoring period based on the monitoring data and the allowable range.
2. Based on the operation history of the monitoring target, further includes a total monitoring number estimation means for calculating and outputting an estimated value of the total number of monitoring times in the monitoring period,
   The quality evaluation apparatus according to claim 1, wherein:
3. The total monitoring number estimating means calculates a range of the total monitoring number corresponding to a specific confidence interval based on a tee distribution (t distribution) based on the number of monitoring in a plurality of monitoring periods included in the operation history. The quality evaluation device according to claim 2, wherein the calculated range of the total number of monitoring is output as an estimated value of the total number of times of monitoring.
4. The quality evaluation apparatus according to claim 1, further comprising: a violation prediction unit that calculates and outputs a time at which the occurrence of the violation action is predicted based on the compliance rate.
5. The quality evaluation apparatus according to any one of claims 1 to 4, further comprising: a compliance rate aggregation unit that generates and outputs information in which a plurality of compliance rates are aggregated.
6. Computer
   The monitoring based on the violation condition that defines the condition of the violation operation to be monitored, and the total number of monitoring calculated based on the monitoring data acquisition interval corresponding to the violation condition and the monitoring period corresponding to the violation condition Calculate and output an allowable range indicating the allowable range of the amount of monitoring data that does not meet the monitoring reference value, which is the data reference value,
   A quality evaluation method for calculating and outputting a compliance rate with respect to the violation condition at an arbitrary point in the middle of the monitoring period based on the monitoring data and the allowable range.
7. Computer
   The quality evaluation method according to claim 6, wherein an estimated value of the total number of monitoring times in the monitoring period is calculated and output based on the operation history of the monitoring target.
8. The calculation of the estimated value of the total number of monitoring is based on the number of monitoring in a plurality of monitoring periods included in the operation history, and the total number of monitoring corresponding to a specific confidence interval based on a tee distribution (t distribution). The quality evaluation method according to claim 7, wherein a range is calculated, and the calculated range of the total number of monitoring is output as an estimated value of the total number of monitoring times.
9. Computer
   The quality evaluation method according to claim 6, wherein a time at which the occurrence of the violating action is predicted is calculated and output based on the compliance rate.
10. On the computer,
    The monitoring based on the violation condition that defines the condition of the violation operation to be monitored, and the total number of monitoring calculated based on the monitoring data acquisition interval corresponding to the violation condition and the monitoring period corresponding to the violation condition Processing for calculating and outputting an allowable range indicating an allowable range of the amount of monitoring data that does not satisfy the monitoring reference value that is the data reference value;
    A non-volatile recording medium storing a program for calculating and outputting a compliance rate with respect to the violation condition at an arbitrary point in the monitoring period based on the monitoring data and the allowable range.

Images