JP2010257272A - Service evaluation system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve such the problem that a conventional technology is provided for making a service operator or an IDC analyze operating situations from an operation log in a data center, and not for making a use service user inspect whether or not SLA is satisfied. <P>SOLUTION: A user stores information of the definition items and thresholds of SLA in an SLA definition information DB 61 in advance. The specific monitoring items are converted into a monitoring item management DB 62. The monitor items are represented by a service time, a service operation rate, a mean restoration time, an online response time, and the number of simultaneous connection users or the like. Thus, a service user can inspect whether or not the items specified in the SLA are operated within a determined range from a user side by installing a system for monitoring the operating situations/using situations of a service between a software service providing company and a service user. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention is a method for evaluating the service quality of software services such as ASP and SaaS, and is particularly intended for the audit of SLA that guarantees the service quality.

  In recent years, the number of ASP and SaaS operators has increased greatly, and with the widespread use of them, many have questioned the reliability and availability of ASP and SaaS. Along with this, the Ministry of Economy, Trade and Industry announced the “SLA Guidelines for SaaS and ASP” in January 2008 with the aim of promoting stable and reliable SaaS and ASP for further spread of SaaS usage. This guideline shows a guideline for contracting SaaS and ASP.

  Further, in the prior art, the IDC collects and analyzes the service operation status log, suggests improvement to the service operator, and provides feedback to the IDC system facility improvement location.

JP 2008-123183 A

  The prior art is a system installed on the IDC side, and the IDC or software service provider company verifies the operation status in the IDC and improves its operation method. It was not possible to grasp the operating status of the service. Also, it was not a technology that was conscious of monitoring with items corresponding to SLA.

  By setting a threshold value for standard SLA items in SaaS, service level management (SLM) is automated and the burden on the service usage manager is reduced.

  In the present invention, by installing a system for monitoring the operating status / usage status of a service between a software service provider and a service user, the service user can operate within the default range for the items specified in the SLA. It is possible to audit from the user side.

It is a figure explaining the service evaluation system concerning this invention embodiment. 2 is a diagram showing a configuration of a service monitoring system 300. FIG. It is a figure which shows the storage format of SLA definition information DB61. It is a figure which shows the storage format of the monitoring item management DB62. It is a figure which shows the storage format of operation log DB63. It is a figure which shows the storage format of response log DB64. It is a figure which shows the storage format of simultaneous connection user DB65. It is a figure which shows the storage format of utilization track record DB. The content of the SLA definition information DB 61 is converted into the monitoring item management DB 62, and the supplementary part is a flowchart showing a processing procedure for performing an initial setting process supplemented by the service use manager. It is a flowchart which shows the process sequence which performs operation confirmation and preserve | saves to operation log DB63. It is a flowchart which shows the process sequence which performs a response confirmation process and preserve | saves in response log DB64. It is a flowchart which shows the process sequence which checks a simultaneous connection user and updates a simultaneous connection user DB. It is a flowchart which shows the process sequence which analyzes operation | movement log DB63, response log DB64, and simultaneous connection user DB65, and preserve | saves in utilization track record DB66. It is a flowchart which shows the process sequence which compares the monitoring item DB62 and the utilization performance DB66, and warns when there is a suspicion of SLA violation.

  FIG. 1 is a diagram for explaining a service evaluation system according to an embodiment of the present invention. This system comprises a software service provider 100, the Internet 200, a service monitoring system 300, a service usage manager terminal 400, and a service usage terminal 500. The service monitoring system 300, the service usage manager terminal 400, and the service usage terminal 500 are either LAN or WAN. The software service provider 100 and the service monitoring system 300 are connected to each other via the Internet 200.

  FIG. 2 shows the configuration of the service monitoring system 300. The service monitoring system 300 is composed of six types of processing units and six types of DBs as described below.

  The communication processing unit 31 is a processing unit that collects all the communications of the service monitoring system 300, and the operation confirmation processing unit 32 confirms whether the host operating the software service of the software service provider 100 is operating. The response confirmation processing unit 33 is a processing unit that confirms the response of the software service of the software service provider 100, and the simultaneous connection user confirmation processing unit 34 is connected to the software service of the software service provider 100 at the same time. The log analysis processing unit 35 is a processing unit that stores the results of analyzing the contents of the operation log DB 63 and the response log DB 64 of the software service in the usage record storage DB 66. The warning processing unit 36 includes a monitoring item management DB 62. Check the contents of the use record storage DB66, to see if there is a suspicion of SLA violation, a processing unit to alert the service management terminal 400 if there is a suspicion of violation. The SLA definition information DB 61 is a DB that stores the contents of the SLA. The monitoring item management DB 62 is a DB that converts the SLA definition information DB 61 and manages the monitoring items and the monitoring method of the system, and the operation log DB 63. Is a DB that stores the operating status of the host that is operating the software service, the response log DB 64 is a DB that stores the confirmation result of the response of the software service, and the simultaneous connection user DB 65 uses the software service simultaneously. The usage record storage DB 66 is a DB that stores the results of analysis of the operation log DB 63 and the response log DB 64 of the software service by the analysis processing unit 35.

  FIG. 3 shows the data format of the SLA definition information DB 61 that is a DB for storing the contents of the SLA, and is composed of service level items 6101 and setting values 6102.

  FIG. 4 shows the data format of the monitoring item management DB 62 that manages the monitoring items of the present system generated by converting the SLA definition information DB 61, and includes a monitoring item 6201, a setting value 6202, and a monitoring method 6203.

  FIG. 5 shows the data format of the operation log DB 63 that stores the operation status of the host operating the software service, and is composed of a date and time 6301, a response time 6302, and a host name 6303.

  FIG. 6 shows the data format of the response log DB 64 that stores the result of confirming the response of the software service. The response log DB 64 includes a date and time 6401, a response time 6402, a connection source 6403, a URL 6404, and the number of simultaneous connection users 6405.

  FIG. 7 shows the data format of the simultaneous connection user DB 65 that stores the number of users who are simultaneously using the software service. The data format includes an item number 6501, a last access date 6502, a session ID 6503, and a status 6504.

  FIG. 8 shows the data format of the usage record storage DB 66 for storing the results of analysis of the contents of the software service operation log DB 63 and the response log DB 64 by the analysis processing unit 35. The analysis date and time 6601, service time 6602, and service operation rate 6603 are shown. , Average recovery time 6604, online response time (via network) 6605, and the maximum number of simultaneously connected users 6606.

  FIG. 9 is a flowchart showing a processing procedure for converting the contents of the SLA definition information DB 61 into the monitoring item management DB 62 and performing the initial setting process supplemented by the service use manager as the supplementary part.

  In step 3101, the contents of the SLA defined by the software service used by the service manager for each item in the SLA definition information DB 61 are input to the SLA definition information DB 61.

  In step 3102, data is read from the SLA definition information DB 61 and converted into a monitoring item management DB in subsequent steps.

  In step 3103, “24 hours 365 days” is read from the service time 6103, and the set value 6202 of the service time 6204 is set to 00: 00-24: 00 (start time−end time).

  In step 3104, “99% or more” is read from the service operation rate 6104, and the setting value 6202 of the service operation rate 6205 is set to “99%”.

  In step 3105, “within 1 hour” is read from the average recovery time 6105, and the set value 6202 of the average recovery time 6206 is set to “within 1 hour”.

  In step 3106, “within 10 seconds” is read from the online response time (via the network) 6107 and the set value 6202 of the online response time (via the network) 6207 is set to “within 10 seconds”.

  In step 3107, “50 users” of the number of simultaneously connected users 6108 is read, and the setting value 6202 of the number of simultaneously connected users 6208 is set to “50 users”.

  In the subsequent steps, monitoring information not included in the SLA definition information DB is input to the monitoring item management DB 62.

  In step 3108, the service use manager inputs the host name of the software service provider 100 in the setting value 6202 of the service provider host 6209.

  In step 3109, the service use manager inputs a host name of a well-known site to be referred to in order to confirm the operating state of the service provider as the setting value 6202 of the reference site host 6210.

  In step 3110, the service use administrator inputs a command for accessing the host, a monitoring interval, and a timeout time in the monitoring method 6203 for the service operation rate 6205.

  In step 3111, the service use administrator inputs a command for accessing the host, a monitoring interval, and a timeout time in the monitoring method 6203 of the online response time (via netarc) 6207.

  In step 3112, a time limit for forcibly logging out while the service user is logged in is input.

  In step 3113, the data converted in steps 3203 to 3207 and the data in step 3108 to step 3112 are stored and updated in the input monitoring item management DB 62.

  FIG. 10 is a flowchart showing a processing procedure for confirming the operation and storing it in the operation log DB 63.

  In step 3201, the service provider host 6209 is pinged by the monitoring method 6205 every 0.6 seconds of the monitoring interval of the service operation rate 6205 monitoring method 6203 to confirm the response time. If timed out, the response time is set to “−”.

  In step 3202, the monitoring method 6205 is pinged to the reference site host 6210 every 0.6 seconds of the monitoring interval of the service operation rate 6205 monitoring method 6203. If timed out, the response time is set to “−”.

  In step 3203, the monitoring result date and time 6301, response time 6302, and host name 6303 are stored in the operation log DB 63.

  FIG. 11 is a flowchart showing a processing procedure for confirming a response and storing it in the response log DB 64.

  In step 3301, a connection request from the service using terminal 500 is accepted.

  In step 3302, the simultaneous connection user confirmation process is called.

  In step 3303, it is determined whether the number of simultaneously connected users confirmed in step 3302 exceeds the number of simultaneously connected users 6208 in 62 of the monitoring item management DB. If the limit is not exceeded, the process proceeds to step 3305.

  In step 3304, the service using terminal is warned that the upper limit of the number of simultaneously connected users has been reached, and the process ends without connecting to the software service provider 100.

  In step 3305, connection is made to the software service provider 100 in response to a connection request from the service using terminal 500.

  In step 3306, the simultaneous connection user confirmation process is called based on the connection result in step 3305.

  In step 3307, the response result from the support service provider 100 is transmitted to the service using terminal 500. If the response is a timeout, the fact is transmitted to the service using terminal 500. In this case, the response time is “−”.

  In step 3308, the response result date and time 6401, response time 6402, connection source 6403 (IP address of the service using terminal 500 from which the connection request is issued), URL 6404, and number of simultaneously connected users 6405 are stored in the response log DB 64.

  FIG. 12 is a flowchart showing a processing procedure for checking the simultaneous connection user and updating the simultaneous connection user DB.

  In step 3401, it is determined whether or not it is a confirmation process of the number of users who are simultaneously logged in to the software service provider 100. If the number of simultaneously connected users is confirmed, the process proceeds to step 3402; otherwise, the process proceeds to step 3403.

  In step 3402, the simultaneous connection user DB 65 is confirmed, the number of simultaneous connection users is calculated, and the process ends.

  In step 3403, it is determined whether or not the connection request from the service using terminal 500 is a login process to the software service provider 100. In the case of a login process, the process proceeds to step 3404. Otherwise, go to step 3405.

  In step 3404, the session ID of the connection request from the service using terminal 500 is confirmed and registered in the simultaneous connection user DB 65 as a new record.

  In step 3405, it is determined whether or not the connection request from the service using terminal 500 is a request processing to the software service provider 100. If the request is a request processing, the process proceeds to step 3406. Otherwise, the process proceeds to step 3407.

  In step 3406, the session ID of the connection request from the service using terminal 500 is confirmed, the record of the simultaneous connection user DB 65 is updated, and the process ends.

  In step 3407, it is determined whether the connection request from the service using terminal 500 is a logout process to the software service provider 100. If the logout process is a logout process, the process proceeds to step 3408. Otherwise, the process proceeds to step 3409.

  In step 3408, the session ID of the connection request from the service using terminal 500 is confirmed, the record in the simultaneous connection user DB 65 is deleted, and the process ends.

  In step 3409, it is confirmed whether there is a session ID that has reached the upper limit of the no-response logout time 6211 without responding while logged in among the users registered in the simultaneous connection user DB 65. If there is a session ID, the process proceeds to step 3410. If finished.

  In step 3410, the session ID of the connection request from the service using terminal 500 is confirmed, and the record in the simultaneous connection user DB 65 is deleted. If necessary, logout processing of the session with the ID of the session is performed and the process is terminated.

  FIG. 13 is a flowchart showing a processing procedure for analyzing the operation log DB 63 and the response log DB 64 and storing them in the usage record DB 66.

  In step 3501, the first operating time 0:00 and the last operating time 24:00 between 0:00 and 24:00 on the previous day are stored from the operation log DB (63).

  In step 3502, the total number of timeout times (response time “-”) between 0:00 and 24:00 the previous day from the operation log DB (63) (however, the reference site host at the same time has timed out) In this case, the service operation rate 6205 of the monitoring item management DB 62 is multiplied by 0.6 seconds, and divided by 24 hours.

  In step 3503, the service log DB 63 is searched for the service stop time and service restart time from 0:00 to 24:00 on the previous day. Recovery time = Average of recovery time obtained by “service restart time−service stop time” is calculated.

  In step 3504, the maximum value of the response time of the response log 64 from 0:00 to 24:00 on the previous day is calculated.

  In step 3505, the maximum value of the number of simultaneously connected users 6405 in the response log 64 from 0:00 to 24:00 on the previous day is calculated.

  In step 3506, the analysis date / time 6601, service time 6602, service operation rate 6603, average recovery time 6604, online response time (via network) 6605, and maximum number of simultaneously connected users 6606 are stored in the usage record DB 66.

  FIG. 14 is a flowchart showing a processing procedure for comparing the monitoring item DB 62 and the usage record DB 66 and giving a warning when there is a suspicion of SLA violation.

  In step 3601, the contents of the monitoring item management DB 62 and the usage record DB 66 are compared to audit whether there is a suspicion of SLA violation. Specifically, whether the service time 6602 satisfies the condition of the service time 6204 is audited and whether the service operation rate 6603 satisfies the condition of the service operation rate 6205 is audited, and whether the average recovery time 6604 satisfies the condition of the average recovery time 6206 Auditing is performed to check whether the online response time (via the network) 6605 satisfies the condition of the online response time (via the network) 6207, and whether the maximum number of simultaneously connected users 6606 satisfies the condition of the simultaneously connected users 6208.

  In step 3602, it is determined whether there is a suspicion of SLA violation as a result of the comparison in step 3601. If there is no doubt, the process ends. If there is a doubt, the process proceeds to step 3603.

  In step 3603, the service usage manager terminal 400 is notified that there is a suspicion of SLA violation by a predetermined method.

  DESCRIPTION OF SYMBOLS 300 ... Service monitoring system, 31 ... Communication processing part, 32 ... Operation confirmation processing part, 33 ... Response confirmation processing part, 34 ... Simultaneous connection user confirmation processing part, 35 ... Log analysis processing part, 36 ... Auditing / warning processing part, 61 ... SLA definition information DB, 62 ... monitoring item management DB, 63 ... operation log DB, 64 ... response log DB, 65 ... simultaneous connection user DB, 66 ... use result DB.

Claims (5)

  A service evaluation system that enables a software service user to perform an audit using a system set on the user side to determine whether the software service user is operating within the scope of the contract set for the items specified in the SLA.   A service evaluation system that can audit software services from the perspective of SLA guidelines.   When verifying the operation status, a service evaluation system that eliminates communication problems by comparing the response time of a well-known and highly trusted representative site, and confirms the more accurate operation rate.   A service evaluation system that measures the response time for actual business request processing and can audit online response time in actual operation.   Service evaluation system that has a function to analyze and report usage status regularly (approximately once a day) and automatically warn if there is a suspicion of SLA violation in the analysis result.

Images