JP2022087371A - Monitoring device, monitoring method, and program - Google Patents

Abstract

To provide a monitoring device that controls monitoring in consideration of an operating environment of each server.SOLUTION: In a maintenance management system, a monitoring device 100 comprises: acquisition means 101 for acquiring hardware monitoring information for each server, an individual failure rate due to operating time and operating environment; calculation means 102 for calculating the individual failure rate based on the hardware monitoring information for each server, the operating time, and an operating environment temperature of the server; and determination means 103 for determining a monitoring interval for a server based on the individual failure rate.SELECTED DRAWING: Figure 1

Description

This disclosure relates to server monitoring.

Various services via the Internet are realized by running a server with a program installed in a computer. However, the computer is not always running stably, and the server may go down or slow down due to an unexpected failure. It is important to constantly monitor the server and take prompt action in the event of a failure in order to provide stable services.

In order to detect the occurrence of a failure of an electronic device, the electronic device is monitored through a network. Patent Document 1 discloses that a maintenance-managed device such as a personal computer or a server, or a monitoring device notifies the occurrence of a failure in the maintenance-managed device. Patent Document 2 discloses that a device to be managed such as a printer is monitored at a predetermined monitoring interval according to a usage situation.

As a technique related to the present disclosure, Patent Document 3 describes an operation / maintenance support system for determining the presence or absence of an abnormality in a device in consideration of information on the cumulative operating status of the device and parameters unique to each device. Is disclosed.

Japanese Unexamined Patent Publication No. 2016-081374 Japanese Unexamined Patent Publication No. 2014-053027 Japanese Unexamined Patent Publication No. 2010-271905

The failure rate of electronic devices changes depending on the operating environment such as temperature and humidity. However, in Patent Document 2, monitoring and control of electronic devices are not performed in consideration of the operating environment of the devices.

It is an object of the present disclosure to provide a monitoring device or the like that controls monitoring in consideration of the operating environment of the server.

The monitoring device according to the present disclosure determines the monitoring interval for the server based on the acquisition means for acquiring the individual failure rate due to the hardware, operating time, and operating environment for each server, and the individual failure rate. Means and.

The monitoring method according to the present disclosure acquires the individual failure rate due to the hardware, operating time, and operating environment for each server, and determines the monitoring interval for the server based on the individual failure rate.

The program according to the present disclosure acquires the individual failure rate due to the hardware, operating time, and operating environment of each server, and executes a process of determining the monitoring interval for the server on the computer based on the individual failure rate. Let me.

According to the present disclosure, monitoring can be controlled in consideration of the operating environment of the server.

It is a schematic diagram which shows the structure of the maintenance management system which concerns on 1st Embodiment. It is a hardware schematic diagram of a server 300. It is a sequence diagram which shows the operation example of the maintenance management system. It is a graph which shows an example of a relational model. It is a graph which shows another example of a relational model. It is a block diagram which shows the structure of the monitoring apparatus 100 which concerns on 2nd Embodiment. It is a schematic diagram which shows the arrangement example of the calculation apparatus 120. It is a graph which shows other example of the relationship between an individual failure rate and a determined monitoring interval. It is a flowchart which shows the operation example of the monitoring apparatus 100 which concerns on 2nd Embodiment. It is a block diagram which shows the example of the hardware composition of the computer 500.

In this disclosure, a monitoring device for monitoring a plurality of electronic devices constituting a large-scale system for maintenance management will be described. As an example of the electronic device to be monitored, a monitoring device for monitoring a server will be taken as an example, and the following embodiments will be described with reference to the drawings. However, the monitored target may include electronic devices other than the server.

[First Embodiment]
(Constitution)
FIG. 1 is a schematic diagram showing a configuration of a maintenance management system according to the first embodiment. The maintenance management system includes a monitoring device 100 and one or more server racks 200 (200_1, ..., 200_y) to be monitored. Each server rack 200 includes one or more servers 300 (300_1, 300_2, ..., 300_x) and a temperature sensor 400. The monitoring device 100 is communicably connected to each server 300 and the temperature sensor 400 by the network 010. The number of servers 300 for each server rack 200 can be changed arbitrarily.

The monitoring device 100 monitors the presence or absence of a failure in the server 300. In the first embodiment, the monitoring device 100 includes an acquisition unit 101, a calculation unit 102, and a determination unit 103.

The acquisition unit 101 acquires hardware monitoring information, operation information, and information on the operating environment for each server 300. The hardware monitoring information indicates hardware monitoring data for determining an abnormality tendency of each component inside the server 300. The operation information indicates the operation status of the server 300, such as the operation time from the initial arrangement or the number of years of use. The information regarding the operating environment is, for example, information indicating the temperature or humidity in the server rack 200, or the temperature of the server 300. Further, the acquisition unit 101 acquires the individual failure rate based on the hardware, operation information, and operating environment of each server 300 calculated by the calculation unit 102.

The calculation unit 102 calculates the individual failure rate P for each server 300 based on the hardware monitoring information, the operation information, and the information regarding the operating environment acquired by the acquisition unit 101. The calculation of the individual failure rate P will be described later.

The determination unit 103 determines the monitoring interval for each server 300 based on the relationship model between the individual failure rate of each server 300 and the monitoring interval. The monitoring interval is a time interval after the monitoring device 100 executes the failure monitoring process for the server 300 until the monitoring process is executed again. As a monitoring process, the monitoring device 100 may transmit an input / output request to the server 300, for example, and confirm whether the response time to the input / output request is within a normal range. Alternatively, the monitoring device 100 may confirm whether the monitoring data of each component of the server 300 is within a normal range as a monitoring process.

The relationship model between the individual failure rate and the monitoring interval may show a decreasing tendency of the monitoring interval corresponding to the increase in the individual failure rate. The relational model may be represented by a relational expression between the individual failure rate and the monitoring interval, or may be represented by a table. The relational model may be obtained by machine learning with the objective variable as the network load and, for example, the failure rate, the monitoring interval, the number of servers, and the network bandwidth as the explanatory variables.

In the first embodiment, the temperature sensor 400 measures the temperature inside the server rack 200. The temperature sensor 400 transmits the measured temperature to the monitoring device 100 as information regarding the operating environment of the servers 300_1, ..., 300_x in the server rack 200.

When the temperature of each server 300 is used as the information regarding the operating environment, one temperature sensor 400 may be installed for each server 300. The temperature sensor 400 transmits the measured temperature of each server to the monitoring device 100.

In the server rack 200, other sensors may be installed depending on the type of information regarding the operating environment acquired by the acquisition unit 101. When humidity is used as information regarding the operating environment, the description regarding the temperature sensor 400 can be replaced with the humidity sensor in the following description.

FIG. 2 is a schematic hardware diagram of the server 300. It includes a CPU (Central Processing Unit) 301, a memory 302, an HDD (hard disk drive) 303, an e-support 304, and a BMC (Baseboard Management Controller) 309. The BMC 309 acquires monitoring data from the hardware of the server 300 (each component and element constituting inside the server 300) and transmits it to the monitoring device 100 as hardware monitoring information.

(motion)
FIG. 3 is a sequence diagram showing an operation example of the maintenance management system. In FIG. 3, only one server 300 is shown for simplification, but each server 300 operates in the same manner.

<< Calculation of failure rate due to hardware >>
The acquisition unit 101 of the monitoring device 100 acquires hardware monitoring information from the server 300 via the network 010 and the BMC 309. The calculation unit 102 calculates the failure rate _Ph due to hardware based on the monitoring information acquired by the acquisition unit 101 (step S101).

The failure rate _Ph due to hardware is the probability of failure of individual devices. The failure rate _Ph is calculated, for example, between 0 and 1 (or 0% to 100%).

The hardware monitoring information includes monitoring data corresponding to a plurality of monitored components such as CPU 301, memory 302, HDD 303, e-support 304, and power supply. The monitoring data regarding the CPU 301 is, for example, the CPU usage rate and the temperature. As the monitoring data related to the memory 302, for example, the number of errors in ECC (Error checking and correcting) is used. As the monitoring data related to the HDD 303, for example, inspection values by SMART (Self-Monitoring, Analysis and Reporting Technology) are used. The monitoring data regarding the power supply indicates, for example, the redundancy state of the power supply.

The calculation unit 102 determines, for example, whether or not there is an abnormality tendency based on the value of the monitoring data acquired for each hardware failure target component. When the monitoring data exceeds a predetermined threshold value, the calculation unit 102 determines that the component has an abnormal tendency. Alternatively, the calculation unit 102 may determine that the component has an abnormality tendency when the failure rate of the component calculated based on the value of the monitoring data of the component is equal to or higher than a predetermined threshold value. When the monitoring information includes information about 100 parts, the calculation unit 102 increases the failure rate Ph by _+0.01 (or + 1%) by, for example, the number of parts having a tendency to be abnormal. The failure rate _Ph to be increased may be adjusted for each component based on the importance of the component included in the monitoring information and the value of the measured data.

<< Calculation of failure rate due to operating environment >>
The acquisition unit 101 of the monitoring device 100 acquires the measured temperature of the temperature sensor 400 installed in the server rack 200 as information regarding the operating environment of the server 300. The calculation unit 102 calculates the failure rate _Pe due to the operating environment based on the temperature acquired by the acquisition unit 101 (step S102).

The failure rate _Pe due to the operating environment is the probability of failure calculated based on the difference between the environment in which the server 300 is installed and the stable operating conditions. The failure rate _Pe is calculated, for example, between 0 and 1 (0% to 100%).

The calculation unit 102 may, for example, 0 (or 0%) if the temperature is the median of the assumed operating temperature, 1 (or 100%) if the temperature is equal to or higher than the upper limit of the temperature, and 1 (or 100%) if the temperature is equal to or lower than the lower limit of the temperature. %). The calculation unit 102 may calculate the failure rate _Pe in consideration of the rate of change in the operating environment temperature, the duration of the temperature, or the number of times the predetermined upper limit value or lower limit value is exceeded.

The acquisition unit 101 may acquire the temperature distribution of the server room in which the server 300 is arranged from an external device. The acquisition unit 101 may acquire the operating environment temperature of each server 300 based on the temperature distribution of the server room and the arrangement of the servers 300.

<< Calculation of failure rate due to operating time >>
The server 300 records and accumulates the total operating time (total energizing time) from the initial arrangement. The acquisition unit 101 acquires the operating time of the server 300 from the server 300 as operation information. The calculation unit 102 calculates the failure rate _Pt based on the operating time (step S103).

The failure rate _Pt due to the operating time is the probability of failure calculated based on the bathtub curve of the device showing the life characteristic of the server 300 due to the total operating time. The failure rate _Pt is 0 (0%) in the unused state, and the upper limit of the operating time is 1 (100%).

<< Calculation and acquisition of individual failure rate P >>
The calculation unit 102 comprehensively calculates the failure rate _{Ph due to the hardware, the failure rate P e due} to the operating environment, and the failure rate P _t due to the operating time calculated for each server 300. In consideration, the individual failure rate P is calculated (step S104). The acquisition unit 101 acquires the individual failure rate P calculated by the calculation unit 102 (step S105).

The order of steps S101 to 103 can be changed.

Calculation example of individual failure rate P 1: The calculation unit 102 may give priority or weight to three failure rates _Ph , P _e , and P _t . The individual failure rate P is expressed by, for example, the following equation.
P = W _h x P _h + We x P _e + W _t x P _{t (} weighting constants W _h , _We , W _t > 0)
Calculation Example 2: Individual Failure Rate P: When the failure rates _Ph and P _t increase to a predetermined value, the calculation unit 102 may set the weight _We of the failure rate P _e to be small. That is, a server having a high failure rate _{Ph due to hardware and a high failure rate P t due} to operating time can calculate a high individual failure rate P even if the failure rate _Pe due to the operating environment is low. good. This allows the server to be monitored at short monitoring intervals.

<< Determination of monitoring interval >>
The determination unit 103 determines the monitoring interval for the server 300 based on the relationship model between the individual failure rate of the server 300 and the monitoring interval (step S106). FIG. 4 is a graph showing an example of a relational model. In FIG. 4, the relational model between the individual failure rate P and the monitoring interval I is represented by a straight line, but may be a curve. Further, in FIG. 4, a continuous value is set for the monitoring interval I as the individual failure rate increases, but the monitoring interval I may be discontinuous (discrete). The monitoring interval I may be expressed by a different relational expression depending on the range of the individual failure rate P.

As the individual failure rate P approaches 100, the monitoring interval I may be set shorter than the normal monitoring interval of a general monitoring device. Further, as the individual failure rate P is closer to 0, the monitoring interval I may be set longer than the normal monitoring interval of a general monitoring device.

The monitoring device 100 monitors the server 300 at a monitoring interval determined for each server 300.

(effect)
According to the first embodiment, monitoring can be controlled in consideration of the operating environment of the server 300. The reason is that the acquisition unit 101 of the monitoring device 100 acquires the individual failure rate due to the hardware, operating time, and operating environment of each server, and the determination unit 103 determines the monitoring interval for the server 300 based on the individual failure rate. To decide.

In system maintenance, many electronic devices are monitored by one monitoring device. As the scale of the system increases, the number of electronic devices to be monitored increases. Further, each time the number of monitoring items is increased, the access for the number of electronic devices increases, so that the network load and the load of the monitoring device increase. Therefore, in a large system, the network load and the load of the monitoring device can be reduced by uniformly increasing the monitoring interval.

However, if the monitoring interval is long, the time from the occurrence of the failure to the detection of the failure by the monitoring device becomes long. If a failure cannot be detected immediately, the utilization rate of the entire system will be reduced, and the quality of service will be reduced.

According to the first embodiment, the determination unit 103 monitors many servers by determining the monitoring interval of the server 300 whose individual failure rate P is lower than the standard of the predetermined failure rate to be longer than the standard of the predetermined monitoring interval. The load on the network 010 can be reduced and the monitoring load can be reduced.

Further, according to the first embodiment, the determination unit 103 makes the monitoring interval of the server 300, which has an individual failure rate P higher than the standard of the predetermined failure rate, shorter than the standard of the predetermined monitoring interval, so that the actual failure occurs. The time from the occurrence to the detection of the failure by the monitoring device 100 can be shortened, and the delay in the detection of the failure can be prevented.

(Modification example)
FIG. 5 is a graph showing another example of the relational model. In FIG. 5, the relational model is represented by a different relational expression depending on whether the individual failure rate P is smaller than the reference failure rate P ₀ or greater than or equal to the reference failure rate P ₀ .

When P ≤ P ₀ , I = _IA
When 0 <P < _{P 0} , I = IB
In the example of FIG. 5, for example, the monitoring interval _IA may be set to 2 to 3 minutes, and the monitoring interval _IB may be set to 1 hour. For example, the determination unit 103 may set the average value or the median of the individual failure rates of the servers 300_1, 300_2, ..., _{300_x} of each server rack 200 in a predetermined period as the reference failure rate P0. Further, in the determination unit 103, the reference failure rate P ₀ may be set by using the above-mentioned machine learning.

[Second Embodiment]
In the first embodiment, the case where the monitoring device 100 includes a calculation unit for calculating the individual failure rate P has been described. In the second embodiment, a case where another device has the function of the calculation unit 102 according to the first embodiment and the monitoring device 100 acquires the individual failure rate P calculated by the calculation device will be described. The same description as in the first embodiment will be omitted in the description of the second embodiment.

(Constitution)
FIG. 6 is a block diagram showing the configuration of the monitoring device 100 according to the second embodiment. The monitoring device 100 according to the second embodiment includes an acquisition unit 121 and a determination unit 122. The monitoring device 100 according to the second embodiment can be replaced with the monitoring device 100 in FIG.

FIG. 7 is a schematic diagram of the maintenance management system according to the second embodiment, showing an arrangement example of the calculation device 120. As shown in FIG. 7, one probability calculation device 120 may be communicably connected to the monitoring device 100, each server 300 of each server rack 200, and the temperature sensor 400. The calculation device 120 acquires hardware monitoring information and operation information from each server 300, and acquires information on the operating environment from the temperature sensor 400.

The acquisition unit 121 of the monitoring device 100 acquires the hardware of each server 300, the operation information, and the individual failure rate based on the operating environment from the external calculation device 120.

The determination unit 122 determines the monitoring interval for the server based on the individual failure rate of each server 300.

The determination unit 122 determines the monitoring interval of the server 300 as the first monitoring interval when the individual failure rate is higher than the predetermined reference failure rate, and when the individual failure rate is lower than the reference failure rate, the monitoring interval of the server 300. May be determined as a second monitoring interval that is longer than the first monitoring interval. When the individual failure rate is equal to the reference failure rate, the determination unit 122 determines the monitoring interval at any time equal to or greater than the first monitoring interval and equal to or less than the second monitoring interval.

The relationship between the individual failure rate and the monitoring interval determined by the determination unit 122 may be shown by the graph of FIG. 4 or FIG. FIG. 8 is a graph showing another example of the relationship between the individual failure rate and the determined monitoring interval. In FIG. 8, the monitoring intervals I ₁ to I ₅ are determined corresponding to the reference failure rates P ₁ to P ₄ . _Each of the reference failure rates P1 to _P4 is an embodiment of a predetermined reference failure rate. The monitoring intervals I ₁ to I ₅ are embodiments of the first or second monitoring interval, respectively.

The monitoring device 100 monitors the server 300 according to the determined monitoring interval. Alternatively, the monitoring device 100 transmits a determined monitoring interval to another monitoring processing execution device (not shown), and causes the monitoring processing execution device to monitor the server 300.

In FIG. 7, one calculation device 120 is arranged, but one calculation device 120 may be arranged for each server rack 200. At this time, the calculation device 120 calculates the individual failure rate of each server 300 in the server rack 200. Alternatively, the function of the calculation device 120 may be provided in each server 300. At this time, each server 300 acquires information on the operating environment from the temperature sensor 400 and calculates the individual failure rate of its own device. The monitoring device 100 acquires the individual failure rate of each server 300 from each of the plurality of calculation devices 120.

(motion)
FIG. 9 is a flowchart showing an operation example of the monitoring device 100 according to the second embodiment. The acquisition unit 121 acquires the individual failure rate due to the hardware, operating time, and operating environment for each server 300 (step S121). The determination unit 122 determines the monitoring interval for the server 300 based on the individual failure rate of each server 300 (step S122).

(effect)
According to the second embodiment, monitoring can be controlled in consideration of the operating environment of the server 300. The reason is that the acquisition unit 121 of the monitoring device 100 acquires the individual failure rate due to the hardware, operating time, and operating environment of each server, and the determination unit 122 sets the monitoring interval for the server 300 based on the individual failure rate. To decide.

(Modification example)
In addition to the individual failure rate, the determination unit 122 may further determine the monitoring interval for the server based on the ratio of the number of servers 300 for each monitoring interval or the upper limit number. That is, the determination unit 122 has a reference failure rate so that, for example, when the number of servers 300 monitored in the first monitoring interval reaches a predetermined upper limit, the remaining servers 300 are monitored in the longer second monitoring interval. To determine.

According to this modification, the request to reduce the network load and the monitoring load when monitoring many servers, and the time from the actual occurrence of the failure to the detection of the failure by the monitoring device 100 are shortened. It can respond to both requests to prevent delays in fault detection.

[Hardware configuration]
In each of the above-described embodiments, each component of the monitoring device 100 indicates a block of functional units. Some or all of the components of each device may be realized by any combination of the computer 500 and the program.

FIG. 10 is a block diagram showing an example of the hardware configuration of the computer 500. Referring to FIG. 10, the computer 500 includes, for example, a CPU (Central Processing Unit) 501, a ROM (Read Only Memory) 502, a RAM (Random Access Memory) 503, a program 504, a storage device 505, a drive device 507, and a communication interface 508. , Input device 509, input / output interface 511, and bus 512.

The program 504 includes an instruction for realizing each function of each device. The program 504 is stored in the ROM 502, the RAM 503, and the storage device 505 in advance. The CPU 501 realizes each function of each device by executing the instruction included in the program 504. For example, the function of the monitoring device 100 is realized by the CPU 501 of the monitoring device 100 executing the instruction included in the program 504. Further, the RAM 503 may store data processed in each function of each device. For example, the hardware monitoring information and the operating environment temperature used by the calculation unit 102 of the monitoring device 100 may be stored in the RAM 503 of the computer 500.

The drive device 507 reads and writes the recording medium 506. The communication interface 508 provides an interface with a communication network. The input device 509 is, for example, a mouse, a keyboard, or the like, and receives input of information from the administrator of the maintenance management system. The output device 510 is, for example, a display, and outputs (displays) information to the administrator. The input / output interface 511 provides an interface with peripheral devices. Bus 512 connects each component of these hardware. The program 504 may be supplied to the CPU 501 via the communication network, or may be stored in the recording medium 506 in advance, read by the drive device 507, and supplied to the CPU 501.

The hardware configuration shown in FIG. 10 is an example, and components other than these may be added or may not include some components.

There are various modifications in the method of realizing each device. For example, each device may be realized by any combination of a computer and a program, which are different for each component. Further, a plurality of components included in each device may be realized by any combination of one computer and a program.

Further, a part or all of each component of each device may be realized by a general-purpose or dedicated circuitry including a processor or the like, or a combination thereof. These circuits may be composed of a single chip or a plurality of chips connected via a bus. A part or all of each component of each device may be realized by the combination of the circuit or the like and the program described above.

Further, when a part or all of each component of each device is realized by a plurality of computers, circuits, etc., the plurality of computers, circuits, etc. may be centrally arranged or distributed.

Further, at least a part of the monitoring device 100 may be provided in the SaaS (Software as a Service) format. That is, at least a part of the functions for realizing the monitoring device 100 may be executed by software executed via the network.

Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above embodiments. Various changes that can be understood by those skilled in the art can be made to the structure and details of the present disclosure within the scope of the present disclosure. Also, the configurations in each embodiment can be combined with each other as long as they do not deviate from the scope of the present disclosure.

100 Monitoring device 101, 121 Acquisition unit 102 Calculation unit 103, 122 Decision unit 120 Calculation device 200 Server rack 300 Server 400 Temperature sensor 500 Computer

Claims (8)

An acquisition method for acquiring the individual failure rate due to the hardware, operating time, and operating environment of each server, and
A determination means for determining the monitoring interval for the server based on the individual failure rate, and
A monitoring device equipped with. The acquisition means acquires the individual failure rate based on the hardware monitoring information for each server, the operating time, and the operating environment temperature of the server in the server rack.
The monitoring device according to claim 1. Further provided with a calculation means for calculating the individual failure rate based on the hardware monitoring information for each server, the operating time, and the operating environment temperature of the server.
The monitoring device according to claim 1 or 2. The determination means determines the monitoring interval for each server based on a relational model showing a decreasing tendency of the monitoring interval corresponding to an increase in the individual failure rate.
The monitoring device according to any one of claims 1 to 3. The determination means is
When the individual failure rate is higher than a predetermined reference failure rate, the monitoring interval of the server is determined to be the first monitoring interval.
When the individual failure rate is lower than the reference failure rate, the monitoring interval of the server is determined to be a second monitoring interval longer than the first monitoring interval.
The monitoring device according to any one of claims 1 to 4. The monitoring device according to any one of claims 1 to 5, wherein the determination means further determines the monitoring interval based on the upper limit number of the servers monitored at the monitoring interval. Obtain the individual failure rate due to the hardware, operating time, and operating environment of each server.
The monitoring interval for the server is determined based on the individual failure rate.
Monitoring method. Obtain the individual failure rate due to the hardware, operating time, and operating environment of each server.
A program that causes a computer to execute a process of determining a monitoring interval for the server based on the individual failure rate.

Images