WO2017118133A1 - 一种云系统内部虚拟机的异常检测方法 - Google Patents

一种云系统内部虚拟机的异常检测方法 Download PDF

Info

Publication number
WO2017118133A1
WO2017118133A1 PCT/CN2016/101891 CN2016101891W WO2017118133A1 WO 2017118133 A1 WO2017118133 A1 WO 2017118133A1 CN 2016101891 W CN2016101891 W CN 2016101891W WO 2017118133 A1 WO2017118133 A1 WO 2017118133A1
Authority
WO
WIPO (PCT)
Prior art keywords
virtual machine
state
abnormal
abnormality
probability
Prior art date
Application number
PCT/CN2016/101891
Other languages
English (en)
French (fr)
Inventor
韩德志
毕坤
韩昊宸
王军
谢柏林
陈付梅
Original Assignee
上海海事大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海海事大学 filed Critical 上海海事大学
Priority to US15/736,227 priority Critical patent/US10616268B2/en
Publication of WO2017118133A1 publication Critical patent/WO2017118133A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
    • H04L63/1441—Countermeasures against malicious traffic
    • H04L63/145—Countermeasures against malicious traffic the attack involving the propagation of malware through the network, e.g. viruses, trojans or worms
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
    • G06F11/0706—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation the processing taking place on a specific hardware platform or in a specific software environment
    • G06F11/0709—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation the processing taking place on a specific hardware platform or in a specific software environment in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
    • G06F11/0706—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation the processing taking place on a specific hardware platform or in a specific software environment
    • G06F11/0712—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation the processing taking place on a specific hardware platform or in a specific software environment in a virtual computing platform, e.g. logically partitioned systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
    • G06F11/0751—Error or fault detection not based on redundancy
    • G06F11/0754—Error or fault detection not based on redundancy by exceeding limits
    • G06F11/076—Error or fault detection not based on redundancy by exceeding limits by exceeding a count or rate limit, e.g. word- or bit count limit
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
    • G06F11/0766—Error or fault reporting or storing
    • G06F11/0772—Means for error signaling, e.g. using interrupts, exception flags, dedicated error registers
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
    • G06F11/0793—Remedial or corrective actions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/3055—Monitoring arrangements for monitoring the status of the computing system or of the computing system component, e.g. monitoring if the computing system is on, off, available, not available
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
    • G06F21/55—Detecting local intrusion or implementing counter-measures
    • G06F21/554—Detecting local intrusion or implementing counter-measures involving event detection and direct action
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N7/00—Computing arrangements based on specific mathematical models
    • G06N7/01—Probabilistic graphical models, e.g. probabilistic networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N7/00—Computing arrangements based on specific mathematical models
    • G06N7/08—Computing arrangements based on specific mathematical models using chaos models or non-linear system models
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
    • H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
    • H04L63/1416—Event detection, e.g. attack signature detection
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
    • H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
    • H04L63/1425—Traffic logging, e.g. anomaly detection
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/40—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass for recovering from a failure of a protocol instance or entity, e.g. service redundancy protocols, protocol state redundancy or protocol service redirection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G06F2009/45575—Starting, stopping, suspending or resuming virtual machine instances
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G06F2009/45587—Isolation or security of virtual machine instances
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G06F2009/45591—Monitoring or debugging support
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G06F2009/45595—Network integration; Enabling network access in virtual machine instances
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2201/00—Indexing scheme relating to error detection, to error correction, and to monitoring
    • G06F2201/815—Virtual

Definitions

  • the present invention relates to the field of network technologies, and in particular, to an abnormality detecting method for a virtual machine inside a cloud system.
  • Cloud service providers use commercial virtualization software such as VMware and vSphere to build different types of virtualized infrastructure, including private and public cloud systems, where data from potentially distributed systems can be distributed across hundreds of interconnected computers. Equipment and other physical machines.
  • an enterprise rents a computing resource and a storage resource of a cloud service provider, that is, a cloud tenant.
  • a cloud service provider has multiple virtual machines for different users in one physical host of the cloud system, and the virtual machines have a virtual machine abnormality (virus or external attack) in the cloud system.
  • a virtual machine abnormality virus or external attack
  • the cloud system Assuring that the cloud system is of great significance to the availability of a normal virtual machine, it includes two aspects: First, it provides a reasonable resource allocation service for the normal use of the virtual machine under normal circumstances; The availability of other normal virtual machines in the cloud system under abnormal conditions, that is, the detection of abnormal virtual machines and elimination of abnormalities to ensure the normal use of other normal virtual machines.
  • the method for detecting an abnormality of a virtual machine in a cloud system ensures the real-time and reliability of the abnormal virtual machine in the cloud system, reduces the impact of the anomaly detection on the performance of the entire cloud system, and ensures the normal virtual machine of the cloud system. Availability to users.
  • the technical solution of the present invention is to provide an abnormality detecting method for a virtual machine inside a cloud system:
  • the state attribute information of each virtual machine in the cloud system is collected by the virtual machine state attribute information search module, and is transmitted to the hidden semi-Markov model HsMM online detection module for detection in real time;
  • the hidden semi-Markov model HsMM online detection module detects a virtual machine with abnormal behavior, and transmits state attribute information of the virtual machine with abnormal behavior to the virtual machine abnormality detecting and processing system;
  • the virtual machine abnormality detecting and processing system detects the virtual machine with abnormal behavior, and the abnormal virtual machine that does not reach the abnormality level of the abnormality indicator eliminates the abnormality and issues a warning prompt to the corresponding cloud tenant; If the behavior of the abnormal indicator is abnormal, the virtual machine is alerted to the corresponding cloud tenant and the virtual machine is shut down.
  • the abnormality detecting method of the virtual machine inside the cloud system includes the following processes:
  • Step 1 The virtual machine state attribute information collecting module collects state attribute value items of the virtual machines in the cloud system under normal conditions; the normal state refers to a state in which there is no virus inside the virtual machine and no external attacks;
  • Step 2 Using the state attribute value item of the virtual machine in a normal state as an observation sequence, training the hidden semi-Markov model HsMM and designing an implicit semi-Markov model HsMM online detection algorithm;
  • Step 3 The virtual machine state attribute information collecting module collects state information of each virtual machine during online work according to a preset time interval, and transmits the state information to the hidden semi-Markov model HsMM online detection module in real time;
  • Step 4 The hidden semi-Markov model HsMM online detection module detects the state behavior of each virtual machine online based on the corresponding algorithm obtained in step 2, calculates the probability probability of the state behavior and the Mahalanobis distance, and thereby judges the virtual machine. Abnormal behavior;
  • Step 5 Compare the Mahalanobis distance calculated according to the online behavior of each virtual machine with the preset threshold value Q, and determine whether the Mahalanobis distance of the virtual machine online behavior is greater than a preset threshold value Q:
  • step 6 If yes, go to step 6; if no, go to step 3;
  • Step 6 Start a cloud system abnormality detecting and processing system, and perform an abnormality detection on the virtual machine whose detection result is greater than the preset threshold value Q;
  • Step 7 Determine whether the abnormality indicator of the abnormality detecting virtual machine in step 6 is greater than the maximum threshold value of the abnormal detection and processing E max :
  • the abnormality detecting and processing system eliminates the abnormality and sends a warning prompt to the cloud tenant, and then proceeds to step 3;
  • Step 8 The abnormality detecting and processing system alerts the cloud tenant of the virtual machine whose abnormal rate is greater than or equal to E max and closes the virtual machine.
  • the lightweight virtual machine state attribute information collection module collects and transmits the state information of the virtual machine in real time.
  • the lightweight hidden semi-Markov model HsMM online detection algorithm can quickly detect the virtual machine with abnormal state behavior. Its detection speed is much faster than the commonly used virtual machine anomaly detection software. Because the traditional anomaly detection method is detected periodically after a virtual machine abnormality is found.
  • the abnormality detection and processing system composed of multiple heterogeneous detection engines is used to detect the virtual machine with abnormal behavior. Anomaly detection greatly improves the accuracy of the abnormal virtual machine judgment inside the cloud system.
  • Figure 2 is a block diagram of the cloud system.
  • FIG. 1 is a flowchart of an abnormality detecting method of a virtual machine in a cloud system provided by the present invention
  • FIG. 2 is a structural diagram of a cloud system in the present invention.
  • the cloud system of the present invention includes an application server cluster and various types of storage devices.
  • the application server cluster is equipped with a cloud operating system, virtualization software, virtual machine state attribute information collection module, hidden semi-Markov model HsMM online detection module, anomaly detection and processing system, and virtual machines established for cloud tenants.
  • the cloud operating system is an operating system supported by cloud computing and cloud storage technologies, and is an overall management operation system of a cloud computing back-end data center. It refers to basic hardware resources and a stand-alone operating system framed in servers, storage, and networks.
  • a cloud platform integrated management system that manages massive amounts of basic hardware and software resources on top of basic software such as middleware and databases.
  • the virtualization software allows a main server to establish and execute one or more virtualized environments, such as virtualization software such as VMware and Xen.
  • the virtual machine state attribute information collecting module is configured to invoke a virtual machine state in the cloud system.
  • the hidden semi-Markov model HsMM online detection module includes an HsMM online detection algorithm for detecting a hidden semi-Markov model of virtual machine online behavior abnormality.
  • the abnormality detecting and processing system is composed of a heterogeneous detection engine produced by different manufacturers, and can effectively detect internal viruses and external attacks of abnormal virtual machines, including open source and non-open source detection engines.
  • the virtual machine anomaly detection and processing process in the present invention includes:
  • the virtual machine state attribute information search module collects the state attribute information of each virtual machine, and transmits it to the hidden semi-Markov model HsMM online detection module in real time;
  • the HsMM online detection module quickly detects the virtual machine with abnormal behavior based on the corresponding algorithm, and transmits the relevant information to the virtual machine anomaly detection and processing system;
  • the virtual machine abnormality detection and processing system detects the virtual machine with abnormal behavior, eliminates the abnormality of the abnormally virtual virtual machine, and sends an alarm message to the corresponding cloud tenant, and issues an alarm and closes the severely abnormal virtual machine.
  • the double abnormality detection of the invention greatly improves the accuracy of abnormal virtual machine abnormality determination within the cloud system, and fully guarantees the high availability of the normal virtual machine.
  • an abnormality detecting method for a virtual machine in a cloud system includes the following steps:
  • Step 1 The virtual machine state attribute information collecting module collects state information of each virtual machine in the cloud system under normal conditions.
  • the normal state refers to a state in which there is no virus inside the virtual machine and no external attacks.
  • the state information of the virtual machine is composed of attribute value items that reflect the normal or abnormal operation of the virtual machine, including virtual machine CPU utilization, GPU utilization, I/O waiting time, and memory utilization, and the like. Dynamic changes over time.
  • Step 2 using the state attribute value item of the virtual machine in a normal state as an observation sequence, training hidden The semi-Markov model HsMM and the hidden semi-Markov model HsMM online detection algorithm (detailed below).
  • Step 3 The virtual machine state attribute information collecting module collects state information of each virtual machine in the cloud system during online work according to a preset time interval, and transmits the state information to the hidden semi-Markov model HsMM online detection module in real time.
  • Step 4 The hidden semi-Markov model HsMM online detection module detects the state behavior of each virtual machine of the cloud system online based on the corresponding algorithm obtained in step 2, calculates the probability probability and the Mahalanobis distance of the state behavior, and judges the virtual The behavior of the machine is abnormal.
  • the abnormal behavior of the virtual machine refers to abnormality in performance indicators such as CPU utilization, GPU utilization, I/O waiting time, and memory utilization of the virtual machine, or virtual machine resources are exhausted or performance is gradually deteriorated. Case.
  • Step 5 Comparing the Mahalanobis distance calculated according to the online behavior of each virtual machine with the preset threshold value Q, determining whether the Mahalanobis distance of the online behavior of the virtual machine is greater than a preset threshold value Q: if yes, proceeding to the step 6; If no, go to step 3.
  • the preset threshold value Q is a minimum standard for the cloud system virtual machine behavior abnormality detection result to be accepted.
  • Step 6 Start a cloud system abnormality detection and processing system, and perform an abnormality detection on the virtual machine whose detection result is greater than the threshold Q.
  • the abnormality detection is a comprehensive internal virus detection and external attack detection on a virtual machine with abnormal behavior.
  • Step 7 Determine whether the virtual machine abnormality index obtained in step 6 is greater than the maximum threshold value of the abnormal detection and processing E max : if it is greater than or equal to E max , then go to step 8; if less than E max , the abnormality detection and processing system eliminates Abnormal and send a warning to the cloud tenant and go to step 3.
  • the maximum threshold value E max of the abnormality detection and processing is the maximum abnormality indicator that the abnormality detection and processing system can handle.
  • Step 8 The abnormality detecting and processing system alerts the cloud tenant of the virtual machine whose abnormal rate is greater than or equal to E max and closes the virtual machine.
  • the following describes the process of training the hidden semi-Markov model HsMM and obtaining the hidden semi-Markov model HsMM online algorithm.
  • ⁇ g represents the probability that the hidden Markov model is in the state s g at the first moment;
  • the probability of k , K is the maximum length of the observed sequence;
  • P is the state persistence probability matrix
  • q t s g ] represents the model The probability of being in state s g at the tth moment and also continuing for d moments under state s g , where D represents the maximum time the state continues.
  • the attribute value status of each virtual machine in the cloud system includes:
  • State 2 One or more of the CPU, GPU, and memory utilization of each virtual machine is higher than 30% but less than 50%, and the rest is less than 30%, and the I/O wait time is normal;
  • the state 1234 is a normal state and the state 56 is an abnormal state.
  • the process of training the hidden semi-Markov model HsMM by using the attribute value state observation sequence in the normal state of the virtual machine includes:
  • H is the total number of virtual machine state observation sequences
  • T * is the length of the corresponding observation sequence
  • G is the total number of model states
  • D is the maximum time the state continues
  • H is the total number of virtual machine state observations
  • T * is the length of the corresponding sequence.
  • the initial likelihood logarithmic probability distribution of the normal virtual machine and the likelihood logarithm probability calculation formula of the online virtual machine are obtained, and the simplified Mahalanobis distance can be used to measure the initial likelihood logarithmic probability distribution of the normal virtual machine.
  • the distance between the probability of the logarithmic logarithm of the online virtual machine in the cloud system and the simplified Mahalanobis distance is shown in equation (15):
  • the value of d in formula (15) reflects the abnormal degree of virtual machine online state behavior in the cloud system.
  • a threshold Q representing the normal behavior of the virtual machine can be defined.
  • d ⁇ Q the state behavior of the virtual machine can be judged.
  • d>Q it can be judged that the state of the virtual machine is abnormal, that is, a virus may occur or an external attack may occur.
  • the Mahalanobis distance is a method proposed by the Indian mathematician P.C. Mahalanobis to calculate the distance between two points of correlation.
  • the abnormality detection and processing system in the cloud system is started to perform abnormality detection.
  • the severity index E of the virtual machine behavior abnormality is obtained.
  • i , E i p 1 + p 2 ;
  • p 1 is an indicator that can reflect the virus inside the abnormal virtual machine. If the virus inside the abnormal virtual machine can be processed, p 1 takes a value of 0. If the virus inside the abnormal virtual machine cannot be processed, p 1 The value is 1;
  • p 2 is an indicator that can be processed by an external attack of an abnormal virtual machine. If an abnormal external attack can be processed, p 2 takes a value of 0. If the abnormal external attack of the virtual machine cannot be processed, p 2 takes a value of 1.
  • the lightweight virtual machine state attribute information search module collects the state information of each virtual machine and transmits it to the HsMM online detection module of the hidden semi-Markov model, which is detected by the HsMM online detection algorithm based on the hidden semi-Markov model.
  • the state of the three virtual machines is abnormal, and the abnormality detection and processing system in the cloud system is started to detect and process the abnormalities of the three virtual machines.
  • the present invention only needs to detect and eliminate abnormalities of three abnormal virtual machines, and the traditional method needs to detect all virtual machines, including abnormality detection of virtual machines with actual normal and actual abnormalities. Anomaly detection and processing takes up resources and working hours of the virtual machine.
  • the present invention can fully guarantee the real-time and accuracy of cloud system virtual machine anomaly detection and processing, and ensure the availability of a normal virtual machine.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Computer Security & Cryptography (AREA)
  • Computing Systems (AREA)
  • Quality & Reliability (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Computer Hardware Design (AREA)
  • Mathematical Physics (AREA)
  • Artificial Intelligence (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Mathematical Optimization (AREA)
  • Pure & Applied Mathematics (AREA)
  • Mathematical Analysis (AREA)
  • Computational Mathematics (AREA)
  • Algebra (AREA)
  • Medical Informatics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Probability & Statistics with Applications (AREA)
  • Nonlinear Science (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Virology (AREA)
  • Debugging And Monitoring (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

一种云系统内部虚拟机的异常检测方法，通过搜集云系统中正常虚拟机的状态信息来训练隐半马尔可夫模型HsMM，并设计相应算法来检测并计算云系统中各虚拟机在线时资源动态变化行为的或然概率和马氏距离。若对某一虚拟机在线检测结果的马氏距离大于预设门限值，说明该虚拟机的活动情况异常，则启动云系统内部的异常检测和处理系统对该虚拟机进行异常检测和处理。若检测到某虚拟机的异常率小于异常检测和处理的最大门限值时，消除异常后向该虚拟的云租户发警告提示；否则，向该虚拟机的云租户报警并关闭该虚拟机。该方法能实时检测云系统内部虚拟机的异常行为，占用系统资源少，能充分保证云系统内部虚拟机的高可用性和安全性。

Description

一种云系统内部虚拟机的异常检测方法 技术领域
本发明涉及网络技术领域，具体涉及一种云系统内部虚拟机的异常检测方法。
背景技术
越来越多的公司和企业通过迁移其部分的信息技术基础设施到云服务提供商来降低他们的成本，例如含有分布式存储基础设施的数据中心和其他类型的云计算系统的广泛使用。云服务提供商使用Vmware、vSphere等商用虚拟化软件来建立各种不同类型的虚拟化基础设施，包括私有云和公有云系统，这些云系统的数据可能分布在数百个相互连接的计算机，存储设备和其他物理机器上。
在公有云系统或私有云系统中，企业是租用云服务提供商的计算资源和存储资源，即云租户。当企业把他们的数据存放到云系统时，他们的数据己处于潜在的安全威胁之中。例如，云服务提供商在云系统的某一台物理主机中为不同用户建有多个虚拟机，这多个虚拟机只要有一个虚拟机异常(有病毒或被外部攻击)，处于云系统中同一台物理主机的其他虚拟机数据就存在安全威胁；
异常虚拟机的存在对于与之共享同一物理主机的其他虚拟机的正常运行构成很大的威胁，它将阻碍云系统为正常虚拟机提供服务。检测异常虚拟机的存在对云安全提出了挑战，目前针对云系统内部虚拟机的异常检测和处理方法比较少，并且现有的防御技术也没有考虑到云系统内部虚拟机活动的动态变化情况，因此存在一定的局限性。
保证云系统对正常虚拟机的可用性具有十分重要意义，它包括两方面的内容：一是正常情况下为虚拟机的正常使用提供合理的资源分配服务；二是 异常情况下云系统内部其他正常虚拟机的可用性，即通过检测方法检测出异常虚拟机并消除异常，保证其他正常虚拟机的正常使用。
发明的公开
本发明提供的一种云系统内部虚拟机的异常检测方法，保障了云系统内部异常虚拟机判断的实时性和可靠性，减少了异常检测对整个云系统性能的影响，确保云系统正常虚拟机对用户的可用性。
为了达到上述目的，本发明的技术方案是提供一种云系统内部虚拟机的异常检测方法：
通过虚拟机状态属性信息搜索模块搜集云系统内部的各虚拟机的状态属性信息，实时传给隐半马尔可夫模型HsMM在线检测模块进行检测；
所述隐半马尔可夫模型HsMM在线检测模块检测出行为异常的虚拟机，并将行为异常的虚拟机的状态属性信息传给虚拟机异常检测和处理系统；
所述虚拟机异常检测和处理系统对行为异常的虚拟机进行检测，对异常程度没有达到所设异常指标的行为异常虚拟机，消除异常并向对应的云租户发出警告提示；对异常程度达到所设异常指标的行为异常虚拟机，则向对应的云租户发出报警并关闭该虚拟机。
所述云系统内部虚拟机的异常检测方法，包含以下过程：
步骤1、虚拟机状态属性信息搜集模块搜集云系统内部各虚拟机在正常状态下的状态属性值项；所述正常状态是指虚拟机内部没有病毒和外部没有各种攻击的状态；
步骤2、将虚拟机在正常状态下的状态属性值项作为观测序列，训练隐半马尔可夫模型HsMM并设计隐半马尔可夫模型HsMM在线检测算法；
步骤3、虚拟机状态属性信息搜集模块按事先设置的时间间隔，搜集各虚拟机在线工作时的状态信息，并实时传给隐半马尔可夫模型HsMM在线检测模块；
步骤4、隐半马尔可夫模型HsMM在线检测模块基于步骤2中得到的相应算法，在线检测各虚拟机的状态行为，计算其状态行为的或然概率和马氏距离，以此判断虚拟机的行为异常情况；
步骤5、将根据每个虚拟机在线行为计算得到的马氏距离与预设门限值Q比对，判断虚拟机在线行为的马氏距离是否大于预设门限值Q：
若是，则转到步骤6；若否，则转到步骤3；
步骤6、启动云系统异常检测和处理系统，对检测结果大于预设门限值Q的虚拟机进行异常检测；
步骤7、判断步骤6中异常检测的虚拟机的异常指标是否大于异常检测和处理的最大门限值Emax：
若异常指标大于等于Emax，则转步骤8；
若异常指标小于Emax，则异常检测和处理系统消除异常并向云租户发警告提示后转步骤3；
步骤8、异常检测和处理系统向异常率大于等于Emax的虚拟机的云租户报警并关闭该虚拟机。
本发明具有以下优点及效果：
1、保障了云系统内异常虚拟机判断的实时性。通过轻量级的虚拟机状态属性信息搜集模块实时搜集并传递虚拟机的状态信息，轻量级的隐半马尔可夫模型HsMM在线检测算法能快速检测状态行为异常的虚拟机。其检测速度远远快于常用的虚拟机异常检测软件。因为传统的异常检测方法是定期或发现有虚拟机异常后再检测。
2、提高了云系统内部异常虚拟机判断的准确性。通过轻量级的隐半马尔可夫模型HsMM在线检测算法检测出行为异常的虚拟机后，再启动由多种异构检测引擎组成的异常检测和处理系统对行为异常的虚拟机进行检测，双重异常检测大大提高了云系统内部异常虚拟机判断的准确性。
3、充分保证了正常虚拟机的可用性。一方面使用轻量级的虚拟机状态 属性信息搜集模块和轻量级的隐半马尔可夫模型HsMM在线检测算法，对正常虚拟机的工作没有影响；另一方面使用异常检测和处理系统只对行为异常的虚拟机进行检测，不会占用正常虚拟机的资源和时间，所以本发明充分保证了正常虚拟机的可用性。
附图的简要说明
为了更完全地理解本发明及其优点，现在结合附图并参照描述，其中：
图1是本发明的一个整体流程图。
图2是云系统结构图。
实现本发明的最佳方式
为了使本发明实现的技术手段、创作特征、达成目的与功效易于明白了解，下面结合图示与具体实施例，进一步阐述本发明提出的一种云系统内部虚拟机异常检测方法。
图1所示是本发明提供的一种云系统内部虚拟机的异常检测方法的流程图；图2所示是本发明中云系统的结构图。
本发明所述的云系统，包括应用服务器集群和各种类型的存储设备。应用服务器集群中安装有云操作系统、虚拟化软件、虚拟机状态属性信息搜集模块、隐半马尔可夫模型HsMM在线检测模块、异常检测和处理系统，以及为云租户建立的虚拟机等。
所述的云操作系统是以云计算、云存储技术作为支撑的操作系统，是云计算后台数据中心的整体管理运营系统，它是指构架于服务器、存储、网络等基础硬件资源和单机操作系统、中间件、数据库等基础软件之上的、管理海量的基础硬件、软件资源的云平台综合管理系统。
所述的虚拟化软件是可以让一部主体服务器建立与执行一至多个虚拟化环境，如VMware,Xen等虚拟化软件。
所述的虚拟机状态属性信息搜集模块是通过调用云系统中的虚拟机状态 属性信息搜集功能而实现的轻量级软件模块；所述的轻量级软件模块是指运行时占用较少系统资源并对整个系统影响很小的应用软件。
所述的隐半马尔可夫模型HsMM在线检测模块，其内部包含有能检测虚拟机在线行为异常的隐半马尔可夫模型HsMM在线检测算法。
所述的异常检测和处理系统，是由不同厂家生产的异构检测引擎组成，能高效检测行为异常虚拟机的内部病毒和外部攻击，包括开源的和非开源的检测引擎。
本发明中的虚拟机异常检测和处理过程，包括：
(1)虚拟机状态属性信息搜索模块搜集各虚拟机的状态属性信息，实时传给隐半马尔可夫模型HsMM在线检测模块；
(2)隐半马尔可夫模型HsMM在线检测模块基于相应的算法，快速检测出行为异常的虚拟机，并将相关信息传给虚拟机异常检测和处理系统；
(3)虚拟机异常检测和处理系统对行为异常的虚拟机进行检测，对异常较轻的虚拟机消除异常并向对应的云租户发警报信息，对严重异常的虚拟机发出报警并关闭。
从上面的处理过程可知，本发明采用双重异常检测大大提高了云系统内部异常虚拟机异常判断的准确性，同时充分保证了正常虚拟机的高可用性。
如图1所示，本发明所述云系统内部虚拟机的异常检测方法，包含以下步骤：
步骤1、虚拟机状态属性信息搜集模块搜集云系统内部各虚拟机在正常状态下的状态信息。
所述的正常状态是指虚拟机内部没有病毒和外部没有各种攻击的状态。
所述虚拟机的状态信息，是由能反映虚拟机工作正常或异常的属性值项组成，包括虚拟机的CPU利用率、GPU的利用率、I/O等待时间和内存利用率等，以及他们随时间的动态变化情况。
步骤2、用虚拟机在正常状态下的状态属性值项作为观测序列，训练隐 半马尔可夫模型HsMM并设计隐半马尔可夫模型HsMM在线检测算法(下文详述)。
步骤3、虚拟机状态属性信息搜集模块按事先设置的时间间隔，搜集云系统内部各虚拟机在线工作时的状态信息，并实时传给隐半马尔可夫模型HsMM在线检测模块。
步骤4、隐半马尔可夫模型HsMM在线检测模块基于步骤2中得到的相应算法，在线检测云系统各虚拟机的状态行为，计算其状态行为的或然概率和马氏距离，以此判断虚拟机的行为异常情况。
所述虚拟机的行为异常情况，是指虚拟机的CPU利用率、GPU的利用率、I/O等待时间以及内存利用率等性能指标变化出现异常，或出现虚拟机资源耗尽或性能逐步恶化的情况。
步骤5、将根据每个虚拟机在线行为计算的马氏距离与预设门限值Q比对，判断虚拟机在线行为的马氏距离是否大于预设门限值Q：若是，则转到步骤6；若否，则转到步骤3。所述的预设门限值Q为云系统虚拟机行为异常检测结果被接受的最低标准。
步骤6、启动云系统异常检测和处理系统，对检测结果大于门限Q的虚拟机进行异常检测。所述的异常检测是对行为异常的虚拟机进行全面的内部病毒检测和外部攻击检测。
步骤7、判断步骤6中得到的虚拟机异常指标是否大于异常检测和处理的最大门限值Emax：若大于等于Emax，则转步骤8；若小于Emax，则异常检测和处理系统消除异常并向云租户发警告提示后转步骤3。所述的异常检测和处理的最大门限值Emax是异常检测和处理系统能处理的最大异常指标。
步骤8、异常检测和处理系统向异常率大于等于Emax的虚拟机的云租户报警并关闭该虚拟机。
以下对训练隐半马尔可夫模型HsMM，得到隐半马尔可夫模型HsMM在线算法的过程进行说明。
所述的隐半马尔可夫模型用λ＝{S,π,A,B,P}来表示，其各个参数定义如下：
S为模型状态集合，S＝{s1,s2,…,sG}，其中sg(1≤g≤G)表示隐马尔可夫模型在第t个时刻可能所处的状态，G为模型状态(State)的总个数；
π为初始状态概率矩阵，π＝{πg}，πg＝Pr[q1＝sg]，1≤g≤G，其中∑gπg＝1。πg表示隐马尔可夫模型在第1个时刻处于状态sg的概率；
A为状态转移概率矩阵，A＝{agi}，agi＝Pr[qt+1＝si|qt＝sg]，1≤g,i≤G，其中∑iagi＝1，qt表示隐马尔可夫模型在第t个时刻所处的状态，agi表示隐马尔可夫模型在第t个时刻从状态sg跳转到状态si的概率；
B为观测值概率矩阵，B＝{bg(vk)}，bg(vk)＝Pr[Ot＝Vk|qt＝sg]，1≤k≤K，1≤g≤G，其中Ot表示隐马尔可夫模型在第t个时刻的观测量，bg(vk)表示当隐马尔可夫模型在第t个时刻处于状态sg下，观测量Ot＝Vk的概率，K为观测序列的最大长度；
P为状态持续概率矩阵，P＝{pg(d)；1≤d≤D,1≤g≤G}，pg(d)＝Pr[τt＝d|qt＝sg]表示模型在第t个时刻处于状态sg，且还将在状态sg下持续d个时刻的概率，其中D表示状态持续的最大时间。
其中，云系统内部的各虚拟机的属性值状态包括：
状态①：各虚拟机的CPU、GPU和内存的利用率低于30％，I/O等待时间较短；
状态②：各虚拟机的CPU、GPU和内存的利用率有一种或多种高于30％但低于50％，其余低于30％，I/O等待时间正常；
状态③：各虚拟机的CPU、GPU和内存的利用率有一种或多种高于50％但低于80％，其余低于50％，I/O等待时间正常；
状态④：各虚拟机的CPU、GPU和内存的利用率有一种高于80％但低于90％，其余低于80％，并且变化正常，I/O等待时间较长；
状态⑤：各虚拟机的CPU、GPU和内存的利用率有一种或多种高于80％但低于90％，其余低于80％，并且变化出现异常，I/O等待时间较长或超长；
状态⑥：各虚拟机的CPU、GPU和内存的利用率有一种或多种高于90％，其余低于80％，并且变化异常，I/O等待时间超长；
所述的状态①②③④属于正常状态，状态⑤⑥属于异常状态。
本发明中用虚拟机正常状态下的属性值状态观测序列来训练隐半马尔可夫模型HsMM的过程，包括：
S1、计算出每个虚拟机的状态信息观测序列O(h)(1≤h≤H)的前向变量
Figure PCTCN2016101891-appb-000001
它表示虚拟机在前t个观测量
Figure PCTCN2016101891-appb-000002
到达虚拟机状态属性信息搜集模块时，虚拟机在状态sg持续停留d个时刻的概率，其中1≤t≤T*，前向变量的定义式如(1)所示：
Figure PCTCN2016101891-appb-000003
其中，H为虚拟机状态观测值序列的总个数，T*为相应观测序列的长度。
S2、根据公式(2)计算出每个正常虚拟机的观测序列相对于HsMM的或然对数概率Ph，1≤h≤H；同理，可以根据公式(3)求出在线虚拟机的或然对数概率
Figure PCTCN2016101891-appb-000004
在此基础上可以根据公式(4)计算出所有正常虚拟机的观测序列相对于HsMM的或然对数概率PH，这里的PH即为正常虚拟机的或然对数概率值构成的初始或然对数概率分布，
Figure PCTCN2016101891-appb-000005
Figure PCTCN2016101891-appb-000006
Figure PCTCN2016101891-appb-000007
其中G为模型状态的总个数，D为状态持续的最大时间，H为虚拟机状态观测值序列的总个数，T*为相应序列的长度。
S3、根据式(5)、(6)所示，分别计算出正常虚拟机的初始或然对数概率分布PH的平均值μ和标准差σ：
Figure PCTCN2016101891-appb-000008
Figure PCTCN2016101891-appb-000009
S4、求出每个虚拟机观测序列O(h)(1≤h≤H)的后向变量
Figure PCTCN2016101891-appb-000010
它表示虚拟机的第t个观测量
Figure PCTCN2016101891-appb-000011
到达虚拟机状态属性信息搜集模块时，虚拟机在状态sg持续停留d个时刻的情况下，产生
Figure PCTCN2016101891-appb-000012
的概率，后向变量的定义式如式(7)所示：
Figure PCTCN2016101891-appb-000013
S5、由前向变量
Figure PCTCN2016101891-appb-000014
和后向变量
Figure PCTCN2016101891-appb-000015
我们可以得到状态跳转联合概率
Figure PCTCN2016101891-appb-000016
状态持续联合概率
Figure PCTCN2016101891-appb-000017
状态和观测值联合概率
Figure PCTCN2016101891-appb-000018
它们的定义式分别如式(8)、(9)、(10)所示：
Figure PCTCN2016101891-appb-000019
Figure PCTCN2016101891-appb-000020
Figure PCTCN2016101891-appb-000021
S6、训练HsMM的模型参数：在训练模型参数之前，要给HsMM的模型参数赋初值，由于模型参数A,P,B,π的初值对模型训练的影响比较小，所以可令agi＝1/(G‐1)，πg＝1/G，pg(d)＝1/D，bg(vk)＝1/(G‐1)，另外令状态自跳转概率agg＝0，然后根据下式(11)‐(14)进行模型参数的更新，在公式(15)中，当ot＝vk时，δ(ot-vk)＝1，否则δ(ot-vk)＝0，
Figure PCTCN2016101891-appb-000022
Figure PCTCN2016101891-appb-000023
Figure PCTCN2016101891-appb-000024
Figure PCTCN2016101891-appb-000025
S7、判断第S2项中求到的PH是否趋向于一个稳定的值，若是，则得到HsMM的模型参数集λ，模型训练结束；否则的话，重复以上第S1项到第S6项的过程。
根据模型训练得到了正常虚拟机的初始或然对数概率分布和在线虚拟机的或然对数概率计算公式，就可以用简化的马氏距离来衡量正常虚拟机的初始或然对数概率分布和云系统中在线虚拟机的或然对数概率之间的距离，简化的马氏距离如公式(15)所示：
Figure PCTCN2016101891-appb-000026
公式(15)中d的取值反映了云系统中虚拟机在线状态行为的异常程度，这里可以定义一个代表虚拟机正常行为的阈值Q，当d≤Q时，可以判断虚拟机的状态行为是正常的；当d>Q时，则可以判断该虚拟机的状态行为异常，即可能出现病毒或遭受外部攻击。所述的马氏距离是由印度数学家马哈拉诺比斯(P.C.Mahalanobis)提出的一种计算具有相关性两点之间距离的方法。
得到行为异常的虚拟机后，启动云系统中异常检测和处理系统进行异常检测，根据对异常虚拟机内部病毒检测结果和外部攻击检测结果进行评估，得出该虚拟机行为异常的严重程度指数Ei，Ei＝p1+p2；p1是反映异常虚拟机内部病毒可处理的指标，如果异常虚拟机内部病毒可处理，p1取值0，如果异常虚拟机内部病毒不可处理p1取值1；p2是反映异常虚拟机外部攻击可处理的指标，如果异常虚拟机外部攻击可处理，p2取值0，如果异常虚拟机外部攻击不可处理p2取值1。
若虚拟机行为异常的严重程度指数Ei<Emax＝1，则消除异常后给异常虚拟机的云租户发警告通知；虚拟机行为异常的严重程度指数Ei≥Emax＝1则会给异常虚拟机的云租户发报警并关闭该虚拟机。
下面用一个实例来阐述本发明所述的方法。
假设云系统设置有100台虚拟机，其中1台由于病毒引起虚拟机工作状态异常，另两台由于外部攻击引起虚拟机工作状态异常。轻量级的虚拟机状态属性信息搜索模块搜集各虚拟机的状态信息并传给隐半马尔可夫模型HsMM在线检测模块后，后者基于隐半马尔可夫模型HsMM在线检测算法很快检测出这3台虚拟机的状态行为异常，并启动云系统中异常检测和处理系统对3台虚拟机进行异常检测和处理。
轻量级的虚拟机状态属性信息搜索模块和隐半马尔可夫模型HsMM在线检测算法工作对所有虚拟机几乎没有影响，在这里我们忽略其影响。我们还假设由病毒和外攻击引起的虚拟机异常是可以消除而不需要关闭虚拟机，并且每台虚拟机异常消除需要时间为10分钟，则3台虚拟机消除异常需要：3×10＝30分钟。
传统的虚拟机异常消除方法是启动云系统中的异常检测和处理系统对所有虚拟机进行检测，则消除3台虚拟机异常需要的总时间为：100×10＝1000分钟。
另外，本发明只需要对3台异常虚拟机进行检测和消除异常，而传统的方法需要对所有的虚拟机进行检测，包括实际正常和实际异常的虚拟机进行异常检测。异常检测和处理需要占用虚拟机的资源和工作时间。
从上面实例看，本发明与传统方法相比，可充分保证云系统虚拟机异常检测和处理的实时性和准确性，并保证正常虚拟机的可用性。
尽管本发明的内容已经通过上述优选实施例作了详细介绍，但应当认识到上述的描述不应被认为是对本发明的限制。在本领域技术人员阅读了上述内容后，对于本发明的多种修改和替代都将是显而易见的。因此，本发明的保护范围应由所附的权利要求来限定。

Claims (7)

  1. 一种云系统内部虚拟机的异常检测方法，其特征在于，
    通过虚拟机状态属性信息搜索模块搜集云系统内部的各虚拟机的状态属性信息，实时传给隐半马尔可夫模型HsMM在线检测模块进行检测；
    所述隐半马尔可夫模型HsMM在线检测模块检测出行为异常的虚拟机，并将行为异常的虚拟机的状态属性信息传给虚拟机异常检测和处理系统；
    所述虚拟机异常检测和处理系统对行为异常的虚拟机进行检测，对异常程度没有达到所设异常指标的行为异常虚拟机，消除异常并向对应的云租户发出警告提示；对异常程度达到所设异常指标的行为异常虚拟机，则向对应的云租户发出报警并关闭该虚拟机。
  2. 如权利要求1所述云系统内部虚拟机的异常检测方法，其特征在于，包含以下过程：
    步骤1、虚拟机状态属性信息搜集模块搜集云系统内部各虚拟机在正常状态下的状态属性值项；所述正常状态是指虚拟机内部没有病毒和外部没有各种攻击的状态；
    步骤2、将虚拟机在正常状态下的状态属性值项作为观测序列，训练隐半马尔可夫模型HsMM并设计隐半马尔可夫模型HsMM在线检测算法；
    步骤3、虚拟机状态属性信息搜集模块按事先设置的时间间隔，搜集各虚拟机在线工作时的状态信息，并实时传给隐半马尔可夫模型HsMM在线检测模块；
    步骤4、隐半马尔可夫模型HsMM在线检测模块基于步骤2中得到的相应算法，在线检测各虚拟机的状态行为，计算其状态行为的或然概率和马氏距离，以此判断虚拟机的行为异常情况；
    步骤5、将根据每个虚拟机在线行为计算得到的马氏距离与预设门限值 Q比对，判断虚拟机在线行为的马氏距离是否大于预设门限值Q：
    若是，则转到步骤6；若否，则转到步骤3；
    步骤6、启动云系统异常检测和处理系统，对检测结果大于预设门限值Q的虚拟机进行异常检测；
    步骤7、判断步骤6中异常检测的虚拟机的异常指标是否大于异常检测和处理的最大门限值Emax：
    若异常指标大于等于Emax，则转步骤8；
    若异常指标小于Emax，则异常检测和处理系统消除异常并向云租户发警告提示后转步骤3；
    步骤8、异常检测和处理系统向异常率大于等于Emax的虚拟机的云租户报警并关闭该虚拟机。
  3. 如权利要求2所述云系统内部虚拟机的异常检测方法，其特征在于，
    所述虚拟机的状态属性值项，包括虚拟机的CPU利用率、GPU的利用率、I/O等待时间和内存利用率，以及他们随时间的动态变化情况；
    虚拟机处在第一状态到第四状态中任意一种状态时属于正常状态，处在第五状态或第六状态时属于异常状态，其中：
    第一状态时，虚拟机的CPU、GPU和内存的利用率低于30％，具有比正常I/O等待时间短的第一I/O等待时间；
    第二状态时，虚拟机的CPU、GPU和内存的利用率有一种或多种高于30％但低于50％，其余低于30％，并具有处于正常I/O等待时间的第二I/O等待时间；
    第三状态时，虚拟机的CPU、GPU和内存的利用率有一种或多种高于50％但低于80％，其余低于50％，并具有第二I/O等待时间；
    第四状态时，虚拟机的CPU、GPU和内存的利用率有一种高于80％但低于90％，其余低于80％，且变化正常，并具有比正常I/O等待时间长的第三 I/O等待时间；
    第五状态时，虚拟机的CPU、GPU和内存的利用率有一种或多种高于80％但低于90％，其余低于80％，且变化异常，并具有第三I/O等待时间或具有比第三I/O等待时间长的第四I/O等待时间；
    第六状态时，虚拟机的CPU、GPU和内存的利用率有一种或多种高于90％，其余低于80％，且变化异常，并具有第四I/O等待时间。
  4. 如权利要求2或3所述云系统内部虚拟机的异常检测方法，其特征在于，
    所述隐半马尔可夫模型用λ＝{S,π,A,B,P}来表示，其各个参数定义如下：
    S为模型状态集合，S＝{s1,s2,…,sG}，其中sg(1≤g≤G)表示隐马尔可夫模型在第t个时刻可能所处的状态，G为模型状态的总个数；
    π为初始状态概率矩阵，π＝{πg}，πg＝Pr[q1＝sg]，1≤g≤G，其中∑gπg＝1；πg表示隐马尔可夫模型在第1个时刻处于状态sg的概率；
    A为状态转移概率矩阵，A＝{agi}，agi＝Pr[qt+1＝si|qt＝sg]，1≤g，i≤G，其中∑iagi＝1，qt表示隐马尔可夫模型在第t个时刻所处的状态，agi表示隐马尔可夫模型在第t个时刻从状态sg跳转到状态si的概率；
    B为观测值概率矩阵，B＝{bg(vk)}，bg(vk)＝Pr[Ot＝Vk|qt＝sg]，1≤k≤K，1≤g≤G，其中Ot表示隐马尔可夫模型在第t个时刻的观测量，bg(vk)表示当隐马尔可夫模型在第t个时刻处于状态sg下，观测量Ot＝Vk的概率，K为观测序列的最大长度；
    P为状态持续概率矩阵，P＝{pg(d)；1≤d≤D，1≤g≤G}，pg(d)＝Pr[τt＝d|qt＝sg]表示模型在第t个时刻处于状态sg，且还将在状态sg下持续d个时刻的概率，其中D表示状态持续的最大时间。
  5. 如权利要求4所述云系统内部虚拟机的异常检测方法，其特征在于，
    通过虚拟机在正常状态下的状态属性值项的观测序列，来训练隐半马尔可夫模型HsMM的过程，包括：
    S1、计算出每个虚拟机的状态信息观测序列O(h)(1≤h≤H)的前向变量
    Figure PCTCN2016101891-appb-100001
    它表示虚拟机在前t个观测量
    Figure PCTCN2016101891-appb-100002
    到达虚拟机状态属性信息搜集模块时，虚拟机在状态sg持续停留d个时刻的概率，其中1≤t≤T*，前向变量的定义式为：
    Figure PCTCN2016101891-appb-100003
    其中，H为虚拟机状态观测值序列的总个数，T*为相应观测序列的长度；
    S2、分别计算出每个正常虚拟机的观测序列相对于HsMM的或然对数概率Ph，1≤h≤H，在线虚拟机的或然对数概率
    Figure PCTCN2016101891-appb-100004
    以及所有正常虚拟机的观测序列相对于HsMM的或然对数概率PH，这里的PH即为正常虚拟机的或然对数概率值构成的初始或然对数概率分布：
    Figure PCTCN2016101891-appb-100005
    Figure PCTCN2016101891-appb-100006
    Figure PCTCN2016101891-appb-100007
    S3、分别计算出正常虚拟机的初始或然对数概率分布PH的平均值μ和标准差σ：
    Figure PCTCN2016101891-appb-100008
    Figure PCTCN2016101891-appb-100009
    S4、求出每个虚拟机观测序列O(h)(1≤h≤H)的后向变量
    Figure PCTCN2016101891-appb-100010
    它表示虚拟机的第t个观测量
    Figure PCTCN2016101891-appb-100011
    到达虚拟机状态属性信息搜集模块时，虚拟机在状态sg持续停留d个时刻的情况下，产生
    Figure PCTCN2016101891-appb-100012
    的概率，后向变量的定义式：
    Figure PCTCN2016101891-appb-100013
    S5、根据前向变量
    Figure PCTCN2016101891-appb-100014
    和后向变量
    Figure PCTCN2016101891-appb-100015
    分别求出状态跳转联合概率
    Figure PCTCN2016101891-appb-100016
    状态持续联合概率
    Figure PCTCN2016101891-appb-100017
    状态和观测值联合概率
    Figure PCTCN2016101891-appb-100018
    Figure PCTCN2016101891-appb-100019
    Figure PCTCN2016101891-appb-100020
    Figure PCTCN2016101891-appb-100021
    S6、给HsMM的模型参数赋初值，令agi＝1/(G‐1)，πg＝1/G，pg(d)＝1/D，bg(vk)＝1/(G‐1)，另外令状态自跳转概率agg＝0，然后根据下式进行模型参数的更新；当ot＝vk时，δ(ot-vk)＝1，否则δ(ot-vk)＝0；
    Figure PCTCN2016101891-appb-100022
    Figure PCTCN2016101891-appb-100023
    Figure PCTCN2016101891-appb-100024
    Figure PCTCN2016101891-appb-100025
    S7、判断第S2项中求到的PH是否趋向于一个稳定的值：
    若是，则得到HsMM的模型参数集λ，模型训练结束；若否，重复以上第S1项到第S6项的过程。
  6. 如权利要求5所述云系统内部虚拟机的异常检测方法，其特征在于，
    通过简化的马氏距离来衡量正常虚拟机的初始或然对数概率分布和云系统中在线虚拟机的或然对数概率之间的距离，简化的马氏距离为：
    Figure PCTCN2016101891-appb-100026
    其中，d的取值反映了云系统中虚拟机在线状态行为的异常程度；定义代表虚拟机正常行为的预设门限值Q，当d≤Q时，判断虚拟机的状态行为是正常的；当d>Q时，判断该虚拟机的状态行为异常，即可能出现病毒或遭受外部攻击。
  7. 如权利要求6所述云系统内部虚拟机的异常检测方法，其特征在于，
    检测出行为异常的虚拟机后，启动云系统中异常检测和处理系统进行异常检测，根据对行为异常的虚拟机内部病毒检测结果和外部攻击检测结果进行评估，得出该虚拟机行为异常的严重程度指数Ei，Ei＝p1+p2；
    其中，p1是反映异常虚拟机内部病毒可处理的指标，如果异常虚拟机内部病毒可处理，p1取值0；如果异常虚拟机内部病毒不可处理p1取值1；
    p2是反映异常虚拟机外部攻击可处理的指标，如果异常虚拟机外部攻击可处理，p2取值0；如果异常虚拟机外部攻击不可处理，p2取值1；
    若虚拟机行为异常的严重程度指数Ei<Emax＝1，则消除异常后给行为异常的虚拟机的云租户发警告提示；若虚拟机行为异常的严重程度指数Ei≥Emax＝1，则给行为异常的虚拟机的云租户发报警并关闭该虚拟机。
PCT/CN2016/101891 2016-01-07 2016-10-12 一种云系统内部虚拟机的异常检测方法 WO2017118133A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US15/736,227 US10616268B2 (en) 2016-01-07 2016-10-12 Anomaly detection method for the virtual machines in a cloud system

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610008093.1A CN105511944B (zh) 2016-01-07 2016-01-07 一种云系统内部虚拟机的异常检测方法
CN201610008093.1 2016-01-07

Publications (1)

Publication Number Publication Date
WO2017118133A1 true WO2017118133A1 (zh) 2017-07-13

Family

ID=55719950

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/101891 WO2017118133A1 (zh) 2016-01-07 2016-10-12 一种云系统内部虚拟机的异常检测方法

Country Status (3)

Country Link
US (1) US10616268B2 (zh)
CN (1) CN105511944B (zh)
WO (1) WO2017118133A1 (zh)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108900556A (zh) * 2018-08-24 2018-11-27 海南大学 基于HMM和混沌模型的DDoS攻击检测方法
CN110598802A (zh) * 2019-09-26 2019-12-20 腾讯科技（深圳）有限公司 一种内存检测模型训练的方法、内存检测的方法及装置
US10684909B1 (en) 2018-08-21 2020-06-16 United States Of America As Represented By Secretary Of The Navy Anomaly detection for preserving the availability of virtualized cloud services
CN112381110A (zh) * 2020-10-10 2021-02-19 神华北电胜利能源有限公司 基于模糊规则的煤矿生产多维度数据异常概率预测方法
CN115225536A (zh) * 2022-06-17 2022-10-21 上海仪电（集团）有限公司中央研究院 一种基于无监督学习的虚拟机异常检测方法及系统

Families Citing this family (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105511944B (zh) 2016-01-07 2018-09-28 上海海事大学 一种云系统内部虚拟机的异常检测方法
CN106775929B (zh) * 2016-11-25 2019-11-26 中国科学院信息工程研究所 一种虚拟化平台安全监控方法及系统
CN108228308B (zh) * 2016-12-21 2021-07-06 中国电信股份有限公司 虚拟机的监控方法以及装置
US10728261B2 (en) * 2017-03-02 2020-07-28 ResponSight Pty Ltd System and method for cyber security threat detection
CN108255536A (zh) * 2017-04-27 2018-07-06 平安科技（深圳）有限公司 基于功能开关的应用程序控制方法和服务器
EP3407273A1 (de) * 2017-05-22 2018-11-28 Siemens Aktiengesellschaft Verfahren und anordnung zur ermittlung eines anomalen zustands eines systems
CN107844406A (zh) * 2017-10-25 2018-03-27 千寻位置网络有限公司 分布式系统的异常检测方法及系统、服务终端、存储器
CN108376103A (zh) * 2018-02-08 2018-08-07 厦门集微科技有限公司 一种云平台的资源平衡控制方法及服务器
CN108491251A (zh) * 2018-04-15 2018-09-04 肖恒念 一种云服务器虚拟机管理方法和云服务器
US11366680B2 (en) * 2018-07-19 2022-06-21 Twistlock, Ltd. Cloud native virtual machine runtime protection
CN110969222A (zh) * 2018-09-29 2020-04-07 西门子股份公司 信息提供方法和系统
US11126493B2 (en) * 2018-11-25 2021-09-21 Aloke Guha Methods and systems for autonomous cloud application operations
CN110097037B (zh) * 2019-05-22 2021-10-01 天津联图科技有限公司 智能监测方法、装置、存储介质及电子设备
CN110187990B (zh) * 2019-05-31 2021-11-16 东北大学 一种基于模式转移的虚拟机混合备用动态可靠性评估方法
CN110233941B (zh) * 2019-06-28 2021-03-30 厦门大学嘉庚学院 一种基于马尔科夫模型的热线话务信道预测方法及系统
CN110825579B (zh) * 2019-09-18 2022-03-08 平安科技（深圳）有限公司 服务器性能监控方法、装置、计算机设备及存储介质
CN112988527A (zh) * 2019-12-13 2021-06-18 中国电信股份有限公司 Gpu管理平台异常检测方法、装置以及存储介质
CN111131304B (zh) * 2019-12-31 2022-01-11 嘉兴学院 面向云平台大规模虚拟机细粒度异常行为检测方法和系统
CN111431895B (zh) * 2020-03-20 2022-04-22 宁波和利时信息安全研究院有限公司 系统异常处理方法、装置及系统
JP2022162903A (ja) * 2021-04-13 2022-10-25 パナソニックＩｐマネジメント株式会社 生産設備監視システムおよび生産設備監視方法
CN113342534B (zh) * 2021-06-29 2024-01-02 天翼云科技有限公司 图形处理资源调配方法、装置、设备及存储介质
US20230145484A1 (en) * 2021-11-09 2023-05-11 Sap Se Software defined anomaly detection for workloads
CN116016298B (zh) * 2023-01-04 2024-04-09 重庆邮电大学 一种基于隐半马尔可夫模型的5g通信协议异常检测方法

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6868325B2 (en) * 2003-03-07 2005-03-15 Honeywell International Inc. Transient fault detection system and method using Hidden Markov Models
CN103428026A (zh) * 2012-05-14 2013-12-04 国际商业机器公司 用于共享动态云中的问题确定和诊断的方法和系统
CN103442076A (zh) * 2013-09-04 2013-12-11 上海海事大学 一种云存储系统的可用性保障方法
CN104461821A (zh) * 2014-11-03 2015-03-25 浪潮(北京)电子信息产业有限公司 一种虚拟机监控预警方法及系统
CN105045648A (zh) * 2015-05-08 2015-11-11 北京航空航天大学 IaaS云环境下物理主机资源状态预测方法
CN105511944A (zh) * 2016-01-07 2016-04-20 上海海事大学 一种云系统内部虚拟机的异常检测方法
CN105791286A (zh) * 2016-03-01 2016-07-20 上海海事大学 云端虚拟环境的异常检测和处理方法

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7499842B2 (en) * 2005-11-18 2009-03-03 Caterpillar Inc. Process model based virtual sensor and method
TWI349867B (en) * 2008-05-20 2011-10-01 Univ Nat Cheng Kung Server and system and method for automatic virtual metrology
US9047441B2 (en) * 2011-05-24 2015-06-02 Palo Alto Networks, Inc. Malware analysis system
US9417901B2 (en) * 2013-01-11 2016-08-16 Electronics And Telecommunications Research Institute Switch and method for guaranteeing quality of service of multi-tenant cloud service and system having the same switch
CN105446792B (zh) * 2014-08-27 2019-09-24 联想(北京)有限公司 一种虚拟机的部署方法、部署装置和管理节点

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6868325B2 (en) * 2003-03-07 2005-03-15 Honeywell International Inc. Transient fault detection system and method using Hidden Markov Models
CN103428026A (zh) * 2012-05-14 2013-12-04 国际商业机器公司 用于共享动态云中的问题确定和诊断的方法和系统
CN103442076A (zh) * 2013-09-04 2013-12-11 上海海事大学 一种云存储系统的可用性保障方法
CN104461821A (zh) * 2014-11-03 2015-03-25 浪潮(北京)电子信息产业有限公司 一种虚拟机监控预警方法及系统
CN105045648A (zh) * 2015-05-08 2015-11-11 北京航空航天大学 IaaS云环境下物理主机资源状态预测方法
CN105511944A (zh) * 2016-01-07 2016-04-20 上海海事大学 一种云系统内部虚拟机的异常检测方法
CN105791286A (zh) * 2016-03-01 2016-07-20 上海海事大学 云端虚拟环境的异常检测和处理方法

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10684909B1 (en) 2018-08-21 2020-06-16 United States Of America As Represented By Secretary Of The Navy Anomaly detection for preserving the availability of virtualized cloud services
CN108900556A (zh) * 2018-08-24 2018-11-27 海南大学 基于HMM和混沌模型的DDoS攻击检测方法
CN108900556B (zh) * 2018-08-24 2021-02-02 海南大学 基于HMM和混沌模型的DDoS攻击检测方法
CN110598802A (zh) * 2019-09-26 2019-12-20 腾讯科技（深圳）有限公司 一种内存检测模型训练的方法、内存检测的方法及装置
CN111078479A (zh) * 2019-09-26 2020-04-28 腾讯科技（深圳）有限公司 一种内存检测模型训练的方法、内存检测的方法及装置
CN112381110A (zh) * 2020-10-10 2021-02-19 神华北电胜利能源有限公司 基于模糊规则的煤矿生产多维度数据异常概率预测方法
CN115225536A (zh) * 2022-06-17 2022-10-21 上海仪电（集团）有限公司中央研究院 一种基于无监督学习的虚拟机异常检测方法及系统
CN115225536B (zh) * 2022-06-17 2024-02-27 上海仪电（集团）有限公司中央研究院 一种基于无监督学习的虚拟机异常检测方法及系统

Also Published As

Publication number Publication date
US20180309770A1 (en) 2018-10-25
CN105511944B (zh) 2018-09-28
CN105511944A (zh) 2016-04-20
US10616268B2 (en) 2020-04-07

Similar Documents

Publication Publication Date Title
WO2017118133A1 (zh) 一种云系统内部虚拟机的异常检测方法
US11171970B2 (en) System and method for reducing false positive security events
US11818014B2 (en) Multi-baseline unsupervised security-incident and network behavioral anomaly detection in cloud-based compute environments
US7685251B2 (en) Method and apparatus for management of virtualized process collections
WO2018159337A1 (ja) プロファイル生成装置、攻撃検知装置、プロファイル生成方法、および、プロファイル生成プログラム
US8516499B2 (en) Assistance in performing action responsive to detected event
Wang et al. Containerguard: A real-time attack detection system in container-based big data platform
WO2013055311A1 (en) Methods and systems for identifying action for responding to anomaly in cloud computing system
CN116049146B (zh) 一种数据库故障处理方法、装置、设备及存储介质
EP3206367A1 (en) Techniques for detecting attacks in a publish-subscribe network
US9280386B1 (en) Identifying task instance outliers based on metric data in a large scale parallel processing system
CN109165296A (zh) 工业物联网资源知识图谱构建方法、可读存储介质和终端
US9929921B2 (en) Techniques for workload toxic mapping
US9560027B1 (en) User authentication
Yan et al. Padm: Page rank-based anomaly detection method of log sequences by graph computing
Chang et al. Implementation of ransomware prediction system based on weighted-KNN and real-time isolation architecture on SDN Networks
CN109766243B (zh) 一种基于幂函数的多核主机性能监控方法
KR102348357B1 (ko) 동적인 분석 플랜을 이용하는 edr 장치 및 방법
Wei-wu et al. An efficient parallel anomaly detection algorithm based on hierarchical clustering
Sugaya et al. A lightweight anomaly detection system for information appliances
US20240143746A1 (en) Context aware behavioral anomaly detection in computing systems
JP7176630B2 (ja) 検知装置、検知方法および検知プログラム
CN107040554B (zh) 一种防御cc攻击的方法
CN116112339B (zh) 一种根因告警的定位方法、装置、设备及介质
Zheng et al. EHMM-CT: an online method for failure prediction in cloud computing systems

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16883262

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 15736227

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16883262

Country of ref document: EP

Kind code of ref document: A1