WO2016045489A1 - 云环境下的虚拟机负载评估系统、方法以及服务节点 - Google Patents

云环境下的虚拟机负载评估系统、方法以及服务节点 Download PDF

Info

Publication number
WO2016045489A1
WO2016045489A1 PCT/CN2015/088868 CN2015088868W WO2016045489A1 WO 2016045489 A1 WO2016045489 A1 WO 2016045489A1 CN 2015088868 W CN2015088868 W CN 2015088868W WO 2016045489 A1 WO2016045489 A1 WO 2016045489A1
Authority
WO
WIPO (PCT)
Prior art keywords
virtual machines
virtual machine
kmeans algorithm
center
virtual
Prior art date
Application number
PCT/CN2015/088868
Other languages
English (en)
French (fr)
Inventor
柴洪峰
吴承荣
华锦芝
王振方
杨阳
张骏
Original Assignee
中国银联股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中国银联股份有限公司 filed Critical 中国银联股份有限公司
Priority to US15/512,191 priority Critical patent/US10528378B2/en
Priority to EP15845103.9A priority patent/EP3200076A4/en
Publication of WO2016045489A1 publication Critical patent/WO2016045489A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
    • G06F11/3006—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G06F11/3409—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment for performance assessment
    • G06F11/3433—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment for performance assessment for load management
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061—Partitioning or combining of resources
    • G06F9/5077—Logical partitioning of resources; Management or configuration of virtualized resources
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
    • G06F11/301—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system is a virtual computing platform, e.g. logically partitioned systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G06F2009/4557—Distribution of virtual machine instances; Migration and load balancing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2201/00—Indexing scheme relating to error detection, to error correction, and to monitoring
    • G06F2201/815—Virtual
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061—Partitioning or combining of resources
    • G06F9/5072—Grid computing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5083—Techniques for rebalancing the load in a distributed system

Definitions

  • the present invention relates to cloud monitoring technologies and, more particularly, to load evaluation of virtual machines in a cloud environment.
  • Virtualization technology in cloud computing provides a new idea for traditional problem solutions in some aspects, but virtualization technology as an emerging technology, its work in virtual machine resource performance management and detection. There are still many shortcomings. In a 11-year paper "Toward an architecture for monitoring private clouds,” the author also pointed out the importance of cloud monitoring. Although there are many solutions for cloud management deployment, cloud monitoring technology has not kept pace.
  • the performance characteristics of a system are determined by its own characteristics and the load characteristics of the application services running on it, which quantitatively describes a basic system feature. How to judge the load characteristics of the application service to understand the environment in which the entire system is located is of great significance.
  • the present application provides a virtual machine load evaluation system in a cloud environment, including: a monitoring data collection module configured to be from a cloud data center or Collecting monitoring data at a plurality of virtual machines; and monitoring a data processing module coupled to the monitoring data collection module and configured to: utilize a first KMeans algorithm for each of the one or more virtual machines
  • the monitoring data is aggregated, each group has a center point coordinate, and the performance characterization of each virtual machine in the one or more virtual machines is calculated according to the grouping and the center point coordinates of each group, that is, the first feature a value is obtained by using the second KMeans algorithm to aggregate the one or more virtual machines based on the first feature value of each virtual machine to determine each of the one or more virtual machines a feature type and a performance load value; wherein a type parameter K in the first KMeans algorithm is greater than a type parameter in the second KMeans algorithm, and an initial center of the first KMeans algorithm is selected such that an initial class is between The distance as
  • the monitoring data is associated with a central processing unit, memory, hard disk input or output, and network input or output of one or more virtual machines.
  • the feature types of the virtual machine include CPU-intensive, Network-intensive, and Memory-intensive.
  • the type parameter K in the first KMeans algorithm is equal to 4, and the type parameter in the second KMeans algorithm is equal to 3.
  • the initial center of the first KMeans algorithm is selected as P1 (100, 0, 0), P2 (0, 100, 0), P3 (0, 0, 100), and P4 (0, 0, 0).
  • the proportion of samples in each group is A%, B%, C%, D%, and the coordinates of the center point of each group are (X1, Y1, Z1), (X2) , Y2, Z2), (X3, Y3, Z3) and (X4, Y4, Z4), then the first characteristic value is represented by the following equation:
  • the monitoring data processing module is further configured to redistribute the virtual machines according to performance load values in the one or more virtual machines to achieve efficient utilization of resources.
  • a method for load assessment of a virtual machine in a cloud environment including: using a first KMeans algorithm for each of one or more virtual machines in a cloud data center
  • the monitoring data obtained from the one or more virtual machines is aggregated, each group having a center point coordinate, and each of the one or more virtual machines is calculated according to the grouping and the center point coordinates of each group Performance characterization of the virtual machines, ie, the first eigenvalues; based on the first eigenvalues of each virtual machine, the second KMeans algorithm is used to aggregate the one or more virtual machines to determine the one Or a feature type of each virtual machine in the plurality of virtual machines and a performance load value; wherein a type parameter K in the first KMeans algorithm is greater than a type parameter in the second KMeans algorithm, and the first KMeans algorithm
  • the initial center is selected such that the initial categories are kept as far apart as possible, and the initial center of the second Kmeans algorithm is selected as the initial of
  • the monitoring data is associated with a central processing unit of one or more virtual machines, a memory, a hard disk input or output, and a network input or output.
  • the feature types of the virtual machine include CPU-intensive, Network-intensive, and Memory-intensive.
  • the type parameter K in the first KMeans algorithm is equal to 4, and the type parameter in the second KMeans algorithm is equal to 3.
  • the initial center of the first KMeans algorithm is selected as P1 (100, 0, 0), P2 (0, 100, 0), P3 (0, 0, 100), and P4 (0, 0, 0).
  • the proportion of the samples in each group is A%, B%, C%, D%, and the coordinates of the center points of each group are (X1, Y1, Z1), (X2, Y2, Z2), respectively. , (X3, Y3, Z3) and (X4, Y4, Z4), then the first characteristic value is represented by the following equation:
  • the above method may further include redistributing the virtual machines according to performance load values in the one or more virtual machines to achieve efficient utilization of resources.
  • a service node including the virtual machine load evaluation system described above is provided.
  • This application adopts KMeans clustering algorithm to realize the evaluation and monitoring of the virtual machine load macroscopic situation, and overcomes the accuracy of the traditional KMeans algorithm.
  • the accuracy of the clustering results is affected by many factors (including initial center selection, noise, isolated points). Defects such as handling or not).
  • FIG. 1 is a schematic diagram of a cloud platform application environment including a virtual machine load evaluation system, in accordance with an embodiment of the present application;
  • FIG. 2 is a schematic structural diagram of a virtual machine load evaluation system according to an embodiment of the present application.
  • FIG. 1 is a schematic diagram of a cloud platform application environment including a virtual machine load evaluation system, in accordance with an embodiment of the present application.
  • a user deploys related application services on one or more virtual machine instances of a cloud data center 140, such as Openstack.
  • the data related to the application service is the object to be collected by the virtual machine load evaluation system 130 in the service node 120, including data sources such as CPU, Memory, disk I/O, and network IO, and then the collected data is collected by the virtual machine load evaluation system 130.
  • the analysis is processed and the analysis results are returned to the front end display module 110.
  • the monitoring data collection module 210 in the virtual machine load evaluation system 130 is responsible for collecting data from the cloud data center 140 and providing the collected data to the monitoring data processing module 220.
  • the monitoring data processing module processes and analyzes the data according to the scheme of virtual machine performance data processing described below, and returns the analysis result to the front end display module 110.
  • the virtual machine performance data processing is mainly divided into two parts.
  • the first part is to use the improved KMeans algorithm in the cloud environment to calculate the eigenvalue of the virtual machine, and obtain the performance characterization (characteristic value) of the virtual machine through the performance load evaluation algorithm.
  • the second part is to calculate the characteristic performance of all virtual machines. It is mainly divided into three categories: CPU, Network and Memory. Based on this, the virtual machine performance load value is obtained, and the application type is evaluated according to the clustering result.
  • the type parameter K of the improved KMeans algorithm is chosen to be four. This mainly considers the algorithm to be sensitive to “noise” and “isolated point”. Therefore, based on the performance of the virtual machine features of three common cloud environment platforms, a cluster is added, and noise and isolated points are included in the cluster (most noise and The isolated point comes from the virtual machine instance with no load in some cases), avoiding the influence of such points on the final clustering result, thus improving the accuracy of the clustering result.
  • the principle of the initial center selection in the improved KMeans algorithm is that in order to maintain the obviousness of the cluster, the centers that must be selected are far apart from each other. The scheme can well eliminate the influence of edge points and noise points, and can adapt to the imbalance of density distribution of actual categories in the dataset, and finally obtain better clustering effect, so as to accurately obtain the utilization of virtual machine resources.
  • the assessment is divided into two phases.
  • K feature value types
  • the initial center selection adopts the maximum and minimum distance selection center method. The principle of this method is to keep the initial categories as far as possible, and the virtual machine eigenvalues are more obvious in the cloud environment. So the type center selects P1(100,0,0)P2(0,100,0)P3(0,0,100)P4(0,0,0).
  • the Kmeans algorithm is applied to the feature vectors of all the virtual machines obtained in the first stage.
  • the virtual machines can be aggregated and grouped.
  • the aggregation points of each group are still gathered from the initial three points as in the first stage.
  • three aggregation points represent three characteristics: CPU-intensive, Network-intensive, and Memory-intensive. According to the class to which each virtual machine feature vector belongs, it is possible to grasp the application type from a macro perspective, thereby realizing the effective formulation of the resource utilization strategy.
  • the technical solution uses the improved KMeans algorithm to quantify the performance load of the virtual machine. Compared with the load evaluation that only relies on the virtual machine running macro state, single threshold control, and lack of quantization processing, the solution is comprehensively analyzed through specific data. Comprehensive and accurate system quantitative evaluation can effectively determine the type of application from specific data, quantify and refine the load characteristics, and improve the load evaluation accuracy by nearly 20%.
  • this technical solution can obtain the current system macro application type through cluster analysis (for example, CPU-occupied application cluster, memory-occupied application cluster, network bandwidth-occupied application cluster) Configuring optimization of virtual machine resources and physical machine resources based on application type and virtual machine resources.
  • cluster analysis for example, CPU-occupied application cluster, memory-occupied application cluster, network bandwidth-occupied application cluster
  • the technical solution can fully utilize virtual machine resources.
  • the application services will behave differently at runtime, and the user may not be able to quantitatively or qualitatively describe the characteristics of the application services in advance.
  • Through the evaluation system it is possible to specifically quantify the specific characteristics of each application, and based on the application characteristic values, Applications allocate different virtual machine types, which ensures that the application service will not degrade the performance of the virtual machine due to insufficient virtual machine resources, reduce the quality of the service, and enable the application service to fully utilize the resources of the virtual machine to provide corresponding services. Service, reduce the cost of the application, and make the cloud resources fully used, avoiding the waste of cloud resources.
  • the technical solution can realize full utilization of physical machine resources.
  • the eigenvalues ⁇ 1, ⁇ 2, ... ⁇ M of the M applications are calculated.
  • the performance of the corresponding three types of virtual machines can be finally calculated. load.
  • the virtual machine VM1VM2...VMM running the application service runs irregularly on the N physical machine nodes.
  • the same type of virtual machine such as CPU-intensive, runs on the same physical node.
  • the CPU resources of the node are very high, and the Memory resources and Network resources are rarely used. As a result, resources are not evenly distributed, resulting in waste.
  • the performance load values of all virtual machines are: Load(V1)Load(V2).....Load(Vm), and the virtual machines are divided into three types ( For example, CPU-intensive, Network-intensive, and Memory-intensive, the virtual machine can be redistributed according to the performance load value of the virtual machine, that is, an effective virtual machine migration is performed to solve the waste or shortage of the physical machine resources caused by the above. Case.
  • the present application applies the method of cluster analysis to a cloud environment to characterize the type of virtual machine for application service.
  • This solves the shortcomings that the clustering result of the traditional KMeans algorithm is related to the selection of the initial center, often ending with local optimization, and all points are not assigned to a certain cluster in the allocation step.
  • the application is also designed to improve the Kmeans algorithm to make it more suitable for cloud environments.
  • the feature types of the virtual machine include three types: CPU-intensive, Network-intensive, and Memory-intensive.
  • those skilled in the art can increase or decrease the type of the virtual machine or dynamically expand the type of the virtual machine according to actual needs.
  • the virtual machine has four feature types, namely, CPU-intensive, Network-intensive, Memory-intensive, and hard disk. Intensive.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Quality & Reliability (AREA)
  • Mathematical Physics (AREA)
  • Computer Hardware Design (AREA)
  • Debugging And Monitoring (AREA)

Abstract

公开了一种云环境下的虚拟机负载评估系统，包括：监控数据采集模块，其配置成从云数据中心内的一个或多个虚拟机处采集监控数据；以及监控数据处理模块，与所述监控数据采集模块耦合，并配置成：针对所述一个或多个虚拟机中的每一个，利用第一KMeans算法来将所述监控数据进行聚集分组，每一个组具有一中心点坐标，并根据所述分组和各组的中心点坐标计算所述一个或多个虚拟机中每个虚拟机的性能表征，即第一特征值；基于每个虚拟机的所述第一特征值，利用第二KMeans算法来将所述一个或多个虚拟机进行聚集分组，从而确定所述一个或多个虚拟机中每个虚拟机的特征类型以及性能负载值。还公开了云环境下对虚拟机进行负载评估的方法以及一种服务节点。

Description

云环境下的虚拟机负载评估系统、方法以及服务节点 技术领域
本发明涉及云监控技术，并且更具体来说，涉及云环境下虚拟机的负载评估。
背景技术
云计算中虚拟化技术，在某些方面对传统问题解决方案提供了一种新的思路，但虚拟化技术作为一门新兴的技术，它在虚拟机资源性能管理与检测等方面所做的工作仍然有许多的不足之处。11年的一篇论文《Toward an architecture for monitoring private clouds》中，作者同样指出云监控的重要性，尽管对云的管理部署的解决方案有许多，但是云监控的技术却没有跟上步伐。
一个系统的性能特征是由其本身特征以及运行在上面的应用服务的负载特征决定的，它定量描述了一个基本的系统特征。如何去判断应用服务的负载特征从而了解整个系统所处的环境具有重要意义。
目前在虚拟机负载评估方面，并没有精确的解决方案，大多基于人为或者虚拟机宏观运行状态表现判断负载情况(例如：通过用户数量预测一个负载今后运行的情况与大致性能需求)，负载状态评估相对单一、简略。
发明内容
传统机器负载评估通常采用机器本身的系统数据进行评分，通过预先设定的参数来对机器本身进行评估。这种方法在云环境下并不适用，因为云环境下存在大量虚拟机，用户与管理员真正需要了解的不仅是各个虚拟机性能上的差异性，更需要的是整体上的一个较为精确地宏观对比，从而了解具体应用对资源的占用情况。
为解决上述问题，本申请提供了一种云环境下的虚拟机负载评估系统，包括：监控数据采集模块，其配置成从云数据中心内的一个或 多个虚拟机处采集监控数据；以及监控数据处理模块，与所述监控数据采集模块耦合，并配置成：针对所述一个或多个虚拟机中的每一个，利用第一KMeans算法来将所述监控数据进行聚集分组，每一个组具有一中心点坐标，并根据所述分组和各组的中心点坐标计算所述一个或多个虚拟机中每个虚拟机的性能表征，即第一特征值；基于每个虚拟机的所述第一特征值，利用第二KMeans算法来将所述一个或多个虚拟机进行聚集分组，从而确定所述一个或多个虚拟机中每个虚拟机的特征类型以及性能负载值；其中，所述第一KMeans算法中的类型参数K大于所述第二KMeans算法中的类型参数，并且所述第一KMeans算法的初始中心被选择为使得初始类别之间保持尽可能远的距离，而所述第二Kmeans算法的初始中心被选择为所述第一KMeans算法的初始中心的子集。
在上述虚拟机负载评估系统中，所述监控数据与一个或多个虚拟机的中央处理单元、存储器、硬盘输入或输出以及网络输入或输出关联。
在上述虚拟机负载评估系统中，所述虚拟机的特征类型包括CPU密集型、Network密集型以及Memory密集型。
在上述虚拟机负载评估系统中，所述第一KMeans算法中的类型参数K等于4，而所述第二KMeans算法中的类型参数等于3。
在上述虚拟机负载评估系统中，所述第一KMeans算法的初始中心被选择为P1(100,0,0)、P2(0,100,0)、P3(0,0,100)以及P4(0,0,0)。
在上述虚拟机负载评估系统中，每一组中样本的占比为A％、B％、C％、D％，而每一组的中心点坐标分别为(X1,Y1,Z1)，(X2,Y2,Z2)，(X3,Y3,Z3)以及(X4,Y4,Z4)，则所述第一特征值由下面的等式来表示：
Figure PCTCN2015088868-appb-000001
Figure PCTCN2015088868-appb-000002
在上述虚拟机负载评估系统中，所述监控数据处理模块进一步被配置成根据所述一个或多个虚拟机中的性能负载值来对虚拟机进行重新分布，实现资源的有效利用。
根据本申请的另一个方面，提供了一种云环境下对虚拟机进行负载评估的方法，包括：针对云数据中心内的一个或多个虚拟机中的每一个，利用第一KMeans算法来将从所述一个或多个虚拟机处获得的监控数据进行聚集分组，每一个组具有一中心点坐标，并根据所述分组和各组的中心点坐标计算所述一个或多个虚拟机中每个虚拟机的性能表征，即第一特征值；基于每个虚拟机的所述第一特征值，利用第二KMeans算法来将所述一个或多个虚拟机进行聚集分组，从而确定所述一个或多个虚拟机中每个虚拟机的特征类型以及性能负载值；其中，所述第一KMeans算法中的类型参数K大于所述第二KMeans算法中的类型参数，并且所述第一KMeans算法的初始中心被选择为使得初始类别之间保持尽可能远的距离，而所述第二Kmeans算法的初始中心被选择为所述第一KMeans算法的初始中心的子集。
在上述方法中，所述监控数据与一个或多个虚拟机的中央处理单元、存储器、硬盘输入或输出以及网络输入或输出关联。
在上述方法中，所述虚拟机的特征类型包括CPU密集型、Network密集型以及Memory密集型。
在上述方法中，所述第一KMeans算法中的类型参数K等于4，而所述第二KMeans算法中的类型参数等于3。
在上述方法中，所述第一KMeans算法的初始中心被选择为P1(100,0,0)、P2(0,100,0)、P3(0,0,100)以及P4(0,0,0)。
在上述方法中，每一组中样本的占比为A％、B％、C％、D％，每一组的中心点坐标分别为(X1,Y1,Z1)，(X2,Y2,Z2)，(X3,Y3,Z3)以及(X4,Y4,Z4)，则所述第一特征值由下面的等式来表示：
Figure PCTCN2015088868-appb-000003
Figure PCTCN2015088868-appb-000004
上述方法还可包括根据所述一个或多个虚拟机中的性能负载值来对虚拟机进行重新分布，实现资源的有效利用。
根据本申请的又一个方面，提供了一种包括前面所述的虚拟机负载评估系统的服务节点。
本申请通过采用KMeans聚类算法实现虚拟机负载宏观情况的评估监测，并且克服了传统的KMeans算法所存在的聚类结果的准确性受很多因素的影响(包括初始中心的选择，噪声、孤立点的处理与否)等缺陷。
附图说明
在参照附图阅读了本发明的具体实施方式以后，本领域技术人员将会更清楚地了解本发明的各个方面。本领域技术人员应当理解的是：这些附图仅仅用于配合具体实施方式说明本发明的技术方案，而并非意在对本发明的保护范围构成限制。
图1是根据本申请的实施例，包含虚拟机负载评估系统的云平台应用环境的示意图；
图2是根据本申请的实施例，虚拟机负载评估系统的结构示意图。
具体实施方式
下面介绍的是本发明的多个可能实施例中的一些，旨在提供对本发明的基本了解，并不旨在确认本发明的关键或决定性的要素或限定所要保护的范围。容易理解，根据本发明的技术方案，在不变更本发明的实质精神下，本领域的一般技术人员可以提出可相互替换的其它实现方式。因此，以下具体实施方式以及附图仅是对本发明的技术方案的示例性说明，而不应当视为本发明的全部或者视为对本发明技术方案的限定或限制。
图1是根据本申请的实施例，包含虚拟机负载评估系统的云平台应用环境的示意图。在图1中，用户在诸如Openstack等云数据中心140的一个或多个虚拟机实例上部署相关应用服务。应用服务的有关数据就是服务节点120中虚拟机负载评估系统130所要采集的对象，包括CPU、Memory、disk I/O以及network IO等数据源，然后就虚拟机负载评估系统130对采集的数据进行处理分析，并将分析结果返回到前端展示模块110。
图2是根据本申请的实施例，虚拟机负载评估系统的结构示意图。结合图1，虚拟机负载评估系统130中的监控数据采集模块210负责从云数据中心140采集数据并将采集数据提供给监控数据处理模块220。该监控数据处理模块按照下面所述的虚拟机性能数据处理的方案对数据进行处理分析，并将分析结果返回到前端展示模块110。
虚拟机性能数据处理主要分为两部分，第一部分是利用云环境下改进的KMeans算法来计算出虚拟机的特征值，通过性能负载评估算法得出虚拟机近期的性能表征(特征值)。第二部分是计算出所有虚拟机的特征表现，主要分为三类即CPU、Network和Memory限制型，并在此基础之上得出虚拟机性能负载值，从而依据聚类结果评估应用类型。
在一个具体的实施例中，改进的KMeans算法的类型参数K被选择为4。这主要考虑了算法对“噪声”和“孤立点”敏感，故在常见三种云环境平台虚拟机特征表现的基础上再加一簇，将噪声与孤立点等纳入该簇(大多数噪声与孤立点来自于某些情况下没有负载的虚拟机实例)，避免此类点对最终聚类结果的影响，从而提高聚类结果的准确性。改进的KMeans算法中初始的中心选择遵循的原则是：为保持聚类的明显性，必须选取的中心相互距离都较远。该方案能够很好地排除类边缘点和噪声点的影响，并且能够适应数据集中各个实际类别密度分布不平衡的情况，最终获得较好的聚类效果，从而准确得出虚拟机资源利用情况。
改进的KMeans算法是具体算法描述如下：
1)在所有的样本点数据集A中依据已知条件，有目的性的(传统Kmeans算法初始中心的选择具有随机性)选择K个样本点，将K个样本点的值分别赋给初始时的聚类中心
Figure PCTCN2015088868-appb-000005
2)当第j次迭代时，对样本点A中所有样本点Pt(t＝1,2,3….,n),依次计算到各个聚类中心
Figure PCTCN2015088868-appb-000006
的欧式距离d(t,i)
Figure PCTCN2015088868-appb-000007
3)找出Pt关于
Figure PCTCN2015088868-appb-000008
的最小的距离，将Pt划入到关于
Figure PCTCN2015088868-appb-000009
距离的最小的簇中
4)更新各个簇的聚类中心点
Figure PCTCN2015088868-appb-000010
5)计算数据集A中所有的点的平方误差Ei，并与前一次的误差Ei-1比较
Figure PCTCN2015088868-appb-000011
若|Ei+1-Ei|<δ则算法结束，否则转入2)进行再一次迭代。
在一个实施例中，评估分为两个阶段。在第一阶段中，根据数据样本，对每台虚拟机进行改进后KMeans运算，这次运算需要确定两个变量，一个是特征值类型个数K(依照上述分析K＝4),一个是K个初始中心点的选取。初始中心选取采用最大最小距离选中心法，该方法的原则是使初始类别之间尽可能地保持到足够远的距离，在云环境下虚拟机特征值表现比较明显。所以类型中心选择P1(100,0,0)P2(0,100,0)P3(0,0,100)P4(0,0,0)。
最后可以把所有样本点分为4类：
每一类样本的占比 每一类的中心点坐标
A％ (X1,Y1,Z1)
B％ (X2,Y2,Z2)
C％ (X3,Y3,Z3)
D％ (X4,Y4,Z4)
根据这4类可以得出：其中有一类聚集了大量样本点，这个聚集点可以作为表征这台虚拟机近期的特征。
Figure PCTCN2015088868-appb-000012
在第二阶段中，对第一阶段得到的所有虚拟机的特征向量再运用一次Kmeans算法，这一次的K＝3，因为所有虚拟机的特征向量是通过大量样本点得出的，所以基本可以忽略接近0的特征向量(噪声点)。此时就可以把虚拟机进行聚集分组。而且每组的聚集点还是像第一阶段时一样由初始的三个点聚集而来。并且三个聚集点表征出三种特性：CPU密集型，Network密集型，Memory密集型。根据每个虚拟机特征向量所属的类就能够从宏观上把握应用类型，从而实现资源利用策略的有效制定。
本技术方案对虚拟机的性能负载采用改进的KMeans算法进行量化处理，相对于目前仅依靠虚拟机运行宏观状态、单一阀值控制、欠缺量化处理的负载评估，本方案通过具体数据综合分析更为全面、精确的实现系统量化评估，能从具体数据上有效判定应用所属类型，量化、细化负载特征，负载评估准确率提升约近20％。
除却有效提升虚拟机性能评估的准确性之外，本技术方案通过聚类分析能够得出当前系统宏观应用类型(例如，CPU占用型应用集群、内存占用型应用集群、网络带宽占用型应用集群)，基于应用类型及虚拟机资源情况对虚拟机资源以及物理机资源做出配置优化。
本技术方案可实现虚拟机资源的充分利用。云环境下当不同的用户需要部署不一样的应用服务，应用服务在运行时会表现为不同的特征，用户事先或许并不能定量或定性描述应用服务的特征。通过该评估系统，能够具体量化每种应用的具体特征，依据应用特征值，为各 种应用分配不同的虚拟机类型，这样保证了应用服务即不会因为虚拟机资源不足，导致虚拟机性能下降，降低服务的质量，又可以使应用服务可以充分利用虚拟机的资源来提供相应的服务，降低的申请时使用的费用，且使云资源得到充分的使用，避免了云资源的浪费。
本技术方案可实现物理机资源的充分利用。通过数据分析算出M种应用的特征值λ1，λ2......λM.通过这些应用服务标记出应用服务的具体特征后，就可以最后计算出相应的三种类型的虚拟机的性能负载。而在现有技术中，运行应用服务的虚拟机VM1VM2......VMM无规律地运行在N台物理机节点上。在这种情况下，就有可能同一种类型的虚拟机：如CPU密集型的都运行在同一物理节点上。导致该节点CPU资源占用很高，而Memory资源，Network资源确很少使用，造成资源的分配不均，导致浪费，更可能导致Nodei上的所有虚拟机因为资源的抢占，而使虚拟机的服务质量下降。而通过对虚拟机(应用服务)进行性能负载的量化，所有的虚拟机的性能负载值为:Load(V1)Load(V2)…..Load(Vm),并且虚拟机分为三种类型(例如CPU密集型、Network密集型以及Memory密集型)，就可以依据虚拟机的性能负载值来对虚拟机进行重新分布，即进行有效的虚拟机迁移，来解决上述所导致物理机资源浪费或不足的情况。
综上所述，本申请将聚类分析的方法应用到云环境下，用来表征面向应用服务的虚拟机的类型。这解决了传统的KMeans算法的聚类结果与初始中心的选择有关、经常以局部最优结束、所有的点在分配步骤中都未分到某个簇而得到空簇等缺点。本申请还设计改进了Kmeans算法，使之更适用于云环境。
需要指出的是，在上面的实施例中，虚拟机的特征类型包括CPU密集型、Network密集型以及Memory密集型这三种类型。但是，本领域技术人员可以根据实际需要增加或减少虚拟机的类型或可动态扩展虚拟机的类型。例如，在另外的实施例中，虚拟机的特征类型包括四种，即CPU密集型、Network密集型、Memory密集型以及硬盘 密集型。
上文中，参照附图描述了本发明的具体实施方式。但是，本领域中的普通技术人员能够理解，在不偏离本发明的精神和范围的情况下，还可以对本发明的具体实施方式作各种变更和替换。这些变更和替换都落在本发明权利要求书所限定的范围内。

Claims (15)

  1. 一种云环境下的虚拟机负载评估系统，包括：
    监控数据采集模块，其配置成从云数据中心内的一个或多个虚拟机处采集监控数据；以及
    监控数据处理模块，与所述监控数据采集模块耦合，并配置成：
    针对所述一个或多个虚拟机中的每一个，利用第一KMeans算法来将所述监控数据进行聚集分组，每一个组具有一中心点坐标，并根据所述分组和各组的中心点坐标计算所述一个或多个虚拟机中每个虚拟机的性能表征，即第一特征值；
    基于每个虚拟机的所述第一特征值，利用第二KMeans算法来将所述一个或多个虚拟机进行聚集分组，从而确定所述一个或多个虚拟机中每个虚拟机的特征类型以及性能负载值；
    其中，所述第一KMeans算法中的类型参数K大于所述第二KMeans算法中的类型参数，并且所述第一KMeans算法的初始中心被选择为使得初始类别之间保持尽可能远的距离，而所述第二Kmeans算法的初始中心被选择为所述第一KMeans算法的初始中心的子集。
  2. 如权利要求1所述的虚拟机负载评估系统，其中，所述监控数据与一个或多个虚拟机的中央处理单元、存储器、硬盘输入或输出以及网络输入或输出关联。
  3. 如权利要求1所述的虚拟机负载评估系统，其中，所述虚拟机的特征类型包括CPU密集型、Network密集型以及Memory密集型。
  4. 如权利要求1所述的虚拟机负载评估系统，其中，所述第一KMeans算法中的类型参数K等于4，而所述第二KMeans算法中的类型参数等于3。
  5. 如权利要求4所述的虚拟机负载评估系统，其中，所述第一KMeans算法的初始中心被选择为P1(100,0,0)、P2(0,100,0)、P3(0,0,100)以及P4(0,0,0)。
  6. 如权利要求4所述的虚拟机负载评估系统，其中，每一组中样本的占比为A％、B％、C％、D％，而每一组的中心点坐标分别为(X1,Y1,Z1)，(X2,Y2,Z2)，(X3,Y3,Z3)以及(X4,Y4,Z4)，则所述第一特征值由下面的等式来表示：
    Figure PCTCN2015088868-appb-100001
  7. 如权利要求1所述的虚拟机负载评估系统，其中，所述监控数据处理模块进一步被配置成根据所述一个或多个虚拟机中的性能负载值来对虚拟机进行重新分布，实现资源的有效利用。
  8. 一种云环境下对虚拟机进行负载评估的方法，包括：
    针对云数据中心内的一个或多个虚拟机中的每一个，利用第一KMeans算法来将从所述一个或多个虚拟机处获得的监控数据进行聚集分组，每一个组具有一中心点坐标，并根据所述分组和各组的中心点坐标计算所述一个或多个虚拟机中每个虚拟机的性能表征，即第一特征值；
    基于每个虚拟机的所述第一特征值，利用第二KMeans算法来将所述一个或多个虚拟机进行聚集分组，从而确定所述一个或多个虚拟机中每个虚拟机的特征类型以及性能负载值；
    其中，所述第一KMeans算法中的类型参数K大于所述第二KMeans算法中的类型参数，并且所述第一KMeans算法的初始中心被选择为使得初始类别之间保持尽可能远的距离，而所述第二Kmeans算法的初始中心被选择为所述第一KMeans算法的初始中心的子集。
  9. 如权利要求8所述的方法，其中，所述监控数据与一个或多个虚拟机的中央处理单元、存储器、硬盘输入或输出以及网络输入或输出关联。
  10. 如权利要求8所述的方法，其中，所述虚拟机的特征类型包括CPU 密集型、Network密集型以及Memory密集型。
  11. 如权利要求8所述的方法，其中，所述第一KMeans算法中的类型参数K等于4，而所述第二KMeans算法中的类型参数等于3。
  12. 如权利要求11所述的方法，其中，所述第一KMeans算法的初始中心被选择为P1(100,0,0)、P2(0,100,0)、P3(0,0,100)以及P4(0,0,0)。
  13. 如权利要求11所述的方法，其中，每一组中样本的占比为A％、B％、C％、D％，每一组的中心点坐标分别为(X1,Y1,Z1)，(X2,Y2,Z2)，(X3,Y3,Z3)以及(X4,Y4,Z4)，则所述第一特征值由下面的等式来表示：
    Figure PCTCN2015088868-appb-100002
  14. 如权利要求8所述的方法，还包括：根据所述一个或多个虚拟机中的性能负载值来对虚拟机进行重新分布，实现资源的有效利用。
  15. 一种服务节点，其包括如权利要求1至7所述的虚拟机负载评估系统。
PCT/CN2015/088868 2014-09-22 2015-09-02 云环境下的虚拟机负载评估系统、方法以及服务节点 WO2016045489A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US15/512,191 US10528378B2 (en) 2014-09-22 2015-09-02 System and method for load estimation of virtual machines in a cloud environment and serving node
EP15845103.9A EP3200076A4 (en) 2014-09-22 2015-09-02 System and method for load estimation of virtual machines in a cloud environment and serving node

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201410489705.4A CN105511953B (zh) 2014-09-22 2014-09-22 云环境下的虚拟机负载评估系统、方法以及服务节点
CN201410489705.4 2014-09-22

Publications (1)

Publication Number Publication Date
WO2016045489A1 true WO2016045489A1 (zh) 2016-03-31

Family

ID=55580272

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2015/088868 WO2016045489A1 (zh) 2014-09-22 2015-09-02 云环境下的虚拟机负载评估系统、方法以及服务节点

Country Status (4)

Country Link
US (1) US10528378B2 (zh)
EP (1) EP3200076A4 (zh)
CN (1) CN105511953B (zh)
WO (1) WO2016045489A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113747270A (zh) * 2021-09-06 2021-12-03 蓝鲸智云智能科技南京有限公司 一种云数据中心物联管控系统

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10509685B2 (en) * 2015-12-14 2019-12-17 VCE IP Holding Company, LLC Methods, systems, and computer readable mediums for workload clustering
CN106027685A (zh) * 2016-08-02 2016-10-12 合肥奇也信息科技有限公司 一种基于云计算机系统的高峰访问的方法
CN106230944A (zh) * 2016-08-02 2016-12-14 合肥奇也信息科技有限公司 一种基于云计算机系统的高峰访问的运行装置
CN107995651A (zh) * 2016-10-27 2018-05-04 中国电信股份有限公司 用于获取基站负载信息的方法、虚拟基站和mec服务器
US20180157557A1 (en) * 2016-12-02 2018-06-07 Intel Corporation Determining reboot time after system update
CN108459905B (zh) * 2017-02-17 2022-01-14 华为技术有限公司 资源池容量规划方法及服务器
CN110647381B (zh) * 2019-09-18 2023-06-06 上海擎创信息技术有限公司 一种虚拟机资源均衡和部署优化方法
CN111737078B (zh) * 2020-05-12 2021-11-23 华南理工大学 基于负载类型的自适应云服务器能耗测算方法、系统及设备
TWI767530B (zh) * 2021-01-22 2022-06-11 緯穎科技服務股份有限公司 產生運算裝置的參考配置的電子裝置和方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103092751A (zh) * 2012-12-13 2013-05-08 华中科技大学 一种云环境下基于用户行为模型的web应用性能测试系统
US20130145367A1 (en) * 2011-09-27 2013-06-06 Pneuron Corp. Virtual machine (vm) realm integration and management
CN103475538A (zh) * 2013-09-02 2013-12-25 南京邮电大学 一种基于多接口的自适应的云服务测试方法
CN103576829A (zh) * 2012-08-01 2014-02-12 复旦大学 基于混合遗传算法的云计算虚拟机动态调度方法
CN103595780A (zh) * 2013-11-08 2014-02-19 中国人民解放军理工大学 基于消重的云计算资源调度方法

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1818746A1 (en) * 2006-02-10 2007-08-15 ALSTOM Technology Ltd Method of condition monitoring
CN101354663A (zh) * 2007-07-25 2009-01-28 联想(北京)有限公司 应用于虚拟机系统的真实cpu资源的调度方法及调度装置
US10797953B2 (en) * 2010-10-22 2020-10-06 International Business Machines Corporation Server consolidation system
CN102508718B (zh) * 2011-11-22 2015-04-15 杭州华三通信技术有限公司 一种虚拟机负载均衡方法和装置
US20130185729A1 (en) 2012-01-13 2013-07-18 Rutgers, The State University Of New Jersey Accelerating resource allocation in virtualized environments using workload classes and/or workload signatures

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130145367A1 (en) * 2011-09-27 2013-06-06 Pneuron Corp. Virtual machine (vm) realm integration and management
CN103576829A (zh) * 2012-08-01 2014-02-12 复旦大学 基于混合遗传算法的云计算虚拟机动态调度方法
CN103092751A (zh) * 2012-12-13 2013-05-08 华中科技大学 一种云环境下基于用户行为模型的web应用性能测试系统
CN103475538A (zh) * 2013-09-02 2013-12-25 南京邮电大学 一种基于多接口的自适应的云服务测试方法
CN103595780A (zh) * 2013-11-08 2014-02-19 中国人民解放军理工大学 基于消重的云计算资源调度方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3200076A4 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113747270A (zh) * 2021-09-06 2021-12-03 蓝鲸智云智能科技南京有限公司 一种云数据中心物联管控系统
CN113747270B (zh) * 2021-09-06 2024-01-02 蓝鲸智云智能科技南京有限公司 一种云数据中心物联管控系统

Also Published As

Publication number Publication date
CN105511953B (zh) 2019-04-05
EP3200076A4 (en) 2018-05-23
CN105511953A (zh) 2016-04-20
US10528378B2 (en) 2020-01-07
US20170286147A1 (en) 2017-10-05
EP3200076A1 (en) 2017-08-02

Similar Documents

Publication Publication Date Title
WO2016045489A1 (zh) 云环境下的虚拟机负载评估系统、方法以及服务节点
CN102694868B (zh) 一种集群系统实现及任务动态分配方法
CN107220108B (zh) 一种实现云数据中心负载均衡的方法和系统
A. El-Moursy et al. Multi-dimensional regression host utilization algorithm (MDRHU) for host overload detection in cloud computing
WO2016134542A1 (zh) 虚拟机的迁移方法、装置及设备
Al-Dulaimy et al. Type-aware virtual machine management for energy efficient cloud data centers
US20130030761A1 (en) Statistically-based anomaly detection in utility clouds
Fu et al. Predicted affinity based virtual machine placement in cloud computing environments
Tan et al. Exploiting resource usage patterns for better utilization prediction
US20150019722A1 (en) Determining, managing and deploying an application topology in a virtual environment
JP6411911B2 (ja) コンピュータ実行可能な方法、前記方法を行わせる命令を記憶している非一時的なコンピュータ可読記憶媒体、及び前記方法を行なうためのコンピュータシステム
Jena et al. Performance evaluation of energy efficient power models for digital cloud
WO2020119627A1 (zh) 应用于分布式容器云平台的异常检测与定位方法及装置
Karimi et al. A new fuzzy approach for dynamic load balancing algorithm
US20200233724A1 (en) Workload placement in a cluster computing environment using machine learning
US20150286493A1 (en) Virtual-machine placement based on information from multiple data centers
CN109976879A (zh) 一种基于资源使用曲线互补的云计算虚拟机放置方法
Canali et al. An adaptive technique to model virtual machine behavior for scalable cloud monitoring
Ismaeel et al. Real-time energy-conserving vm-provisioning framework for cloud-data centers
Wang et al. Effects of correlation-based VM allocation criteria to cloud data centers
US10817401B1 (en) System and method for job-to-queue performance ranking and resource matching
Canali et al. Agate: Adaptive gray area-based technique to cluster virtual machines with similar behavior
US20180284870A1 (en) Method and apparatus for energy efficiency rating evaluation for physical server
Moldovan et al. On analyzing elasticity relationships of cloud services
CN114741160A (zh) 一种基于平衡能耗与服务质量的动态虚拟机整合方法和系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15845103

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 15512191

Country of ref document: US

REEP Request for entry into the european phase

Ref document number: 2015845103

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 2015845103

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE