WO2012100544A1 - Method, device and cluster system for virtual machine migration based on network data flow direction - Google Patents

Method, device and cluster system for virtual machine migration based on network data flow direction Download PDF

Info

Publication number
WO2012100544A1
WO2012100544A1 PCT/CN2011/079733 CN2011079733W WO2012100544A1 WO 2012100544 A1 WO2012100544 A1 WO 2012100544A1 CN 2011079733 W CN2011079733 W CN 2011079733W WO 2012100544 A1 WO2012100544 A1 WO 2012100544A1
Authority
WO
WIPO (PCT)
Prior art keywords
virtual machine
migration
cluster
service node
virtual
Prior art date
Application number
PCT/CN2011/079733
Other languages
French (fr)
Chinese (zh)
Inventor
叶川
许建辉
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to CN201110025843.3 priority Critical
Priority to CN 201110025843 priority patent/CN102082692B/en
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2012100544A1 publication Critical patent/WO2012100544A1/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00Arrangements for maintenance or administration or management of packet switching networks
    • H04L41/08Configuration management of network or network elements
    • H04L41/0803Configuration setting of network or network elements
    • H04L41/0823Configuration optimization
    • H04L41/083Configuration optimization for network speed increase
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5083Techniques for rebalancing the load in a distributed system
    • G06F9/5088Techniques for rebalancing the load in a distributed system involving task migration

Abstract

Provided are a method, device and cluster system for virtual machine migration based on network data flow direction. A network traffic monitoring module of the cluster system is used for interacting with a VMM to obtain network traffic information about each virtual machine on the current service node; a virtual machine deployment adjusting module is used for collecting the network traffic information about each virtual machine from the network traffic monitoring modules on a plurality of service nodes in the cluster system, determining a data flow direction relationship among the virtual machines in the cluster system according to the network traffic information, and issuing a virtual machine migration command or outputting a corresponding virtual machine migration proposal to the VMM on which the determined virtual machine to be migrated runs according to a virtual machine migration strategy and the data flow direction relationship among the virtual machines in the cluster system; and a Virtual Machine Monitor (VMM) unit is used for receiving the virtual machine migration command issued by the virtual machine deployment adjusting module and migrating the virtual machines to be migrated onto a target service node. Therefore, the communication traffic among the virtual machines in the cluster system can be effectively reduced, reducing the network burden.

Description

基于网络数据流向的虚拟机迁移方法、 设备和集群系统 技术领域  Virtual machine migration method, device and cluster system based on network data flow

本发明涉及信息技术领域, 特别涉及一种基于网络数据流向的虚拟机迁 移方法、 设备和集群系统。 背景技术  The present invention relates to the field of information technology, and in particular, to a virtual machine migration method, device, and cluster system based on network data flow. Background technique

随着计算机技术的发展, 虚拟化技术得到了越来越广泛的应用。 其中, 应用最多的是虚拟化分割技术, 即在一台物理服务器上运行多个虚拟机 ( Virtual Machine; VM ), 在物理服务器和多个 VM之间, 通过虚拟机监视 器(Virtual Machine Monitor; VMM )管理中央处理单元 ( Central Processing Unit; CPU ), 内存、 存储、 网络和其它外设资源, VMM 对这些物理资源进 行调度、 管理, 为多个 VM分配这些物理资源, 实现各个 VM之间的相互隔 离运行。  With the development of computer technology, virtualization technology has been more and more widely used. Among them, the most used is the virtualization segmentation technology, that is, running multiple virtual machines (VMs) on one physical server, between the physical server and multiple VMs, through the virtual machine monitor (Virtual Machine Monitor; VMM) manages the central processing unit (CPU), memory, storage, network, and other peripheral resources. The VMM schedules and manages these physical resources, and allocates these physical resources to multiple VMs to achieve Run isolated from each other.

为了有效利用集群系统中各物理服务器上的资源, 使各物理服务器上负 载均衡,通常采用虚拟机迁移的方法,将负载较多的物理服务器上的部分 VM 迁移到其他物理服务器上。  In order to effectively utilize the resources on the physical servers in the cluster system and balance the load on each physical server, a virtual machine migration method is used to migrate some VMs on the physical servers with more load to other physical servers.

现有的虚拟机迁移方法, 根据集群系统中各物理服务器上的资源利用情 况来进行, 如果某台物理服务器的物理资源超过设定阈值, 则将其上运行的 的一台或多台虚拟机迁移到其他物理服务器上, 迁移的目的物理服务器通常 资源利用率不高, 或者与中央处理器兼容。 然而, 现有技术进行虚拟机迁移 后, 可能会增加集群系统内虚拟机之间的通信流量, 从而增加网络负担。 发明内容  The existing virtual machine migration method is performed according to resource utilization on each physical server in the cluster system. If the physical resource of a physical server exceeds a set threshold, one or more virtual machines running on the virtual machine Migrating to other physical servers, the destination physical server for migration is usually not resource efficient or compatible with the central processor. However, after the virtual machine migration in the prior art, communication traffic between virtual machines in the cluster system may be increased, thereby increasing the network load. Summary of the invention

本发明实施例提供了一种基于网络数据流向的虚拟机迁移方法、 设备和 集群系统, 用以降低现有技术进行虚拟机迁移所带来的集群系统内的虚拟机 之间的通信流量, 从而减少网络负担。 The embodiment of the invention provides a virtual machine migration method, device and cluster system based on network data flow, which is used to reduce virtual machines in the cluster system caused by virtual machine migration in the prior art. The communication traffic between them reduces the network load.

本发明实施例提供一种集群系统, 包括: 管理节点和多台业务节点, 其 中每台业务节点包括硬件层, 运行在所述硬件层之上的虚拟机监控单元 An embodiment of the present invention provides a cluster system, including: a management node and multiple service nodes, where each service node includes a hardware layer, and a virtual machine monitoring unit running on the hardware layer

VMM , 运行在所述虚拟机监控单元 VMM之上的至少一个虚拟机; 每台业务 节点上进一步部署有网络流量监控模块, 所述管理节点上部署有虚拟机部署 调整模块, 其中: a VMM, at least one virtual machine running on the virtual machine monitoring unit VMM; a network traffic monitoring module is further deployed on each service node, and a virtual machine deployment adjustment module is deployed on the management node, where:

所述网络流量监控模块用于: 通过与所述虚拟机监控单元 VMM 交互, 获取当前业务节点上每个虚拟机的网络流量信息; 所述虚拟机部署调整模块用于: 向所述集群系统内的所述多台业务节点 上的网络流量监控模块收集所述虚拟机的网络流量信息, 根据所述虚拟机的 网络流量信息确定所述集群系统内虚拟机之间的数据流向关系, 根据虚拟机 迁移策略和所述集群系统内虚拟机之间的数据流向关系, 向其上运行有所确 定的待迁移虚拟机的 VMM 下发虚拟机迁移命令或者输出对应的虚拟机迁移 建议;  The network traffic monitoring module is configured to: obtain network traffic information of each virtual machine on the current service node by interacting with the virtual machine monitoring unit VMM; the virtual machine deployment adjustment module is configured to: The network traffic monitoring module on the multiple service nodes collects network traffic information of the virtual machine, and determines a data flow relationship between the virtual machines in the cluster system according to the network traffic information of the virtual machine, according to the virtual machine. The migration policy and the data flow relationship between the virtual machines in the cluster system are sent to the VMM on which the determined virtual machine to be migrated is executed, or the corresponding virtual machine migration proposal is output;

其上运行有所确定的待迁移虚拟机的所述虚拟机监控单元 VMM用于: 接收所述虚拟机部署调整模块下发的虚拟机迁移命令, 根据所述虚拟机迁移 命令将所述待迁移虚拟机迁移到目标业务节点上, 使得存在数据流向关系的 多个虚拟机宿主在同一业务节点上。  The virtual machine monitoring unit VMM that runs the determined virtual machine to be migrated is configured to: receive a virtual machine migration command delivered by the virtual machine deployment adjustment module, and move the to-be-migrated according to the virtual machine migration command The virtual machine is migrated to the target service node, so that multiple virtual machines with data flows to the relationship are hosted on the same service node.

本发明实施例还提供一种业务节点, 包括: 硬件层, 运行在所述硬件层 之上的虚拟机监控单元 VMM ,运行在所述虚拟机监控单元 VMM之上的至少 一个虚拟机, 所述业务节点还进一步部署有网络流量监控模块;  The embodiment of the present invention further provides a service node, including: a hardware layer, a virtual machine monitoring unit VMM running on the hardware layer, and at least one virtual machine running on the virtual machine monitoring unit VMM, The service node is further deployed with a network traffic monitoring module;

所述网络流量监控模块用于: 通过与所述 VMM 交互, 获取本业务节点 上每个虚拟机的网络流量信息, 所述虚拟机的网络流量信息包括: 虚拟机标 识信息;  The network traffic monitoring module is configured to: obtain, by interacting with the VMM, network traffic information of each virtual machine on the service node, where the network traffic information of the virtual machine includes: virtual machine identification information;

所述虚拟机监控单元 VMM用于: 在收到虚拟机迁移命令后, 根据所述 虚拟机迁移命令将其上运行的待迁移虚拟机迁移到对应的目标业务节点上, 使得存在数据流向关系的多个虚拟机宿主在同一业务节点上, 其中, 所述虚 拟机迁移命令是管理节点根据向集群系统内的各业务节点上的网络流量监控 模块所收集的网络流量信息, 确定所述集群系统内虚拟机之间的数据流向关 系, 根据虚拟机迁移策略和所述集群系统内虚拟机之间的数据流向关系向其 上运行有所确定的待迁移虚拟机的 VMM下发的。 The virtual machine monitoring unit VMM is configured to: after receiving the virtual machine migration command, according to the The virtual machine migration command migrates the virtual machine to be migrated to the corresponding target service node, so that the multiple virtual machine hosts that have the data flow to the relationship are on the same service node, where the virtual machine migration command is the management node. Determining, according to network traffic information collected by the network traffic monitoring module on each service node in the cluster system, a data flow relationship between the virtual machines in the cluster system, according to the virtual machine migration policy and the virtual machine in the cluster system The data flow relationship is delivered to the VMM on which the determined virtual machine to be migrated runs.

本发明实施例提供一种管理节点, 包括:  An embodiment of the present invention provides a management node, including:

网络流量收集单元, 用于向所述集群系统内的各业务节点收集宿主在所 述业务节点上的虚拟机的网络流量信息;  a network traffic collection unit, configured to collect, by each service node in the cluster system, network traffic information of a virtual machine hosted on the service node;

虚拟机部署调整单元, 用于根据所述虚拟机的网络流量信息确定所述集 群系统内虚拟机之间的数据流向关系, 根据虚拟机迁移策略和所述集群系统 内虚拟机之间的数据流向关系, 向其上运行有所确定的待迁移虚拟机的虚拟 机监控单元 VMM下发虚拟机迁移命令或者输出对应的虚拟机迁移建议。  a virtual machine deployment adjustment unit, configured to determine, according to network traffic information of the virtual machine, a data flow relationship relationship between virtual machines in the cluster system, according to a virtual machine migration policy and a data flow direction between the virtual machines in the cluster system The virtual machine migration command is sent to the virtual machine monitoring unit VMM that runs the determined virtual machine to be migrated or the corresponding virtual machine migration proposal is output.

本发明实施例还提供一种基于网络数据流向的虚拟机迁移方法, 包括: 向集群系统内的各业务节点收集宿主在所述业务节点上的虚拟机的网络 流量信息;  The embodiment of the present invention further provides a virtual machine migration method based on network data flow, including: collecting network traffic information of a virtual machine hosted on the service node to each service node in the cluster system;

根据所述虚拟机的网络流量信息确定所述集群系统内虚拟机之间的数据 流向关系;  Determining, according to network traffic information of the virtual machine, a data flow relationship relationship between virtual machines in the cluster system;

根据虚拟机迁移策略和所述集群系统内虚拟机之间的数据流向关系, 向 其上运行有所确定的待迁移虚拟机的虚拟机监控单元 VMM 下发虚拟机迁移 命令或者输出对应的虚拟机迁移建议。  Distributing a virtual machine migration command or outputting a corresponding virtual machine to the virtual machine monitoring unit VMM running the determined virtual machine to be migrated according to the virtual machine migration policy and the data flow relationship between the virtual machines in the cluster system Migration recommendations.

本发明实施例提供的基于网络数据流向的虚拟机迁移方法、 设备和集群 系统, 根据集群系统内的多台业务节点上的虚拟机的网络流量信息确定所述 集群系统内虚拟机之间的数据流向关系, 根据虚拟机迁移策略和集群系统内 虚拟机之间的数据流向关系下发虚拟机迁移命令或者输出对应的虚拟机迁移 建议; 相应的, 响应于该虚拟机迁移命令, 将所述待迁移虚拟机迁移到对应 的目标业务节点上, 使得存在数据流向关系的多个虚拟机宿主在同一业务节 点上, 从而减少集群系统中虚拟机之间经过物理网卡的数据传输, 能够有效 降低集群系统内的虚拟机之间的通信流量, 减少网络负担。 附图说明 The method, device, and cluster system for virtual machine migration based on network data flow according to embodiments of the present invention determine data between virtual machines in the cluster system according to network traffic information of virtual machines on multiple service nodes in the cluster system. The flow direction relationship, the virtual machine migration command is sent according to the virtual machine migration policy and the data flow relationship relationship between the virtual machines in the cluster system, or the corresponding virtual machine migration proposal is output; correspondingly, in response to the virtual machine migration command, the standby Migrate virtual machines to migrate to corresponding On the target service node, the multiple virtual machines that have data flow to the relationship are hosted on the same service node, thereby reducing the data transmission between the virtual machines in the cluster system through the physical network card, thereby effectively reducing the virtual machines in the cluster system. Communication traffic, reducing network load. DRAWINGS

为了更清楚地说明本发明实施例或现有技术中的技术方案, 下面将对实 施例或现有技术描述中所需要使用的附图作简单地介绍, 显而易见地, 下面 描述中的附图仅仅是本发明的一些实施例, 对于本领域普通技术人员来讲, 在不付出创造性劳动的前提下, 还可以根据这些附图获得其他的附图。  In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the description of the prior art will be briefly described below. Obviously, the drawings in the following description are only It is a certain embodiment of the present invention, and other drawings can be obtained from those skilled in the art without any creative work.

图 1为本发明提供的集群系统第一实施例的结构示意图;  1 is a schematic structural diagram of a first embodiment of a cluster system according to the present invention;

图 2为本发明提供的业务节点第一实施例的结构示意图;  2 is a schematic structural diagram of a first embodiment of a service node according to the present invention;

图 3为本发明提供的各业务节点生成数据流向表的方法流程图; 图 4为本发明提供的管理节点第一实施例的结构示意图;  3 is a flowchart of a method for generating a data flow direction table by each service node according to the present invention; FIG. 4 is a schematic structural diagram of a first embodiment of a management node provided by the present invention;

图 5为本发明提供的管理节点第二实施例的结构示意图;  FIG. 5 is a schematic structural diagram of a second embodiment of a management node according to the present invention; FIG.

图 6为本发明提供的基于网络数据流向的虚拟机迁移方法第一实施例的 流程图;  6 is a flowchart of a first embodiment of a virtual machine migration method based on network data flow according to the present invention;

图 7为本发明实施例提供的根据虚拟机的网络流量信息确定集群系统内 虚拟机之间的数据流向关系的方法流程图;  FIG. 7 is a flowchart of a method for determining a data flow relationship relationship between virtual machines in a cluster system according to network traffic information of a virtual machine according to an embodiment of the present invention;

图 8为本发明实施例提供的对收集的所述虚拟机的网络流量信息进行匹 配处理的方法流程图;  FIG. 8 is a flowchart of a method for performing matching processing on collected network traffic information of the virtual machine according to an embodiment of the present invention;

图 9为本发明实施例提供的根据集群系统的流向匹配记录确定满足虚拟 机迁移策略的待迁移虚拟机的方法流程图。 具体实施方式  FIG. 9 is a flowchart of a method for determining a virtual machine to be migrated that satisfies a virtual machine migration policy according to a flow matching record of a cluster system according to an embodiment of the present invention. detailed description

下面将结合本发明实施例中的附图, 对本发明实施例中的技术方案进行 清楚、 完整地描述, 显然, 所描述的实施例仅仅是本发明一部分实施例, 而 不是全部的实施例。 基于本发明中的实施例, 本领域普通技术人员在没有做 出创造性劳动前提下所获得的所有其他实施例, 都属于本发明保护的范围。 The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and Not all embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present invention without departing from the inventive scope are the scope of the present invention.

图 1为本发明提供的集群系统第一实施例的结构示意图, 如图 1所示, 该集群系统包括: 管理节点 1和多台业务节点 2, 其中, 每台业务节点 2包 括硬件层 21,运行在硬件层 21之上的虚拟机监控单元 VMM22,运行在虚拟 机监控单元 VMM22之上的至少一个虚拟机 23; 每台业务节点 2上进一步部 署有网络流量监控模块 24,管理节点 1上部署有虚拟机部署调整模块 11 ,其 中:  1 is a schematic structural diagram of a first embodiment of a cluster system according to the present invention. As shown in FIG. 1, the cluster system includes: a management node 1 and a plurality of service nodes 2, wherein each service node 2 includes a hardware layer 21, The virtual machine monitoring unit VMM22 running on the hardware layer 21 runs at least one virtual machine 23 above the virtual machine monitoring unit VMM22; each of the service nodes 2 is further deployed with a network traffic monitoring module 24, which is deployed on the management node 1. There is a virtual machine deployment adjustment module 11 in which:

网络流量监控模块 24用于: 通过与虚拟机监控单元 VMM22交互, 获取 当前业务节点 2上每个虚拟机 23的网络流量信息;  The network traffic monitoring module 24 is configured to: obtain network traffic information of each virtual machine 23 on the current service node 2 by interacting with the virtual machine monitoring unit VMM22;

虚拟机部署调整模块 11用于:向集群系统内的多台业务节点 2上的网络 流量监控模块 24收集虚拟机 23的网络流量信息, 根据虚拟机 23的网络流 量信息确定集群系统内虚拟机 23之间的数据流向关系,根据虚拟机迁移策略 和集群系统内虚拟机 23之间的数据流向关系,向其上运行有所确定的待迁移 虚拟机 23的 VMM22下发虚拟机迁移命令或者输出对应的虚拟机迁移建议; 需要说明的是, 虚拟机 23之间的数据流向关系可以用源虚拟机标识信 息、 目标虚拟机标识信息及对应流量大小表示。  The virtual machine deployment adjustment module 11 is configured to: collect network traffic information of the virtual machine 23 from the network traffic monitoring module 24 on the multiple service nodes 2 in the cluster system, and determine the virtual machine in the cluster system according to the network traffic information of the virtual machine 23 The data flow relationship between the virtual machine migration policy and the virtual machine 23 in the cluster system is used to deliver a virtual machine migration command or output corresponding to the VMM 22 on which the determined virtual machine 23 to be migrated is running. The virtual machine migration proposal; it should be noted that the data flow relationship between the virtual machines 23 can be represented by the source virtual machine identification information, the target virtual machine identification information, and the corresponding traffic size.

其上运行有所确定的待迁移虚拟机的虚拟机监控单元 VMM22用于: 接 收虚拟机部署调整模块 11下发的虚拟机迁移命令,根据所述虚拟机迁移命令 将待迁移虚拟机 23迁移到目标业务节点 2上,使得存在数据流向关系的多个 虚拟机 23宿主在同一业务节点上。  The virtual machine monitoring unit VMM22 running the determined virtual machine to be migrated is configured to: receive the virtual machine migration command delivered by the virtual machine deployment adjustment module 11, and migrate the virtual machine 23 to be migrated according to the virtual machine migration command. On the target service node 2, the plurality of virtual machines 23 having the data flow to the relationship are hosted on the same service node.

在集群系统中, 每台业务节点 2为一台物理服务器, 该物理服务器的底 层为硬件层 21 ,硬件层 21主要包括 CPU、 内存、硬盘以及网卡等硬件资源。 运行在硬件层 21之上的为虚拟机监控单元 VMM22, VMM22承担对硬件层 21 中的硬件资源进行调度、 分配和管理工作。 VMM22之上运行多个虚拟机 VM23, VMM22为多个虚拟机 23提供虚拟的硬件环境, 保证多个虚拟机 23 相互隔离同时运行。 In the cluster system, each service node 2 is a physical server. The bottom layer of the physical server is the hardware layer 21, and the hardware layer 21 mainly includes hardware resources such as a CPU, a memory, a hard disk, and a network card. Running above the hardware layer 21 is a virtual machine monitoring unit VMM 22, which is responsible for scheduling, distributing, and managing hardware resources in the hardware layer 21. Running multiple virtual machines on top of VMM22 The VM23 and the VMM 22 provide a virtual hardware environment for the plurality of virtual machines 23 to ensure that the plurality of virtual machines 23 are isolated from each other and operate at the same time.

由于 VMM22直接面向物理服务器上的硬件层 21 , 并且 VMM22能够获 知所管理的各个虚拟机的运行情况, 因此, VMM22能够获知所在业务节点 2 上每个虚拟机 23的网络流量信息。 其中, 该网络流量信息可以包括: 虚拟机 标识信息, 例如: 虚拟机名和 /虚拟机的 IP地址等信息。 还可以包括: 流入 源互联网协议 I P地址以及流出目标 I P地址, 即可以从网络流量信息中反映 出存在数据流向关系的虚拟机对。 进一步的, 网络流量信息中还可以包括流 入源 IP地址对应的流量, 流出目标 IP地址对应的流量, 以便进一步获知数 据流向关系的虚拟机对之间的流量大小, 以便在下发命令或者给出迁移建议 时对存在数据流向关系的虚拟机对之间的流量大小加以考虑。  Since the VMM 22 directly faces the hardware layer 21 on the physical server, and the VMM 22 can learn the operation status of each managed virtual machine, the VMM 22 can know the network traffic information of each virtual machine 23 on the service node 2. The network traffic information may include: virtual machine identification information, such as: a virtual machine name and/or a virtual machine IP address. It may also include: flowing into the source Internet Protocol IP address and the outbound target IP address, that is, the virtual machine pair having the data flow relationship relationship may be reflected from the network traffic information. Further, the network traffic information may include the traffic corresponding to the source IP address and the traffic corresponding to the destination IP address, so as to further know the traffic volume between the virtual machine pairs of the data flow relationship, so as to issue a command or give a migration. It is recommended to consider the amount of traffic between pairs of virtual machines that have data flow relationships.

本发明实施例提供的集群系统中, 在所有业务节点 2上均部署网络流量 监控模块 24, 网络流量监控模块 24可以通过与 VMM22的交互获取各个虚 拟机的网络流量信息。  In the cluster system provided by the embodiment of the present invention, the network traffic monitoring module 24 is deployed on all the service nodes 2, and the network traffic monitoring module 24 can obtain the network traffic information of each virtual machine through interaction with the VMM 22.

本发明实施例提供的集群系统中, 还设置了一个管理节点 1 , 该管理节 点 1可以是集群系统中一台独立的物理服务器, 或者, 所述管理节点 1还具 有业务节点 2的功能, 或者所述管理节点 1为业务节点 2上的虚拟机 23。 该 管理节点 1上部署有虚拟机部署调整模块 11 , 该虚拟机部署调整模块 11可 以周期性或定时地向集群系统内的多台业务节点 2上的网络流量监控模块 24 收集虚拟机 23的网络流量信息, 根据虚拟机 23的网络流量信息确定集群系 统内虚拟机 23之间的数据流向关系。 其中, 数据流向关系表示了虚拟机 23 之间的网络数据走向, 确定产生流量的虚拟机对。虚拟机部署调整模块 11获 取集群系统内虚拟机 23之间数据流向关系后,可以根据虚拟机迁移策略和集 群系统内虚拟机 23之间数据流向关系确定能减轻网络压力的最优迁移策略, 从而向其上运行有所确定的待迁移虚拟机 23的 VMM22下发虚拟机迁移命令 或者输出对应的虚拟机迁移建议。 其中, 可以尽可能少的迁移数量为前提, 判断存在数据流量的虚拟机对 能够迁移到一台物理服务器上, 即对处理器是否兼容等情况加以考虑。 另外, 可以对用户的需求加以考虑。如果能够获知各虚拟机对之间的数据流量大小, 则还可以对数据流量大小加以考虑, 获得最优迁移策略。 可以直接向其上运 行有所确定的待迁移虚拟机 23的 VMM22下发迁移命令,也可以向其上运行 有所确定的待迁移虚拟机 23的 VMM22提供迁移建议。 其上运行有所确定的待迁移虚拟机 23的虚拟机监控单元 VMM22, 接收 到虚拟机部署调整模块 11下发的虚拟机迁移命令,可以根据虚拟机迁移命令 将待迁移虚拟机 23迁移到目标业务节点 2上,使得存在数据流向关系的多个 虚拟机 23宿主在同一业务节点上。 In the cluster system provided by the embodiment of the present invention, a management node 1 is further provided, and the management node 1 may be an independent physical server in the cluster system, or the management node 1 also has the function of the service node 2, or The management node 1 is a virtual machine 23 on the service node 2. A virtual machine deployment adjustment module 11 is deployed on the management node 1, and the virtual machine deployment adjustment module 11 can periodically or periodically collect the network of the virtual machine 23 from the network traffic monitoring module 24 on the multiple service nodes 2 in the cluster system. The traffic information determines the data flow relationship between the virtual machines 23 in the cluster system based on the network traffic information of the virtual machine 23. The data flow relationship indicates the network data direction between the virtual machines 23, and determines the virtual machine pair that generates the traffic. After obtaining the data flow relationship between the virtual machines 23 in the cluster system, the virtual machine deployment adjustment module 11 can determine an optimal migration strategy capable of reducing network pressure according to the relationship between the virtual machine migration policy and the data flow relationship between the virtual machines 23 in the cluster system. The VMM 22 running the determined virtual machine 23 to be migrated delivers a virtual machine migration command or outputs a corresponding virtual machine migration proposal. Among them, the number of migrations can be as small as possible, and it can be judged that the virtual machine pair with data traffic can be migrated to one physical server, that is, whether the processor is compatible or not. In addition, the needs of the user can be considered. If you can know the data traffic between each virtual machine pair, you can also consider the data traffic size to obtain the optimal migration strategy. The migration command may be directly sent to the VMM 22 on which the determined virtual machine 23 to be migrated is run, or may be provided to the VMM 22 on which the determined virtual machine 23 to be migrated is running. The virtual machine monitoring unit VMM22 on which the determined virtual machine 23 to be migrated is executed, receives the virtual machine migration command delivered by the virtual machine deployment adjustment module 11, and can migrate the virtual machine 23 to be migrated to the target according to the virtual machine migration command. On the service node 2, the plurality of virtual machines 23 having the data flow to the relationship are hosted on the same service node.

本实施例提供的集群系统, 根据集群系统内的多台业务节点上的虚拟机 的网络流量信息确定所述集群系统内虚拟机之间的数据流向关系, 根据虚拟 机迁移策略和集群系统内虚拟机之间的数据流向关系下发虚拟机迁移命令或 者输出对应的虚拟机迁移建议; 相应的, 响应于该虚拟机迁移命令, 将所述 待迁移虚拟机迁移到对应的目标业务节点上, 使得存在数据流向关系的多个 虚拟机宿主在同一业务节点上, 从而减少集群系统中虚拟机之间经过物理网 卡的数据传输, 能够有效降低集群系统内的虚拟机之间的通信流量, 减少网 络负担。  The cluster system provided in this embodiment determines the data flow relationship between the virtual machines in the cluster system according to the network traffic information of the virtual machines on the multiple service nodes in the cluster system, according to the virtual machine migration policy and the virtual system in the cluster system. Corresponding to the virtual machine migration command, or the corresponding virtual machine migration proposal is sent to the data flow relationship between the machines; correspondingly, the virtual machine to be migrated is migrated to the corresponding target service node in response to the virtual machine migration command, so that Multiple virtual machines with data flow relationship are hosted on the same service node, thereby reducing data transmission between virtual machines in the cluster system through physical network cards, which can effectively reduce communication traffic between virtual machines in the cluster system and reduce network load. .

在前一实施例的基础上, 作为一种可行的实施方式, 业务节点 2上部署 的网络流量监控模块 24可与 VMM22集成于一体,从而减少网络流量监控模 块 24与 VMM22之间数据交互的开销。 具体可以将网络流量监控模块 24集 成到 VMM22内部, 由 VMM22直接与虚拟机部署调整模块 11进行交互。  On the basis of the previous embodiment, as a feasible implementation manner, the network traffic monitoring module 24 deployed on the service node 2 can be integrated with the VMM 22, thereby reducing the overhead of data interaction between the network traffic monitoring module 24 and the VMM 22. . Specifically, the network traffic monitoring module 24 can be integrated into the VMM 22, and the VMM 22 directly interacts with the virtual machine deployment adjustment module 11 .

管理节点 1 可以为一台独立的物理服务器, 也可以具有业务节点 2的功 能, 还可以是某个业务节点 2上的虚拟机 23, 该业务节点 2为独立的物理服 务器。 另夕卜,虚拟机部署调整模块 11可与集群系统内的一业务节点 2上部署的 网络流量监控模块 24集成于一体,即可以在集群系统中的一业务节点 2上设 置一主控装置, 该主控装置即执行网络流量监控模块 24的操作, 又执行虚拟 机部署调整模块 11的操作, 从而减小网络流量监控模块 24和虚拟机部署调 整模块 11之间的数据交互开销。 进一步的, 本发明实施例提供了一种集群系统内各设备执行虚拟机迁移 的较佳的实施方式: The management node 1 may be an independent physical server, or may have the function of the service node 2, or may be the virtual machine 23 on a certain service node 2, and the service node 2 is an independent physical server. In addition, the virtual machine deployment adjustment module 11 can be deployed on a service node 2 in the cluster system. The network traffic monitoring module 24 is integrated, that is, a master device can be set on a service node 2 in the cluster system, and the master device performs the operation of the network traffic monitoring module 24, and executes the virtual machine deployment adjustment module 11 The operation reduces the data interaction overhead between the network traffic monitoring module 24 and the virtual machine deployment adjustment module 11. Further, an embodiment of the present invention provides a preferred implementation manner for performing virtual machine migration by devices in a cluster system:

VMM22用于: 取样获取虚拟机列表中的虚拟机 23的数据包, 根据数据 包的 IP信息和对应数量,统计对应虚拟机 23的流入源 IP地址及对应的流量 数据、 流出目标 IP地址及对应的流量数据, 并将相应的统计结果输出给网络 流量监控模块 24, 以及在接收虚拟机部署调整模块 11 下发的虚拟机迁移命 令后,根据虚拟机迁移命令将待迁移虚拟机 23迁移到对应的目标业务节点 2 上;  The VMM 22 is configured to: obtain a data packet of the virtual machine 23 in the virtual machine list, and collect statistics on the inbound source IP address and the corresponding traffic data, the outflow destination IP address, and the corresponding virtual machine 23 according to the IP information and the corresponding quantity of the data packet. The traffic data is outputted to the network traffic monitoring module 24, and after the virtual machine migration command sent by the virtual machine deployment adjustment module 11 is received, the virtual machine 23 to be migrated is migrated according to the virtual machine migration command. Target on business node 2;

网络流量监控模块 24具体用于: 统计超过流量阈值的虚拟机列表, 并将 虚拟机列表输出给 VMM22; 根据来自 VMM22 的统计结果得到数据流向记 录, 该数据流向记录包括虚拟机标识信息, 流入源 IP地址以及对应的流量数 据、 流出目标 IP地址以及对应的流量数据, 虚拟机标识信息包括虚拟机名和 虚拟机 IP地址;  The network traffic monitoring module 24 is specifically configured to: collect a virtual machine list that exceeds the traffic threshold, and output the virtual machine list to the VMM 22; obtain a data flow direction record according to the statistical result from the VMM 22, where the data flow direction record includes the virtual machine identification information, and the inflow source The IP address and the corresponding traffic data, the outgoing destination IP address, and the corresponding traffic data, and the virtual machine identification information includes a virtual machine name and a virtual machine IP address;

具体的, 根据来自 VMM22的统计结果生成或更新数据流向表(数据流 向记录的一种表现载体), 数据流向表包括虚拟机标识信息, 流入源 IP地址 以及对应的流量数据、 流出目标 I P地址以及对应的流量数据, 虚拟机标识信 息包括虚拟机名和虚拟机 IP地址;  Specifically, the data flow direction table (a performance carrier of the data flow to the record) is generated or updated according to the statistical result from the VMM 22, and the data flow direction table includes the virtual machine identification information, the inflow source IP address, the corresponding flow data, the outflow destination IP address, and Corresponding traffic data, the virtual machine identification information includes a virtual machine name and a virtual machine IP address;

需要说明的是, 这里的数据流向表指的是用于记录虚拟机标识信息, 流 入源 IP地址以及对应的流量数据、 流出目标 IP地址以及对应的流量数据的 载体, 虽然这里表达为表, 但不限于表, 包含了其他载体形式。 其中,流量阈值用于判断当前业务节点 2上某个虚拟机 23的流量是否过 大的下限值, 该流量阈值可由用户自定义或由集群系统给出建议值。 如果某个虚拟机 23流量超过阈值,则 VMM可以在当前业务节点 2的底 层定时或周期性地取样获取该虚拟机 23的数据包, 根据数据包的 IP信息和 对应数量统计虚拟机流量的流入源 IP、流出目标 IP以及对应的流量数据, 并 这些统计数据打包提交至网络流量监控模块 24, 网络流量监控模块 24, 根 据统计结果生成数据流向表, 需要注意的是, 只有超过流量阈值的虚拟机才 会被统计入数据流向表。网络流量监控模块 24还对该数据流向表进行维护以 及定期进行更新。 It should be noted that the data flow direction table herein refers to a carrier for recording virtual machine identification information, a source IP address and corresponding traffic data, an outgoing destination IP address, and corresponding traffic data, although expressed as a table, Not limited to tables, it contains other forms of carrier. The traffic threshold is used to determine whether the traffic of a virtual machine 23 on the current service node 2 is excessively large. The traffic threshold may be customized by the user or given a recommended value by the cluster system. If the traffic of a virtual machine 23 exceeds the threshold, the VMM may sample the data packet of the virtual machine 23 at the bottom layer of the current service node 2 or periodically, and count the inflow of the virtual machine traffic according to the IP information of the data packet and the corresponding quantity. The source IP address, the outbound destination IP address, and the corresponding traffic data are forwarded to the network traffic monitoring module 24, and the network traffic monitoring module 24 generates a data flow direction table according to the statistical result. Note that only the virtual traffic threshold is exceeded. The machine will be counted into the data flow table. The network traffic monitoring module 24 also maintains the data flow table and periodically updates it.

其中, 数据流向表可以为结构体数组, 该结构体数组包括虚拟机标识信 息, 流入源 I P地址以及对应的流量数据、 流出目标 I P地址以及对应的流量 数据。 数据流向表还可以采用其他数据结构, 而无论采用哪种数据结构, 只 要能够反应出各虚拟机 23的流入源 IP地址以及对应的流量数据, 流出目标 I P地址以及对应的流量数据即可。  The data flow direction table may be an array of structures, and the array of the structure includes virtual machine identification information, an inflow source IP address, a corresponding flow data, an outflow destination IP address, and corresponding flow data. The data flow table can also adopt other data structures, and no matter which data structure is used, it can reflect the inflow source IP address of each virtual machine 23 and the corresponding traffic data, and the target IP address and corresponding traffic data can be flowed out.

以下详细给出网络流量监控模块 24与 VMM22交互获取数据流向表的过 程:  The following is a detailed description of the process of the network traffic monitoring module 24 interacting with the VMM 22 to obtain a data flow table:

网络流量监控模块 24可以定时或周期性向 VMM22获取当前业务节点 2 上的所有虚拟机 23的网络流量信息, 具体包括流入源互联网协议 IP地址及 对应的流量、 流出目标 I P地址以及对应的流量, 虚拟机标识信息包括虚拟机 名和虚拟机 IP地址; 网络流量监控模块 24遍历虚拟机 23的网络流量数据, 如果发现某个虚拟机 23 的流出或流入流量超过流量阈值, 则记录该虚拟机 23, 如果符合条件的虚拟机 23数量为 0, 本次监控流程结束。  The network traffic monitoring module 24 can periodically or periodically obtain the network traffic information of all the virtual machines 23 on the current service node 2 from the VMM 22, including the inbound source Internet Protocol IP address and the corresponding traffic, the outgoing destination IP address, and the corresponding traffic. The machine identification information includes the virtual machine name and the virtual machine IP address. The network traffic monitoring module 24 traverses the network traffic data of the virtual machine 23. If the outflow or inflow traffic of a virtual machine 23 exceeds the traffic threshold, the virtual machine 23 is recorded. The number of eligible virtual machines 23 is 0, and the monitoring process ends.

网络流量监控模块 24 将流量超过阈值的虚拟机 23 告知 VMM22 , VMM22监控一定时间内对应虚拟机 23的所有数据包, 并逐一对其分析, 统 计每个数据包的流入源 IP地址、 流出目标 IP地址, 并把相同数据包进行计 数, 从而计算出该时间段内每个流入源 IP地址流入该虚拟机 23、 该虚拟机 23流出目标 IP地址的流量大小。 VMM22把统计结果打包输出给网络流量监 控模块 24, 网络流量监控 24将 VMM22统计结果填充到数据流向表中。 如 果虚拟机 23未存在于数据流向表中, 则在该数据流向表中创建新的表项, 并 4巴对应虚拟机信息、 流入源 IP地址以及对应的流量数据、 流出目标 IP地址 以及对应的流量数据填入新表项; 如果虚拟机 23已经在数据流向表中, 则更 新该数据流向表中的相应数据; 更新完毕后, 将当前周期内流量超过阈值的 虚拟机 23与数据流向表已经存在的表项——对比,把已经不存在数据流量的 表项从数据流向表中剔除。 The network traffic monitoring module 24 informs the VMM 22 of the virtual machine 23 whose traffic exceeds the threshold. The VMM 22 monitors all the data packets corresponding to the virtual machine 23 in a certain period of time, analyzes them one by one, and counts the inflow source IP address and the outbound destination IP of each data packet. The address, and the same data packet is counted, thereby calculating the traffic volume of each incoming source IP address flowing into the virtual machine 23 and the virtual machine 23 flowing out of the destination IP address during the time period. The VMM 22 packs the statistical result and outputs it to the network traffic monitoring module 24, and the network traffic monitoring 24 fills the VMM22 statistical result into the data flow direction table. Such as If the virtual machine 23 does not exist in the data flow direction table, a new entry is created in the data flow table, and the virtual machine information, the inflow source IP address, the corresponding traffic data, the outgoing destination IP address, and the corresponding The traffic data is filled in the new entry; if the virtual machine 23 is already in the data flow to the table, the data flow is updated to the corresponding data in the table; after the update is completed, the virtual machine 23 with the traffic exceeding the threshold in the current period and the data flow to the table have been The existing entry - comparison, removes the entry that does not have data traffic from the data flow to the table.

需要说明的是,网络流量监控模块 24与 VMM22可以采用多种交互方式, 既可以由 VMM22提供接口, 由网络流量监控模块 24主动调用; 也可以由 VMM22主动向网络流量监控模块 24主动推送; 网络流量监控模块 24可以 逐个也可以批量地向 VMM22获取虚拟机 23的网络流量数据和数据包统计结 果, 而遍历虚拟机流量数据的既可以是网络流量监控模块 24 , 也可以是 VMM22。  It should be noted that the network traffic monitoring module 24 and the VMM 22 can adopt multiple interaction modes, and the interface can be provided by the VMM 22 and actively invoked by the network traffic monitoring module 24; or the VMM 22 can actively push to the network traffic monitoring module 24; The traffic monitoring module 24 may obtain the network traffic data and the packet statistics result of the virtual machine 23 in batches from the VMM 22, and the traversal of the virtual machine traffic data may be the network traffic monitoring module 24 or the VMM 22.

虚拟机部署调整模块 11具体用于:向集群系统内的各业务节点 2上的网 络流量监控模块 24获取数据流向记录; 对数据流向记录进行匹配处理,得到 集群系统内虚拟机 23之间的数据流向关系, 集群系统内虚拟机 23之间的数 据流向关系用流向匹配记录表示, 包括: 源虚拟机标识信息、 目标虚拟机标 识信息及对应流量大小; 并根据集群系统的流向匹配记录确定满足虚拟机迁 移策略的待迁移虚拟机 23 , 向其上运行有所确定的待迁移虚拟机 23 的 VMM22下发虚拟机迁移命令或者输出对应的虚拟机迁移建议;  The virtual machine deployment adjustment module 11 is specifically configured to: obtain a data flow direction record to the network traffic monitoring module 24 on each service node 2 in the cluster system; perform matching processing on the data flow record to obtain data between the virtual machines 23 in the cluster system. In the flow relationship, the data flow relationship between the virtual machines 23 in the cluster system is represented by a flow matching record, including: source virtual machine identification information, target virtual machine identification information, and corresponding traffic size; and determining that the virtuality is satisfied according to the flow matching record of the cluster system The virtual machine 23 to be migrated from the machine migration policy, and the virtual machine migration command is sent to the VMM 22 of the virtual machine 23 to be migrated, or the corresponding virtual machine migration proposal is output;

具体的, 定期向集群系统内的各业务节点 2 上的网络流量监控模块 24 获取数据流向表(数据流向记录的一种表现载体); 根据数据流向表确定集群 系统内虚拟机 23之间的数据流向关系及对应流量,生成或更新集群系统的流 向匹配表, 流向匹配表包括: 存在数据流向关系的源虚拟机标识信息、 目标 虚拟机标识信息及对应流量大小; 并根据所述流向匹配表确定满足虚拟机迁 移策略的待迁移虚拟机 23 , 向其上运行有所确定的待迁移虚拟机 23 的 VMM22下发虚拟机迁移命令或者输出对应的虚拟机迁移建议; 需要说明的是, 这里的流向匹配表指的是用于记录源虚拟机标识信息、 目标虚拟机标识信息、 流量大小的载体, 虽然这里表达为表, 但不限于表, 包含了其他载体形式。 Specifically, the data flow direction table (the data flow direction record representation carrier) is obtained periodically from the network traffic monitoring module 24 on each service node 2 in the cluster system; and the data between the virtual machines 23 in the cluster system is determined according to the data flow direction table. The flow direction relationship and the corresponding traffic, the flow direction matching table of the cluster system is generated or updated, and the flow direction matching table includes: source virtual machine identification information, target virtual machine identification information, and corresponding traffic size of the data flow direction relationship; and determining according to the flow direction matching table The virtual machine 23 to be migrated to meet the virtual machine migration policy, and the VMM 22 that runs the determined virtual machine 23 to be delivered sends a virtual machine migration command or outputs a corresponding virtual machine migration proposal. It should be noted that the flow matching table herein refers to a carrier for recording source virtual machine identification information, target virtual machine identification information, and traffic size. Although expressed as a table, it is not limited to a table, and includes other carrier forms.

其中, 所述虚拟机迁移策略包括: 虚拟机迁移数量小于等于设定阈值, 和 / 或, 根据减少的网络流量大小确定迁移顺序, 和 /或, 满足虚拟机迁移数量最小 化和减少的网络流量最大化的平衡关系, 和 /或, 将存在数据流向关系的虚拟机 23迁移到同一目标业务节点 2, 所述目标业务节点 2为根据集群系统内存在数 据流向关系的多个虚拟机 23各自宿主的业务节点 2的物理资源情况所确定的。  The virtual machine migration policy includes: the number of virtual machine migrations is less than or equal to a set threshold, and/or, determining a migration sequence according to the reduced network traffic size, and/or meeting network traffic that minimizes and reduces the number of virtual machine migrations. Maximizing the balance relationship, and/or migrating the data flow to the relationship virtual machine 23 to the same target service node 2, the target service node 2 being the respective hosts of the plurality of virtual machines 23 according to the data flow relationship relationship in the cluster system The physical resource situation of the service node 2 is determined.

所述虚拟机迁移策略可由用户自定义,如果用户事先设定了虚拟机 23与 物理机或虚拟机 23之间的亲和关系,那么在计算待迁移列表时, 需要对这部 分虚拟机 23进行特殊判断。  The virtual machine migration policy may be customized by the user. If the user has previously set the affinity relationship between the virtual machine 23 and the physical machine or the virtual machine 23, the virtual machine 23 needs to be performed when calculating the to-be-migrated list. Special judgment.

其中, 源虚拟机为发送数据流量的虚拟机 23, 目标虚拟机为接收数据流 量的虚拟机 23, 源虚拟机和目标虚拟机的标识信息可以为: 虚拟机名和 /虚拟 机的 IP地址等信息。  The source virtual machine is a virtual machine 23 that sends data traffic, and the target virtual machine is a virtual machine 23 that receives data traffic. The identification information of the source virtual machine and the target virtual machine may be: virtual machine name and/or virtual machine IP address and other information. .

流向匹配表可以为结构体数组, 该结构体数组包括存在数据流向关系的 源虚拟机标识信息、 目标虚拟机标识信息及对应流量。 流向匹配表还可以采 用其他数据结构, 而无论采用哪种数据结构, 只要能够反应出存在数据流向 关系的源虚拟机标识信息、 目标虚拟机标识信息及对应流量即可。  The flow matching table may be an array of structures, the source array includes source virtual machine identification information, target virtual machine identification information, and corresponding traffic. The flow matching table can also adopt other data structures, and no matter which data structure is used, it can reflect the source virtual machine identification information, the target virtual machine identification information, and the corresponding traffic in the presence of the data flow relationship.

虚拟机部署调整模块 11 根据数据流向表生成流向匹配表具体可以采用 以下方法: 如果虚拟机部署调整模块 11遍历到虚拟机 A, 虚拟机部署调整模 块 11首先找到虚拟机 A对应的数据流向表(称为流向表 A ) , 从中获取虚拟 机 A的 IP地址信息, 然后在除了流向表 A外的所有数据流向表的流入源 IP 地址和流出目标 IP地址列表中查找虚拟机 A的 IP地址信息。 如果在虚拟机 B的数据流向表(称为流向表 B ) 中找到虚拟机 A的 IP地址信息, 接下来则 反过来在流向表 A的流入源 IP地址和目标流出 IP地址列表中查找虚拟机 B 的 IP地址信息, 如果也找到匹配的数据, 并且两者互为流入和流出关系, 流 量数据误差在 5%以内 , 则可以视为虚拟机 A与虚拟机 B存在流向关系 , 如 果虚拟机 B在虚拟机 A的目标流出 IP地址列表中, 则认为数据流向是虚拟 机 A->虚拟机 B, 反之则为虚拟机 B->虚拟机 A。 如果虚拟机 B的 IP地址信 息在流向表 A中找不到匹配数据, 或者是流量数据误差很大, 则可视为虚拟 机 A和虚拟机 B之间不存在流向关系,则可以继续寻找与虚拟机 A存在流向 关系的下一个虚拟机, 直至遍历完所有业务节点 2的数据流向表。 The virtual machine deployment adjustment module 11 may generate the flow direction matching table according to the data flow direction table. The following method may be adopted: If the virtual machine deployment adjustment module 11 traverses to the virtual machine A, the virtual machine deployment adjustment module 11 first finds the data flow direction table corresponding to the virtual machine A ( Referred to as the flow table A), the IP address information of the virtual machine A is obtained therefrom, and then the IP address information of the virtual machine A is searched for in the flow source IP address and the outbound destination IP address list of all the data flow except the flow table A. If the IP address information of the virtual machine A is found in the data flow direction table of the virtual machine B (referred to as the flow direction table B), then the virtual machine is searched for in the flow source IP address and the target outgoing IP address list of the flow table A. B's IP address information, if the matching data is also found, and the two are inflow and outflow relationships, the flow If the error of the data is less than 5%, it can be considered that the virtual machine A has a flow relationship with the virtual machine B. If the virtual machine B is in the target IP address list of the virtual machine A, the data flow is considered to be virtual machine A->virtual. Machine B, otherwise VM->VM A. If the IP address information of the virtual machine B cannot find matching data in the flow direction table A, or the traffic data error is large, it can be regarded that there is no flow relationship between the virtual machine A and the virtual machine B, and the search can continue. Virtual machine A has the next virtual machine in the flow relationship until the data flow direction table of all service nodes 2 is traversed.

以上仅给出了一种根据数据流向表生成流向匹配表的方法, 但并不以此 作为对本发明的限制。  Only a method for generating a flow matching table based on a data flow table is given above, but is not intended to limit the present invention.

在获取流向匹配表后, 虚拟机部署调整模块 11 可以以流向匹配表为依 据, 根据虚拟机迁移策略计算出能减轻集群压力的最优虚拟机迁移顺序。 该 流向匹配表给出了虚拟机 23之间的流量关系,而将存在数据流量的虚拟机对 迁移到哪个业务节点 2上取决于虚拟机迁移策略。 而虚拟机迁移策略可以尽 量少迁移虚拟机 23为前提, 对于有数据流量的两个虚拟机 23: VM1 ->VM2, 可以先判断 VM1、 VM2所在业务节点 2的物理硬件资源是否充足,优先迁移 到资源充足的业务节点 2上, 如果虚拟机对所在的业务节点 2的物理资源均 不充足, 则可以放弃迁移。 对于有数据流量的多个虚拟机 23 : VM1->VM2->VM3或 VM1->VM2&VM3 ( VM2&VM3表示这两个虚拟机 23 位于同一业务节点 2上) , 如果其中有两个虚拟机 23在同一业务节点 2上, 则优先迁移剩余的一个虚拟机 23到该业务节点 2上。 而在业务节点 2资源 不足的情况下, 优先把网络流量比较大的虚拟机 23 迁移到一台业务节点 2 上。  After obtaining the flow matching table, the virtual machine deployment adjustment module 11 can calculate the optimal virtual machine migration order according to the virtual machine migration policy according to the virtual machine migration policy. The flow direction matching table gives the traffic relationship between the virtual machines 23, and the migration of the virtual machine pair with the data traffic to which service node 2 depends on the virtual machine migration policy. The virtual machine migration policy can minimize the migration of the virtual machine 23. For the two virtual machines 23 with data traffic: VM1 -> VM2, you can first determine whether the physical hardware resources of the service node 2 where VM1 and VM2 are located are sufficient, and the migration is preferred. On the service node 2 with sufficient resources, if the physical resources of the service node 2 where the virtual machine is located are insufficient, the migration can be abandoned. For multiple virtual machines 23 with data traffic: VM1->VM2->VM3 or VM1->VM2&VM3 (VM2&VM3 indicates that the two virtual machines 23 are on the same service node 2), if two of the virtual machines 23 are in the same On the service node 2, the remaining one virtual machine 23 is preferentially migrated to the service node 2. In the case that the service node 2 has insufficient resources, the virtual machine 23 with relatively large network traffic is preferentially migrated to one service node 2.

作为一种较佳的实施方式,虚拟机部署调整模块 11可以根据流向匹配表 生成虚拟机迁移加权表, 虚拟机迁移加权表包括流向匹配表中的所有虚拟机 23的标识信息、 目标业务节点 2的标识信息及对应权值, 权值为虚拟机 23 与目标业务节点 2上的各虚拟机 23之间的数据流量之和。  As a preferred implementation, the virtual machine deployment adjustment module 11 may generate a virtual machine migration weighting table according to the flow direction matching table, where the virtual machine migration weighting table includes the identification information of all the virtual machines 23 in the flow matching table, and the target service node 2 The identification information and the corresponding weight, the weight is the sum of the data traffic between the virtual machine 23 and each virtual machine 23 on the target service node 2.

若虚拟机迁移加权表中存在一个最大权值, 则从虚拟机迁移加权表中选 取权值最大的虚拟机 23加入虚拟机迁移列表;若虚拟机迁移加权表中存在多 个最大权值,则将虚拟机 23内存值与目标业务节点 2剩余内存值比值最大的 虚拟机 23加入虚拟机迁移列表; If there is a maximum weight in the virtual machine migration weighting table, select from the virtual machine migration weighting table. The virtual machine 23 with the largest weight is added to the virtual machine migration list; if there are multiple maximum weights in the virtual machine migration weighting table, the virtual machine 23 with the largest virtual machine 23 memory value and the remaining service value of the target service node 2 is added. Virtual machine migration list;

根据虚拟机迁移列表的顺序向其上运行有所述权值最大的虚拟机 23 或 者所述比值最大的虚拟机 23的 VMM22下发虚拟机迁移命令或者输出对应的 迁移建议。  The virtual machine migration command or the corresponding migration proposal is delivered to the VMM 22 of the virtual machine 23 with the highest value of the virtual machine 23 or the virtual machine 23 with the highest ratio.

另外, 虚拟机部署调整模块 11还用于: 根据虚拟机迁移列表中已迁移的 虚拟机 23, 对虚拟机迁移加权表进行更新。  In addition, the virtual machine deployment adjustment module 11 is further configured to: update the virtual machine migration weighting table according to the migrated virtual machine 23 in the virtual machine migration list.

具体的,虚拟机部署调整模块 11可以首先生成一张空的虚拟机迁移加权 表。 该虚拟机迁移加权表用来存储虚拟机 23迁移到某台业务节点 2的权值, 以此来区分不同的迁移所带来的不同效果。 该虚拟机迁移加权表的表项中记 录了虚拟机标识、 业务节点的标识信息及对应权值, 其中, 业务节点的标识 信息可以为: IP地址和 /或主机名等信息。 权值为对应虚拟机 23迁移到对应 业务节点上所带来的流量数据的加权和。  Specifically, the virtual machine deployment adjustment module 11 may first generate an empty virtual machine migration weighting table. The virtual machine migration weighting table is used to store the weights that the virtual machine 23 migrates to a certain service node 2, so as to distinguish different effects brought by different migrations. The virtual machine identifier, the identifier information of the service node, and the corresponding weight are recorded in the entry of the virtual machine migration weighting table. The identifier information of the service node may be: an IP address and/or a host name. The weight is a weighted sum of the traffic data brought by the virtual machine 23 to migrate to the corresponding service node.

虚拟机部署调整装置 11可以遍历流向匹配表中的每一项,把可能存在的 迁移行为的相应数据添加到迁移加权表中。 例如: 流向匹配表中存在流向关 系: 虚拟机 A->虚拟机 B, 对应流量为 X , 虚拟机 A迁到虚拟机 B所在业务 节点 2或是虚拟机 B迁到虚拟机 A所在业务节点 2的效果相同 ,虚拟机部署 调整模块 11分别判断虚拟机 A和虚拟机 B所在业务节点 2的剩余资源能否 满足虚拟机 23的迁移要求,然后把对应流量 X分别累加到满足要求的表项中。  The virtual machine deployment adjustment means 11 can traverse each item in the flow matching table to add corresponding data of the migration behavior that may exist to the migration weighting table. For example, there is a flow relationship in the flow matching table: virtual machine A-> virtual machine B, the corresponding traffic is X, virtual machine A moves to the service node 2 where virtual machine B is located, or virtual machine B moves to the service node 2 where virtual machine A is located. The effect is the same. The virtual machine deployment adjustment module 11 determines whether the remaining resources of the service node 2 where the virtual machine A and the virtual machine B are located can meet the migration requirement of the virtual machine 23, and then the corresponding traffic X is respectively added to the entry that meets the requirements. .

虚拟机部署调整模块 11遍历虚拟机迁移加权表,选取该表中权值最大的 一次迁移行为, 如果存在多个最大权值, 则可以按照贪心算法, 选取虚拟机 The virtual machine deployment adjustment module 11 traverses the virtual machine migration weighting table, and selects a migration behavior with the largest weight in the table. If there are multiple maximum weights, the virtual machine may be selected according to the greedy algorithm.

23内存值与目标业务节点剩余内存值比值最大的一项。 23 The largest value of the ratio of the memory value to the remaining memory value of the target service node.

虚拟机部署调整模块 11 把选取出来的当前最优的迁移对应的虚拟机标 识信息加入虚拟机迁移列表。  The virtual machine deployment adjustment module 11 adds the selected virtual machine identification information corresponding to the current optimal migration to the virtual machine migration list.

虚拟机部署调整模块 11根据流向匹配表,把与加入虚拟机迁移列表的虚 拟机 23相关的权值从虚拟机迁移加权表中去除; 同时,虚拟机部署调整模块 11根据集群系统中各业务节点 2的剩余资源状况, 增减相关表项的权值; 如 果某个表项权值减为 0, 则把相关表项从加权表中删除。 The virtual machine deployment adjustment module 11 adds the virtual table to the virtual machine migration list according to the flow matching table. The weight associated with the virtual machine migration weighting table is removed from the virtual machine migration weighting table; at the same time, the virtual machine deployment adjustment module 11 increases or decreases the weight of the related entry according to the remaining resource status of each service node 2 in the cluster system; If the weight of the item is reduced to 0, the related item is deleted from the weighting table.

虚拟机部署调整模块 11判断虚拟机迁移加权表是否还存在表项,是则继 续添加下一个次优的迁移对应的虚拟机标识信息,否则待迁移虚拟机 23选取 结束。  The virtual machine deployment adjustment module 11 determines whether the virtual machine migration weighting table still has an entry, and then continues to add the virtual machine identification information corresponding to the next suboptimal migration, otherwise the virtual machine to be migrated 23 is selected to end.

其中, 虚拟机部署调整模块 11可以根据用户预先设定的迁移数量, 按照 虚拟机迁移列表中的按序向对应业务节点 2上的 VMM22下发虚拟机迁移命 令, 或者向 VMM22推送虚拟机迁移建议。  The virtual machine deployment adjustment module 11 may deliver a virtual machine migration command to the VMM 22 on the corresponding service node 2 according to the number of migrations preset by the user, or push the virtual machine migration proposal to the VMM 22 according to the order in the virtual machine migration list. .

另外, 虚拟机迁移命令或者所述虚拟机迁移策略每次中选取的虚拟机迁 移数量可以由用户预先自定义, 一旦如果加入虚拟机迁移列表中的虚拟机数 量达到用户设定的上限, 则可以停止选取待迁移的虚拟机迁移的选取, 直接 进入下个迁移虚拟机的周期。  In addition, the number of virtual machine migrations selected by the virtual machine migration command or the virtual machine migration policy may be pre-defined by the user, once the number of virtual machines in the virtual machine migration list reaches the upper limit set by the user, Stop selecting the migration of the VM to be migrated and go directly to the next migration VM.

与数据流向表和流向匹配表相类似的, 虚拟机迁移加权表和虚拟机迁移 列表均可以为结构体数组, 或者其他数据结构。  Similar to the data flow table and the flow matching table, the virtual machine migration weighting table and the virtual machine migration list can both be an array of structures, or other data structures.

图 2为本发明提供的业务节点第一实施例的结构示意图, 如图 2所示, 该业务节点包括: 硬件层 21 , 运行在硬件层 21 之上的虚拟机监控单元 2 is a schematic structural diagram of a first embodiment of a service node according to the present invention. As shown in FIG. 2, the service node includes: a hardware layer 21, and a virtual machine monitoring unit running on the hardware layer 21.

VMM22, 运行在虚拟机监控单元 VMM22之上的至少一个虚拟机 23, 业务 节点还进一步部署有网络流量监控模块 24; VMM22, at least one virtual machine 23 running on the virtual machine monitoring unit VMM22, the service node is further deployed with a network traffic monitoring module 24;

其中, 网络流量监控模块 24用于: 通过与 VMM22交互, 获取本业务节 点上每个虚拟机 23的网络流量信息;  The network traffic monitoring module 24 is configured to: obtain, by interacting with the VMM 22, network traffic information of each virtual machine 23 on the service node;

虚拟机监控单元 VMM22用于: 在收到虚拟机迁移命令后, 根据虚拟机 迁移命令将其上运行的待迁移虚拟机 23迁移到对应的目标业务节点上,使得 存在数据流向关系的多个虚拟机 23宿主在同一业务节点上, 其中,虚拟机迁 移命令是管理节点根据向集群系统内的各业务节点上的网络流量监控模块 The virtual machine monitoring unit VMM22 is configured to: after receiving the virtual machine migration command, migrate the virtual machine 23 to be migrated running on the corresponding target service node according to the virtual machine migration command, so that there are multiple virtual flows of the data to the relationship. The machine 23 is hosted on the same service node, wherein the virtual machine migration command is a network node monitoring module of the management node according to each service node in the cluster system.

24所收集的网络流量信息,确定集群系统内虚拟机 23之间的数据流向关系, 根据虚拟机迁移策略和集群系统内虚拟机 23之间的数据流向关系向其上运 行有所确定的待迁移虚拟机 23的 VMM22下发的。 24 collected network traffic information to determine the data flow relationship between the virtual machines 23 in the cluster system, According to the data flow relationship between the virtual machine migration policy and the virtual machine 23 in the cluster system, the VMM 22 of the virtual machine 23 to be migrated is sent to the virtual machine 23 to be migrated.

在此基础上, 网络流量监控模块 24可以与 VMM22集成于一体。  Based on this, the network traffic monitoring module 24 can be integrated with the VMM 22.

进一步的, VMM22具体用于: 取样获取虚拟机列表中的虚拟机 23的数 据包, 根据数据包的 IP信息和对应数量, 统计对应虚拟机 23的流入源 IP地 址及对应的流量数据、 流出目标 IP地址及对应的流量数据, 并将相应的统计 结果输出给网络流量监控模块 24, 以及在收到虚拟机迁移命令后, 根据虚拟 机迁移命令将其上运行的待迁移虚拟机 23迁移到对应的目标业务节点上; 网络流量监控模块 24具体用于: 统计超过流量阈值的虚拟机列表, 并将 所述虚拟机列表输出给所述 VMM22;根据来自所述 VMM22的统计结果得到 数据流向记录, 所述数据流向记录包括虚拟机标识信息, 流入源 IP地址以及 对应的流量数据、 流出目标 IP地址以及对应的流量数据, 所述虚拟机标识信 息包括虚拟机名和虚拟机 IP地址;  Further, the VMM 22 is specifically configured to: obtain a data packet of the virtual machine 23 in the virtual machine list, and collect statistics on the inflow source IP address and the corresponding traffic data and the outflow target of the corresponding virtual machine 23 according to the IP information and the corresponding quantity of the data packet. The IP address and the corresponding traffic data are output to the network traffic monitoring module 24, and after the virtual machine migration command is received, the virtual machine 23 to be migrated running on the virtual machine is migrated to the corresponding virtual machine migration command. The network traffic monitoring module 24 is configured to: collect a virtual machine list that exceeds the traffic threshold, and output the virtual machine list to the VMM 22; and obtain a data flow direction record according to the statistical result from the VMM 22, The data flow direction record includes virtual machine identification information, a source IP address and corresponding traffic data, an outbound destination IP address, and corresponding traffic data, where the virtual machine identification information includes a virtual machine name and a virtual machine IP address;

具体的, 统计超过流量阈值的虚拟机列表, 并将虚拟机列表输出给 VMM22; 根据来自 VMM22的统计结果生成或更新数据流向记录, 数据流向 记录包括虚拟机标识信息, 流入源 IP地址及对应的流量数据、 流出目标 IP 地址及对应的流量数据, 虚拟机标识信息包括虚拟机名和虚拟机 IP地址。  Specifically, the virtual machine list exceeding the traffic threshold is counted, and the virtual machine list is output to the VMM 22; the data flow direction record is generated or updated according to the statistical result from the VMM 22, and the data flow direction record includes the virtual machine identification information, the inflow source IP address, and the corresponding Traffic data, outgoing destination IP address, and corresponding traffic data. The virtual machine identification information includes the virtual machine name and the virtual machine IP address.

其中, 各业务节点生成数据流向表(数据流向记录的一种表现载体) 的 具体过程可参见图 3所示, 包括:  The specific process of each service node generating a data flow direction table (a data flow to a performance carrier of the record) can be seen in FIG. 3, including:

a、 虚拟机监控单元 VMM获取虚拟机的网络流量;  a, the virtual machine monitoring unit VMM obtains the network traffic of the virtual machine;

b、 网络流量监控模块判断流量是否超过阈值, 是, 则执行 c, 否, 则流 程结束。  b. The network traffic monitoring module determines whether the traffic exceeds the threshold. If yes, execute c. Otherwise, the process ends.

c、 虚拟机监控单元 VMM对虚拟机的数据包进行分析;  c. The virtual machine monitoring unit VMM analyzes the data packets of the virtual machine;

d、 网络流量监控模块根据 VMM的分析结果更新数据流向表。  d. The network traffic monitoring module updates the data flow direction table according to the analysis result of the VMM.

具体的,网络流量监控模块 24可以定时或周期性向 VMM22获取当前业 务节点 2上的所有虚拟机 23的网络流量信息,具体包括流入源互联网协议 IP 地址及对应的流量、 流出目标 IP地址以及对应的流量, 虚拟机标识信息包括 虚拟机名和虚拟机 IP地址;网络流量监控模块 24遍历虚拟机 23的网络流量 数据,如果发现某个虚拟机 23的流出或流入流量超过流量阈值, 则记录该虚 拟机 23, 如果符合条件的虚拟机 23数量为 0, 本次监控流程结束。 Specifically, the network traffic monitoring module 24 can obtain the network traffic information of all the virtual machines 23 on the current service node 2 to the VMM 22 periodically or periodically, including the inbound source Internet Protocol IP. The address and the corresponding traffic, the outgoing destination IP address, and the corresponding traffic, the virtual machine identification information includes the virtual machine name and the virtual machine IP address; the network traffic monitoring module 24 traverses the network traffic data of the virtual machine 23, if a virtual machine 23 is found If the outbound or inbound traffic exceeds the traffic threshold, the virtual machine 23 is recorded. If the number of the qualified virtual machines 23 is 0, the monitoring process ends.

网络流量监控模块 24将流量超过阈值的虚拟机 23告知 VMM22 , VMM22 监控一定时间内对应虚拟机 23的所有数据包, 并逐一对其分析, 统计每个数 据包的流入源 IP地址、 流出目标 IP地址, 并把相同数据包进行计数, 从而 计算出该时间段内每个流入源 IP地址流入该虚拟机 23、该虚拟机 23流出目 标 IP地址的流量大小。 VMM22把统计结果打包输出给网络流量监控模块 24, 网络流量监控 24将 VMM22统计结果填充到数据流向表中。 如果虚拟机 23 未存在于数据流向表中, 则在该数据流向表中创建新的表项, 并把对应虚拟 机信息、 流入源 I P地址以及对应的流量数据、 流出目标 I P地址以及对应的 流量数据填入新表项; 如果虚拟机 23已经在数据流向表中, 则更新该数据流 向表中的相应数据; 更新完毕后,将当前周期内流量超过阈值的虚拟机 23与 数据流向表已经存在的表项——对比, 把已经不存在数据流量的表项从数据 流向表中剔除。  The network traffic monitoring module 24 informs the VMM 22 of the virtual machine 23 whose traffic exceeds the threshold. The VMM 22 monitors all the data packets corresponding to the virtual machine 23 in a certain period of time, analyzes them one by one, and counts the inflow source IP address and the outbound destination IP of each data packet. The address, and the same data packet is counted, thereby calculating the traffic volume of each incoming source IP address flowing into the virtual machine 23 and the virtual machine 23 flowing out of the destination IP address during the time period. The VMM 22 packs the statistical result and outputs it to the network traffic monitoring module 24, and the network traffic monitoring 24 fills the VMM22 statistical result into the data flow direction table. If the virtual machine 23 does not exist in the data flow direction table, create a new entry in the data flow table, and input the corresponding virtual machine information, the inflow source IP address, the corresponding traffic data, the outgoing destination IP address, and the corresponding traffic. The data is filled in with a new entry; if the virtual machine 23 is already in the data flow to the table, the data is updated to the corresponding data in the table; after the update is completed, the virtual machine 23 with the traffic exceeding the threshold in the current period and the data flow table already exist. The entry - comparison, removes the entry that does not have data traffic from the data flow to the table.

本实施例提供的业务节点, 其上包括的各个功能模块以及各个功能模块 的具体功能及操作过程均可参见本发明提供的集群系统中业务节点的相关描 述, 不再赘述。  For the service modules provided in this embodiment, the specific functions and operation procedures of the function modules and the function modules of the present invention can be referred to the related description of the service nodes in the cluster system provided by the present invention, and details are not described herein.

图 4为本发明提供的管理节点第一实施例的结构示意图, 如图 4所示, 包括: 网络流量收集单元 111和虚拟机部署调整单元 112;  4 is a schematic structural diagram of a first embodiment of a management node according to the present invention. As shown in FIG. 4, the network traffic collection unit 111 and the virtual machine deployment adjustment unit 112 are included;

网络流量收集单元 111 , 用于向集群系统内的各业务节点收集宿主在业 务节点上的虚拟机的网络流量信息;  The network traffic collection unit 111 is configured to collect network traffic information of the virtual machine hosted on the service node to each service node in the cluster system.

虚拟机部署调整单元 112, 用于根据虚拟机的网络流量信息确定集群系 统内虚拟机之间的数据流向关系, 根据虚拟机迁移策略和集群系统内虚拟机 之间的数据流向关系, 向其上运行有所确定的待迁移虚拟机的虚拟机监控单 元 VMM下发虚拟机迁移命令或者输出对应的虚拟机迁移建议。 The virtual machine deployment adjustment unit 112 is configured to determine, according to the network traffic information of the virtual machine, a data flow relationship relationship between the virtual machines in the cluster system, and according to the relationship between the virtual machine migration policy and the data flow relationship between the virtual machines in the cluster system, Run a virtual machine monitoring list with a certain VM to be migrated The virtual machine migration command or the corresponding virtual machine migration proposal is sent by the VMM.

需要说明的是, 网络流量收集单元 111和虚拟机部署调整单元 112可以 集成于一体, 即在管理节点上设置一个虚拟机部署调整模块。  It should be noted that the network traffic collection unit 111 and the virtual machine deployment adjustment unit 112 may be integrated, that is, a virtual machine deployment adjustment module is set on the management node.

图 5为本发明提供的管理节点第二实施例的结构示意图, 如图 5所示, 包括: 网络流量收集单元 111和虚拟机部署调整单元 112;  5 is a schematic structural diagram of a second embodiment of a management node according to the present invention. As shown in FIG. 5, the network traffic collection unit 111 and the virtual machine deployment adjustment unit 112 are included;

在此基础上, 进一步的,  On this basis, further,

网络流量收集单元 111具体用于: 向所述集群系统内各业务节点上的网 络流量监控模块获取数据流向记录,所述数据流向记录包括虚拟机标识信息, 流入源互联网协议 IP地址以及对应的流量数据、 流出目标 IP地址以及对应 的流量数据;  The network traffic collection unit 111 is specifically configured to: obtain a data flow direction record to the network traffic monitoring module on each service node in the cluster system, where the data flow direction record includes virtual machine identification information, an inflow source Internet Protocol IP address, and a corresponding traffic. Data, outgoing destination IP address, and corresponding traffic data;

具体的, 向集群系统内各业务节点上的网络流量监控模块获取数据流向 表, 该数据流向表包括虚拟机标识信息, 流入源互联网协议 IP地址以及对应 的流量数据、 流出目标 I P地址以及对应的流量数据;  Specifically, the data flow direction module is obtained from the network traffic monitoring module on each service node in the cluster system, where the data flow direction table includes the virtual machine identification information, the source Internet protocol IP address, the corresponding traffic data, the outgoing destination IP address, and the corresponding Traffic data;

相应的, 虚拟机部署调整单元 112具体用于: 对所述数据流向记录进行 匹配处理, 得到所述集群系统内虚拟机之间的数据流向关系, 所述集群系统 内虚拟机之间的数据流向关系用流向匹配记录表示, 包括: 源虚拟机标识信 息、 目标虚拟机标识信息及对应流量大小; 并根据所述集群系统的流向匹配 记录确定满足虚拟机迁移策略的待迁移虚拟机, 向其上运行有所确定的待迁 移虚拟机的 VMM下发虚拟机迁移命令或者输出对应的虚拟机迁移建议; 具体的, 根据数据流向表确定集群系统内虚拟机之间的数据流向关系, 生成或更新集群系统的流向匹配表, 该流向匹配表包括: 存在数据流向关系 的源虚拟机标识信息、 目标虚拟机标识信息及对应流量; 并 居流向匹配表 确定满足虚拟机迁移策略的待迁移虚拟机, 向其上运行有所确定的待迁移虚 拟机的 VMM 下发虚拟机迁移命令或者输出对应的虚拟机迁移建议; 其中, 该虚拟机迁移策略包括: 虚拟机迁移数量小于等于设定阈值, 和 /或, 根据减 少的网络流量大小确定迁移顺序, 和 /或, 满足虚拟机迁移数量最小化和减少 的网络流量最大化的平衡关系, 和 /或, 将存在数据流向关系的虚拟机迁移到 同一目标业务节点, 目标业务节点为根据集群系统内存在数据流向关系的多 个虚拟机各自宿主的业务节点的物理资源情况所确定的。 Correspondingly, the virtual machine deployment adjustment unit 112 is specifically configured to: perform matching processing on the data flow to the record, obtain a data flow relationship relationship between the virtual machines in the cluster system, and data flow between the virtual machines in the cluster system. The relationship is represented by the flow matching record, including: source virtual machine identification information, target virtual machine identification information, and corresponding traffic size; and determining, according to the flow matching record of the cluster system, the virtual machine to be migrated that satisfies the virtual machine migration policy, and The VMM that runs the determined VM to be migrated delivers the VM migration command or outputs the corresponding VM migration proposal. Specifically, the data flow direction table determines the data flow relationship between the VMs in the cluster system, and generates or updates the cluster. a flow matching table of the system, where the flow matching table includes: source virtual machine identification information of the data flow direction relationship, target virtual machine identification information, and corresponding traffic; and the flow matching table determines the virtual machine to be migrated that satisfies the virtual machine migration policy, The VMM that runs the determined VM to be migrated is delivered. The virtual machine migration policy includes: the virtual machine migration policy is less than or equal to a set threshold, and/or the migration sequence is determined according to the reduced network traffic size, and/or, Minimize and reduce the number of virtual machine migrations The network traffic is maximized in a balanced relationship, and/or, the virtual machine that has the data flow to the relationship is migrated to the same target service node, and the target service node is the service node hosted by each of the plurality of virtual machines according to the data flow relationship relationship in the cluster system. The physical resource situation is determined.

进一步的, 虚拟机部署调整单元 112可以包括:  Further, the virtual machine deployment adjustment unit 112 may include:

流向匹配子单元 1121 , 用于对所述数据流向记录进行匹配处理, 得到所 述集群系统内虚拟机之间的数据流向关系, 所述集群系统内虚拟机之间的数 据流向关系用流向匹配记录表示, 包括: 源虚拟机标识信息、 目标虚拟机标 识信息及对应流量大小;  And a flow matching matching sub-unit 1121, configured to perform matching processing on the data flow to the record, to obtain a data flow relationship relationship between the virtual machines in the cluster system, and a data flow direction matching flow record between the virtual machines in the cluster system The indication includes: source virtual machine identification information, target virtual machine identification information, and corresponding traffic size;

具体的, 根据数据流向表确定集群系统内虚拟机之间的数据流向关系, 生成或更新集群系统的流向匹配表,每个表项记录一组虚拟机数据流向关系, 包括源虚拟机标识信息、 目标虚拟机标识信息及对应流量大小。  Specifically, the data flow direction relationship between the virtual machines in the cluster system is determined according to the data flow direction table, and the flow direction matching table of the cluster system is generated or updated, and each entry records a set of virtual machine data flow direction relationships, including source virtual machine identification information, Target virtual machine identification information and corresponding traffic size.

迁移决策子单元 1122, 用于根据所述集群系统的流向匹配记录(具体可 以是流向匹配表 )生成虚拟机迁移加权表, 该虚拟机迁移加权表包括流向匹 配记录中的所有虚拟机的标识信息、 目标业务节点的标识信息及对应权值, 该权值为所述虚拟机与目标业务节点上的各虚拟机之间的数据流量之和; 若 虚拟机迁移加权表中存在一个最大权值, 则从虚拟机迁移加权表中选取权值 最大的虚拟机加入虚拟机迁移列表; 若虚拟机迁移加权表中存在多个最大权 值, 则将虚拟机内存值与目标业务节点剩余内存值比值最大的虚拟机加入虚 拟机迁移列表; 根据虚拟机迁移列表向其上运行有权值最大的虚拟机或者比 值最大的虚拟机的 VMM 下发虚拟机迁移命令或者输出对应的虚拟机迁移建 议。  The migration decision sub-unit 1122 is configured to generate a virtual machine migration weighting table according to the flow direction matching record (specifically, a flow direction matching table) of the cluster system, where the virtual machine migration weighting table includes identification information of all virtual machines in the matching matching record. The identification information of the target service node and the corresponding weight, the weight is the sum of the data traffic between the virtual machine and each virtual machine on the target service node; if there is a maximum weight in the virtual machine migration weighting table, The virtual machine with the largest weight is selected from the virtual machine migration weighting table to join the virtual machine migration list. If there are multiple maximum weights in the virtual machine migration weighting table, the virtual machine memory value and the target service node remaining memory value ratio are the largest. The virtual machine is added to the virtual machine migration list. The virtual machine migration command or the corresponding virtual machine migration proposal is delivered to the VMM that runs the virtual machine with the highest value or the virtual machine with the highest value.

其中, 管理节点为独立的物理服务器, 或者, 管理节点还具有业务节点 的功能, 或者管理节点为业务节点上的虚拟机; 该业务节点可以为独立的物 理服务器。  The management node is an independent physical server, or the management node has the function of a service node, or the management node is a virtual machine on the service node; the service node can be an independent physical server.

本实施例提供的管理节点, 其上包括的各个功能模块以及各个功能模块 的具体功能及操作过程均可参见本发明提供的集群系统中管理节点的相关描 述, 不再赘述。 For the specific functions and operation procedures of the management modules provided in this embodiment, and the specific functions and operation procedures of the respective functional modules, refer to the related description of the management nodes in the cluster system provided by the present invention. Said, no longer repeat them.

图 6为本发明提供的基于网络数据流向的虚拟机迁移方法第一实施例的 流程图, 如图 6所示, 该方法包括:  FIG. 6 is a flowchart of a first embodiment of a virtual machine migration method based on network data flow according to the present invention. As shown in FIG. 6, the method includes:

5101、 向集群系统内的各业务节点收集宿主在业务节点上的虚拟机的网 络流量信息;  5101. Collect network traffic information of a virtual machine hosted on the service node to each service node in the cluster system.

5102、 根据虚拟机的网络流量信息确定集群系统内虚拟机之间的数据流 向关系;  5102. Determine, according to network traffic information of the virtual machine, a data flow relationship between virtual machines in the cluster system.

5103、 根据虚拟机迁移策略和所述集群系统内虚拟机之间的数据流向关 系, 向其上运行有所确定的待迁移虚拟机的虚拟机监控单元 VMM 下发虚拟 机迁移命令或者输出对应的虚拟机迁移建议。  5103. Send a virtual machine migration command or output corresponding to the virtual machine monitoring unit VMM that runs the determined virtual machine to be migrated according to the virtual machine migration policy and the data flow relationship between the virtual machines in the cluster system. Virtual machine migration recommendations.

本实施例以上步骤的执行主体为虚拟机部署调整模块, 该虚拟机部署调 整模块可以部署在集群系统中的管理节点上。 该网络流量监控模块可与所在 业务节点上的 VMM交互, 获取所在业务节点上每个虚拟机的网络流量信息。  The execution body of the above steps is the virtual machine deployment adjustment module, and the virtual machine deployment adjustment module can be deployed on the management node in the cluster system. The network traffic monitoring module can interact with the VMM on the service node to obtain network traffic information of each virtual machine on the service node.

虚拟机部署调整模块与集群系统中各业务节点 (具体可以是业务节点上 部署的网络流量监控模块)进行数据交互, 收集宿主在各业务节点上的虚拟 机的网络流量信息。 其中, 该网络流量信息可以包括: 虚拟机标识信息, 例 如: 虚拟机名和 /虚拟机的 IP地址等信息。 还可以包括: 流入源互联网协议 IP地址以及流出目标 IP地址,即可以从网络流量信息中反映出存在数据流向 关系的虚拟机对。 进一步的, 网络流量信息中还可以包括流入源 I P地址对应 的流量, 流出目标 IP地址对应的流量, 以便进一步获知数据流向关系的虚拟 机对之间的流量大小, 以便在下发命令或者给出迁移建议时对存在数据流向 关系的虚拟机对之间的流量大小加以考虑。  The virtual machine deployment adjustment module performs data interaction with each service node in the cluster system (specifically, the network traffic monitoring module deployed on the service node), and collects network traffic information of the virtual machine hosted on each service node. The network traffic information may include: virtual machine identification information, such as: a virtual machine name and/or a virtual machine IP address. It may also include: flowing into the source Internet Protocol IP address and the outgoing destination IP address, that is, the virtual machine pair having the data flow relationship relationship may be reflected from the network traffic information. Further, the network traffic information may include the traffic corresponding to the source IP address and the traffic corresponding to the destination IP address, so as to further know the traffic volume between the virtual machine pairs of the data flow relationship, so as to issue a command or give a migration. It is recommended to consider the amount of traffic between pairs of virtual machines that have data flow relationships.

虚拟机部署调整模块可以周期性或定时地向集群系统内的多台业务节点 收集其上宿主的虚拟机的网络流量信息, 根据虚拟机的网络流量信息确定集 群系统内虚拟机之间的数据流向关系。 其中, 数据流向关系表示了虚拟机之 间的网络数据走向, 确定产生流量的虚拟机对。 虚拟机部署调整模块获取集 群系统内虚拟机之间的数据流向关系后, 可以根据虚拟机迁移策略和集群系 统内虚拟机之间的数据流向关系确定能减轻网络压力的最优迁移策略, 从而 向其上运行有所确定的待迁移虚拟机的 VMM 下发虚拟机迁移命令或者输出 对应的虚拟机迁移建议。 The virtual machine deployment adjustment module can periodically or periodically collect network traffic information of the virtual machine hosted by the multiple service nodes in the cluster system, and determine the data flow direction between the virtual machines in the cluster system according to the network traffic information of the virtual machine. relationship. The data flow relationship indicates the network data direction between the virtual machines, and determines the virtual machine pair that generates the traffic. Virtual machine deployment adjustment module acquisition set After the data flow relationship between the virtual machines in the group system, the optimal migration strategy that can alleviate the network pressure can be determined according to the relationship between the virtual machine migration policy and the data flow relationship between the virtual machines in the cluster system, thereby determining the operation on the network. The VMM of the VM to be migrated delivers the VM migration command or outputs the corresponding VM migration proposal.

其中, 可以尽可能少的迁移数量为前提, 判断存在数据流量的虚拟机对 能够迁移到一台物理服务器上, 即对处理器是否兼容等情况加以考虑。 另外, 可以对用户的需求加以考虑,如果能够获知各虚拟机对之间的数据流量大小, 则还可以对数据流量大小加以考虑, 获得最优迁移策略。 可以直接向其上运 行有所确定的待迁移虚拟机的 VMM 下发迁移命令, 也可以向其上运行有所 确定的待迁移虚拟机的 VMM提供迁移建议。  Among them, the number of migrations with as few as possible can be judged, and it can be judged that the virtual machine pair with data traffic can be migrated to one physical server, that is, whether the processor is compatible or not. In addition, the user's needs can be considered. If the data traffic size between the virtual machine pairs can be known, the data traffic size can also be considered to obtain an optimal migration strategy. You can directly send the migration command to the VMM on which the VM to be migrated is running, or you can provide the migration proposal to the VMM on which the VM to be migrated is running.

虚拟机监控单元 VMM接收到虚拟机部署调整模块下发的虚拟机迁移命 令, 可以将待迁移虚拟机迁移到目标业务节点上, 使得存在数据流向关系的 多个虚拟机宿主在同一业务节点上。  The virtual machine monitoring unit receives the virtual machine migration command from the virtual machine deployment and adjustment module. The virtual machine can be migrated to the target service node, so that multiple virtual machines with data flow to the relationship are hosted on the same service node.

本实施例提供的基于网络数据流向的虚拟机迁移方法, 根据集群系统内 的多台业务节点上的虚拟机的网络流量信息确定所述集群系统内虚拟机之间 的数据流向关系, 根据虚拟机迁移策略和集群系统内虚拟机之间的数据流向 关系下发虚拟机迁移命令或者输出对应的虚拟机迁移建议; 相应的, 响应于 该虚拟机迁移命令, 将所述待迁移虚拟机迁移到对应的目标业务节点上, 使 得存在数据流向关系的多个虚拟机宿主在同一业务节点上, 从而减少集群系 统中虚拟机之间经过物理网卡的数据传输, 能够有效降低集群系统内的虚拟 机之间的通信流量, 减少网络负担。  The virtual machine migration method based on the network data flow provided by the embodiment determines the data flow relationship between the virtual machines in the cluster system according to the network traffic information of the virtual machines on the multiple service nodes in the cluster system, according to the virtual machine. The migration policy and the data flow relationship between the virtual machines in the cluster system are sent to the virtual machine migration command or the corresponding virtual machine migration proposal is output; correspondingly, the virtual machine to be migrated is migrated to the corresponding one in response to the virtual machine migration command. On the target service node, the multiple virtual machines that have data flow to the relationship are hosted on the same service node, thereby reducing the data transmission between the virtual machines in the cluster system through the physical network card, thereby effectively reducing the virtual machines in the cluster system. Communication traffic, reducing network load.

在前一实施例的基础上, 参见图 7, S102中根据虚拟机的网络流量信息 确定集群系统内虚拟机之间的数据流向关系的过程可以具体为:  On the basis of the previous embodiment, referring to FIG. 7, the process of determining the data flow relationship between the virtual machines in the cluster system according to the network traffic information of the virtual machine in S102 may be specifically as follows:

S201、 对收集的所述虚拟机的网络流量信息进行匹配处理;  S201. Perform matching processing on collected network traffic information of the virtual machine.

所述网络流量信息包括虚拟机标识信息, 流入源互联网协议 IP地址以及 对应的流量数据、 流出目标 IP地址以及对应的流量数据, 所述虚拟机标识信 息包括虚拟机名和虚拟机 IP地址; The network traffic information includes virtual machine identification information, an inbound source Internet Protocol IP address, and corresponding traffic data, an outbound destination IP address, and corresponding traffic data, and the virtual machine identification information. The information includes the virtual machine name and the virtual machine IP address;

S202、 根据匹配处理结果得到集群系统内虚拟机之间的数据流向关系; 所述集群系统内虚拟机之间的数据流向关系用流向匹配记录表示, 包括: 源虚拟机标识信息、 目标虚拟机标识信息及对应流量大小;  S202. Obtain a data flow relationship relationship between virtual machines in the cluster system according to the matching processing result. The data flow relationship between the virtual machines in the cluster system is represented by a flow matching record, including: source virtual machine identification information, target virtual machine identifier. Information and corresponding traffic size;

S103中根据虚拟机迁移策略和集群系统内虚拟机之间的数据流向关系, 向其上运行有所确定的待迁移虚拟机的虚拟机监控单元 VMM 下发虚拟机迁 移命令或者输出对应的虚拟机迁移建议, 包括:  S103 delivers a virtual machine migration command or outputs a corresponding virtual machine to the virtual machine monitoring unit VMM that runs the determined virtual machine to be migrated according to the data flow relationship between the virtual machine migration policy and the virtual machine in the cluster system. Migration recommendations, including:

根据集群系统的流向匹配记录确定满足虚拟机迁移策略的待迁移虚拟 机, 向其上运行有所确定的待迁移虚拟机的 VMM 下发虚拟机迁移命令或者 输出对应的虚拟机迁移建议。  Determine the VM to be migrated that meets the VM migration policy based on the flow matching record of the cluster system, and send the VM migration command to the VMM that runs the determined VM to be migrated or output the corresponding VM migration proposal.

参见图 8、 本实施例给出对收集的所述虚拟机的网络流量信息进行匹配 处理的一种较佳的实现方式:  Referring to FIG. 8, a preferred implementation manner for performing matching processing on the collected network traffic information of the virtual machine is given in this embodiment:

5301、 获取当前虚拟机的 IP地址信息;  5301. Obtain an IP address information of a current virtual machine.

遍历集群系统内的每个虚拟机, 如果虚拟机部署调整模块遍历到虚拟机 A, 虚拟机部署调整模块首先找到虚拟机 A对应的数据流向表(称为流向表 A ) , 从中获取虚拟机 A的 IP地址信息。 其中, 数据流向表即为网络流量信 息, 该数据流向表中包括: 虚拟机标识信息、 流入源 IP地址以及对应的流量 数据、 流出目标 IP地址以及对应的流量数据。  Traversing each virtual machine in the cluster system, if the virtual machine deployment adjustment module traverses to virtual machine A, the virtual machine deployment adjustment module first finds the data flow direction table corresponding to virtual machine A (referred to as flow direction table A), and obtains virtual machine A from it. IP address information. The data flow direction table is network traffic information, and the data flow direction table includes: virtual machine identification information, inflow source IP address and corresponding traffic data, outbound destination IP address, and corresponding traffic data.

5302、该 IP地址信息是否存在于其他数据流向表中,是, 则执行 S303、 否则不再执行其他操作;  5302. Whether the IP address information exists in another data flow direction table, if yes, execute S303, otherwise, no other operations are performed;

具体是,在除了流向表 A外的所有数据流向表的流入源 IP地址和流出目 标 IP地址列表中查找虚拟机 A的 IP地址信息。 如果在虚拟机 B的数据流向 表(称为流向表 B ) 中找到虚拟机 A的 IP地址信息。  Specifically, the IP address information of the virtual machine A is searched for in the inflow source IP address and the outbound destination IP address list of all the data flow directions except the flow to the table A. If the data flow direction table of virtual machine B (called flow table B) is found, the IP address information of virtual machine A is found.

5303、 对应虚拟机的 IP地址信息是否存在本数据流向表中且流向是否 匹配, 是, 执行 S304, 否则返回执行 S302。  5303. Whether the IP address information of the corresponding virtual machine exists in the data flow direction table and the flow direction matches, yes, execute S304, otherwise return to execute S302.

在流向表 A的流入源 IP地址和目标流出 IP地址列表中查找虚拟机 B的 IP地址信息, 如果也找到匹配的数据, 并且两者互为流入和流出关系, 流量 数据误差在 5%以内, 则可以视为虚拟机 A与虚拟机 B存在流向关系, 如果 虚拟机 B在虚拟机 A的目标流出 IP地址列表中, 则认为数据流向是虚拟机 A虚拟机 B, 反之则为虚拟机 B虚拟机 A。 如果虚拟机 B的 IP地址信息 在流向表 A中找不到匹配数据, 或者是流量数据误差很大, 则可视为虚拟机 A和虚拟机 B之间不存在流向关系, 则可以继续寻找与虚拟机 A存在流向关 系的下一个虚拟机, 直至遍历完所有业务节点的数据流向表。 Find the virtual machine B in the flow source IP address and the target outgoing IP address list of the flow table A. IP address information, if the matching data is also found, and the two are inflow and outflow relationships, and the traffic data error is within 5%, it can be regarded as the flow relationship between virtual machine A and virtual machine B. If virtual machine B is virtual If the target A of the machine A flows out of the IP address list, the data flow direction is considered to be virtual machine A virtual machine B, and vice versa is virtual machine B virtual machine A. If the IP address information of the virtual machine B cannot find the matching data in the flow direction table A, or the traffic data error is large, it can be regarded that there is no flow relationship between the virtual machine A and the virtual machine B, and the search can continue. Virtual machine A has the next virtual machine in the flow direction until it traverses the data flow table of all service nodes.

S304、 确定匹配关系。  S304. Determine a matching relationship.

以上仅给出了一种对收集的所述虚拟机的网络流量信息进行匹配处理, 但并不以此作为对本发明的限制。  Only the matching processing of the collected network traffic information of the virtual machine is given above, but it is not intended to limit the present invention.

进一步的, 虚拟机迁移策略可以包括: 虚拟机迁移数量小于等于设定阈 值, 和 /或, 根据减少的网络流量大小确定迁移顺序, 和 /或, 满足虚拟机迁移 数量最小化和减少的网络流量最大化的平衡关系, 和 /或, 将存在数据流向关 系的虚拟机迁移到同一目标业务节点, 该目标业务节点为根据集群系统内存 在数据流向关系的多个虚拟机各自宿主的业务节点的物理资源情况所确定 的。  Further, the virtual machine migration policy may include: the number of virtual machine migrations is less than or equal to a set threshold, and/or determining a migration order according to the reduced network traffic size, and/or meeting network traffic that minimizes and reduces the number of virtual machine migrations. Maximizing the balance relationship, and/or migrating the data flow to the relationship virtual machine to the same target service node, the target service node being the physics of the service node hosted by each of the plurality of virtual machines according to the data flow relationship relationship in the cluster system The resource situation is determined.

作为一个较佳的实施方案, S103中根据集群系统的流向匹配记录确定 满足虚拟机迁移策略的待迁移虚拟机, 向其上运行有所确定的待迁移虚拟机 的 VMM下发虚拟机迁移命令或者输出对应的虚拟机迁移建议, 可以为: 根据集群系统的流向匹配记录(即集群系统内虚拟机之间的数据流向关 系)生成虚拟机迁移加权表, 虚拟机迁移加权表包括所述流向匹配表中的所 有虚拟机的标识信息、 目标业务节点的标识信息及对应权值, 该权值为虚拟 机与目标业务节点上的各虚拟机之间的数据流量之和;  As a preferred implementation, in S103, the VM to be migrated that meets the virtual machine migration policy is determined according to the flow matching record of the cluster system, and the virtual machine migration command is sent to the VMM on which the determined virtual machine to be migrated is run or The virtual machine migration weighting table is generated according to the flow direction matching record of the cluster system (that is, the data flow relationship between the virtual machines in the cluster system), and the virtual machine migration weighting table includes the flow direction matching table. The identification information of all the virtual machines in the medium, the identification information of the target service node, and the corresponding weight, the weight is the sum of the data traffic between the virtual machine and each virtual machine on the target service node;

根据虚拟机迁移加权表中的权值大小将对应的虚拟机加入虚拟机迁移列 表;  Add the corresponding VM to the VM migration list according to the weight in the VM migration weight table.

根据虚拟机迁移列表向其上运行有所确定的待迁移虚拟机的 VMM 下发 虚拟机迁移命令或者输出对应的迁移建议。 According to the VM migration list, the VMM that runs the determined VM to be migrated is delivered. The virtual machine migration command or the corresponding migration proposal.

其中, 根据虚拟机迁移加权表中的权值大小将对应的虚拟机加入虚拟机 迁移列表可以具体包括:  The adding the virtual machine to the virtual machine migration list according to the weight in the virtual machine migration weighting table may specifically include:

若虚拟机迁移加权表中存在一个最大权值, 则从虚拟机迁移加权表中选 取权值最大的虚拟机加入虚拟机迁移列表; 若虚拟机迁移加权表中存在多个 最大权值, 则将虚拟机内存值与目标业务节点剩余内存值比值最大的虚拟机 加入虚拟机迁移列表。  If there is a maximum weight in the virtual machine migration weighting table, the virtual machine with the largest weight is selected from the virtual machine migration weighting table to join the virtual machine migration list; if there are multiple maximum weights in the virtual machine migration weighting table, The virtual machine with the largest ratio of virtual machine memory value to the remaining memory value of the target service node is added to the virtual machine migration list.

另外, 若虚拟机迁移列表中任一虚拟机对应的目的业务节点的剩余内存 不足, 则对虚拟机迁移列表中下一虚拟机进行迁移。  In addition, if the remaining memory of the destination service node corresponding to any virtual machine in the virtual machine migration list is insufficient, the next virtual machine in the virtual machine migration list is migrated.

参见图 9, 本实施例给出根据所述集群系统的流向匹配记录确定满足虚 拟机迁移策略的待迁移虚拟机,向其上运行有所确定的待迁移虚拟机的 VMM 下发虚拟机迁移命令或者输出对应的虚拟机迁移建议的一种可行的实施方 式, 具体的:  Referring to FIG. 9, the embodiment provides that the virtual machine to be migrated is determined according to the flow matching record of the cluster system, and the virtual machine migration command is delivered to the VMM that runs the determined virtual machine to be migrated. Or output a feasible implementation of the corresponding virtual machine migration proposal, specifically:

S401、 初始化虚拟机迁移列表;  S401. Initialize a virtual machine migration list.

虚拟机部署调整模块可以首先生成一张空的虚拟机迁移加权表。 该虚拟 机迁移加权表用来存储虚拟机迁移到某台业务节点的权值, 以此来区分不同 的迁移所带来的不同效果。该虚拟机迁移加权表的表项中记录了虚拟机标识、 业务节点的标识信息以及对应权值。 其中, 业务节点的标识信息可以为 IP地 址和 /或主机名等信息, 权值为对应虚拟机迁移到对应业务节点上所带来的流 量数据的加权和。  The virtual machine deployment adjustment module can first generate an empty virtual machine migration weighting table. The virtual machine migration weighting table is used to store the weight of the virtual machine migrated to a service node, so as to distinguish the different effects brought by different migrations. The virtual machine identifier, the identification information of the service node, and the corresponding weight are recorded in the entry of the virtual machine migration weighting table. The identification information of the service node may be information such as an IP address and/or a host name, and the weight is a weighted sum of the traffic data brought by the virtual machine to the corresponding service node.

S402、 计算每个虚拟机迁移权值;  S402. Calculate a migration weight of each virtual machine.

虚拟机部署调整装置可以遍历流向匹配表中的每一项, 把可能存在的迁 移行为的相应数据添加到迁移加权表中。 例如: 流向匹配表中存在流向关系: 虚拟机 A→虚拟机 B , 对应流量为 X , 虚拟机 A迁到虚拟机 B所在业务节点 或是虚拟机 B迁到虚拟机 A所在业务节点的效果相同, 虚拟机部署调整模块 分别判断虚拟机 A和虚拟机 B所在业务节点的剩余资源能否满足虚拟机的迁 移要求, 然后把对应流量 X分别累加到满足要求的表项中。 The virtual machine deployment adjustment device can traverse each item in the flow matching table and add corresponding data of the migration behavior that may exist to the migration weighting table. For example, there is a flow relationship in the flow matching table: virtual machine A→virtual machine B, the corresponding traffic is X, virtual machine A moves to the service node where virtual machine B is located, or virtual machine B moves to the same service node where virtual machine A is located. The virtual machine deployment adjustment module determines whether the remaining resources of the service node where the virtual machine A and the virtual machine B are located can satisfy the virtual machine migration. Move the request, and then add the corresponding traffic X to the entry that meets the requirements.

5403、 选出权值最大的一次迁移;  5403. Selecting a migration with the largest weight;

虚拟机部署调整模块遍历虚拟机迁移加权表, 选取该表中权值最大的一 次迁移行为, 如果存在多个最大权值, 则可以按照贪心算法, 选取虚拟机内 存值与目标业务节点剩余内存值比值最大的一项。  The virtual machine deployment adjustment module traverses the virtual machine migration weighting table, and selects a migration behavior with the largest weight in the table. If there are multiple maximum weights, the virtual machine memory value and the remaining service memory value of the target service node may be selected according to the greedy algorithm. The one with the largest ratio.

5404、 将选出的迁移对应的虚拟机标识加入虚拟机迁移列表;  5404. Add the VM ID corresponding to the selected migration to the VM migration list.

虚拟机部署调整模块把选取出来的当前最优的迁移对应的虚拟机标识信 息加入虚拟机迁移列表。  The virtual machine deployment adjustment module adds the selected virtual machine identification information corresponding to the current optimal migration to the virtual machine migration list.

5405、 更新虚拟机迁移加权表;  5405. Update a virtual machine migration weighting table.

虚拟机部署调整模块根据流向匹配表, 把与加入虚拟机迁移列表的虚拟 机相关的权值从虚拟机迁移加权表中去除; 同时, 虚拟机部署调整模块根据 集群系统中各业务节点的剩余资源状况, 增减相关表项的权值; 如果某个表 项权值减为 0 , 则把相关表项从加权表中删除。  The virtual machine deployment adjustment module removes the weight associated with the virtual machine that joins the virtual machine migration list from the virtual machine migration weighting table according to the flow matching table. Meanwhile, the virtual machine deployment adjustment module is configured according to the remaining resources of each service node in the cluster system. Status, increase or decrease the weight of the related entry; If the weight of an entry is reduced to 0, the related entry is removed from the weighting table.

5406、 判断虚拟机迁移加权表是否为空, 是, 则执行 S407, 否, 则返 回执行 S403;  5406, determining whether the virtual machine migration weighting table is empty, if yes, executing S407, otherwise, returning to execute S403;

虚拟机部署调整模块判断虚拟机迁移加权表是否还存在表项, 是则继续 添加下一个次优的迁移对应的虚拟机标识信息,否则待迁移虚拟机选取结束。  The virtual machine deployment adjustment module determines whether the virtual machine migration weighting table still has an entry. If yes, the virtual machine identification information corresponding to the next suboptimal migration is added. Otherwise, the virtual machine to be migrated is selected.

其中, 虚拟机部署调整模块可以根据用户预先设定的迁移数量, 按照虚 拟机迁移列表中的按序向其上运行有所确定的待迁移虚拟机的 VMM 下发虚 拟机迁移命令, 或者向该 VMM推送虚拟机迁移建议。  The virtual machine deployment adjustment module may deliver a virtual machine migration command to the VMM that runs the determined virtual machine to be migrated according to the number of migrations preset by the user, or according to the order in the virtual machine migration list. VMM pushes virtual machine migration recommendations.

5407、 下达对应虚拟机的迁移命令。  5407. Release a migration command corresponding to the virtual machine.

作为对虚拟机迁移方案的计算方法的补充需要说明的是, 虚拟机迁移命 令或者所述虚拟机迁移策略每次中选取的虚拟机迁移数量可以由用户预先自 定义, 一旦如果加入虚拟机迁移列表中的虚拟机数量达到用户设定的上限, 则可以停止选取待迁移的虚拟机迁移的选取, 直接进入下个迁移虚拟机的周 期。 本实施例提供的基于网络数据流向的虚拟机迁移方法, 可以适用于视频 教学、 可视会议或在线点播等多种对网络传输要求比较高的应用环境。 确定 集群系统中不同业务节点上存在网络数据流量的虚拟机对, 将不同业务节点 上的虚拟机尽量迁移至同一业务节点(即同一物理服务器), 使虚拟机之间的 数据交换转变为域间通信, 无需经过物理网卡, 能够有效降低集群网络内的 虚拟机之间的通信流量, 减少网络负担。 In addition to the calculation method of the virtual machine migration solution, it should be noted that the virtual machine migration command or the virtual machine migration policy selected in the virtual machine migration policy may be pre-defined by the user in advance, once the virtual machine migration list is added. If the number of virtual machines in the user reaches the upper limit set by the user, you can stop selecting the migration of the VM to be migrated and go directly to the next migration VM. The virtual machine migration method based on the network data flow provided in this embodiment can be applied to various application environments with high requirements for network transmission, such as video teaching, video conferencing, or online on-demand. Determining the virtual machine pairs with network data traffic on different service nodes in the cluster system, and migrating the virtual machines on different service nodes to the same service node (that is, the same physical server), so that the data exchange between the virtual machines is changed to inter-domain. Communication eliminates the need for physical network cards, which effectively reduces communication traffic between virtual machines in the cluster network and reduces network load.

本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流 程, 是可以通过计算机程序来指令相关的硬件来完成, 所述的程序可存储于 一计算机可读取存储介质中, 该程序在执行时, 可包括如上述各方法的实施 例的流程。 其中, 所述的存储介质可为磁碟、 光盘、 只读存储记忆体 ( Read-Only Memory, ROM )或随机存者 i己忆体( Random Access Memory, RAM )等。  A person skilled in the art can understand that all or part of the process of implementing the above embodiment method can be completed by a computer program to instruct related hardware, and the program can be stored in a computer readable storage medium. In execution, the flow of an embodiment of the methods as described above may be included. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

最后应说明的是: 以上实施例仅用以说明本发明的技术方案, 而非对其 限制; 尽管参照前述实施例对本发明进行了详细的说明, 本领域的普通技术 人员应当理解: 其依然可以对前述各实施例所记载的技术方案进行修改, 或 者对其中部分技术特征进行等同替换; 而这些修改或者替换, 并不使相应技 术方案的本质脱离本发明各实施例技术方案的精神和范围。  It should be noted that the above embodiments are only for explaining the technical solutions of the present invention, and are not intended to be limiting; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those skilled in the art that: The technical solutions described in the foregoing embodiments are modified, or some of the technical features are equivalently replaced. The modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

权 利 要求 Rights request
1、 一种集群系统, 其特征在于, 包括: 管理节点和多台业务节点, 其中 每台业务节点包括硬件层, 运行在所述硬件层之上的虚拟机监控单元 VMM , 运行在所述虚拟机监控单元 VMM之上的至少一个虚拟机; 每台业务节点上 进一步部署有网络流量监控模块, 所述管理节点上部署有虚拟机部署调整模 块, 其中:  A cluster system, comprising: a management node and a plurality of service nodes, wherein each service node includes a hardware layer, and a virtual machine monitoring unit VMM running on the hardware layer runs in the virtual At least one virtual machine above the machine monitoring unit VMM; a network traffic monitoring module is further deployed on each service node, and a virtual machine deployment adjustment module is deployed on the management node, where:
所述网络流量监控模块用于: 通过与所述虚拟机监控单元 VMM 交互, 获取当前业务节点上每个虚拟机的网络流量信息; 所述虚拟机部署调整模块用于: 向所述集群系统内的所述多台业务节点 上的网络流量监控模块收集所述虚拟机的网络流量信息, 根据所述虚拟机的 网络流量信息确定所述集群系统内虚拟机之间的数据流向关系, 根据虚拟机 迁移策略和所述集群系统内虚拟机之间的数据流向关系, 向其上运行有所确 定的待迁移虚拟机的 VMM 下发虚拟机迁移命令或者输出对应的虚拟机迁移 建议;  The network traffic monitoring module is configured to: obtain network traffic information of each virtual machine on the current service node by interacting with the virtual machine monitoring unit VMM; the virtual machine deployment adjustment module is configured to: The network traffic monitoring module on the multiple service nodes collects network traffic information of the virtual machine, and determines a data flow relationship between the virtual machines in the cluster system according to the network traffic information of the virtual machine, according to the virtual machine. The migration policy and the data flow relationship between the virtual machines in the cluster system are sent to the VMM on which the determined virtual machine to be migrated is executed, or the corresponding virtual machine migration proposal is output;
其上运行有所确定的待迁移虚拟机的所述虚拟机监控单元 VMM用于: 接收所述虚拟机部署调整模块下发的虚拟机迁移命令, 根据所述虚拟机迁移 命令将所述待迁移虚拟机迁移到对应的目标业务节点上, 使得存在数据流向 关系的多个虚拟机宿主在同一业务节点上。  The virtual machine monitoring unit VMM that runs the determined virtual machine to be migrated is configured to: receive a virtual machine migration command delivered by the virtual machine deployment adjustment module, and move the to-be-migrated according to the virtual machine migration command The virtual machine is migrated to the corresponding target service node, so that multiple virtual machine hosts with data flow to the relationship are on the same service node.
2、 根据权利要求 1所述的集群系统, 其特征在于, 所述管理节点为独立 的物理服务器, 或者, 所述管理节点还具有业务节点的功能, 或者所述管理 节点为业务节点上的虚拟机; 所述业务节点为独立的物理服务器。  The cluster system according to claim 1, wherein the management node is an independent physical server, or the management node further has a function of a service node, or the management node is a virtual node on a service node. The service node is an independent physical server.
3、 根据权利要求 1所述的集群系统, 其特征在于, 所述业务节点上部署 的网络流量监控模块与所述 VMM集成于一体。  The clustering system according to claim 1, wherein the network traffic monitoring module deployed on the service node is integrated with the VMM.
4、 根据权利要求 1所述的集群系统, 其特征在于, 所述虚拟机部署调整 模块与所述集群系统内的一业务节点上部署的所述网络流量监控模块集成于 一体。 The cluster system according to claim 1, wherein the virtual machine deployment adjustment module is integrated with the network traffic monitoring module deployed on a service node in the cluster system. One.
5、 根据权利要求 1 , 2或 4任一项所述的集群系统, 其特征在于, 所述 VMM具体用于: 取样获取虚拟机列表中的虚拟机的数据包,根据所述数据包 的 IP信息和对应数量, 统计对应虚拟机的流入源 IP地址及对应的流量数据、 流出目标 IP地址及对应的流量数据,并将相应的统计结果输出给所述网络流 量监控模块,以及在接收所述虚拟机部署调整模块下发的虚拟机迁移命令后, 根据所述虚拟机迁移命令将所述待迁移虚拟机迁移到对应的目标业务节点 上;  The cluster system according to any one of claims 1 to 2, wherein the VMM is specifically configured to: sample and acquire a data packet of a virtual machine in a virtual machine list, according to an IP of the data packet. The information and the corresponding quantity, the corresponding source IP address of the corresponding virtual machine and the corresponding traffic data, the outflow destination IP address, and the corresponding traffic data are counted, and the corresponding statistical result is output to the network traffic monitoring module, and the receiving After the virtual machine migration command is delivered by the virtual machine deployment adjustment module, the virtual machine to be migrated is migrated to the corresponding target service node according to the virtual machine migration command.
所述网络流量监控模块具体用于: 统计超过流量阈值的虚拟机列表, 并 将所述虚拟机列表输出给所述 VMM;根据来自所述 VMM的统计结果得到数 据流向记录, 所述数据流向记录包括虚拟机标识信息, 流入源 IP地址以及对 应的流量数据、 流出目标 I P地址以及对应的流量数据, 所述虚拟机标识信息 包括虚拟机名和虚拟机 IP地址;  The network traffic monitoring module is specifically configured to: collect a virtual machine list that exceeds a traffic threshold, and output the virtual machine list to the VMM; and obtain a data flow direction record according to a statistical result from the VMM, where the data flow records The virtual machine identification information includes the virtual machine identification information, the inbound source IP address, the corresponding traffic data, the outgoing destination IP address, and the corresponding traffic data, where the virtual machine identification information includes a virtual machine name and a virtual machine IP address;
所述虚拟机部署调整模块具体用于: 向所述集群系统内的各业务节点上 的网络流量监控模块获取所述数据流向记录; 对所述数据流向记录进行匹配 处理, 得到所述集群系统内虚拟机之间的数据流向关系, 所述集群系统内虚 拟机之间的数据流向关系用流向匹配记录表示, 包括: 源虚拟机标识信息、 目标虚拟机标识信息及对应流量大小; 并根据所述集群系统的流向匹配记录 确定满足虚拟机迁移策略的待迁移虚拟机, 向其上运行有所确定的待迁移虚 拟机的 VMM 下发虚拟机迁移命令或者输出对应的虚拟机迁移建议; 其中, 所述虚拟机迁移策略包括: 虚拟机迁移数量小于等于设定阈值, 和 /或, 根据 减少的网络流量大小确定迁移顺序, 和 /或, 满足虚拟机迁移数量最小化和减 少的网络流量最大化的平衡关系, 和 /或, 将存在数据流向关系的虚拟机迁移 到同一目标业务节点, 所述目标业务节点为根据集群系统内存在数据流向关 系的多个虚拟机各自宿主的业务节点的物理资源情况所确定的。  The virtual machine deployment adjustment module is specifically configured to: obtain the data flow direction record from a network traffic monitoring module on each service node in the cluster system; perform matching processing on the data flow record to obtain the cluster system The data flow relationship between the virtual machines, the data flow relationship between the virtual machines in the cluster system is represented by the flow matching record, including: source virtual machine identification information, target virtual machine identification information, and corresponding traffic size; The flow matching record of the cluster system determines the virtual machine to be migrated that meets the virtual machine migration policy, and delivers the virtual machine migration command or the corresponding virtual machine migration proposal to the VMM that runs the determined virtual machine to be migrated; The virtual machine migration policy includes: the number of virtual machine migrations is less than or equal to a set threshold, and/or, determining a migration order according to the reduced network traffic size, and/or maximizing the virtual machine migration quantity to minimize and reduce network traffic maximization. Balance the relationship, and / or, there will be a virtual flow of data to the relationship Machine migration service node to the same destination, the target cluster service node according to a plurality of system memory data flow in relation to each virtual machine host physical resources determined by the service node.
6、 根据权利要求 5所述的集群系统, 其特征在于, 所述虚拟机部署调整 模块具体用于: 向所述集群系统内的各业务节点上的网络流量监控模块获取 所述数据流向记录; 对所述数据流向记录进行匹配处理, 得到所述集群系统 内虚拟机之间的数据流向关系, 所述集群系统内虚拟机之间的数据流向关系 用流向匹配记录表示, 包括: 源虚拟机标识信息、 目标虚拟机标识信息及对 应流量大小; 根据所述集群系统的流向匹配记录生成虚拟机迁移加权表, 所 述虚拟机迁移加权表包括所述流向匹配表中的所有虚拟机的标识信息、 目标 业务节点的标识信息及对应权值, 所述权值为所述虚拟机与目标业务节点上 的各虚拟机之间的数据流量之和; 若所述虚拟机迁移加权表中存在一个最大 权值, 则从所述虚拟机迁移加权表中选取权值最大的虚拟机加入虚拟机迁移 列表; 若所述虚拟机迁移加权表中存在多个最大权值, 则将虚拟机内存值与 目标业务节点剩余内存值比值最大的虚拟机加入虚拟机迁移列表; 根据所述 虚拟机迁移列表向其上运行有所述权值最大的虚拟机或者所述比值最大的虚 拟机的 VMM 下发虚拟机迁移命令或者输出对应的虚拟机迁移建议。 6. The cluster system according to claim 5, wherein the virtual machine deployment adjustment The module is specifically configured to: obtain, by the network traffic monitoring module on each service node in the cluster system, the data flow direction record; perform matching processing on the data flow to record, and obtain data between virtual machines in the cluster system. The flow direction relationship, the data flow between the virtual machines in the cluster system is represented by a flow matching record, including: source virtual machine identification information, target virtual machine identification information, and corresponding traffic size; generating according to the flow matching record of the cluster system a virtual machine migration weighting table, where the virtual machine migration weighting table includes identification information of all virtual machines in the flow matching table, identification information of the target service node, and corresponding weights, where the weight is the virtual machine and the target The sum of the data traffic between the virtual machines on the service node; if there is a maximum weight in the virtual machine migration weighting table, the virtual machine with the largest weight is selected from the virtual machine migration weighting table to join the virtual machine. Migration list; if there are multiple maximum weights in the virtual machine migration weighting table, the virtual machine memory The virtual machine whose value is the largest ratio of the remaining memory value of the target service node is added to the virtual machine migration list; according to the virtual machine migration list, the virtual machine with the largest weight or the virtual machine with the largest ratio is run under the VMM Send a VM migration command or output a corresponding VM migration proposal.
7、 一种业务节点, 其特征在于, 包括: 硬件层, 运行在所述硬件层之上 的虚拟机监控单元 VMM ,运行在所述虚拟机监控单元 VMM之上的至少一个 虚拟机, 所述业务节点还进一步部署有网络流量监控模块;  A service node, comprising: a hardware layer, a virtual machine monitoring unit VMM running on the hardware layer, and at least one virtual machine running on the virtual machine monitoring unit VMM, The service node is further deployed with a network traffic monitoring module;
所述网络流量监控模块用于: 通过与所述 VMM 交互, 获取本业务节点 上每个虚拟机的网络流量信息;  The network traffic monitoring module is configured to: obtain network traffic information of each virtual machine on the service node by interacting with the VMM;
所述虚拟机监控单元 VMM用于: 在收到虚拟机迁移命令后, 根据所述 虚拟机迁移命令将其上运行的待迁移虚拟机迁移到对应的目标业务节点上, 使得存在数据流向关系的多个虚拟机宿主在同一业务节点上, 其中, 所述虚 拟机迁移命令是管理节点根据向集群系统内的各业务节点上的网络流量监控 模块所收集的网络流量信息, 确定所述集群系统内虚拟机之间的数据流向关 系, 根据虚拟机迁移策略和所述集群系统内虚拟机之间的数据流向关系向其 上运行有所确定的待迁移虚拟机的 VMM下发的。  The virtual machine monitoring unit VMM is configured to: after receiving the virtual machine migration command, migrate the virtual machine to be migrated running on the virtual machine to the corresponding target service node according to the virtual machine migration command, so that the data flow relationship exists. The plurality of virtual machines are hosted on the same service node, wherein the virtual machine migration command is determined by the management node according to network traffic information collected by the network traffic monitoring module on each service node in the cluster system, and the cluster system is determined. The data flow relationship between the virtual machines is delivered to the VMM on which the VM to be migrated is running according to the virtual machine migration policy and the data flow relationship between the virtual machines in the cluster system.
8、 根据权利要求 7所述的业务节点, 其特征在于, 所述网络流量监控模 块与所述 VMM集成于一体。 8. The service node according to claim 7, wherein the network traffic monitoring mode The block is integrated with the VMM.
9、根据权利要求 7所述的业务节点,其特征在于,所述 VMM具体用于: 取样获取虚拟机列表中的虚拟机的数据包,根据所述数据包的 IP信息和对应 数量, 统计对应虚拟机的流入源 IP地址及对应的流量数据、 流出目标 IP地 址及对应的流量数据, 并将相应的统计结果输出给所述网络流量监控模块, 以及在收到所述虚拟机迁移命令后, 根据所述虚拟机迁移命令将其上运行的 待迁移虚拟机迁移到对应的目标业务节点上;  The service node according to claim 7, wherein the VMM is specifically configured to: sample and acquire a data packet of a virtual machine in a virtual machine list, and perform statistical correspondence according to the IP information and the corresponding quantity of the data packet. The source IP address of the virtual machine, the corresponding traffic data, the outgoing destination IP address, and the corresponding traffic data, and output corresponding statistics to the network traffic monitoring module, and after receiving the virtual machine migration command, The virtual machine to be migrated running on the virtual machine is migrated to the corresponding target service node according to the virtual machine migration command;
所述网络流量监控模块具体用于: 统计超过流量阈值的虚拟机列表, 并 将所述虚拟机列表输出给所述 VMM;根据来自所述 VMM的统计结果得到本 业务节点的数据流向记录, 所述数据流向记录包括虚拟机标识信息, 流入源 IP地址以及对应的流量数据、流出目标 IP地址以及对应的流量数据, 所述虚 拟机标识信息包括虚拟机名和虚拟机 IP地址。  The network traffic monitoring module is specifically configured to: collect a virtual machine list that exceeds a traffic threshold, and output the virtual machine list to the VMM; and obtain a data flow direction record of the service node according to a statistical result from the VMM. The data flow direction record includes virtual machine identification information, a source IP address and corresponding traffic data, an outgoing destination IP address, and corresponding traffic data, where the virtual machine identification information includes a virtual machine name and a virtual machine IP address.
10、 一种管理节点, 其特征在于, 包括:  10. A management node, comprising:
网络流量收集单元, 用于向所述集群系统内的各业务节点收集宿主在所 述业务节点上的虚拟机的网络流量信息;  a network traffic collection unit, configured to collect, by each service node in the cluster system, network traffic information of a virtual machine hosted on the service node;
虚拟机部署调整单元, 用于根据所述虚拟机的网络流量信息确定所述集 群系统内虚拟机之间的数据流向关系, 根据虚拟机迁移策略和所述集群系统 内虚拟机之间的数据流向关系, 向其上运行有所确定的待迁移虚拟机的虚拟 机监控单元 VMM下发虚拟机迁移命令或者输出对应的虚拟机迁移建议。  a virtual machine deployment adjustment unit, configured to determine, according to network traffic information of the virtual machine, a data flow relationship relationship between virtual machines in the cluster system, according to a virtual machine migration policy and a data flow direction between the virtual machines in the cluster system The virtual machine migration command is sent to the virtual machine monitoring unit VMM that runs the determined virtual machine to be migrated or the corresponding virtual machine migration proposal is output.
11、 根据权利要求 10 所述的管理节点, 其特征在于, 所述网络流量收 集单元具体用于向所述集群系统内各业务节点上的网络流量监控模块获取数 据流向记录, 所述数据流向记录包括虚拟机标识信息, 流入源互联网协议 IP 地址以及对应的流量数据、 流出目标 I P地址以及对应的流量数据;  The management node according to claim 10, wherein the network traffic collection unit is configured to acquire a data flow direction record to a network traffic monitoring module on each service node in the cluster system, where the data flow is recorded. Including virtual machine identification information, inflow source Internet Protocol IP address and corresponding traffic data, outgoing destination IP address, and corresponding traffic data;
所述虚拟机部署调整单元具体用于:对所述数据流向记录进行匹配处理, 得到所述集群系统内虚拟机之间的数据流向关系, 所述集群系统内虚拟机之 间的数据流向关系用流向匹配记录表示, 包括: 源虚拟机标识信息、 目标虚 拟机标识信息及对应流量大小; 并根据所述集群系统的流向匹配记录确定满 足虚拟机迁移策略的待迁移虚拟机, 向其上运行有所确定的待迁移虚拟机的The virtual machine deployment adjustment unit is specifically configured to: perform matching processing on the data flow to record, obtain a data flow relationship relationship between virtual machines in the cluster system, and use a data flow relationship relationship between virtual machines in the cluster system. The flow direction matching record indicates, including: source virtual machine identification information, target virtual The virtual machine identification information and the corresponding traffic volume are determined; and the virtual machine to be migrated that meets the virtual machine migration policy is determined according to the flow matching record of the cluster system, and the determined virtual machine to be migrated is run thereon
VMM下发虚拟机迁移命令或者输出对应的虚拟机迁移建议; 其中, 所述虚拟 机迁移策略包括: 虚拟机迁移数量小于等于设定阈值, 和 /或, 根据减少的网 络流量大小确定迁移顺序, 和 /或, 满足虚拟机迁移数量最小化和减少的网络 流量最大化的平衡关系, 和 /或, 将存在数据流向关系的虚拟机迁移到同一目 标业务节点, 所述目标业务节点为根据集群系统内存在数据流向关系的多个 虚拟机各自宿主的业务节点的物理资源情况所确定的。 The VMM sends a virtual machine migration command or outputs a corresponding virtual machine migration proposal. The virtual machine migration policy includes: the virtual machine migration quantity is less than or equal to a set threshold, and/or, according to the reduced network traffic size, determining a migration sequence. And/or, satisfying a balance between maximizing virtual network migration and reducing network traffic maximization, and/or migrating a data flow to a relationship virtual machine to a same target service node, the target service node being a cluster system The memory is determined by the physical resource of the service node hosted by each of the plurality of virtual machines in the relationship.
12、 根据权利要求 11 所述的管理节点, 其特征在于, 所述虚拟机部署 调整单元包括:  The management node according to claim 11, wherein the virtual machine deployment adjustment unit comprises:
流向匹配子单元, 用于对所述数据流向记录进行匹配处理, 得到所述集 群系统内虚拟机之间的数据流向关系, 所述集群系统内虚拟机之间的数据流 向关系用流向匹配记录表示, 包括: 源虚拟机标识信息、 目标虚拟机标识信 息及对应流量大小;  And a flow matching matching sub-unit, configured to perform matching processing on the data flow to the record, to obtain a data flow relationship relationship between the virtual machines in the cluster system, and the data flow relationship between the virtual machines in the cluster system is represented by a flow matching record , including: source virtual machine identification information, target virtual machine identification information, and corresponding traffic size;
迁移决策子单元, 用于根据所述集群系统的流向匹配记录生成虚拟机迁 移加权表, 所述虚拟机迁移加权表包括所述流向匹配记录中的所有虚拟机的 标识信息、 目标业务节点的标识信息及对应权值, 所述权值为所述虚拟机与 目标业务节点上的各虚拟机之间的数据流量之和; 若所述虚拟机迁移加权表 中存在一个最大权值, 则从所述虚拟机迁移加权表中选取权值最大的虚拟机 加入虚拟机迁移列表; 若所述虚拟机迁移加权表中存在多个最大权值, 则将 虚拟机内存值与目标业务节点剩余内存值比值最大的虚拟机加入虚拟机迁移 列表; 根据所述虚拟机迁移列表向其上运行有所述权值最大的虚拟机或者所 述比值最大的虚拟机的 VMM 下发虚拟机迁移命令或者输出对应的虚拟机迁 移建议。  a migration decision sub-unit, configured to generate a virtual machine migration weighting table according to the flow direction matching record of the cluster system, where the virtual machine migration weighting table includes identifier information of all virtual machines in the flow matching record, and identifier of the target service node Information and a corresponding weight, the weight is a sum of data traffic between the virtual machine and each virtual machine on the target service node; if there is a maximum weight in the virtual machine migration weighting table, The virtual machine with the largest weight is selected in the virtual machine migration weighting table to join the virtual machine migration list. If there are multiple maximum weights in the virtual machine migration weighting table, the virtual machine memory value is compared with the remaining service value of the target service node. The virtual machine is added to the virtual machine migration list, and the virtual machine migration command or the VMM that runs the virtual machine with the largest weight or the virtual machine with the largest value is sent according to the virtual machine migration list. Virtual machine migration recommendations.
13、 根据权利要求 10至 12任一项所述的管理节点, 其特征在于, 所述 管理节点为独立的物理服务器, 或者, 所述管理节点还具有业务节点的功能, 或者所述管理节点为业务节点上的虚拟机。 The management node according to any one of claims 10 to 12, wherein the management node is an independent physical server, or the management node further has a function of a service node. Or the management node is a virtual machine on a service node.
14、 一种基于网络数据流向的虚拟机迁移方法, 其特征在于, 包括: 向集群系统内的各业务节点收集宿主在所述业务节点上的虚拟机的网络 流量信息;  A virtual machine migration method based on network data flow, comprising: collecting network traffic information of a virtual machine hosted on the service node to each service node in the cluster system;
根据所述虚拟机的网络流量信息确定所述集群系统内虚拟机之间的数据 流向关系;  Determining, according to network traffic information of the virtual machine, a data flow relationship relationship between virtual machines in the cluster system;
根据虚拟机迁移策略和所述集群系统内虚拟机之间的数据流向关系, 向 其上运行有所确定的待迁移虚拟机的虚拟机监控单元 VMM 下发虚拟机迁移 命令或者输出对应的虚拟机迁移建议。  Distributing a virtual machine migration command or outputting a corresponding virtual machine to the virtual machine monitoring unit VMM running the determined virtual machine to be migrated according to the virtual machine migration policy and the data flow relationship between the virtual machines in the cluster system Migration recommendations.
15、 根据权利要求 14 所述的方法, 其特征在于, 所述根据所述虚拟机 的网络流量信息确定所述集群系统内虚拟机之间的数据流向关系, 包括: 对收集的所述虚拟机的网络流量信息进行匹配处理, 所述网络流量信息 包括虚拟机标识信息 , 流入源互联网协议 I P地址以及对应的流量数据、 流出 目标 IP地址以及对应的流量数据,所述虚拟机标识信息包括虚拟机名和虚拟 机 IP地址;  The method according to claim 14, wherein the determining, according to network traffic information of the virtual machine, a data flow relationship relationship between virtual machines in the cluster system, comprising: collecting the virtual machine The network traffic information is matched, and the network traffic information includes virtual machine identification information, an inbound source Internet Protocol IP address, and corresponding traffic data, an outgoing destination IP address, and corresponding traffic data, where the virtual machine identification information includes a virtual machine. Name and virtual machine IP address;
根据所述匹配处理结果得到集群系统内虚拟机之间的数据流向关系, 所 述集群系统内虚拟机之间的数据流向关系用流向匹配记录表示, 包括: 源虚 拟机标识信息、 目标虚拟机标识信息及对应流量大小;  The data flow relationship between the virtual machines in the cluster system is obtained according to the result of the matching processing, and the data flow relationship between the virtual machines in the cluster system is represented by a flow matching record, including: source virtual machine identification information, target virtual machine identifier Information and corresponding traffic size;
所述根据虚拟机迁移策略和所述集群系统内虚拟机之间的数据流向关 系, 向其上运行有所确定的待迁移虚拟机的虚拟机监控单元 VMM 下发虚拟 机迁移命令或者输出对应的虚拟机迁移建议, 包括:  Transmitting a virtual machine migration command or outputting a corresponding virtual machine monitoring unit VMM to the virtual machine monitoring unit VMM that runs the determined virtual machine to be migrated according to the data flow relationship between the virtual machine migration policy and the virtual machine in the cluster system Virtual machine migration recommendations, including:
根据所述集群系统的流向匹配记录确定满足虚拟机迁移策略的待迁移虚 拟机, 向其上运行有所确定的待迁移虚拟机的 VMM 下发虚拟机迁移命令或 者输出对应的虚拟机迁移建议。  And determining, according to the flow matching record of the cluster system, the virtual machine to be migrated that meets the virtual machine migration policy, and sending a virtual machine migration command to the VMM that runs the determined virtual machine to be migrated or outputting a corresponding virtual machine migration proposal.
16、 根据权利要求 14或 15所述的方法, 其特征在于,  16. Method according to claim 14 or 15, characterized in that
所述虚拟机迁移策略包括虚拟机迁移数量小于等于设定阈值, 和 /或, 根 据减少的网络流量大小确定迁移顺序, 和 /或, 满足虚拟机迁移数量最小化和 减少的网络流量最大化的平衡关系, 和 /或, 将存在数据流向关系的虚拟机迁 移到同一目标业务节点, 所述目标业务节点为根据集群系统内存在数据流向 关系的多个虚拟机各自宿主的业务节点的物理资源情况所确定的。 The virtual machine migration policy includes that the number of virtual machine migrations is less than or equal to a set threshold, and/or root The migration order is determined according to the reduced network traffic size, and/or the balance of the virtual machine migration is minimized and the network traffic is minimized, and/or the virtual machine with the data flow to the relationship is migrated to the same target service node. The target service node is determined according to a physical resource condition of a service node hosted by each of a plurality of virtual machines in a data flow relationship relationship in the cluster system.
17、 根据权利要求 15 所述的方法, 其特征在于, 所述根据所述集群系 统的流向匹配记录确定满足虚拟机迁移策略的待迁移虚拟机, 向其上运行有 所确定的待迁移虚拟机的 VMM 下发虚拟机迁移命令或者输出对应的虚拟机 迁移建议, 包括:  The method according to claim 15, wherein the determining, according to the flow matching record of the cluster system, the virtual machine to be migrated that meets the virtual machine migration policy, and running the determined virtual machine to be migrated The VMM delivers the VM migration command or outputs the corresponding VM migration proposal, including:
根据所述集群系统的流向匹配记录生成虚拟机迁移加权表, 所述虚拟机 迁移加权表包括所述流向匹配表中的所有虚拟机的标识信息、 目标业务节点 的标识信息及对应权值, 所述权值为所述虚拟机与目标业务节点上的各虚拟 机之间的数据流量之和;  Generating a virtual machine migration weighting table according to the flow direction matching record of the cluster system, where the virtual machine migration weighting table includes identification information of all virtual machines in the flow matching table, identification information of the target service node, and corresponding weights. The weight is the sum of the data traffic between the virtual machine and each virtual machine on the target service node;
根据所述虚拟机迁移加权表中的权值大小将对应的虚拟机加入虚拟机迁 移列表;  Adding a corresponding virtual machine to the virtual machine migration list according to the weight value in the virtual machine migration weighting table;
根据所述虚拟机迁移列表向其上运行有所确定的待迁移虚拟机的 VMM 下发虚拟机迁移命令或者输出对应的迁移建议。  The VM migration command is sent to the VMM on which the VM to be migrated is running, or the corresponding migration proposal is output according to the VM migration list.
18、 根据权利要求 17 所述的方法, 其特征在于, 所述根据所述虚拟机 迁移加权表中的权值大小将对应的虚拟机加入虚拟机迁移列表, 包括:  The method according to claim 17, wherein the adding the corresponding virtual machine to the virtual machine migration list according to the weight value in the virtual machine migration weighting table comprises:
若所述虚拟机迁移加权表中存在一个最大权值, 则从所述虚拟机迁移加 权表中选取权值最大的虚拟机加入虚拟机迁移列表; 若所述虚拟机迁移加权 表中存在多个最大权值, 则将虚拟机内存值与目标业务节点剩余内存值比值 最大的虚拟机加入虚拟机迁移列表。  If the virtual machine migration weighting table has a maximum weight, the virtual machine with the largest weight is selected from the virtual machine migration weighting table to join the virtual machine migration list; if there are multiple virtual machine migration weighting tables The maximum weight is added to the virtual machine migration list by the virtual machine with the largest ratio of the virtual machine memory value to the remaining memory value of the target service node.
19、 根据权利要求 18所述的方法, 其特征在于, 还包括:  The method according to claim 18, further comprising:
若所述虚拟机迁移列表中任一虚拟机对应的目的业务节点的剩余内存不 足, 则对所述虚拟机迁移列表中下一虚拟机进行迁移。  If the remaining memory of the destination service node corresponding to any virtual machine in the virtual machine migration list is insufficient, the next virtual machine in the virtual machine migration list is migrated.
PCT/CN2011/079733 2011-01-24 2011-09-16 Method, device and cluster system for virtual machine migration based on network data flow direction WO2012100544A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201110025843.3 2011-01-24
CN 201110025843 CN102082692B (en) 2011-01-24 2011-01-24 Method and equipment for migrating virtual machines based on network data flow direction, and cluster system

Publications (1)

Publication Number Publication Date
WO2012100544A1 true WO2012100544A1 (en) 2012-08-02

Family

ID=44088449

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2011/079733 WO2012100544A1 (en) 2011-01-24 2011-09-16 Method, device and cluster system for virtual machine migration based on network data flow direction

Country Status (2)

Country Link
CN (1) CN102082692B (en)
WO (1) WO2012100544A1 (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103888417A (en) * 2012-12-21 2014-06-25 中国电信股份有限公司 Network switching method and system for virtual machine system migration
US9081599B2 (en) 2013-05-28 2015-07-14 Red Hat Israel, Ltd. Adjusting transfer rate of virtual machine state in virtual machine migration
US10824456B2 (en) 2018-01-31 2020-11-03 Nutanix, Inc. System traffic analyzers to request co-location of virtual machines in frequent communication
US10831525B2 (en) 2018-11-30 2020-11-10 Nutanix, Inc. Intelligent assignment of virtual machines to compute only or hyper converged nodes
US10846129B2 (en) 2018-11-30 2020-11-24 Nutanix, Inc. Systems and methods for organizing on-demand migration from private cluster to public cloud
US10877791B2 (en) 2018-11-30 2020-12-29 Nutanix, Inc. Live migration of virtual machines between compute only nodes and hyperconverged nodes

Families Citing this family (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102082692B (en) * 2011-01-24 2012-10-17 华为技术有限公司 Method and equipment for migrating virtual machines based on network data flow direction, and cluster system
CN103023799B (en) * 2011-09-27 2016-06-15 日电(中国)有限公司 For central controller and the virtual machine migration method of virtual machine (vm) migration
CN103164253A (en) * 2011-12-16 2013-06-19 鸿富锦精密工业(深圳)有限公司 Virtual machine deployment system and virtual machine deployment method
CN102609309B (en) * 2012-01-19 2018-08-21 南京中兴新软件有限责任公司 A kind of strategy scheduling system and method for cloud computing
CN106789751B (en) * 2012-05-10 2020-07-28 英特尔公司 Network routing based on resource availability
US9081603B2 (en) * 2012-07-09 2015-07-14 Cisco Technology, Inc. Packet forwarding optimization with virtual machine mobility by comparing device identifiers to determine VM movement
US9071587B2 (en) * 2012-08-27 2015-06-30 International Business Machines Corporation Optimizing deployment of virtual machines by pre-generating a virtual machine's first-boot metadata by cloud controller
CN103825834B (en) * 2012-11-16 2017-08-08 新华三技术有限公司 The method and network-switching equipment of flow load sharing are adjusted in a kind of EVI
CN103856548B (en) * 2012-12-07 2017-11-03 华为技术有限公司 Dynamic resource scheduling method and dynamic resource scheduling device
CN104158675B (en) * 2013-05-13 2018-09-11 华为技术有限公司 Calculate node dispositions method, processing node, controller and system
CN103338123B (en) * 2013-06-20 2016-04-06 中国联合网络通信集团有限公司 The defining method of correspondence and device
CN103870340B (en) * 2014-03-06 2017-11-07 华为技术有限公司 Data processing method, control node and stream calculation system in stream calculation system
JP6277853B2 (en) * 2014-05-12 2018-02-14 富士通株式会社 Control device, communication device, and communication method
CN104008012B (en) * 2014-05-30 2017-10-20 长沙麓云信息科技有限公司 A kind of high-performance MapReduce implementation methods based on dynamic migration of virtual machine
CN104063267B (en) * 2014-07-11 2017-11-14 孙强强 A kind of virtual machine traffic monitoring method and system
CN104184813B (en) * 2014-08-20 2018-03-09 杭州华为数字技术有限公司 The load-balancing method and relevant device and group system of virtual machine
CN104270416B (en) * 2014-09-12 2018-03-13 杭州华为数字技术有限公司 Control method for equalizing load and management node
CN104468504B (en) * 2014-10-22 2017-08-15 南京绿云信息技术有限公司 Virtualize the monitoring method and system of network dynamic information safety
CN105677454B (en) * 2014-11-20 2019-08-27 华为技术有限公司 The integration method of computing resource, device and system
CN105991367B (en) * 2015-03-06 2019-10-18 清华大学 The method and system of dependence is communicated between a kind of measurement virtual machine
CN104683186A (en) * 2015-03-11 2015-06-03 杭州华三通信技术有限公司 Network resource scheduling method and device
CN105094944B (en) * 2015-06-10 2018-06-29 中国联合网络通信集团有限公司 A kind of virtual machine migration method and device
CN105183546A (en) * 2015-08-12 2015-12-23 北京因特信安软件科技有限公司 Secure virtual machine migration method based on trustworthy resource pool
CN105119993B (en) * 2015-08-24 2019-02-26 华为技术有限公司 Virtual machine deployment method and device
CN105117280A (en) * 2015-08-24 2015-12-02 用友网络科技股份有限公司 Virtual machine migration device and method
CN106775918A (en) * 2015-11-23 2017-05-31 中国电信股份有限公司 Dispatching method of virtual machine, virtual machine manager and SDN systems
CN106489251B (en) 2015-12-21 2019-10-18 华为技术有限公司 The methods, devices and systems of applied topology relationship discovery
CN106250210A (en) * 2016-08-03 2016-12-21 上海交通大学 Dispatching method of virtual machine under cloud environment
CN107086932A (en) * 2017-05-22 2017-08-22 安徽电信规划设计有限责任公司 A kind of cloud platform virtualizes traffic security monitoring system
CN107733701B (en) * 2017-09-29 2019-11-08 中国联合网络通信集团有限公司 A kind of method and apparatus of deploying virtual machine

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101739287A (en) * 2008-11-13 2010-06-16 国际商业机器公司 Device, system and method for managing virtual machines
CN101876921A (en) * 2009-04-29 2010-11-03 华为技术有限公司 Method, device and system for migration decision-making of virtual machine
CN101937357A (en) * 2009-07-01 2011-01-05 华为技术有限公司 Virtual machine migration decision-making method, device and system
CN102082692A (en) * 2011-01-24 2011-06-01 华为技术有限公司 Method and equipment for migrating virtual machines based on network data flow direction, and cluster system

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101452406B (en) * 2008-12-23 2011-05-18 北京航空航天大学 Cluster load balance method transparent for operating system
US9600332B2 (en) * 2009-04-28 2017-03-21 Cisco Technology, Inc. Server load balancing based on virtual utilization, physical utilization, and feedback

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101739287A (en) * 2008-11-13 2010-06-16 国际商业机器公司 Device, system and method for managing virtual machines
CN101876921A (en) * 2009-04-29 2010-11-03 华为技术有限公司 Method, device and system for migration decision-making of virtual machine
CN101937357A (en) * 2009-07-01 2011-01-05 华为技术有限公司 Virtual machine migration decision-making method, device and system
CN102082692A (en) * 2011-01-24 2011-06-01 华为技术有限公司 Method and equipment for migrating virtual machines based on network data flow direction, and cluster system

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103888417A (en) * 2012-12-21 2014-06-25 中国电信股份有限公司 Network switching method and system for virtual machine system migration
US9081599B2 (en) 2013-05-28 2015-07-14 Red Hat Israel, Ltd. Adjusting transfer rate of virtual machine state in virtual machine migration
US9201676B2 (en) 2013-05-28 2015-12-01 Red Hat Israel, Ltd. Reducing or suspending transfer rate of virtual machine migration when dirtying rate exceeds a convergence threshold
US10824456B2 (en) 2018-01-31 2020-11-03 Nutanix, Inc. System traffic analyzers to request co-location of virtual machines in frequent communication
US10831525B2 (en) 2018-11-30 2020-11-10 Nutanix, Inc. Intelligent assignment of virtual machines to compute only or hyper converged nodes
US10846129B2 (en) 2018-11-30 2020-11-24 Nutanix, Inc. Systems and methods for organizing on-demand migration from private cluster to public cloud
US10877791B2 (en) 2018-11-30 2020-12-29 Nutanix, Inc. Live migration of virtual machines between compute only nodes and hyperconverged nodes

Also Published As

Publication number Publication date
CN102082692A (en) 2011-06-01
CN102082692B (en) 2012-10-17

Similar Documents

Publication Publication Date Title
US10185592B2 (en) Network storage device using dynamic weights based on resource utilization
Aujla et al. Optimal decision making for big data processing at edge-cloud environment: An SDN perspective
Krishnamurthy et al. Pratyaastha: an efficient elastic distributed sdn control plane
Noormohammadpour et al. Datacenter traffic control: Understanding techniques and tradeoffs
US10061786B2 (en) Providing a database as a service in a multi-tenant environment
US9354941B2 (en) Load balancing for single-address tenants
US9389903B2 (en) Method, system and apparatus for creating virtual machine
US9459904B2 (en) NUMA I/O aware network queue assignments
Aslam et al. Load balancing algorithms in cloud computing: A survey of modern techniques
US9992077B2 (en) Optimizing allocation of on-demand resources using performance zones
US9213573B2 (en) Management of prioritizing virtual machines in an operating environment
US9519500B2 (en) Offering network performance guarantees in multi-tenant datacenters
CN104850450B (en) A kind of load-balancing method and system towards mixed cloud application
CN103699445B (en) Task scheduling method, device and system
Agarwal et al. Efficient optimal algorithm of task scheduling in cloud computing environment
US9237131B2 (en) Virtual private storage array service for cloud servers
Ghaznavi et al. Elastic virtual network function placement
US10778756B2 (en) Location of actor resources
JP6199514B2 (en) Scheduling fabric distributed resources
EP2865147B1 (en) Guarantee of predictable and quantifiable network performance
US20190182174A1 (en) Method and system to allocate bandwidth based on task deadline in cloud computing networks
US10387202B2 (en) Quality of service implementation in a networked storage system with hierarchical schedulers
Buyya et al. Software-defined cloud computing: Architectural elements and open challenges
EP2899947A1 (en) Component oriented hybrid cloud operating system architecture and communication method thereof
AU2013277589B2 (en) Offloading virtual machine flows to physical queues

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 11857159

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase in:

Ref country code: DE

122 Ep: pct app. not ent. europ. phase

Ref document number: 11857159

Country of ref document: EP

Kind code of ref document: A1