CN104123171A - Virtual machine migrating method and system based on NUMA architecture - Google Patents
Virtual machine migrating method and system based on NUMA architecture Download PDFInfo
- Publication number
- CN104123171A CN104123171A CN201410256198.XA CN201410256198A CN104123171A CN 104123171 A CN104123171 A CN 104123171A CN 201410256198 A CN201410256198 A CN 201410256198A CN 104123171 A CN104123171 A CN 104123171A
- Authority
- CN
- China
- Prior art keywords
- virtual machine
- node
- physical
- destination node
- migrated
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Memory System Of A Hierarchy Structure (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a virtual machine migrating method and system based on the NUMA architecture. The virtual machine migrating method based on the NUMA architecture comprises the steps that the configuration information of a virtual machine to be migrated is obtained, and a temporary file is generated according to the content in a memory page and a virtual machine register which have already been used by the virtual machine to be migrated and the state of an I/O device; the virtual machine to be migrated is stopped; the node information of all physical nodes in a target host is obtained, and whether a target node exists on the target host is judged according to the node information and a migration command; if the target node exists on the target host, the virtual machine to be migrated is migrated to the target node according to the temporary file and the configuration information; if the target node does not exist on the target host, an error report is sent. According to the virtual machine migrating method and system based on the NUMA architecture, it is guaranteed that virtual nodes of the NUMA topological structure of the virtual machine correspond to the NUMA nodes of a physical machine in a one-to-one mode on the basis of migration, the performance degradation of the virtual machine is small, the NUMA topological structure of the virtual machine can be changed according to the hardware condition of the target host, and thus the virtual machine can be adapted to more diversified hardware environments.
Description
Technical field
The present invention relates to field of computer technology, particularly a kind of virtual machine migration method and system based on NUMA framework.
Background technology
In computer technology, Intel Virtualization Technology is by the computational resource of physics, in addition abstract such as CPU, internal memory, network and storer etc., makes user can more reasonably use a technology of these resources.Now be widely used in some scenes such as Internet service, cloud computing.
Early stage computing machine adopts SMP structure (Symmetric Multi-Processing, symmetrical multiprocessing structure), saves as all CPU in all to share, and each CPU is the arbitrary address of access memory at random.But along with the development of hardware, particularly, along with the increasing of core cpu quantity, existing SMP structure is easily brought memory access conflict, that is to say, different CPU accesses same internal memory simultaneously and causes bus collision, and it is more next obviously that request of access is delayed this phenomenon.So, in high-performance calculation, the computing machine of multi-core does not often adopt SMP structure and adopts NUMA framework (Non Uniform Memory Access Architecture, nonuniform memory access): the CPU of machine and storer can be divided into several nodes, and each node is made up of several CPU and a storage unit.CPU accesses local storer becomes local memory access, and the storer of access far-end becomes far-end memory access, and far-end memory access is by CPU and the direct circuit communication of CPU.Like this, just, the performance cost of having avoided bus collision to bring.But, but bring the unbalanced problem of memory access speed of internal memory: the expense that the expense of far-end memory access will be deposited much larger than local model.This has also directly caused the performance issue of Intel Virtualization Technology conventional under NUMA framework.
Because virtual machine monitor need to be virtual machine hides bottom hardware, the storer that virtual machine obtains is a continuous linear address conventionally, does not understand it and is specifically positioned at which node.Its VCPU obtaining (virtual cpu) does not also identify and is currently operating on which physical node.What therefore, physical machine presented to virtual machine is exactly an indiscriminate SMP architecture.Although the operating system of the operation of virtual machine inside has certain optimization to NUMA architecture, because this type of virtual machine lacks clear and definite understanding to actual physics topology, make these optimizations have no ample scope for abilities.
VNUMA (virtual nonuniform memory access) technology is the technology being born in order to solve problems.It shows and SLIT table by the SRAT in the APIC table of change virtual machine at the beginning of creating virtual machine, thereby creates a virtual machine that can present NUMA architecture.Positive virtual machine is made up of several dummy nodes, and each dummy node and a physical node are strictly bound, and the VCPU of each dummy node only allows to operate on the CPU that specifies physical node.The internal memory that each dummy node need to distribute can only be to the application of fixed physical node.Like this, the virtual NUMA topology of virtual machine is just strict consistent with physics NUMA structure.The operating system of moving on virtual machine only needs just can be optimized far-end memory access problem according to the virtual NUMA topological structure of presenting to it.
This technology also can be with serving new problem: because the dummy node of virtual machine is all strictly bound with physical node.Change the quite difficulty of virtual machine topology of virtual machine.But, in application scenes, such as virtual machine is energy-conservation, need to close some physical cpus, so physical machine.If the dummy node number of virtual machine is greater than the physical node number that Energy Saving Strategy plan is used, Energy Saving Strategy just can not well be carried out, because force, the VCPU on additional dummy node is moved to and on other physical node, just will inevitably destroy the original VNUMA topological structure of virtual machine.Now need the VNUMA topological structure of virtual machine to upgrade accordingly.Existing VNUMA topological structure method is to show and internally deposit into Mobile state to move to realize this function by the SRAT table of dynamic change virtual machine and SLIT.But this method also has distinct disadvantage: the virtual machine after change can only with change before virtual machine be in same physical machine, if subscriber policy will be transferred in another physical machine, cannot solve by this technology.In addition, existing migrating technology is taking node as unit, virtual machine to be moved, and cannot place virutal machine memory neatly according to present load as unit taking page, and the effect of load balancing is restricted.
Summary of the invention
For the deficiencies in the prior art, a kind of virtual machine migration method and system based on NUMA framework that the present invention proposes.
Based on a virtual machine migration method for NUMA framework, comprising:
(1) obtain the configuration information of virtual machine to be migrated, and content in the page, the virtual machine register that have used according to virtual machine to be migrated and I/O equipment state generate temporary file;
(2) close virtual machine to be migrated;
(3) obtain the nodal information of all physical nodes on destination host, and judge according to described nodal information and migration instruction whether destination host exists destination node:
If exist, according to described temporary file and configuration information, by virtual machine (vm) migration to be migrated to destination node;
Otherwise, send error reporting.
In the present invention, the configuration information of virtual machine, temporary file are kept in one or more nodes of network with centralized or distributed method, are generally on the temporary or permanent memory device outside source host (main frame at virtual machine to be migrated place before migration) and destination host.
Described migration instruction should comprise the information such as virtual machine information to be migrated, destination host and destination node number.Receive after this instruction, preserve the relevant information (comprising configuration signal and temporary file) of virtual machine to be migrated, directly regain corresponding virtual machine last time according to the relevant information of preserving at destination host, completed the migration of virtual machine to be migrated.In this virtual machine migration method according to the nodal information selected target node of each physical node in migration instruction and destination host, rebuild VNUMA topological structure, state and the migration instruction matching degree of destination host self are fully taken into account, make the virtual machine after migration can adapt to more various hardware environment, in the process of migration, can not violate the one-to-one principle of the virtual NUMA node of virtual machine and the NUMA node of physical machine (physical host) simultaneously.
Described configuration information comprises total internal memory of virtual machine to be migrated, number and the I/O facility information of VCPU.Configuration information is the basis of virtual machine, can initially dissolve a virtual machine according to above information.
Described step generates temporary file by following steps in (1):
(1-1) page virtual machine to be migrated having been used dumps to temporary file, and in unloading process, the new dirty page producing of mark;
(1-2) add up the quantity of dirty page, and compare with the threshold value of setting:
If the quantity of dirty page is less than setting threshold, suspend virtual machine to be migrated, and by the content in virtual machine register to be migrated and I/O equipment state, the unloading of all dirty pages to temporary file;
Otherwise, return and continue execution step (1-1) to being labeled as dirty page.
In unloading process, virtual machine is still in work, in the time of unloading, still at access memory page (comprise and read and write).In transfer process, due to the effect of write operation, the content in page corresponding to virtual machine to be migrated is being constantly updated, and therefore needs circulation repeatedly to carry out unloading, after meeting some requirements, stops.Because unloading and write operation are parallel, may exist by some pages are carried out and write and unloading operation, now for improving the efficiency that generates temporary file, can first suspend unloading operation, after to be written completing again by this page of unloading to temporary file.
Dirty page refers to the page being modified in unloading process.In circulation each time, for each page, if while this page being carried out to write operation to this page of unloading after, using this page as dirty page, circulate next time and carry out unloading taking dirty page of this circulation as object.
Described threshold value be total internal memory of virtual machine to be migrated number of pages 1%～10%.
Choosing of threshold value should be closely related with source host (the current residing physical machine of virtual machine to be migrated) performance, it should consider the disk read-write speed with source host, and internal storage access speed, to prevent in the process of migration the dirty page of iteration migration continually, affect entire system efficiency.The number of pages of total internal memory is the sum of the page that total internal memory is corresponding, and 1%～10% this threshold value of page sum can adapt to the hardware condition of most of physical machine.
Described nodal information comprises total memory size of physical node and the memory size having used, and the number of physical cpu and the working time of each physical cpu within the time period of setting on this physical node.
In this time period, choose arbitrarily, duration is 50ms～200ms.
Described step (3) determines by following steps whether destination host exists destination node:
(3-1) according to the load balancing degrees L of each physical node on following formula calculating destination host
d:
Wherein,
for the working time of i CPU on this physical node in setting-up time section, t
totalfor the duration of setting-up time section, n is the physical cpu number on this physical node, M
usedfor this physical node has used memory size, M
totalfor total memory size of this physical node;
(3-2), according to load balancing degrees order from small to large, judge successively whether the internal memory of each physical node is more than or equal to the needed internal memory of dummy node:
If be more than or equal to the needed internal memory of dummy node, using this physical node as both candidate nodes, and continue to judge next physical node;
Otherwise, directly judge next physical node;
(3-3) until the number of both candidate nodes equal when destination node number or all physical nodes have all judged to stop:
(a) if the number of the both candidate nodes obtaining after stopping equals destination node number, total number of total number of the physical cpu of more all both candidate nodes and the VCPU of virtual machine to be migrated:
If total number of the physical cpu of all both candidate nodes is more than or equal to total number of the VCPU of virtual machine to be migrated, think and have destination node, and using both candidate nodes as destination node;
Otherwise, think and do not have destination node;
(b) if the number of the both candidate nodes obtaining after stopping is less than destination node number, thinks and do not have destination node;
Described destination node number equals the number of the destination node that moves instruction setting.
Load balancing degrees is lower, and this physical node surplus resources is more.Using load balancing degrees as a condition determining destination node, the physical node that preferably selection load balancing degrees is lower is as destination node, and consider the relation between the internal memory of virtual machine to be migrated and total number and each physical cpu of the VCPU of destination node number and virtual machine to be migrated, farthest ensured the overall performance of NUMA system.
As preferably, described step (3) determines by following steps whether destination host exists destination node:
(S3-1) according to the load balancing degrees L of each physical node on following formula calculating destination host
d:
Wherein,
for the working time of i CPU on this physical node in setting-up time section, t
totalfor the duration of setting-up time section, n is the physical cpu number on this physical node, M
usedfor this physical node has used memory size, M
totalfor total memory size of this physical node;
(S3-2), according to load balancing degrees order from small to large, judge successively whether the internal memory of each physical node is more than or equal to the needed internal memory of dummy node:
If be more than or equal to the needed internal memory of dummy node, using this physical node as both candidate nodes, and continue to judge next physical node;
Otherwise, directly judge next physical node;
(S3-3) until the number of both candidate nodes equal when destination node number or all physical nodes have all judged to stop:
(a) if the number of the both candidate nodes obtaining after stopping equals destination node number, total number of total number of the physical cpu of more all both candidate nodes and the VCPU of virtual machine to be migrated:
If total number of the physical cpu of all both candidate nodes is more than or equal to total number of the VCPU of virtual machine to be migrated, there is destination node in judgement, and using both candidate nodes as destination node;
Otherwise, destination node addend 1 is returned to execution step (3-1), until destination node number stops while being greater than physical node total of destination host, and there is not destination node in judgement;
(b) if the number of the both candidate nodes obtaining after stopping is less than destination node number, destination node is added to 1 and return to execution step (3-1), until destination node number stops while being greater than physical node total of destination host, and there is not destination node in judgement;
The initial value of described destination node number is the number of the destination node of migration instruction setting.
By dynamic adjustment aim nodes, attempt to a certain extent more physical nodes as destination node, can meet better the demand that destination node number is required to undemanding user.In the present invention, at each circulation time, destination node number is 1 from the step value adding, and can adopt in actual applications other step values, can also this be set to changing value from the stepping adding, and actual conditions are dynamically adjusted as required.
In described dummy node is needed, save as total internal memory of virtual machine to be migrated divided by destination node number.
In described step (3) by following steps by virtual machine (vm) migration to be migrated to destination node:
(S1) on destination node, build virtual machine according to configuration information, recover content, I/O equipment state and the page table of the register of virtual machine, and will in page table, be labeled as and skip leaf corresponding to the page table entry of the page having used;
(S2) start the virtual machine building, and recover the internal memory of this virtual machine according to the page having used in temporary file, and cancel the mark that skips leaf of corresponding page table entry.
Transition process is according to definite destination node reconstruct virtual machine to be migrated at destination host.In the time of this virtual machine of reconstruct, the internal memory of virtual machine to be migrated need to be write in the virtual machine of structure, in the present invention, build that directly to start after virtual machine be virtual machine operation, in operational process, carry out internal memory recovery, on the one hand according to certain order by the internal memory of the recovery virtual machine of the page in temporary file, on the other hand, carry out according to access order, if the page table entry corresponding to page of access is for skipping leaf, this page of priority restores.
The present invention also provides a kind of virtual machine (vm) migration system based on NUMA framework, comprising:
Virtual machine monitor, for receiving migration instruction, obtains the configuration information of virtual machine to be migrated, and described configuration information comprises number and the I/O facility information etc. of the memory information of virtual machine, VCPU;
Temporary file maker, for according to the configuration information of virtual machine to be migrated and the page that used, and content and I/O equipment state in virtual machine register to be migrated generate temporary file, and close virtual machine to be migrated producing after temporary file;
Nodal information collector, for obtaining the nodal information of all physical nodes on destination host;
Destination node adaptation, for judging according to described nodal information and migration instruction whether destination host exists destination node;
Virtual machine restorer, for proceeding as follows according to the judged result of destination node adaptation:
If there is destination node in judgement, according to the configuration information of temporary file and virtual machine to be migrated by virtual machine (vm) migration to destination node;
If judgement does not exist destination node, send error reporting.
Compared with prior art, the invention has the advantages that:
(a) on the basis of migration, ensured the strict binding of the virtual NUMA node of virtual machine and the NUMA node (being physical node) of physical machine to make the hydraulic performance decline of virtual machine less;
(b) according to the hardware case of destination host, the virtual NUMA nodes of virtual machine is changed, make virtual machine can adapt to more various hardware environment;
(c) adopt temporary file to record the status information of virtual machine, virtual machine (vm) migration process can be delayed, can be transmitted by far-end, more flexible on time and space.
Brief description of the drawings
Fig. 1 is the process flow diagram of the virtual machine migration method based on NUMA framework of the present embodiment;
Fig. 2 is the structural representation of the virtual machine (vm) migration system based on NUMA framework of the present embodiment.
Embodiment
Describe the present invention below in conjunction with the drawings and specific embodiments.
The MUMA framework of the virtual machine (vm) migration system of the present embodiment based on as shown in Figure 1, this MUMA framework comprises m main frame (physical host, main frame 1 ..., main frame m), if each physical host is provided with physical node, wherein on first main frame (main frame 1), have p physical node (physical node 1, physical node 2 ..., physical node p).As shown in Figure 1, this virtual machine (vm) migration system comprises:
Virtual machine monitor, for receiving migration instruction, obtains the configuration information of virtual machine to be migrated, and this configuration information comprises the memory information of virtual machine, number and the I/O facility information etc. of VCPU;
Temporary file maker, for according to the configuration information of virtual machine to be migrated and the page that used, and content and I/O equipment state in virtual machine register to be migrated generate temporary file, and close virtual machine to be migrated generating after temporary file;
Nodal information collector, for adopting the nodal information that obtains all physical nodes on destination host;
Destination node adaptation, judges according to nodal information and migration instruction whether destination host exists destination node;
Virtual machine restorer, for proceeding as follows according to the judged result of destination node adaptation:
If there is destination node in judgement, according to the configuration information of temporary file and virtual machine to be migrated by virtual machine (vm) migration to destination node;
If judgement does not exist destination node, send error reporting.
The migration instruction of the present embodiment comprises the information such as virtual machine information to be migrated, destination host and destination node number.
As shown in Figure 2, the virtual machine migration method of the present embodiment comprises:
(1) obtain the configuration information of virtual machine to be migrated, and content in the page, the virtual machine register that have used according to virtual machine to be migrated and I/O equipment state generate temporary file.Generate temporary file by following steps:
(1-1) page virtual machine to be migrated having been used dumps to temporary file, and in unloading process, the new dirty page producing of mark;
(1-2) add up the quantity of dirty page, and compare with the threshold value of setting (in the present embodiment for the number of pages of total internal memory 3%):
If the quantity of dirty page is less than setting threshold, suspend virtual machine to be migrated, and by the content in virtual machine register to be migrated and I/O equipment state, the unloading of all dirty pages to temporary file;
Otherwise, return and continue execution step (1-1) to being labeled as dirty page.
In unloading process, virtual machine is still in work, in the time of unloading, still at access memory page (comprise and read and write).In transfer process, due to the effect of write operation, the content in page corresponding to virtual machine to be migrated is being constantly updated, and therefore needs circulation repeatedly to carry out unloading, after meeting some requirements, stops.Because unloading and write operation are parallel, may exist by some pages are carried out and write and unloading operation, now for improving the efficiency that generates temporary file, can first suspend unloading operation, after to be written completing again by this page of unloading to temporary file.
In circulation each time, for each page, if while this page being carried out to write operation to this page of unloading after, using this page as dirty page, circulate next time and carry out unloading taking dirty page of this circulation as object.
Configuration information is the basis of virtual machine, can initially dissolve a virtual machine according to above information.In the present embodiment, configuration information comprises total internal memory of virtual machine to be migrated, number and the I/O facility information of VCPU.
(2) close virtual machine to be migrated, guarantee that content, I/O state and the internal memory of the register of virtual machine to be migrated no longer changes, the state in temporary file and virtual machine to be migrated is consistent.
(3) obtain the nodal information of all physical nodes on destination host, and judge according to nodal information and migration instruction whether destination host exists destination node:
If exist, according to temporary file and configuration information, by virtual machine (vm) migration to be migrated to destination node;
Otherwise, send error reporting.
Wherein, nodal information comprises total memory size of physical node and the memory size having used, and the number of physical cpu and the working time of each physical cpu within the time period of setting on this physical node.
In the present embodiment, determine by following steps whether destination host exists destination node:
(S3-1) according to the load balancing degrees L of each physical node on following formula calculating destination host
d:
Wherein,
for the working time of i CPU on this physical node in setting-up time section, t
totalfor the duration (t in the present embodiment of setting-up time section
total=100ms), n is the physical cpu number on this physical node, M
usedfor this physical node has used memory size, M
totalfor total memory size of this physical node.
(S3-2), according to load balancing degrees order from small to large, judge successively whether the internal memory of each physical node is more than or equal to the needed internal memory of dummy node:
If be more than or equal to the needed internal memory of dummy node, using this physical node as both candidate nodes, and continue to judge next physical node;
Otherwise, directly judge next physical node;
(S3-3) until the number of both candidate nodes equal when destination node number or all physical nodes have all judged to stop:
(a) if the number of the both candidate nodes obtaining after stopping equals destination node number, total number of total number of the physical cpu of more all both candidate nodes and the VCPU of virtual machine to be migrated:
If total number of the physical cpu of all both candidate nodes is more than or equal to total number of the VCPU of virtual machine to be migrated, there is destination node in judgement, and using both candidate nodes as destination node;
Otherwise, destination node addend 1 is returned to execution step (3-1), until destination node number stops while being greater than physical node total of destination host, and there is not destination node in judgement;
(b) if the number of the both candidate nodes obtaining after stopping is less than destination node number, destination node is added to 1 and return to execution step (3-1), until destination node number stops while being greater than physical node total of destination host, and there is not destination node in judgement.
In cyclic process, the initial value of destination node number is the number of the destination node of migration instruction setting.In dummy node is needed, save as total internal memory of virtual machine to be migrated divided by destination node number (this destination node number corresponding to circulation).
In the present embodiment by following steps by virtual machine (vm) migration to be migrated to destination node:
(S1) on destination node, build virtual machine according to configuration information, recover content, I/O equipment state and the page table of the register of virtual machine, and will in page table, be labeled as and skip leaf corresponding to the page table entry of the page having used;
(S2) start the virtual machine building, and recover the internal memory of this virtual machine according to the page having used in temporary file, and cancel the mark that skips leaf of corresponding page table entry.
After structure virtual machine, directly start is virtual machine operation, in operational process, carry out internal memory recovery, on the one hand according to certain order by the internal memory of the recovery virtual machine of the page in temporary file, on the other hand, carry out according to access order, if the page table entry corresponding to page of access is for skipping leaf, this page of priority restores.
Above-described embodiment has been described in detail technical scheme of the present invention and beneficial effect; be understood that and the foregoing is only most preferred embodiment of the present invention; be not limited to the present invention; all any amendments of making within the scope of principle of the present invention, supplement and be equal to replacement etc., within all should being included in protection scope of the present invention.
Claims (10)
1. the virtual machine migration method based on NUMA framework, is characterized in that, comprising:
(1) obtain the configuration information of virtual machine to be migrated, and content in the page, the virtual machine register that have used according to virtual machine to be migrated and I/O equipment state generate temporary file;
(2) close virtual machine to be migrated;
(3) obtain the nodal information of all physical nodes on destination host, and judge according to described nodal information and migration instruction whether destination host exists destination node:
If exist, according to described temporary file and configuration information, by virtual machine (vm) migration to be migrated to destination node;
Otherwise, send error reporting.
2. the virtual machine migration method based on NUMA framework as claimed in claim 1, is characterized in that, described configuration information comprises total internal memory of virtual machine to be migrated, number and the I/O facility information of VCPU.
3. the virtual machine migration method based on NUMA framework as claimed in claim 2, is characterized in that, described step generates temporary file by following steps in (1):
(1-1) page virtual machine to be migrated having been used dumps to temporary file, and in unloading process, the new dirty page producing of mark;
(1-2) add up the quantity of dirty page, and compare with the threshold value of setting:
If the quantity of dirty page is less than setting threshold, suspend virtual machine to be migrated, and by the content in virtual machine register to be migrated and I/O equipment state, the unloading of all dirty pages to temporary file;
Otherwise, return and continue execution step (1-1) to being labeled as dirty page.
4. the virtual machine migration method based on NUMA framework as claimed in claim 3, is characterized in that, 1%～10% of the number of pages of total internal memory that described threshold value is virtual machine to be migrated.
5. the virtual machine migration method based on NUMA framework as claimed in claim 4, it is characterized in that, described nodal information comprises total memory size of physical node and the memory size having used, and the number of physical cpu and the working time of each physical cpu within the time period of setting on this physical node.
6. the virtual machine migration method based on NUMA framework as claimed in claim 5, is characterized in that, described step (3) determines by following steps whether destination host exists destination node:
(3-1) according to the load balancing degrees L of each physical node on following formula calculating destination host
d:
Wherein,
for the working time of i CPU on this physical node in setting-up time section, t
totalfor the duration of setting-up time section, n is the physical cpu number on this physical node, M
usedfor this physical node has used memory size, M
totalfor total memory size of this physical node;
(3-2), according to load balancing degrees order from small to large, judge successively whether the internal memory of each physical node is more than or equal to the needed internal memory of dummy node:
If be more than or equal to the needed internal memory of dummy node, using this physical node as both candidate nodes, and continue to judge next physical node;
Otherwise, directly judge next physical node;
(3-3) until the number of both candidate nodes equal when destination node number or all physical nodes have all judged to stop:
(a) if the number of the both candidate nodes obtaining after stopping equals destination node number, total number of total number of the physical cpu of more all both candidate nodes and the VCPU of virtual machine to be migrated:
If total number of the physical cpu of all both candidate nodes is more than or equal to total number of the VCPU of virtual machine to be migrated, think and have destination node, and using both candidate nodes as destination node;
Otherwise, think and do not have destination node;
(b) if the number of the both candidate nodes obtaining after stopping is less than destination node number, thinks and do not have destination node;
Described destination node number equals the number of the destination node that moves instruction setting.
7. the virtual machine migration method based on NUMA framework as claimed in claim 5, is characterized in that, described step (3) determines by following steps whether destination host exists destination node:
(S3-1) according to the load balancing degrees L of each physical node on following formula calculating destination host
d:
Wherein,
for the working time of i CPU on this physical node in setting-up time section, t
totalfor the duration of setting-up time section, n is the physical cpu number on this physical node, M
usedfor this physical node has used memory size, M
totalfor total memory size of this physical node;
(S3-2), according to load balancing degrees order from small to large, judge successively whether the internal memory of each physical node is more than or equal to the needed internal memory of dummy node:
If be more than or equal to the needed internal memory of dummy node, using this physical node as both candidate nodes, and continue to judge next physical node;
Otherwise, directly judge next physical node;
(S3-3) until the number of both candidate nodes equal when destination node number or all physical nodes have all judged to stop:
(a) if the number of the both candidate nodes obtaining after stopping equals destination node number, total number of total number of the physical cpu of more all both candidate nodes and the VCPU of virtual machine to be migrated:
If total number of the physical cpu of all both candidate nodes is more than or equal to total number of the VCPU of virtual machine to be migrated, there is destination node in judgement, and using both candidate nodes as destination node;
Otherwise, destination node addend 1 is returned to execution step (3-1), until destination node number stops while being greater than physical node total of destination host, and there is not destination node in judgement;
(b) if the number of the both candidate nodes obtaining after stopping is less than destination node number, destination node is added to 1 and return to execution step (3-1), until destination node number stops while being greater than physical node total of destination host, and there is not destination node in judgement;
The initial value of described destination node number is the number of the destination node of migration instruction setting.
8. the virtual machine migration method based on NUMA framework as described in claim 6 or 7, is characterized in that, saves as total internal memory of virtual machine to be migrated divided by destination node number in described dummy node is needed.
9. the virtual machine migration method based on NUMA framework as claimed in claim 1, is characterized in that, in described step (3) by following steps by virtual machine (vm) migration to be migrated to destination node:
(S1) on destination node, build virtual machine according to configuration information, recover content, I/O equipment state and the page table of the register of virtual machine, and will in page table, be labeled as and skip leaf corresponding to the page table entry of the page having used;
(S2) start the virtual machine building, and recover the internal memory of this virtual machine according to the page having used in temporary file, and cancel the mark that skips leaf of corresponding page table entry.
10. the virtual machine (vm) migration system based on NUMA framework, is characterized in that, comprising:
Virtual machine monitor, for receiving migration instruction, obtains the configuration information of virtual machine to be migrated, and described configuration information comprises number and the I/O facility information etc. of the memory information of virtual machine, VCPU;
Temporary file maker, for according to the configuration information of virtual machine to be migrated and the page that used, and content and I/O equipment state in virtual machine register to be migrated generate temporary file, and close virtual machine to be migrated producing after temporary file;
Nodal information collector, for obtaining the nodal information of all physical nodes on destination host;
Destination node adaptation, for judging according to described nodal information and migration instruction whether destination host exists destination node;
Virtual machine restorer, for proceeding as follows according to the judged result of destination node adaptation:
If there is destination node in judgement, according to the configuration information of temporary file and virtual machine to be migrated by virtual machine (vm) migration to destination node;
If judgement does not exist destination node, send error reporting.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410256198.XA CN104123171B (en) | 2014-06-10 | 2014-06-10 | Virtual machine migrating method and system based on NUMA architecture |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410256198.XA CN104123171B (en) | 2014-06-10 | 2014-06-10 | Virtual machine migrating method and system based on NUMA architecture |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104123171A true CN104123171A (en) | 2014-10-29 |
| CN104123171B CN104123171B (en) | 2017-05-10 |
Family
ID=51768595
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410256198.XA Active CN104123171B (en) | 2014-06-10 | 2014-06-10 | Virtual machine migrating method and system based on NUMA architecture |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104123171B (en) |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104461735A (en) * | 2014-11-28 | 2015-03-25 | 杭州华为数字技术有限公司 | Method and device for distributing CPU resources in virtual scene |
| CN104636187A (en) * | 2015-02-15 | 2015-05-20 | 浙江大学 | Virtual machine scheduling method in NUMA (non uniform memory access) architecture and based on load prediction |
| CN105573815A (en) * | 2015-12-10 | 2016-05-11 | 中国联合网络通信集团有限公司 | Virtual machine migration method and device |
| CN107229506A (en) * | 2017-05-27 | 2017-10-03 | 郑州云海信息技术有限公司 | A kind of virtual machine migration method and device |
| CN107807851A (en) * | 2016-09-09 | 2018-03-16 | 华为数字技术（苏州）有限公司 | Moving method and device of a kind of virutal machine memory between NUMA node |
| CN109471702A (en) * | 2018-11-15 | 2019-03-15 | 郑州云海信息技术有限公司 | A kind of management method and device of virtual machine |
| CN111198747A (en) * | 2018-11-20 | 2020-05-26 | 中移(苏州)软件技术有限公司 | Method for allocating memory for virtual machine and virtual machine |
| CN111240797A (en) * | 2019-12-31 | 2020-06-05 | 苏州浪潮智能科技有限公司 | Method and equipment for optimizing performance of virtual machine |
| CN111263406A (en) * | 2019-12-17 | 2020-06-09 | 航天信息股份有限公司 | Load balancing method and system based on virtual access point |
| CN113791870A (en) * | 2021-09-24 | 2021-12-14 | 上海交通大学 | Fine-grained migration method and system for distributed system of WebAssembly virtual machine |
| CN115269120A (en) * | 2022-08-01 | 2022-11-01 | 江苏安超云软件有限公司 | NUMA node scheduling method, device, equipment and storage medium of virtual machine |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102185774A (en) * | 2011-05-10 | 2011-09-14 | 中兴通讯股份有限公司 | Method, monitor and system for seamless migration of virtual machine |
| CN102917055A (en) * | 2012-10-18 | 2013-02-06 | 华为技术有限公司 | Method and device of online migration of virtual machine, and terminal device |
| CN103218260A (en) * | 2013-03-06 | 2013-07-24 | 中国联合网络通信集团有限公司 | Virtual machine migration method and device |
| CN103577249A (en) * | 2013-11-13 | 2014-02-12 | 中国科学院计算技术研究所 | Method and system for virtual machine online migration |
-
2014
- 2014-06-10 CN CN201410256198.XA patent/CN104123171B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102185774A (en) * | 2011-05-10 | 2011-09-14 | 中兴通讯股份有限公司 | Method, monitor and system for seamless migration of virtual machine |
| CN102917055A (en) * | 2012-10-18 | 2013-02-06 | 华为技术有限公司 | Method and device of online migration of virtual machine, and terminal device |
| CN103218260A (en) * | 2013-03-06 | 2013-07-24 | 中国联合网络通信集团有限公司 | Virtual machine migration method and device |
| CN103577249A (en) * | 2013-11-13 | 2014-02-12 | 中国科学院计算技术研究所 | Method and system for virtual machine online migration |
Cited By (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104461735A (en) * | 2014-11-28 | 2015-03-25 | 杭州华为数字技术有限公司 | Method and device for distributing CPU resources in virtual scene |
| CN104461735B (en) * | 2014-11-28 | 2018-03-27 | 杭州华为数字技术有限公司 | A kind of method and apparatus that cpu resource is distributed under virtualization scene |
| CN104636187A (en) * | 2015-02-15 | 2015-05-20 | 浙江大学 | Virtual machine scheduling method in NUMA (non uniform memory access) architecture and based on load prediction |
| CN104636187B (en) * | 2015-02-15 | 2018-01-09 | 浙江大学 | Dispatching method of virtual machine in NUMA architecture based on load estimation |
| CN105573815B (en) * | 2015-12-10 | 2019-02-15 | 中国联合网络通信集团有限公司 | A kind of method and device of virtual machine (vm) migration |
| CN105573815A (en) * | 2015-12-10 | 2016-05-11 | 中国联合网络通信集团有限公司 | Virtual machine migration method and device |
| CN107807851A (en) * | 2016-09-09 | 2018-03-16 | 华为数字技术（苏州）有限公司 | Moving method and device of a kind of virutal machine memory between NUMA node |
| CN107229506B (en) * | 2017-05-27 | 2020-09-18 | 苏州浪潮智能科技有限公司 | Virtual machine migration method and device |
| CN107229506A (en) * | 2017-05-27 | 2017-10-03 | 郑州云海信息技术有限公司 | A kind of virtual machine migration method and device |
| CN109471702A (en) * | 2018-11-15 | 2019-03-15 | 郑州云海信息技术有限公司 | A kind of management method and device of virtual machine |
| CN111198747A (en) * | 2018-11-20 | 2020-05-26 | 中移(苏州)软件技术有限公司 | Method for allocating memory for virtual machine and virtual machine |
| CN111198747B (en) * | 2018-11-20 | 2022-12-16 | 中移(苏州)软件技术有限公司 | Method for allocating memory for virtual machine and virtual machine |
| CN111263406A (en) * | 2019-12-17 | 2020-06-09 | 航天信息股份有限公司 | Load balancing method and system based on virtual access point |
| CN111240797A (en) * | 2019-12-31 | 2020-06-05 | 苏州浪潮智能科技有限公司 | Method and equipment for optimizing performance of virtual machine |
| CN111240797B (en) * | 2019-12-31 | 2022-06-14 | 苏州浪潮智能科技有限公司 | Method and equipment for optimizing performance of virtual machine |
| CN113791870A (en) * | 2021-09-24 | 2021-12-14 | 上海交通大学 | Fine-grained migration method and system for distributed system of WebAssembly virtual machine |
| CN113791870B (en) * | 2021-09-24 | 2024-03-26 | 上海交通大学 | WebAssemble virtual machine distributed system fine granularity migration method and system |
| CN115269120A (en) * | 2022-08-01 | 2022-11-01 | 江苏安超云软件有限公司 | NUMA node scheduling method, device, equipment and storage medium of virtual machine |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104123171B (en) | 2017-05-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104123171A (en) | Virtual machine migrating method and system based on NUMA architecture | |
| US9229878B2 (en) | Memory page offloading in multi-node computer systems | |
| Sahni et al. | A hybrid approach to live migration of virtual machines | |
| CN103955399B (en) | A kind of virtual machine migration method and device and physical host | |
| CN102136993B (en) | A kind of methods, devices and systems of Data Migration | |
| US10452538B2 (en) | Determining task scores reflective of memory access statistics in NUMA systems | |
| US10613957B2 (en) | Achieving balanced execution through runtime detection of performance variation | |
| US10061622B2 (en) | Updating memory topology information for virtual machines | |
| CN104166594B (en) | Control method for equalizing load and relevant apparatus | |
| US9128824B2 (en) | In-place change between transient and persistent state for data structures on non-volatile memory | |
| CN104111897A (en) | Data processing method, data processing device and computer system | |
| CN110427284A (en) | Data processing method, distributed system, computer system and medium | |
| Wang et al. | Ada-Things: An adaptive virtual machine monitoring and migration strategy for internet of things applications | |
| JP2019500705A (en) | Network-attached memory using selective resource migration | |
| CN104636187A (en) | Virtual machine scheduling method in NUMA (non uniform memory access) architecture and based on load prediction | |
| CN103064733A (en) | Cloud computing virtual machine live migration technology | |
| CN104216784A (en) | Hotspot balance control method and related device | |
| US20220050722A1 (en) | Memory pool management | |
| US20180088975A1 (en) | Virtual Machine Live Migration Using Intelligent Order Of Pages To Transfer | |
| CN104102460A (en) | Cloud computing-based memory management method and device | |
| CN104572941A (en) | Data storage method, device and equipment | |
| Kale | Virtual machine migration techniques in cloud environment: A survey | |
| CN105930202A (en) | Migration policy for virtual machine with three thresholds | |
| Dustdar et al. | Rethinking divide and conquer—Towards holistic interfaces of the computing stack | |
| CN102789384B (en) | Method for implementing server operating system applied to Loongson 3B processor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |