CN106293944A - System and optimization method is accessed based on nonuniformity I/O under virtualization multi-core environment - Google Patents

Abstract

The invention discloses and access system and optimization method based on nonuniformity I/O under a kind of virtualization multi-core environment, relate to computer virtualized field, pass through the performance monitoring instrument of amendment including performance detection module and monitor the hardware information of virtual machine and physical host in real time；The hardware information that thread binding module is collected according to described performance detection module, judge that current system is in low-load or high capacity, if be under high load condition, just it is tied to additionally load on relatively low node by the virtual machine thread on node higher for duty factor；If the load of internal memory migration module current system is relatively low, just by node nearest for relevant thread migration to distance network adapter.System of the present invention, set up affinity Optimized model based on the I/O performance under virtualization multi-core environment, the optimization Placement Strategy of real-time dynamic support high bandwidth and low latency is provided for system, thus utilize multi-core resource and the performance of high performance network adapter efficiently, effectively reduce the load of system.

Description

System and optimization method is accessed based on nonuniformity I/O under virtualization multi-core environment

Technical field

The present invention relates to computer virtualized field, particularly relate under a kind of virtualization multi-core environment based on nonuniformity I/ The system of O access and optimization method thereof.

Background technology

Virtualization is the key technology in cloud computing.Virtualization allows multiple computer systems to operate in a physical computing On machine, the hardware resource (CPU, internal memory, I/O equipment etc.) of physical machine is abstracted into be similar to electric power can the money of on-demand use Source, it is provided that use to client.The use of Intel Virtualization Technology, greatly reduces the input that server is bought by small business, with Time also greatly improve the service efficiency of free host, so Intel Virtualization Technology widely exists in current high-performance server The inside, relatively representational virtualization cloud computing example has the EC2 of Amazon and Ali's cloud of Alibaba.

In Intel Virtualization Technology key component be exactly virtual machine monitor (Virtual Machine Management, VMM).Virtual machine monitor be responsible for by host hardware Resource Abstract to virtual machine use, be also responsible for simultaneously virtual machine management and Communication between virtual machine etc..Traditional hardware resource includes cpu resource, and memory source and I/O resource etc., in nonuniformity Under the framework of internal storage access (Non-Uniform Memory Access, NUMA), Intel Virtualization Technology concern is primarily with promoting this Performance after a little hardware resource virtualizations.However as current high performance network technology and the development of CPU multi-core technology, hardware Performance be not the most bottleneck, efficient collaborative between these high-performance hardware is processed into for bottleneck on the contrary, particularly exists Under multi-core environment, process high performance network I/O request the most efficiently, because the least host process delay all can be to virtualization In network application produce huge hydraulic performance decline.

I/O virtualization is one of virtualization important component part, and I/O virtualization is primarily directed to the void of network interface card PCIe function Planization, the quantity of EVM(extended virtual machine) as much as possible in the case of not losing I/O performance.However as high-performance physical machine Being continuously increased of check figure, the node placing physical core is also continuously increased, and many physical core access I/O resource the most efficiently and become more Come the most important.To this end, also been proposed nonuniformity I/O on the basis of based on NUMA architecture to access (Non-Uniform I/O Access, NUIOA) as shown in Figure 1.The node of each physical core is wrapped and is joined directly together this various equipment, and these equipment are not Symmetrical, the node of far-end to access certain equipment can only be by internodal internal connection, thus than the node of near-end Accessing slow, this asymmetric access adds the delay of system, has ultimately resulted in the decline of virtual machine performance.As Fig. 2 is exactly The remote access of one traditional internal memory, but owing to system before does not accounts for the existence of high performance network devices, be The optimisation strategy of system have ignored this key element of network interface card, and particularly under high performance network environment, this performance brought of ignoring is damaged Lose huge.CPU network interface card to be transmitted data on node 2 have to pass through node 2 and node 1, between node 1 and node 0 Internal connection, which not only adds the bandwidth occupancy of internal connection, also add the access delay of data.

Present being normally applied the most all is deployed in high in the clouds, presents networking, distributed characteristic, thus high performance Network transmission reliably plays critical effect, the therefore parent under current multicore architecture to these effectively operations applied Must be in view of the importance of I/O equipment with property modeling.

Present essentially all of virtual machine monitor, as Xen, KVM and VMware ESXi is essentially all employing by one The VCPU of individual virtual machine and all of internal memory are all dispatched on a node keep local IP access, but this method exists very Big defect because the load-balancing technique of system and other technologies can load between dynamic balance CPU and internal memory, cause former This Placement Strategy is disturbed, finally makes strategy fails.The model of existing dynamic placement is all based on NUMA, and they model Focus be the locality of internal memory, or consider internal memory locality and cache hit probability simultaneously, do not account for network I/ The importance of O device.The modeling also having them only considered the affinity between thread and hardware, does not accounts for thread and thread Between affinity, cause modeling accuracy existing problems.

Existing about virtualization multicore architecture under I/O Performance tuning method mainly include thread binding and internal memory migration. Wherein, thread binding refers to that the VCPU thread of the virtual machine by running application program is tied on specific node, internal memory migration Refer to the internal memory migration of virtual machine by running application program on specific node, if by VCPU thread and virutal machine memory It is tied on same node, the affinity between CPU and internal memory so can be allowed to reach maximum, thus promote the property of system Energy.But, existing research is primarily upon CPU and two dimensions of internal memory, have ignored the existence of this key element of network interface card.Carry out new Affinity based on I/O equipment modeling, to reach optimum systematic function.

Therefore, this invention address that exploitation, under NUIOA framework, is developed under one virtualization multi-core environment based on non-uniform Property I/O access system and optimization method, set up affinity Optimization Modeling based on the I/O performance under virtualization multi-core environment, from And utilize multi-core resource and the performance of high performance network adapter efficiently, effectively reduce the load of system, be adapted to current Application under high performance network environment.

Summary of the invention

Because the drawbacks described above of prior art, how the technical problem to be solved is at virtualization multinuclear structure Under frame, outside CPU and two dimensions of internal memory, carry out affinity modeling to based on I/O equipment, provide real-time dynamic for system The optimization Placement Strategy of support high bandwidth and low latency, to reach the overall system performance of optimum.

For achieving the above object, the invention provides under a kind of virtualization multi-core environment and access system based on nonuniformity I/O System, including performance monitoring module, thread binding module and internal memory migration module, described performance detection module is configured to repair The performance monitoring instrument changed monitors the hardware information of virtual machine and physical host in real time；Described thread binding module is configured to root The hardware information collected according to described performance detection module, it is determined that current system is in low-load or high capacity, if be in Under high load condition, just it is tied to additionally load on relatively low node by the virtual machine thread on node higher for duty factor；Institute If stating internal memory migration module, to be configured to the load of current system relatively low, just by relevant thread migration to distance Network adaptation On the node that device is nearest.

Further, described hardware information includes that the application program in virtual machine accesses number of times and the I/O request number of times of page And the real-time cpu load of physical host and internal memory load.

Present invention also offers a kind of optimization method virtualizing and accessing system under multi-core environment based on nonuniformity I/O, Comprise the following steps:

(1) performance monitoring module, thread binding module and internal memory migration module are provided；

(2) access times and the I/O request in the unit interval of virtual machine internal page is simultaneously monitored by performance monitoring module Number of times；

(3) load and the load of internal memory of the CPU of physical host is monitored in real time by performance monitoring module；

(4) when certain node load of physical host is higher than threshold value, described node needs to carry out the migration of thread, it is judged that The formula of condition is:

$\frac{TMT\[T\]\[k\]}{\underset{j=1}{\overset{n}{\Σ}}TMT\[T\]\[j\]}DT\[T\]\[k\]\≤\frac{TMT\[T\]\[p\]}{\underset{j=1}{\overset{n}{\Σ}}TMT\[T\]\[j\]}DT\[T\]\[p\]$

Wherein, the storage and distribution of TMT matrix table timberline journey, DT matrix represents internodal access delay, is migrated by thread T On node K rather than the standard on node P that moves to is to prolong to the average retardation of node K less than to the average access of node P Late；

(5) when certain node load of physical machine is within normal range, then will be distributed over this thread on remote node The hot page migration of access on local node, the judgment formula of described hot page is:

If NodeAcc [n] [i] ＞ 2*NodeAcc [n] [j]

Wherein, NodeAcc [n] [i] represents that node n accesses the number of times of page i；

(6) after the address of the hot page in virtual machine is converted to the physical address of physical machine, call move a page function should With the hot page migration of program to on destination node；

(7) hot page moves after page completes, and continues back at the performance of performance detection module monitoring system.

Because the existing defect of existing multiple nucleus system, the present invention provides one based on virtualization ring under NUIOA framework The support high bandwidth and low latency in border place scheduling strategy in real time, thus improve the performance of the application program run inside virtual machine, As it is shown on figure 3, mainly construct three modules, performance detection module is responsible for collecting some hardware inside virtual machine and physical machine The access times of information, such as page, the real time load situation of I/O request number of times and physical machine, thread binding module is mainly responsible for Carry out the thread binding under high capacity environment according to the hardware information collected, internal memory migration module is mainly responsible for based on low-load feelings Internal memory migration under condition.It is further described below:

Performance detection module monitors the hardware information of virtual machine and physical host in real time by the performance monitoring instrument of amendment, Mainly include that the application program in virtual machine accesses number of times and I/O request number of times, the real time load situation inside physical host of page Etc..

The hardware information that thread binding module is collected according to performance detection module, it is determined that current system is in low-load also It is high capacity, if be under high load condition, it is necessary to the virtual machine thread on node higher for duty factor is tied to separately On outer node, thus reduce load.

Internal memory migration module, if in the case of current system is in than relatively low load, it is possible to by relevant thread Move on the node that distance network adapter is nearest, thus avoid too much remote access, reduce the band of internodal interior company Width takies, and promotes the handling capacity of system.

Access system of the present invention and optimization method are compared with existing modeling method based on NUMA architecture to be had Some advantage below:

(1) consider network adapter and processor node affinity, traditional modeling method with the addition of one Individual dimension, system is better able to reflect the importance of the network equipment under current high performance network environment；

(2) consider the affinity between virtual machine many VCPU thread simultaneously, thread higher for affinity is placed into The data communication between cross-node is reduced in same node point；

(3) have employed more fine-grained modeling matrix, establish access delay matrix and thread internal memory mapping matrix comes Carrying out last placement scheduling decision, fine granularity can more comprehensively embody the accuracy of modeling.

Below with reference to accompanying drawing, the technique effect of design, concrete structure and the generation of the present invention is described further, with It is fully understood from the purpose of the present invention, feature and effect.

Accompanying drawing explanation

Fig. 1 is traditional nonuniformity I/O access architectures schematic diagram；

Fig. 2 is conventional without the remote access schematic diagram considering network interface card key element；

Fig. 3 is the system architecture diagram of a preferred embodiment of the present invention；

Fig. 4 is the optimization method flow chart of a preferred embodiment of the present invention.

Detailed description of the invention

Elaborating embodiments of the invention below in conjunction with the accompanying drawings, the present embodiment is being front with technical solution of the present invention Put and implement, following present detailed embodiment and concrete operating process, but protection scope of the present invention does not limits In following embodiment.

As shown in Figure 4, the system optimization side accessed based on nonuniformity I/O under virtualization multi-core environment of the present invention Method comprises the following steps:

Step 1, simultaneously monitored by performance monitoring module virtual machine internal page access times and in the unit interval I/O please The number of times asked.Recorded the number of times of the page of each node visit by an array, the I/O request number of times of unit interval class is permissible Directly obtain.Load inside monitoring physical host, the load of mainly CPU and the load of internal memory the most in real time.

Step 2, when physical machine certain node load higher than certain threshold value time (this threshold value is relevant to the configuration of system), table This node bright needs to carry out the migration of thread, and concrete moving method is represented by a formula:

$\frac{TMT\[T\]\[k\]}{\underset{j=1}{\overset{n}{\Σ}}TMT\[T\]\[j\]}DT\[T\]\[k\]\≤\frac{TMT\[T\]\[p\]}{\underset{j=1}{\overset{n}{\Σ}}TMT\[T\]\[j\]}DT\[T\]\[p\]$

The storage and distribution of TMT matrix table timberline journey, DT matrix represents internodal access delay, and thread T moves to joint Point K is upper rather than the standard on node P that moves to is to be less than the average access latency of node P to the average retardation of node K.

Step 3, when certain node load of physical machine is within normal range (this threshold value is relevant to the configuration of system), just Will be distributed over the hot page migration of access of this thread on remote node as far as possible to local node, it is possible to use Numactl API carries out moving page, the judgement employing below equation of hot page:

If NodeAcc [n] [i] ＞ 2*NodeAcc [n] [j]

NodeAcc [n] [i] represents that node n accesses the number of times of page i, and formula shows that the node access to web page number of times of maximum is big Page before second largest twice is considered as is all hot page.Then the address of the hot page in virtual machine is converted to physical machine After physical address, a page module of moving call function move_pages, by the hot page migration of application program to on destination node. Hot page moves after page completes, and continues back at the performance of performance detection module monitoring system.

The preferred embodiment of the present invention described in detail above.Should be appreciated that the ordinary skill of this area is without wound The property made work just can make many modifications and variations according to the design of the present invention.Therefore, all technical staff in the art The most on the basis of existing technology by the available technology of logical analysis, reasoning, or a limited experiment Scheme, all should be in the protection domain being defined in the patent claims.

Claims (3)

1. access system based on nonuniformity I/O under a virtualization multi-core environment, it is characterised in that include performance monitoring mould Block, thread binding module and internal memory migration module,

Described performance detection module is configured to the performance monitoring instrument of amendment and monitors virtual machine and physical host in real time Hardware information；

Described thread binding module is configured to the hardware information collected according to described performance detection module, it is determined that current system It is in low-load or high capacity, if be under high load condition, just by the virtual machine thread on node higher for duty factor It is tied to additionally load on relatively low node；

If the load that described internal memory migration module is configured to current system is relatively low, just by relevant thread migration to distance webs On the node that network adapter is nearest.

2. access system based on nonuniformity I/O under virtualization multi-core environment as claimed in claim 1, it is characterised in that institute State the application program that hardware information includes in virtual machine and access the real-time of the number of times of page and I/O request number of times and physical host Cpu load and internal memory load.

3. one kind virtualize under multi-core environment based on nonuniformity I/O access system optimization method, it is characterised in that include with Lower step:

(1) performance monitoring module, thread binding module and internal memory migration module are provided；

(2) by performance monitoring module simultaneously monitor virtual machine internal page access times and in the unit interval I/O request time Number；

(3) load and the load of internal memory of the CPU of physical host is monitored in real time by performance monitoring module；

(4) when certain node load of physical host is higher than threshold value, described node needs to carry out the migration of thread, it is judged that condition Formula be:

$\frac{TMT\[T\]\[k\]}{\underset{j=1}{\overset{n}{\Σ}}TMT\[T\]\[j\]}DT\[T\]\[k\]\≤\frac{TMT\[T\]\[p\]}{\underset{j=1}{\overset{n}{\Σ}}TMT\[T\]\[j\]}DT\[T\]\[p\]$

Wherein, the storage and distribution of TMT matrix table timberline journey, DT matrix represents internodal access delay, and thread T moves to joint Point K is upper rather than the standard on node P that moves to is to be less than the average access latency of node P to the average retardation of node K；

(5) when certain node load of physical machine is within normal range, then will be distributed over the visit of this thread on remote node Ask hot page migration to local node, the judgment formula of described hot page is:

If NodeAcc [n] [i] ＞ 2*NodeAcc [n] [j]

Wherein, NodeAcc [n] [i] represents that node n accesses the number of times of page i；

(6), after the address of the hot page in virtual machine is converted to the physical address of physical machine, calls and move page function and will apply journey The hot page migration of sequence is to on destination node；

(7) hot page moves after page completes, and continues back at the performance of performance detection module monitoring system.