CN101706757A - I/O system and working method facing multi-core platform and distributed virtualization environment - Google Patents
I/O system and working method facing multi-core platform and distributed virtualization environment Download PDFInfo
- Publication number
- CN101706757A CN101706757A CN200910093454A CN200910093454A CN101706757A CN 101706757 A CN101706757 A CN 101706757A CN 200910093454 A CN200910093454 A CN 200910093454A CN 200910093454 A CN200910093454 A CN 200910093454A CN 101706757 A CN101706757 A CN 101706757A
- Authority
- CN
- China
- Prior art keywords
- virtual machine
- special
- territory
- request
- user
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
The invention discloses an I/O system facing a multi-core platform and a distributed virtualization environment. The system comprises privileged virtual machine, an I/O resource management module, at least one dedicated I/O domain, at least one local physical I/O device and a user virtual machine, wherein the privileged virtual machine is used for receiving the resource management command of the administrator and transmitting the resource management command to the I/O resource management module; the I/O resource management module is used for detecting the local physical I/O device and establishing a dedicated I/O domain for each local physical I/O device according to the resource management command; the dedicated I/O domain is used for initializing the local physical I/O device charged at the own risk, and starting to supply I/O service outwards after the initialization of the local physical I/O device is finished; the local physical I/O device is in one-to-one correspondence with the dedicated I/O domains; and the user virtual machine establishes an I/O access channel with the dedicated I/O domain and interacts with the dedicated I/O domain through the I/O access channel.
Description
Technical field
The present invention relates to Intel Virtualization Technology and Computer I/O technical field, particularly involvement aspect to the I/O of multi-core platform and distributed virtualization environment system and method for work.
Background technology
Virtual is the term of a broad sense, is moving on the virtual basis rather than on the real basis typically referring to computing element aspect the computing machine.By Intel Virtualization Technology, single hardware virtual be more virtual component, thereby make that more applications can parallel running, to make full use of the computation capability of multi-core platform.Intel Virtualization Technology can enlarge the capacity of hardware, simplifies the process that reconfigures of software.
Computer I/O technology is a crucial gordian technique all the time in development of computer.The I/O performance that its technical characteristic has determined computing machine has material impact to the overall performance of computing machine.Fundamentally, no matter still in the future now, the I/O technology all will restrict the application and the development of computer technology, especially in high-end calculating field.
To carry out virtually to I/O equipment, just need to monitor the behavior of guest virtual machine, intercept and capture the I/O operation of guest virtual machine,, send to device driver then, finish operation equipment by device driver to its analysis interpretation in addition.Wherein, according to whether revising Guest OS, can be the virtual two big classes that are divided into of local device: the I/O device virtualization of the I/O device virtualization in fully virtualized and half in virtual.
Partly virtual is the notion that Cambridge University proposes, and what accompany with it is the research and development of XEN project, and its appearance is a result of study how to improve the fully virtualized bigger performance loss that brings.Fig. 1 is the virtualized system assumption diagram of I/O among the Xen (a kind of half virtualized architecture).The bottom is a hardware resource among the figure, and Xen VMM (virtual machine monitor) runs on the hardware resource, hardware resource is managed, and virtual several virtual hardware environments that dissolves.The a plurality of Guest OS of operation on Xen VMM.There is not the application layer that the user application of revising runs on Guest OS.Inner nuclear layer operation at Guest OS has the front-end equipment driver, and user application is operated equipment by the front-end equipment driver.And the front-end equipment driver is not really operated equipment, and it is realized real operation of equipment by the rear end equipment driver.The rear end equipment driver is arranged in VMO.VMO is a special Guest OS, and we also are referred to as privileged Guest OS.In the kernel of VMO, moved the service of a rear end equipment driver.Its responsibility is exactly the I/O operation requests that acceptance is sent from other Guest OS, hand to the real device driver among the VMO, finish operation by real device driver, and an operating result that returns from real device driver is returned to the front-end equipment driver equipment.
But when facing multi-core platform, above-mentioned half virtual I/O framework can produce a lot of shortcomings and problem:
1. because the I/O that has introduced under virtual level and half virtualization policy handles framework, application asks I/O request final processed and return result from sending I/O, to pass through many levels, each level all will be handled the I/O request, increase the intermediate treatment level of I/O request greatly, reduced the efficient that I/O handles;
2. because all physical I/O device drives all are arranged in franchise virtual machine Dom0, in case certain physical I/O device drives has problems, may cause the collapse of whole multi-core platform, make and the virtual machine of the One's name is legion that moves on this platform and the collapse that the user uses cause serious consequence;
3. because all physical I/O device drives all are arranged in franchise virtual machine Dom0, all I/O requests all will be transferred to franchise virtual machine Dom0 and carry out, this makes franchise virtual machine Dom0 become the I/O performance bottleneck of system. the computing power of multi-core platform is strong more, the virtual machine that can support and application are many more, this situation is just obvious more. after the quantity of virtual machine that moves on certain multi-core platform and application surpasses the limit that franchise virtual machine Dom0 can support, a lot of virtual machines be applied in the frequent operation suspension of having in the operational process and ask to handle also return results to wait for the I/O that franchise virtual machine Dom0 is sent them. can cause the significant wastage of multi-core platform computing power like this;
4. in above-mentioned half virtual I/O framework down, do not provide good support to the migration of I/O visit, people need could realize the support of migration that I/O in the distributed virtualization environment is visited by the technology of concentrating such as remote I/O resource.If virtual machine or application access is resource on the local physical node, then these virtual machines and use can't be moved on other physical node, and this has also limited the realization of balancing dynamic load and the raising of resource utilization in the distributed computing system.
Summary of the invention
The object of the present invention is to provide I/O system and method for work towards multi-core platform and distributed virtualization environment, succinctly efficient, safe and reliable, I/O access mechanism that extensibility is good in multi-core platform and the distributed virtualization environment have been realized, and can provide efficient I/O access mechanism for numerous virtual machines of parallel running on the multi-core platform and user use, the migration of using for the virtual machine in the distributed virtualization environment and user provides good support.
Be the I/O system that realizes that purpose of the present invention provides towards multi-core platform and distributed virtualization environment, described system comprises: franchise virtual machine, I/O resource management module, at least one special I/O territory, at least one local physical I/O equipment and user virtual machine, wherein:
The privilege virtual machine is used for receiving management person's resource management order, and described resource management order is transmitted to the I/O resource management module;
The I/O resource management module is used for detecting local physical I/O equipment according to described resource management order, for each local physical I/O equipment is created a special I/O territory; And between user virtual machine and special I/O territory, set up the I/O access path;
At least one special I/O territory, local physical I/O the equipment that is used for self-responsibility carries out initialization, after local physical I/O device initialize is finished, this special I/O territory begins externally to provide I/O service: receive the I/O request of access at this local physical I/O equipment that other user virtual machine sends over, after handling, the result is returned to corresponding user virtual machine;
At least one local physical I/O equipment is corresponding one by one with described special I/O territory;
User virtual machine is set up the I/O access path with described special I/O territory, is undertaken alternately by described I/O access path and described special I/O territory.
Described system also comprises:
Home agent field, create by described I/O resource management module, use long-distance user's virtual machine of local physical I/O equipment at moving to other physical node, the request of the local physical I of the visit that the receiving remote user virtual machine is sent/O equipment, and this request is transmitted to corresponding special I/O territory, and receive special I/O territory and handle this I/O request back and send the I/O Request Processing result of returning, and this result is transmitted to corresponding long-distance user's virtual machine.
Described user virtual machine also comprises:
The I/O front-end module is used for being connected with special I/O territory, and the I/O request is sent to corresponding special I/O territory, is handled by special I/O territory, and receives the I/O Request Processing result that special I/O territory is returned.
Described franchise virtual machine comprises:
The administration order interface module, be used for the keeper with carry out mutual, forward command.
For realizing purpose of the present invention, also provide towards the method for work of the I/O system of multi-core platform and distributed virtualization environment, comprise the following steps:
Step 100.I/O system initialization;
The I/O request is sent in application in step 200. user virtual machine, and this I/O asks to give by system call the kernel processes of user virtual machine, after the kernel of user virtual machine carries out necessary processing, this I/O request is issued the I/O front-end module of user virtual machine;
The I/O front-end module of step 300. user virtual machine is after necessary processing is done in request to this I/O, by the I/O access path between user virtual machine and the special I/O territory corresponding with described user virtual machine this I/O request sent to special I/O territory;
After the described special I of step 400./O territory receives this I/O request, handle this I/O request, and result is returned to the I/O front-end module of user virtual machine by the I/O access path between user virtual machine and the special I/O territory corresponding with it;
The described I/O front-end module of step 500. returns to the kernel of user virtual machine with result, and the kernel of user virtual machine returns to result the application of sending this I/O request.
Described step 100 comprises the following steps:
User virtual machine is created according to user's demand by step 120. system, sets up the I/O access path by the I/O resource management module between user virtual machine and special I/O territory;
For realizing purpose of the present invention, also provide towards the method for work of the I/O system of multi-core platform and distributed virtualization environment, comprise the following steps:
Step 100 ' the .I/O system initialization;
Step 400 ' home agent field sends to described special I/O territory by the I/O access path between the special I/O_ territory corresponding with the local physical I/O equipment of desire visit with this remote I/O request;
Step 500 ' after described special I/O territory receives this remote I/O request, handle this remote I/O request, and result is returned to home agent field by the I/O access path between the special I/O_ territory of the local physical I/O equipment correspondence of home agent field and desire visit;
Described step 100 ', comprise the following steps:
Step 140 ' home agent field of .I/O resource management module establishment, system sets up the I/O access path between the I/O of the user virtual machine of moving to long-range physical node front-end module and home agent field, and between home agent field and special I/O territory, set up the I/O access path, and make two I/O access paths associate at the home agent field place.
The invention has the beneficial effects as follows:
1. I/O system of the present invention and construction method thereof have significantly reduced the partly virtual intermediate level and the intermediate treatment number of times that the I/O request needs process of handling down, have therefore significantly reduced processing I/O request to the expense that system brings, and have effectively improved the I/O of system performance; Simultaneously, therefore also significantly reduced the number of times of the operation set redirect that the redirect between the level brings, make and to significantly reduce because of cache miss rate and TLB (TLB:translation Lookaside Buffer, bypass conversion buffered) miss rate that frequently redirect causes between a plurality of processing are gathered.These can both significantly improve the overall performance of computer system, make same system can support more user, and can provide service more efficiently for the user;
2. in original half the I/O framework under virtual, all physical I/O equipment is all managed by Dom0, all I/O requests all need to transfer to Dom0 to be finished, and along with the increase of system's check figure and the increase of user's application, this framework has obviously become the performance bottleneck of system.And in I/O of the present invention system, each physical I/O equipment all has corresponding special I/O domain, is responsible for the relative I/O request of handling specially.Like this, the I/O performance bottleneck of system can be transferred to physical I/O equipment level from the software architecture level.Therefore, the I/O system that the present invention proposes can effectively bring into play the performance of physical I/O equipment, and the problem that can not cause the performance of physical I/O equipment not to be not fully exerted because of the increase of check figure and user's application.Like this, system can make full use of physical I/O equipment, thereby supports more user and application;
3. in the described in front half virtual I/O framework, all physical I/O device drives all is arranged in Dom0, does not have isolation each other.In case certain physical I/O device drives has problems, and may cause the collapse of whole multi-core platform, make and the virtual machine of the One's name is legion that moves on this platform and the collapse that the user uses cause serious consequence.And in the I/O system that the present invention proposes, each physical I/O equipment all has special-purpose I/O domain to be responsible for the relative I/O request of processing specially, has good isolation between each I/Odomain, has effectively improved the reliability of total system;
4. in aforesaid half virtual I/O framework down, it does not provide good support to migration that I/O visits.And in the I/O system that the present invention proposes, user virtual machine and be applied in move to long-range physical node after, still can the I/O resource on the original physical node be conducted interviews by the domain that acts on behalf of on the original physical node, therefore user virtual machine and the migration that is applied in the distributed system have obtained good support, also make intrasystem load balance more effectively to realize, system also can more efficiently utilization be distributed in the computational resource of system's overall situation.
Description of drawings
Fig. 1 is the virtualized system assumption diagram of I/O among the Xen (a kind of half virtualized architecture);
Fig. 2 is the structural representation of the I/O system towards multi-core platform and distributed virtualization environment of the present invention;
Fig. 3 is the structural representation of the I/O system when user virtual machine is moved to other physical node among the present invention;
Fig. 4 is the process flow diagram of the method for work of I/O of the present invention system;
Fig. 5 is the process flow diagram of the method for work of the I/O system when user virtual machine is moved to other physical node among the present invention;
Fig. 6 is the method flow diagram of I/O system initialization among the present invention;
Fig. 7 is the method flow diagram that user virtual machine is moved to the I/O system initialization of long-range physical node among the present invention;
Fig. 8 is the process flow diagram of typical I/O visit among the present invention;
Fig. 9 is the typical I/O browsing process figure in migration back among the present invention.
Embodiment
In order to make purpose of the present invention, technical scheme and advantage clearer,, I/O system and the method for work towards multi-core platform and distributed virtualization environment of the present invention is further elaborated below in conjunction with drawings and Examples.Should be appreciated that specific embodiment described herein only in order to explanation the present invention, and be not used in qualification the present invention.
I/O system and method for work towards multi-core platform and distributed virtualization environment of the present invention, succinctly efficient, safe and reliable, I/O access mechanism that extensibility is good in multi-core platform and the distributed virtualization environment have been realized, and can provide efficient I/O access mechanism for numerous virtual machines of parallel running on the multi-core platform and user use, the migration of using for the virtual machine in the distributed virtualization environment and user provides good support.
Introduce the I/O system towards multi-core platform and distributed virtualization environment of the present invention in detail below in conjunction with above-mentioned target, Fig. 2 is the structural representation of the I/O system towards multi-core platform and distributed virtualization environment of the present invention, as shown in Figure 2, described I/O system comprises:
Privilege virtual machine Dom0 is used for receiving management person's resource management order, and order is transmitted to the I/O resource management module;
Preferably, described franchise virtual machine Dom0 comprises:
The administration order interface module, be used for the keeper with carry out mutual, forward command.
Described administration order interface module has three effects: (1) is used for carrying out alternately forward command with the keeper as the order forwarding interface.(2) collect local node I/O resource information, and provide reference when issuing the resource management order in order to the keeper to the system manager; (3) collect the status information in local I/O territory, and provide, reference when issuing the resource management order with the keeper is monitored in the operation of I/O system in order to the keeper to the keeper.
The I/O resource management module is used for detecting local physical I/O equipment according to described resource management order, for each local physical I/O equipment is created special I/O territory (domain); And between user virtual machine and special I/O domain, set up the I/O access path;
Simultaneously, give the power that these special I/O domain manages one or more physical I/O equipment.The I/O resource management module also will send I/O passage establishment instruction to the application and the special I/O domain of correspondence according to the resources allocation order in the resource management order, makes up the I/O access path.
Preferably, each described local physical I/O equipment all has corresponding special I/O domain, these special I/O domain is based on unusual simple and high-efficient running environment, and wherein only operation is handled relevant code with I/O, reduce considerable intermediate treatment level, improved the efficient that I/O handles.
In half the I/O framework under virtual originally, all physical I/O equipment is all by franchise virtual machine (following represent with Dom0) management, all I/O requests all need to transfer to Dom0 and finish, along with the increase of computing platform check figure and the increase of user's application, this framework has obviously become the performance bottleneck of system.And in I/O framework of the present invention, each physical I/O equipment all has corresponding special I/O domain, is responsible for the relative I/O request of handling specially.Like this, the I/O performance bottleneck of system can be transferred to physical I/O equipment level from the software architecture level, when the software architecture level can also satisfy system requirements, physical I/O equipment might reach the peak value of physical property because of the increase of load, this peak value is that hardware determines, software is powerless to it).Therefore, the I/O framework that the present invention proposes can effectively be brought into play the performance of physical I/O equipment, and the problem that can not cause the performance of physical I/O equipment not to be not fully exerted because of the increase of check figure and user's application.Like this, system can make full use of physical I/O equipment, thereby supports more user and application.
At least one special I/O domain, local physical I/O the equipment that is used for self-responsibility carries out initialization, after local physical I/O device initialize is finished, this special I/O domain begins externally to provide I/O service: receive the I/O request of access at this local physical I/O equipment that other user virtual machine sends over, after handling, the result is returned to corresponding user virtual machine.
At least one local physical I/O equipment is corresponding one by one with described special I/O domain;
User virtual machine is set up the I/O access path with described special I/O domain, is undertaken alternately by described I/O access path and described special I/O domain.
Described user virtual machine also comprises:
The I/O front-end module is used for being connected with special I/O domain, and the I/O request is sent to corresponding special I/O domain, is handled by special I/O domain, and receives the I/O Request Processing result that special I/O domain returns.
In original virtualization system, the I/O front-end module is connected with rear end driver among the Dom0, and the I/O request is sent to the rear end driver of Dom0, is handled by Dom0, and receives the I/O Request Processing result who returns in the rear end driver of Dom0.Therefore the present invention has alleviated the burden of Dom0, avoids virtual machine and be applied in the I/O that operation suspension is sent them with wait Dom0 in the operational process asking to handle and return results, gives full play to the multi-core platform computing power; Simultaneously, avoid to cause the collapse of whole multi-core platform, and then make the virtual machine and the user of the One's name is legion that moves on this platform use all collapses, cause serious consequence because certain physical I/O device drives has problems.
Fig. 3 is the structural representation of the I/O system when user virtual machine is moved to other physical node among the present invention, as shown in Figure 3, use the user virtual machine of local physical I/O equipment at moving to other physical node, the I/O resource management module is also created a home agent field (domain).When migration, system sets up the I/O access path between I/O front-end module of moving to the user virtual machine between the long-range physical node and local agent domain, and between local agent domain and local physical I/O equipment, set up the I/O access path, and make two I/O access paths associate at local agent domain place.
The main task of home agent field (domain) is the request of the local physical I of the receiving remote user virtual machine visit of sending/O equipment, and this request is transmitted to corresponding special I/O domain, and receive special I/O domain and handle this I/O request back and send the I/O Request Processing result of returning, and this result is transmitted to corresponding long-distance user's virtual machine.Like this, even the user virtual machine on the local physical node is walked by migration, also still can continue normally to visit local physical I/O resource.
Corresponding to the I/O system of the present invention towards multi-core platform and distributed virtualization environment, the method of work of this I/O system also is provided. Fig. 4 is the process flow diagram of the method for work of the I/O system towards multi-core platform and distributed virtualization environment of the present invention, as shown in Figure 4, described method comprises the following steps:
Step 100.I/O system initialization;
The I/O request is sent in application in step 200. user virtual machine, and this I/O asks to give by system call the kernel processes of user virtual machine, after the kernel of user virtual machine carries out necessary processing, this I/O request is issued the I/O front-end module of user virtual machine;
The I/O front-end module of step 300. user virtual machine is after necessary processing is done in request to this I/O, by the I/O access path between user virtual machine and the special I/O domain corresponding with described user virtual machine this I/O request sent to special I/O domain;
After the described special I of step 400./O domain receives this I/O request, handle this I/O request, and result is returned to the I/O front-end module of user virtual machine by the I/O access path between user virtual machine and the special I/O domain corresponding with it;
The described I/O front-end module of step 500. returns to the kernel of user virtual machine with result, and the kernel of user virtual machine returns to result the application of sending this I/O request.
Fig. 5 is the process flow diagram of the method for work of the I/O system when user virtual machine is moved to other physical node among the present invention, as shown in Figure 5, after running on user virtual machine on the local physical node and need moving to long-range physical node because of the overall situation, when the I/O request (request of following abbreviation remote I/O) of visit local physical I/O equipment is sent in the application that is arranged in the user virtual machine (following abbreviation long-distance user virtual machine) on the long-range physical node, the method of work of I/O of the present invention system, finish by following step:
Step 100 ' the .I/O system initialization;
Step 200 '. the remote I/O request of visit local physical I/O equipment is sent in the application in long-distance user's virtual machine, the kernel processes of long-distance user's virtual machine is given in this remote I/O request by system call, after the kernel of long-distance user's virtual machine carries out necessary processing, this remote I/O request is issued the I/O front-end module of long-distance user's virtual machine;
After running on user virtual machine on the local physical node and need moving to long-range physical node because of the overall situation, when the I/O request of visit local physical I/O equipment was sent in the application that is arranged in the user virtual machine on the long-range physical node, this remote I/O request was in the disposal route of remote virtual machine inside and feelings processing condition identical (as described in step 200) in this locality.
Step 300 '. the I/O front-end module of remote virtual machine sends to local agent domain to this I/O request by the I/O access path between remote virtual machine and the local agent domain after necessary processing is done in this I/O request;
Step 400 ' local agent domain sends to described special I/Odomain by the I/O access path between the special I/O_domain corresponding with the local physical I/O equipment of desire visit with this remote I/O request;
Step 500 ' after described special I/O domain receives this remote I/O request, handle this remote I/O request, and result is returned to local agent domain by the I/O access path between the special I/O_domain of the local physical I/O equipment correspondence of local agent domain and desire visit;
Step 600 '. local agent domain with this handle the result by and the I/O front-end module of long-distance user's virtual machine between the I/O access path this is handled the I/O front-end module that the result returns to long-distance user's virtual machine;
Step 700 '. described I/O front-end module returns to the kernel of long-distance user's virtual machine with result, and the kernel of long-distance user's virtual machine returns to result the application of sending this remote I/O request.
Described step 700 ' step that result is returned is consistent with the situation in local user's virtual machine, as described in step 500.
Fig. 6 is the method flow diagram of I/O system initialization among the present invention, and as shown in Figure 6, described step 100 comprises the following steps:
User virtual machine is created according to user's demand by step 120. system, sets up the I/O access path by the I/O resource management module between user virtual machine and special I/O domain;
Fig. 7 is the method flow diagram that user virtual machine is moved to the I/O system initialization of long-range physical node among the present invention, as shown in Figure 7, when the user virtual machine on running on local physical node need be moved to long-range physical node because of the overall situation, described method also comprised:
Step 140.I/O resource management module is created a local agent domain, system sets up the I/O access path between the I/O of the user virtual machine of moving to long-range physical node front-end module and local agent domain, and between local agent domain and special I/O domain, set up the I/O access path, and make two I/O access paths associate at local agent domain place.
Described local agent domain, its main task is the request of the local physical I of the receiving remote user virtual machine visit of sending/O equipment, and this request is transmitted to corresponding special I/O domain, and receive special I/O domain and handle this I/O request back and send the I/O Request Processing result of returning, and this result is transmitted to corresponding long-distance user's virtual machine.Like this, even the user virtual machine on the local physical node is walked by migration, also still can continue normally to visit local I/O resource.
As a kind of embodiment, be example with block device I/O below, the process of I/O system handles I/O request of the present invention is described.In typical half virtualized environment, operational management command interface module among the franchise virtual machine Dom0, and by the I/O resource management module physical I/O equipment and special I/Odomain are managed.User virtual machine is undertaken by this I/O access path and special I/O domain after setting up the I/O access path with special I/O domain alternately.
Fig. 8 is the process flow diagram of typical I/O visit among the present invention.As shown in Figure 8, we imagine a typical scene.Suppose that this moment, user virtual machine VM1 moved, one of them application need visit physical block equipment dev1, its corresponding special I/O domain is io_dom1.The process of its typical I/O visit is as follows:
1. the I/O request is sent in the application among the user virtual machine vm1, the kernel processes that this I/O request is given vm1 by system call;
2.Vm1 kernel carry out necessary processing after, this I/O request is issued the I/O front-end module of vm1;
3.Vm1 I/O front-end module after necessary processing is done in request to this I/O, by the I/O access path between vm1 and the io_dom1 this I/O request is sent to special I/O domain io_dom1;
4. after special I/O domain io_dom1 receives this I/O request, handle this I/O request, and result is returned to the I/O front-end module of vm1 by the I/O access path between vm1 and the io_dom1;
5.Vm1 the I/O front-end module result is returned to the kernel of vm1.
6.Vm1 kernel result is returned to the application of sending this I/O request.
In whole process, the entire process process of I/O request only needs by vm1 kernel, I/O front-end module and special I/O domain3 level, through the processing of vm1 kernel, I/O front-end module, special I/Odomain, I/O front-end module and 5 centres of vm1 kernel.
In the I/O framework with respect to prior art described half under virtual, user virtual machine is when creating, to at first in Dom0, create the pairing I/O rear module of user virtual machine, and in user virtual machine, create I/O front-end module, the step of its I/O Request Processing following (we are example with block device I/O visit still):
1. at first use the kernel of request being given user virtual machine by system call;
2. the kernel of user virtual machine is given the I/O front-end module of user virtual machine after treatment the I/O request;
3.I/O front-end module is given the I/O among the Dom0 rear module I/O request more after treatment;
4.Dom0 in the I/O rear module give file system module among the Dom0 I/O request after treatment;
5.Dom0 in file system module give the EM equipment module of the generic block among the Dom0 I/O request after treatment;
6.Dom0 in the generic block EM equipment module give physical I real among the Dom0/O device drives I/O request after treatment, finish real I/O request by it.
7. after I/O asks processed finishing, need return by former road.
In entire I/O processing procedure, I/O request from application be issued to use obtain result need be through 6 levels, through 11 intermediate treatment.This loaded down with trivial details intermediate treatment process greatly reduces the I/O performance of system, thereby has also limited the performance of whole computer system computing power greatly.
And in the novel I/O framework that we propose, the I/O request that application is sent does not need again through so many levels: after using the front end I/O module that arrives user virtual machine, directly transfer to special I/O domain by front end I/O module, handle by special I/O domain, then the result is returned to I/O front-end module in the user virtual machine.Like this, just simplify I/O processing of request process greatly, effectively improved the I/O performance of system.Simultaneously, the minimizing of the intermediate level and intermediate treatment number of times has significantly reduced the level number of hops in the I/O Request Processing process, make because of frequent redirect causes between a plurality of processing set cache miss rate and TLB (TLB:translation Lookaside Buffer, bypass conversion buffered) miss rate significantly reduces, and improved the overall performance of system greatly.The reduction of the lifting of I/O performance and buffer memory miss rate and TLB miss rate makes the computing power of whole computer system to be not fully exerted.
Fig. 9 is the typical I/O browsing process figure in migration back among the present invention, as shown in Figure 9, after running on user virtual machine on the local physical node and need moving to long-range physical node because of the overall situation, the user virtual machine that our hypothesis is moved on other physical node is vm2, its physical I that need visit/O equipment still is devl, and its typical I/O access process is as follows:
1. the I/O request is sent in the application among the user virtual machine vm2, the kernel processes that this I/O request is given vm2 by system call;
2.Vm2 kernel carry out necessary processing after, this I/O request is issued the I/O front-end module of vm2;
3.Vm2 I/O front-end module after necessary processing is done in request to this I/O, by the I/O access path between vm2 and the local agent domain this I/O request is sent to local agent domain;
Local agent domain by and io_dom1 between the I/O access path this I/O request is sent to special I/O domain io_dom1;
5. after special I/O domain io_dom1 receives this I/O request, handle this I/O request, and result is returned to local agent domain by the I/O access path between local agent domain and the io_dom1;
Local agent domain with this handle the result by and the I/O front-end module of vm2 between the I/O access path this is handled the I/O front-end module that the result returns to vm2;
7.Vm2 the I/O front-end module result is returned to the kernel of vm2;
8.Vm2 kernel result is returned to the application of sending this I/O request.
By above processing procedure, behind user virtual machine other physical node in the system of moving to, still can physical I/O equipment of distributing to it originally be conducted interviews. like this, therefore user virtual machine and the migration that is applied in the distributed system have obtained good support, also make intrasystem load balance more effectively to realize, system also can more efficiently utilization be distributed in the computational resource of system's overall situation.
Beneficial effect of the present invention is:
1. I/O system of the present invention and construction method thereof have significantly reduced the partly virtual intermediate level and the intermediate treatment number of times that the I/O request needs process of handling down, have therefore significantly reduced processing I/O request to the expense that system brings, and have effectively improved the I/O of system performance; Simultaneously, therefore also significantly reduced the number of times of the operation set redirect that the redirect between the level brings, make and to significantly reduce because of cache miss rate and TLB (TLB:translation Lookaside Buffer, bypass conversion buffered) miss rate that frequently redirect causes between a plurality of processing are gathered.These can both significantly improve the overall performance of computer system, make same system can support more user, and can provide service more efficiently for the user;
2. in original half the I/O framework under virtual, all physical I/O equipment is all managed by Dom0, all I/O requests all need to transfer to Dom0 to be finished, and along with the increase of system's check figure and the increase of user's application, this framework has obviously become the performance bottleneck of system.And in I/O of the present invention system, each physical I/O equipment all has corresponding special I/O domain, is responsible for the relative I/O request of handling specially.Like this, the I/O performance bottleneck of system can be transferred to physical I/O equipment level from the software architecture level.Therefore, the I/O system that the present invention proposes can effectively bring into play the performance of physical I/O equipment, and the problem that can not cause the performance of physical I/O equipment not to be not fully exerted because of the increase of check figure and user's application.Like this, system can make full use of physical I/O equipment, thereby supports more user and application;
3. in the described in front half virtual I/O framework, all physical I/O device drives all is arranged in Dom0, does not have isolation each other.In case certain physical I/O device drives has problems, and may cause the collapse of whole multi-core platform, make and the virtual machine of the One's name is legion that moves on this platform and the collapse that the user uses cause serious consequence.And in the I/O system that the present invention proposes, each physical I/O equipment all has special-purpose I/O domain to be responsible for the relative I/O request of processing specially, has good isolation between each I/Odomain, has effectively improved the reliability of total system;
4. in aforesaid half virtual I/O framework down, it does not provide good support to migration that I/O visits.And in the I/O system that the present invention proposes, user virtual machine and be applied in move to long-range physical node after, still can the I/O resource on the original physical node be conducted interviews by the domain that acts on behalf of on the original physical node, therefore user virtual machine and the migration that is applied in the distributed system have obtained good support, also make intrasystem load balance more effectively to realize, system also can more efficiently utilization be distributed in the computational resource of system's overall situation.
In conjunction with the drawings to the description of the specific embodiment of the invention, others of the present invention and feature are conspicuous to those skilled in the art.
More than specific embodiments of the invention are described and illustrate it is exemplary that these embodiment should be considered to it, and be not used in and limit the invention, the present invention should make an explanation according to appended claim.
Claims (8)
1. towards the I/O system of multi-core platform and distributed virtualization environment, it is characterized in that described system comprises: franchise virtual machine, I/O resource management module, at least one special I/O territory, at least one local physical I/O equipment and user virtual machine, wherein:
The privilege virtual machine is used for receiving management person's resource management order, and described resource management order is transmitted to the I/O resource management module;
The I/O resource management module is used for detecting local physical I/O equipment according to described resource management order, for each local physical I/O equipment is created a special I/O territory; And between user virtual machine and special I/O territory, set up the I/O access path;
At least one special I/O territory, local physical I/O the equipment that is used for self-responsibility carries out initialization, after local physical I/O device initialize is finished, this special I/O territory begins externally to provide I/O service: receive the I/O request of access at this local physical I/O equipment that other user virtual machine sends over, after handling, the result is returned to corresponding user virtual machine;
At least one local physical I/O equipment is corresponding one by one with described special I/O territory;
User virtual machine is set up the I/O access path with described special I/O territory, is undertaken alternately by described I/O access path and described special I/O territory.
2. the I/O system towards multi-core platform and distributed virtualization environment according to claim 1 is characterized in that described system also comprises:
Home agent field, create by described I/O resource management module, use long-distance user's virtual machine of local physical I/O equipment at moving to other physical node, the request of the local physical I of the visit that the receiving remote user virtual machine is sent/O equipment, and this request is transmitted to corresponding special I/O territory, and receive special I/O territory and handle this I/O request back and send the I/O Request Processing result of returning, and this result is transmitted to corresponding long-distance user's virtual machine.
3. the I/O system towards multi-core platform and distributed virtualization environment according to claim 1 and 2 is characterized in that described user virtual machine also comprises:
The I/O front-end module is used for being connected with special I/O territory, and the I/O request is sent to corresponding special I/O territory, is handled by special I/O territory, and receives the I/O Request Processing result that special I/O territory is returned.
4. the I/O system towards multi-core platform and distributed virtualization environment according to claim 1 is characterized in that, described franchise virtual machine comprises:
The administration order interface module, be used for the keeper with carry out mutual, forward command.
5. towards the method for work of the I/O system of multi-core platform and distributed virtualization environment, it is characterized in that described method comprises the following steps:
Step 100.I/O system initialization;
The I/O request is sent in application in step 200. user virtual machine, and this I/O asks to give by system call the kernel processes of user virtual machine, after the kernel of user virtual machine carries out necessary processing, this I/O request is issued the I/O front-end module of user virtual machine;
The I/O front-end module of step 300. user virtual machine is after necessary processing is done in request to this I/O, by the I/O access path between user virtual machine and the special I/O territory corresponding with described user virtual machine this I/O request sent to special I/O territory;
After the described special I of step 400./O territory receives this I/O request, handle this I/O request, and result is returned to the I/O front-end module of user virtual machine by the I/O access path between user virtual machine and the special I/O territory corresponding with it;
The described I/O front-end module of step 500. returns to the kernel of user virtual machine with result, and the kernel of user virtual machine returns to result the application of sending this I/O request.
6. the method for work of the I/O system towards multi-core platform and distributed virtualization environment according to claim 5 is characterized in that described step 100 comprises the following steps:
Step 110. is when system start-up, and the keeper sends the resource management order by franchise virtual machine to the I/O resource management module, detects local physical I/O equipment, and the I/O resource management module is described for local physical I/O equipment establishment special I/O territory;
User virtual machine is created according to user's demand by step 120. system, sets up the I/O access path by the I/O resource management module between user virtual machine and special I/O territory;
Step 130. is after described special I/O territory is created, and the local physical I/O equipment of self-responsibility carries out initialization, and after initialization was finished, this special I/O territory externally provided I/O service;
7. towards the method for work of the I/O system of multi-core platform and distributed virtualization environment, it is characterized in that described method comprises the following steps:
Step 100 ' the .I/O system initialization;
Step 200 '. the remote I/O request of visit local physical I/O equipment is sent in the application in long-distance user's virtual machine, the kernel processes of long-distance user's virtual machine is given in this remote I/O request by system call, after the kernel of long-distance user's virtual machine carries out necessary processing, this remote I/O request is issued the I/O front-end module of long-distance user's virtual machine;
Step 300 '. the I/O front-end module of remote virtual machine sends to home agent field to this I/O request by the I/O access path between remote virtual machine and the home agent field after necessary processing is done in this I/O request;
Step 400 ' home agent field sends to described special I/O territory by the I/O access path between the special I/O territory corresponding with the local physical I/O equipment of desire visit with this remote I/O request;
Step 500 ' after described special I/O territory receives this remote I/O request, handle this remote I/O request, and result is returned to home agent field by the I/O access path between the special I/O territory of the local physical I/O equipment correspondence of home agent field and desire visit;
Step 600 '. home agent field with this handle the result by and the I/O front-end module of long-distance user's virtual machine between the I/O access path this is handled the I/O front-end module that the result returns to long-distance user's virtual machine;
Step 700 '. described I/O front-end module returns to the kernel of long-distance user's virtual machine with result, and the kernel of long-distance user's virtual machine returns to result the application of sending this remote I/O request.
8. the method for work of the I/O system towards multi-core platform and distributed virtualization environment according to claim 7 is characterized in that described step 100 ', comprise the following steps:
Step 110 '. when system start-up, the keeper sends the resource management order by franchise virtual machine to the I/O resource management module, detects local physical I/O equipment, and the I/O resource management module is described for local physical I/O equipment establishment special I/O territory;
Step 120 '. user virtual machine is created according to user's demand by system, sets up the I/O access path by the I/O resource management module between user virtual machine and special I/O territory;
Step 130 '. after described special I/O territory was created, the local physical I/O equipment of self-responsibility carried out initialization, and after initialization was finished, this special I/O territory externally provided I/O service;
Step 140 ' home agent field of .I/O resource management module establishment, system sets up the I/O access path between the I/O of the user virtual machine of moving to long-range physical node front-end module and home agent field, and between home agent field and special I/O territory, set up the I/O access path, and make two I/O access paths associate at the home agent field place.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2009100934547A CN101706757B (en) | 2009-09-21 | 2009-09-21 | I/O system and working method facing multi-core platform and distributed virtualization environment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2009100934547A CN101706757B (en) | 2009-09-21 | 2009-09-21 | I/O system and working method facing multi-core platform and distributed virtualization environment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101706757A true CN101706757A (en) | 2010-05-12 |
| CN101706757B CN101706757B (en) | 2011-11-16 |
Family
ID=42376983
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2009100934547A Active CN101706757B (en) | 2009-09-21 | 2009-09-21 | I/O system and working method facing multi-core platform and distributed virtualization environment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101706757B (en) |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101976200A (en) * | 2010-10-15 | 2011-02-16 | 浙江大学 | Virtual machine system for input/output equipment virtualization outside virtual machine monitor |
| CN102289620A (en) * | 2011-08-12 | 2011-12-21 | 华南理工大学 | Credible equipment virtualization system and method based on Xen safety computer |
| CN102314372A (en) * | 2010-05-25 | 2012-01-11 | 微软公司 | The configuration of virtual machine I/O multipath |
| CN102317914A (en) * | 2011-08-01 | 2012-01-11 | 华为技术有限公司 | Methods, system and devices for managing virtual resources |
| CN102523153A (en) * | 2011-12-08 | 2012-06-27 | 华中科技大学 | Load balancing method under virtual environment |
| CN102707991A (en) * | 2012-05-17 | 2012-10-03 | 中国科学院计算技术研究所 | Multi-root I/O (Input/Output) virtualization sharing method and system |
| CN103150217A (en) * | 2013-03-27 | 2013-06-12 | 无锡江南计算技术研究所 | Design method of multi-core processor operating system |
| CN105556473A (en) * | 2014-12-27 | 2016-05-04 | 华为技术有限公司 | I/O task processing method, device and system |
| WO2017121273A1 (en) * | 2016-01-12 | 2017-07-20 | 阿里巴巴集团控股有限公司 | Method and device for processing i/o request under kvm virtualization |
| CN107172149A (en) * | 2017-05-16 | 2017-09-15 | 成都四象联创科技有限公司 | Big data instant scheduling method |
| CN107807843A (en) * | 2017-10-26 | 2018-03-16 | 北京百度网讯科技有限公司 | I/O request processing methods, equipment and computer-readable medium in virtual machine |
| CN115048226A (en) * | 2022-08-15 | 2022-09-13 | 南京芯驰半导体科技有限公司 | Virtual control system and method of multi-core heterogeneous processor |
| CN117492967A (en) * | 2023-12-28 | 2024-02-02 | 苏州元脑智能科技有限公司 | Method, device, equipment and medium for managing storage system resources |
-
2009
- 2009-09-21 CN CN2009100934547A patent/CN101706757B/en active Active
Cited By (24)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102314372A (en) * | 2010-05-25 | 2012-01-11 | 微软公司 | The configuration of virtual machine I/O multipath |
| CN102314372B (en) * | 2010-05-25 | 2016-03-23 | 微软技术许可有限责任公司 | For the method and system of virtual machine I/O multipath configuration |
| CN101976200A (en) * | 2010-10-15 | 2011-02-16 | 浙江大学 | Virtual machine system for input/output equipment virtualization outside virtual machine monitor |
| CN101976200B (en) * | 2010-10-15 | 2013-05-08 | 浙江大学 | Virtual machine system for input/output equipment virtualization outside virtual machine monitor |
| CN102317914A (en) * | 2011-08-01 | 2012-01-11 | 华为技术有限公司 | Methods, system and devices for managing virtual resources |
| WO2012109870A1 (en) * | 2011-08-01 | 2012-08-23 | 华为技术有限公司 | Virtual resource management method, system and device |
| CN102317914B (en) * | 2011-08-01 | 2013-10-09 | 华为技术有限公司 | Methods, system and devices for managing virtual resources |
| CN102289620A (en) * | 2011-08-12 | 2011-12-21 | 华南理工大学 | Credible equipment virtualization system and method based on Xen safety computer |
| CN102523153B (en) * | 2011-12-08 | 2014-06-04 | 华中科技大学 | Load balancing method under virtual environment |
| CN102523153A (en) * | 2011-12-08 | 2012-06-27 | 华中科技大学 | Load balancing method under virtual environment |
| CN102707991B (en) * | 2012-05-17 | 2016-03-30 | 中国科学院计算技术研究所 | The many virtual shared method and systems of I/O |
| CN102707991A (en) * | 2012-05-17 | 2012-10-03 | 中国科学院计算技术研究所 | Multi-root I/O (Input/Output) virtualization sharing method and system |
| CN103150217A (en) * | 2013-03-27 | 2013-06-12 | 无锡江南计算技术研究所 | Design method of multi-core processor operating system |
| CN105556473A (en) * | 2014-12-27 | 2016-05-04 | 华为技术有限公司 | I/O task processing method, device and system |
| WO2016101282A1 (en) * | 2014-12-27 | 2016-06-30 | 华为技术有限公司 | Method, device and system for processing i/o task |
| CN106970821B (en) * | 2016-01-12 | 2021-02-02 | 阿里巴巴集团控股有限公司 | Method and device for processing I/O request under KVM virtualization |
| WO2017121273A1 (en) * | 2016-01-12 | 2017-07-20 | 阿里巴巴集团控股有限公司 | Method and device for processing i/o request under kvm virtualization |
| CN106970821A (en) * | 2016-01-12 | 2017-07-21 | 阿里巴巴集团控股有限公司 | A kind of method and apparatus that I/O requests are handled under KVM virtualization |
| CN107172149A (en) * | 2017-05-16 | 2017-09-15 | 成都四象联创科技有限公司 | Big data instant scheduling method |
| CN107807843A (en) * | 2017-10-26 | 2018-03-16 | 北京百度网讯科技有限公司 | I/O request processing methods, equipment and computer-readable medium in virtual machine |
| US10997099B2 (en) | 2017-10-26 | 2021-05-04 | Beijing Baidu Netcom Science And Technology Co., Ltd. | I/O request processing method in virtual machine, device and computer readable medium |
| CN115048226A (en) * | 2022-08-15 | 2022-09-13 | 南京芯驰半导体科技有限公司 | Virtual control system and method of multi-core heterogeneous processor |
| CN117492967A (en) * | 2023-12-28 | 2024-02-02 | 苏州元脑智能科技有限公司 | Method, device, equipment and medium for managing storage system resources |
| CN117492967B (en) * | 2023-12-28 | 2024-03-19 | 苏州元脑智能科技有限公司 | Method, device, equipment and medium for managing storage system resources |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101706757B (en) | 2011-11-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101706757B (en) | I/O system and working method facing multi-core platform and distributed virtualization environment | |
| CN104199718B (en) | A kind of dispatching method of the virtual processor based on NUMA high performance network cache resources affinity | |
| Buyya et al. | Single system image | |
| Mishra et al. | Dynamic resource management using virtual machine migrations | |
| Liu et al. | High Performance VMM-Bypass I/O in Virtual Machines. | |
| CN101430674B (en) | Intraconnection communication method of distributed virtual machine monitoring apparatus | |
| CN102457439B (en) | Virtual switching system and method of cloud computing system | |
| Hirofuchi et al. | Enabling instantaneous relocation of virtual machines with a lightweight vmm extension | |
| US8943294B2 (en) | Software architecture for service of collective memory and method for providing service of collective memory using the same | |
| CN101819564A (en) | Method and device for assisting communication between virtual machines | |
| CN101751284A (en) | I/O resource scheduling method for distributed virtual machine monitor | |
| JP2010267009A (en) | License management system, license management method and computer program | |
| CN101452406A (en) | Cluster load balance method transparent for operating system | |
| CN108228316A (en) | A kind of method and apparatus of encryption device virtualization | |
| CN101278263A (en) | Direct-memory access between input/output device and physical memory within virtual machine environment | |
| CN104615480A (en) | Virtual processor scheduling method based on NUMA high-performance network processor loads | |
| CN102650950A (en) | Platform architecture supporting multi-GPU (Graphics Processing Unit) virtualization and work method of platform architecture | |
| CN102479100A (en) | Pervasive computing environment virtual machine platform and creation method thereof | |
| CN103763403B (en) | Message flux control method and relevant apparatus and calculate node | |
| CN101976200A (en) | Virtual machine system for input/output equipment virtualization outside virtual machine monitor | |
| CN102521015B (en) | Equipment virtualization method under embedded platform | |
| CN106095576A (en) | Under virtualization multi-core environment, nonuniformity I/O accesses resources of virtual machine moving method | |
| CN101488113B (en) | Device driver field implementing method, system and apparatus | |
| CN102426557A (en) | Separated access method and system for PCI (Peripheral Component Interconnect) equipment in virtualization environment | |
| CN105556473A (en) | I/O task processing method, device and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20170209 Address after: 210012 Jiangsu Province, Nanjing city Yuhuatai District Road No. 168 run and Powerise software B center building 11 floor West Patentee after: Jiangsu Run He software incorporated company Address before: 100080 Haidian District, Zhongguancun Academy of Sciences, South Road, No. 6, No. Patentee before: Inst. of Computing Technology, Chinese Academy of Sciences |
|
| TR01 | Transfer of patent right |