CN104123233A - Virtual machine high-capacity memory sharing method - Google Patents
Virtual machine high-capacity memory sharing method Download PDFInfo
- Publication number
- CN104123233A CN104123233A CN201310149915.4A CN201310149915A CN104123233A CN 104123233 A CN104123233 A CN 104123233A CN 201310149915 A CN201310149915 A CN 201310149915A CN 104123233 A CN104123233 A CN 104123233A
- Authority
- CN
- China
- Prior art keywords
- page
- virtual machine
- virtual
- shared
- memory
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Abstract
The invention discloses a virtual machine high-capacity memory sharing method and a system thereof. The virtual machine high-capacity memory sharing method is characterized in that virtual machines read and write a section of shared memory with the capacity of more than 1 G, the memory region can be continuous and can also be discontinuous, but pages must be aligned; according to the operational process, one virtual machine provides the shared memory region and carries out authorization operation, the other virtual machines map the memory in the region to their own address space, and then read-write operation is carried out. The system comprises a virtualized base system expansion module and other modules, the technology of controlling the number of the virtual machines sharing the pages and other technologies are adopted, and a high-capacity memory sharing mechanism of the virtual machines is achieved. The virtual machine high-capacity memory sharing method and the system are suitable for the memory sharing service of the high-capacity remote virtual machine service, and have the wide market application prospects.
Description
Technical field
The present invention relates to the shared method of the large capacity internal memory of a kind of virtual machine, the application program and the virtual machine carrier that run on virtual machine inside have formed a virtual application.While jointly accomplishing a task between a plurality of virtual applications, need to adopt the method to carry out data sharing and synchronous communication, to guarantee the correct stable operation of system.The method is based on Xen virtual platform design, provide between a plurality of virtual applications internal memory share and fast mapping machine-processed, to meet the demand of virtual application cooperation., can shine upon dynamically as required according to the requirements for access of virtual application meanwhile, reduce to greatest extent the pressure that shared drive mechanism is brought to system.
Background technology
In prior art field, virtual platform Xen provides basic technology and the interface of much realizing the method, based on these interfaces, carries out innovative combination, has formed this and has invented described collaboration method.Xen virtual platform provides the DLL (dynamic link library) of sharing a page between virtual machine, and a whole set of event communication mechanism is provided simultaneously.The general Intel Virtualization Technology of this method based on Xen, detailed technical background relates to general virtual, Xen Intel Virtualization Technology, the several aspects of authorization list mechanism and event channel.
general Intel Virtualization Technology
Also referred to as half virtual or accurate virtual (Para Virtualization), the system main foundation that this method adopts is exactly half virtualized virtual machine.Half virtually can provide high performance, and it and Full-virtualization have some similar.This method has used hypervisor to realize the share and access to bottom hardware, also will be integrated in operating system itself with virtual relevant code.Half virtual with fully virtualized identical place is that the privilege level of client computer and hypervisor operation is similar.Difference is exactly the operating system that general virtualized virtual machine need to be revised client computer, does not support general commercial operation system.Therefore general virtual virtual machine has been subject to larger restriction in industry member widespread use, be merely able to the operating system of using some to increase income, be applicable to BSD, Linux, Solaris and other open source operating systems, but the higher performance having due to self is subject to the selection of certain customers.Traditional general virtualization product is mainly the basic Xen of this paper prototype system implementation.Xen, by linux kernel patch installing, also operates on virtual machine management host itself, becomes Domain 0, and other virtual machine is called Domain U, and the kernel of Domain U is the kernel through Xen patch installing.
xen ultimate principle
Xen is as the current topmost virtual machine management program of increasing income (Virtual Machine Monitor), mainly for x86 framework, design, support at present the fully virtualized technology of general virtual and hardware supported, mainly for process file access in the virtual machine of the general virtual pattern of Xen, monitor herein.Xen provides the kernel of revising for general virtualized virtual machine, thereby takies resource seldom and make general virtual virtual machine reach very high performance.The latest edition of Xen is 4.0.0 at present.General virtual aspect, the gordian technique in Xen realization comprises internal memory virtualization, the virtual and CPU Intel Virtualization Technology of IO.Below by respectively for the architectural framework of Xen, gordian technique, general Intel Virtualization Technology and fully virtualized technology are introduced.
Xen virtual environment comprises several main assemblies, and these assemblies are worked in concert and set up the virtualized environment that client disposes.The assembly mainly comprising is Xen Hypervisor, and Domain 0, and Domain U(comprises general virtual virtual machine and the fully virtualized virtual machine of HVM), control and management interface and inter-domain communication mechanism.
Xen Hypervisor is Xen VMM(Virtual Machine Monitor) be basic virtual environment layer, be directly installed on hardware, be responsible for CPU scheduling and the management of different virtual machine memory partitioning.This part, as the core of Xen environment, is virtual machine virtual hardware incessantly, because all virtual machines are all shared same processor resource, Hypervisor is also in charge of the execution of virtual machine.
Domain 0 is a linux kernel of revising, operate in privileged domain a special virtual machine.Can directly access physical I O resource and mutual with other virtual machines, need to be as the virtual machine of first startup.In Domain0, there is multiple rear end to drive, be used for supporting the requests such as the network of other virtual machines and local hard drive.These rear ends drive directly and local network and disk hardware interaction data.
Territory management and Control Component include a series of Linux finger daemon, are used as territory management and control.Basic management and the control of these service supports Xen virtualized environment, operate in the operating system of Domain0.Mainly comprise Xend, xm, xenstore and libxc built-in function, qemu-dm, Xen virtual firmware etc.
Domain U comprises two kinds of virtual machines, general virtual fully virtualized with HVM.The general virtual client computer of Domain U is the target of studying herein.General virtualized virtual machine can only be the Linux revising, Solaris, FreeBSD and other class unix system.General virtualized virtual machine is access hardware directly, must drive by the rear end of aforesaid Domain 0 and go access.Domain U HVM client computer refers to the not system through revising, and need to have hardware virtualization technical support just can start.
Communication mechanism between Domain0 and Domain U is in order to support the multiple driving in Domain U.For example not network enabled and hard disk request of Xen hypervisor, so the general virtualized virtual machine of Domain U must complete by Domain0 the operation requests of network and this class of hard disk request.The general virtual virtual machine Block Device Driver of Domain U receives a request that writes disk, by by passing to Xen Hypervisor with the shared internal memory of Domain0.Between Domain 0 and DomainU, exist an event channel (event channel) to allow both by interruption between asynchronous territory, to communicate by letter.During Domain0 receives, have no progeny and will drive and go the internal memory of accessing local system to read in piece corresponding in general virtual virtual machine with general virtual rear end, and then be written to local disk relevant position.
xen gordian technique
Xen gordian technique mainly comprises internal memory virtualization, and virtual and virtual three parts of CPU of IO, simply introduce these three aspects respectively below.
(1) internal memory virtualization technology
Operating system under non-virtualized environment all thinks that what oneself use is continuous memory headroom, and therefore in order to reach the compatibility of application program, it oneself is to move on continuous memory headroom that Xen must virtual memory space thinks each virtual machine.In addition due to x86, do not provide the TLB(Translation Lookup-aside Buffer of software administration).Therefore after the switching at each address space, all can refresh whole TLB, Xen takies the front 64MB internal memory of each virtual machine address space, and itself and virtual machine are in same address space.In addition, for the page table renewal operation of virtual machine, after the action need Xen writing checking, just can carry out, this mode allows virtual machine internal operating system directly to access physical address, only need to when page table changes, ask Xen to verify.
(2) IO Intel Virtualization Technology
Operation for the general virtual virtual machine of Xen to IO equipment, Xen carries out transfer realization by Domain 0.Domain 0 is a linux kernel of revising, operate in privileged domain a special virtual machine.Can directly access physical I O resource and mutual with other virtual machines, need to be as the virtual machine of first startup.Xen utilizes the mechanism of shared drive ring to support asynchronous IO access, utilizes the mode of authorization list (Grant Table) to support the internal memory between Domain to share and data transmission.When device interrupt produces, Xen utilizes event channel (Event Channel) mechanism pass-along message between Domain.Concrete IO framework will be introduced in detail at further part.
(3) CPU Intel Virtualization Technology
The privilege level that X86-based provides four instructions of Ring0 – Ring3 to carry out.In Xen virtualized environment, Hypervisor operates on the highest privilege level Ring 0, so the operating system in virtual machine need to operate on lower privilege level Ring 1, and application program should operate on Ring 3.While carrying out some instruction in the operating system in virtual machine, its residing Ring 1 privilege level can cause some abnormal harm virtual machine environments, and the responsive instruction that this class is commonly called as need to appropriately be processed by Xen Hypervisor.The instruction of this class comprises some control pages, changes cpu model, the dependent instruction of the operation such as IO equipment, and these privileged instructions are carried out when Ring 1, can be absorbed in Trap, call exception handler.This belongs to a kind of hardware protection mechanism.Hypervisor captures the hypercall that is replaced to Xen after these privilege process and realizes.The process of abnormality processing is that virtual machine produces after abnormal and caught by Hypervisor, calls the abnormality processing function of Xen oneself, then, by the abnormality processing function of returning to target and registering before this to virtual machine, simulates one time abnormality processing.This process expense is larger, so Xen virtual machine provides the mechanism of Fast Trap, by Hypervisor, does not directly enter into Ring 1.Reduce performance loss.
Xen authorization list and event channel technology
The inter-domain communication mechanism of current topmost Xen comprises event channel (Event Channel) mechanism and the authorization list mechanism (Grant Table) of Xen, and this paper monitoring technique below will be used the separate type drive pattern of these two kinds of technique constructions.
(1) event channel (Event Channel)
Xen provides a kind of simple asynchronous mechanism to be called event channel.Each virtual machine (territory) has one group of port can be tied to an event source (such as the port of a kind of physical I RQ or virtual I RQ or another virtual machine).When a pair of port of two virtual machines is bundled in together, can on a territory, by carrying out send, operate, cause an event to be received by aiming field.The basic event communication mechanism that Event Channel provides as Xen, is similar to hardware interrupts.Conventionally only store the information of one, this is made as to 1 by 0 and is representing that relevant event occurs.Event notice is received by a virtual machine, then processes.Asynchronous inter-domain event informing mechanism.Between two territories, set up event channel, can ask a virtual irq.The territory of front end can send notice to the territory of rear end, thereby causes an interruption.The initialization in Xend when driving startup of event channel between two territories.Xc_evtchn_bind_interdomain will produce a hypercall, sets up two event channels between territory, only has privileged domain just can initiate to create the request of event channel.Event is to sending the canonical form of notice between VM or VM from hypervisor.Be similar to the signal on UNIX.In a lot of situations, event has replaced the hardware interrupts of Xen.An interruption is an event trigger that asynchronous hardware is relevant in fact, and an event is also an event trigger that asynchronous virtual machine is relevant.
Event can be divided into three major types: interdomain event, PIRQ, VIRQ.PIRQ refers to physical I RQ.For VIRQ virtual I RQ, be similar to physical I RQ, but be that virtual unit is relevant, most typical example is timer.Timeri virtual unit is similar to the timer of physics, but is maintaining virtual time.Virtual machine can be asked a timer event, obtains an event who is tied to VIRQ_TIMER (VIRQ).For the type of communicating by letter between domian, can comprise two stages.First a domain creates a new event channel, and this is an event channel who there is no binding, then gives the right of some other domain binding.Second domain distributes a new channel and this chennel is tied on the port of first domain of far-end.After connecting foundation, any one domain can notify another one domain to the mode of local port by transmission event.For an event channel who has connected, operate, topmost operation is to send an event.Transmission event is very simple, and the port that only need to set event channel is just passable.Another one operation is to close a port, is used for using to following event channel.Only need to use evtchn_close_t just passable.
Obtain the state of a channel, use EVTCHNOP_status order.In evtchn_status_t structure, need input port and domain.(domain, port) has been bound on an event channel representing a port.Calling HYPERVISOR_sched_op function can dispatch event.Control and use case passage relate to following hypercall event_channel_op (evtchn_op_t * op); In file Xen/include/public/event_channel.h, state.
Current Event channel is mainly used to realize the general virtualized driving of Xen.Refer to the communication mechanism being used as between the front-end driven of Domain U and the driving of the rear end of Domain0.Because writing of driving in Xen comprises two parts, a part is the front-end driven in the DomU of non-level of privilege, and a part is to drive in the rear end of the Dom0 of level of privilege.The driving operational mode of Xen is that event occurs when front-end driven, when need to cause interruption, by by event channel notice, the rear end in Dom0 drives, process after driving and receiving event rear end, such as reading the operations such as user data by authorization list (Grant Table) from shared drive.
As a kind of asynchronous mechanism between Domain, event channel widespread use in virtual driving.The residing domain of front end sends out a notice by event channel mechanism, and rear end will cause once interrupting after receiving, may be physics may be also virtual interruption.Event Channel is a kind of efficient transmission of messages mechanism, is mainly used to allow separated device drives front-end and back-end communication in Xen.Event channel starts the interruption of triggering by the Hypervisor of Xen.These interruptions are all registered in XenStore.These log-on messages of XenStored finger daemon guardian, comprise these interrupting informations, and the Event channel between Dom0 and other DomU connects.Except Event, Xen also provides the communication mechanism of low expense with the form of trap.Be different from event, event mechanism can dynamic creation and binding, but trap has static meaning, direct corresponding hardware interrupts.
(2) authorization list (Grant table)
Event channel is the basic skills of Xen inter-domain communication.But the communication bandwidth being to provide is very little, together with therefore must binding with the communication mechanism of shared drive, make to be used to provide efficient high performance Xen inter-domain communication.Between virtual machine, the mechanism of the safe shared drive page is implemented by giving the access rights of each memory pages special domain.By following hypercall, realize: grant_table_op (unsigned int cmd, void * uop, unsigned int count);
The Grant Tables mechanism of Xen provides a kind of shared method of internal memory between general domain.This shared drive interface is mainly used to realize separated device drives for piece and network I/O.Each domain has the grant table of oneself, and this is one and the shared internal storage data structure of Xen.Grant table allows domain to tell what read-write right other domain of Xen have to the page of this domain.List item in Grant table is quoted by grant reference.Grant reference is an integer, is used for retrieving grant table.Xen allows shared drive between non-privilege level domain.Grant reference comprises the details of sharing page.The mode of shared drive is a kind of the simplest mode that exchanges mass data in IPC.Hypervisor provides a kind of similar mechanism to share the memory pages between virtual machine.
In grant table, two kinds of main operations are mapping and transmits page.Both common ground are all to comprise to the memory headroom that calls domain to insert physical page.Difference is that mapping is still stayed the page in the address space of former domain, and the page in original address space has been deleted in transmission.Mapping is used for setting up internal memory and shares.And transmission is used for Mobile data, also can be used in during balloon drives.
GNTTABOP_map_grant_ref order is used for mapped page frame, according to { grant_ref, domain} couple.Between domain, carry out data transmission.Use GNTTABOP_transfer order.Before a page transmission, recipient need to represent to receive.A very large application of transmission mechanism is exactly that ballon drives, and is used for expanding the memory size of a domain.A virtual machine creates the reference of a transfer in grant table, then notifies dom0, and dom0 transmits the new page into this virtual machine.Need to there be three inputs to represent the page transmitting, the domain in above-mentioned data result, grant reference, and mfn (the internal memory frame data structure that will transmit).Transmission is the good method of a mobile mass data between virtual machine, only need to upgrade page table, so efficiency is very high.Yet for transmission in a small amount, copy mode is more efficient.Copy requirement between Domain must have a domain access originator and destination domain simultaneously.
Authorizing and regain mandate can be by completing the direct control of grant table.Grant table is exactly a structural array in fact.Flag represents which type of right is awarded this memory pages.Current version is supported two kinds of right: GTF_permit_access (reading right) and GTF_accept_transfer (writing right).Before using grant table, need first to obtain access interface, can obtain by GNTTABOP_setup_table order.
Can be by directly the access of grant table being arranged and destroys grant reference.Create grant reference and do not need to involve Xen hypervisor.The domain authorizing calls hypercalls and uses grant reference.Four main operations of Grant table comprise mandate external reference, stop external reference, authorize external transmission and stop external transmission.
Between virtual machine, often to communicate, the transmission of memory pages and to share be more common operation, grant table mechanism proposes for this demand just.Identical with event channel, grant table is still mainly used in writing of separating apparatus driver.Especially the driving of block device and the network equipment.Between the front-end and back-end that this kind equipment drives, exchanges data transmission frequently, is therefore used the mechanism of grant table to make the page share and transmit convenient and safety.Most of block device in Xen and network device driver are all to transmit data by the mechanism of grant table.In addition, in virtual TPM, also used grant table mechanism, as in shared drive ring, for transmitting data between the driving of two of front and back ends.In the request of vTPM and response data structure, just comprise grant reference item.
In the structure of Xenstore, except some use the driving of grant table, under console catalogue, still comprise a ring-ref, represent that the reference of grant table is used by console ring queue.
Summary of the invention
The invention discloses in a kind of large capacity based on Intel Virtualization Technology and have system shared between virtual machine, it is characterized in that, between virtual machine, can jointly read and write the internal memory of one section of shared above capacity of 1G.This section of region of memory can be continuous, can be also discontinuous, but must be page alignment.Operational process is that a virtual machine provides shared drive region, carries out " mandate " operation, and other virtual machines are mapped to own address space to this region memory, then carry out read-write operation.
The cooperation that is virtual application based on virtual platform internal memory shared mechanism facilitates.Described virtual application is the application program that operates in virtual machine inside, and described collaboration method is for sharing the same partial memory of read-write and communicate and synchronously based on a plurality of virtual machines.
Particularly, the invention discloses the large capacity internal memory of a kind of virtual machine technology of sharing, comprise the steps:
S1. on physical server, start virtual platform;
S2. on virtual platform, start some virtual machines as the carrier of application operation;
S3. on virtual platform, start a virtual machine as the carrier of shared drive;
S4. in shared drive carrier inside, move virtual driving, the interface that calling virtual platform provides carries out page Authorized operation, and the page of appointment is licensed to other virtual machine mappings;
S5. shared drive carrier inside is set up event channel, monitors the connection of application virtual machine;
S6. the virtual driving of application virtual machine internal operation, the interface that calling virtual platform provides shines upon the shared drive page, is mapped to kernel or user's state address space of appointment;
S7. many application virtual machines carry out read-write operation to the same partial memory page, thereby communicate and resource sharing fast;
S8. in map operation process, can be undertaken synchronously and communicate by letter by event channel mechanism.
Described a kind of expanding system based on virtual machine shared drive mechanism, is characterized in that, comprises the following stated module:
Virtual basic system expansion, acquiescence virtual platform is because principle of design limits.Isolation guarantee between virtual machine, does not support to share between internal memory virtual machine completely.
1. virtual basic system expansion
A) virtual platform of acquiescence is due to the restriction of principle of design, isolation guarantee between virtual machine, or do not support to share between internal memory virtual machine completely, or only support a small amount of virutal machine memory to share (for example 64M internal memory), cannot meet transmission and the communication requirement of a lot of application mass data.
B) need on virtual platform, expand, thereby allow to support the internal memory that surpasses 1G capacity to share at bottom.The object of expansion module is to expand internal memory to share authorization list, and the ability of making it have is preserved a large amount of mandates and map information.
2. internal memory provides module
A) run on and provide the virtual machine of shared drive inner, will specify the internal memory of physical location to authorize, license to designated virtual machine access.Licensing process can obtain corresponding grant number, and each Authorized operation can be authorized a page.
B) page after mandate is marked as and can allows designated virtual machine access according to authorization list.
C) grant number that Authorized operation produces must be passed to by transmitting module in the memory-mapped module of authorized virtual machine end to be used.
3. memory-mapped module
A) receive and share after grant number, call the interface of mapping, using grant number as parameter, be mapped to kernel spacing or the user's space address of appointment.
B) mapping block can be used the page-map of authorizing to kernel spacing or user's space.
C) after mapping, returning to what obtain is the virtual address of designated space.
D) after mapped page, virtual machine has the authority of read and write to this page, but cannot discharge this part resource, because the owner of this page remains, provides the virtual machine of sharing the page.
4. grant number is transmitted module
A) grant number transmission module plays is providing virtual machine and the use of sharing the page to share the effect of transmitting grant number between page virtual machine.
B) mechanism that transmission module adopts is shared queue's mode.
C) first shared page, transmits the effect that the page is used as transmitting other all shared grant number.
D) each provides between the virtual machine of shared drive and the application virtual machine of use shared drive and exists to transmit a page.
E) grant number of transmitting the page is used nonautomatic mode to transmit, and for example transmit in manual or socket path.
The grant number of the shared page generating f) all will be put into the transmission page, when the transmission page is piled, internal memory provides module to send out message to memory-mapped module by event channel, and memory-mapped module notifies internal memory to provide module can continue to put into new grant number by event channel after just taking out all grant number equally.
G) mode by similar table tennis can a large amount of grant number of circulating transfer, thereby have guaranteed that large internal memory shares the possibility realizing.
Described system needs the problem of considering to comprise in realizing and designing:
1. virtual platform choice of technology requirement
A) virtual platform of selecting must provide basic page management mechanism, allows the page to be accessed by a plurality of virtual machines.
B) in page management, must there is pair mechanism that page access authority is controlled.
2. the shared virtual machine number problem of the page
A) in theory if authorization list put under, can share to numerous virtual machine and use.Be not particularly limited.
B) each project of authorization list comprises provides the virtual machine of mandate ID, target virtual machine ID, the physical page address of authorization page.
C) support a plurality of virtual machines (for example 10) to access the same page, also support a virtual machine to shine upon the shared page that a plurality of virtual machines (for example 10) are authorized.
3. the shared authority problem of the page
A) in order to guarantee maximum dirigibility, system is not distinguished and is treated level of privilege virtual machine and non-level of privilege virtual machine, allows both sides' the page to share and access mutually.
B) access limit acquiescence is given.In Authorized operation, acquiescence is authorized the authority that other virtual machines are read and write this page.
C) acquiescence support is mapped to kernel state and user's state address space.
4. memory-mapped scale restriction
A) under default situations, with the virtual platform of Xen4.0 version, give an example, only allow virtual machine to share 64M internal memory.Computing method are as follows:
I. under default situations, authorization list accounts for 32 pages
Ii. each list item accounts for 8 Byte, can represent a page
Iii. authorization list can hold 32*4K/8=16K list item
Iv. therefore single virtual machine is shared maximum 32*4K/8=16K the page of supporting
V. maximum is shared 16K*4K=64M
B) add after virtual basic system expansion, with the virtual platform of Xen4.0 version, give an example, add expansion module, can allow 4G memory headroom to share.Most importantly the mode of expansion has increased the space of authorization list and the adjustment of corresponding memory address layout.Computing method are as follows:
I. new authorization list accounts for 2048 pages
Ii. each list item accounts for 8 Byte, can represent a page
Iii. authorization list can hold 2048*4K/8=1024K list item
Iv. therefore single virtual machine is shared maximum 2048*4K/8=1024K the page, i.e. the 1024K*4K=4G space supported
If v. sharing to 2 virtual machine theoretical values is 4G/2=2G, sharing to the amount of ram that N virtual machine can be shared is 4/nG.
5. grant number transmission design
A) owing to using and must first be mapped to local address space sharing the page, and must conduct interviews to authorization list during mapping, so the index of authorization list is that grant number must pass to the virtual machine that need to carry out page-map.
B) transfer mode have a variety of, such as socket Internet Transmission etc., but full blast or by sharing the page, transmit, saved between kernel state user state repeatedly data copy.
C) first shared page need to pass to virtual machine by manual or socket mode as transmitting the page.
D) between every pair of share-mapping virtual machine, need to have one to transmit the page.
E) transmit the event channel that the page need to be arranged in pairs or groups and be connected, to guarantee that when the page is filled with the side of collecting can receive that message appropriately processes.
F) foundation of event channel has guaranteed that the work of transmit leg and the side of collecting is synchronous.But belong to obstruction mode, efficiency slightly affects.
6. user's state Address space mappinD
A) must mapping API be exposed to user's state by creating corresponding character device, more conveniently add again the lib storehouse of new user's state, the access of encapsulation to character device.
B) on Xen4.0 platform, the mapping of user's state address space by libxenctrl storehouse API and/dev/gntdev character device realizes.
7. DMA address spatial mappings
A) identical with the non-DMA address space use of kernel state application, can be used for carrying out page mandate and mapping.
8. mapping efficiency
A) under default situations, mapping efficiency is very low, and each page need to once shine upon, and need to adopt Optimization Mechanism.
B) the mapping (enum) data structure packing mapping a plurality of grant number being built, can guarantee that 1G page-map only needs 2-3 time second.
C) prerequisite of packing manner is the grant number that memory-mapped module has received this all action needs, has in advance a large internal memory to carry out the buffer memory of grant number.Under this prerequisite, just can realize, how the usage space of kernel module is limited, needs the operation of packing in batches.
Virtual machine is divided into two kinds of supplier (ProviderVM) and application persons (AppVM) according to residing role's difference in shared procedure.The practical operation role that internal memory authorization module (Grant Module) and memory-mapped module (Map Module) play shared drive, operates in respectively in two kinds of virtual machines.Transmit page actual format for sharing array (Shared Array), be used for depositing the grant number (Grant Index) of transmission, event channel (Event Channel) is used for the synchronous two side's behaviors of sharing array of using.
Idiographic flow is as follows:
1. ProviderVM shares to AppVM by Grant Module by the memory pages being mapped to and uses.
2. in the address space that AppVM retains the page-map of ProviderVM to oneself, when the memory pages for oneself is used.
3. many AppVM share same memory pages region
4. in AppVM, the application of operation is carried out read-write operation to shared memory pages
5. in ProviderVM, the effect of Grant Module is the shared memory pages of mark (Grant Operation), and will quote and be put into (Write Grant Index) in Shared Array.
6. the effect of the Map Module in AppVM is from Shared Array, to read the page to quote (Read Grant Index), and calls Map operation A PI and page-map is arrived to the reserved address space of oneself.
Grant Table and Event Channel mechanism have been used in the realization of whole system on Xen4.0 virtual platform.
Embodiment
The large internal memory sharing method of virtual machine is applied to multi-dummy machine application parallel processing, and as a complete system, this system has the virtual machine (AppVM) that the virtual machine of shared drive (ProviderVM) is provided and runs application.System operation and the flow process realizing are as described below:
1. virtual machine activation: ProviderVM, AppVM1, AppVM2 starts
2. on ProviderVM, specify and need to carry out shared region of memory, obtain the physical page address of this region of memory, as the parameter of importing into of internal memory authorization module
3. the operation in ProviderVM:
A) PCI device drives loads, storage allocation region
B) internal memory authorization module loads
C) in internal memory authorization module, operate:
I. apply for 2 pages, to share structure of arrays initialization, license to respectively AppVM1 and AppVM2, be used for depositing grant number, as transmitting the page
Ii. create event channel example, be used for processing the transmission page and use bipartite synchronization message
Iii. obtain the physical address that needs shared all pages
Iv. the page is authorized, share to respectively AppVM1 and AppVM2
V. by sharing the grant number obtaining, deposit in the transmission page, AppVM1 and AppVM2 can receive these grant number
Vi., after transmitting the page and being filled with, adopt event channel to send message informing AppVM1 and AppVM2 collects processing
Vii. AppVM1 and AppVM2 beam back message after processing, and ProviderVM continues to shine upon and fills and transmit the page
4. the operation in AppVM:
A) memory-mapped module loading, transmits the grant number of the page as start-up parameter
B) in memory-mapped module, operate
I. retain one section of continuous address space for memory-mapped
Ii. the transmission page that mapping obtains
Iii. the event channel that connects the establishment of internal memory authorization module is used for receiving the synchronization message of transmitting the page
Iv. from transmit the page, order reads grant number
V. shine upon the page of grant number appointment to the continuous space having retained
Vi. handle after all grant number of transmitting in the page, by event channel, send out message and allow ProviderVM that new grant number is provided
5. in AppVM, starting application reads and writes shared internal memory;
6. the concurrent service of processing is undertaken synchronously by internal memory shared between multi-dummy machine and communicates by letter.
Claims (10)
1. the large capacity internal memory sharing method of virtual machine, is characterized in that, the internal memory of one section of shared above capacity of 1G of common read-write between described virtual machine, and this section of region of memory can be continuous, also can be discontinuous, but must be page alignment; Operational process is that a virtual machine provides shared drive region, carries out " mandate " operation, and other virtual machines are mapped to own address space to this region memory, then carry out read-write operation;
Described virtual machine provides collaboration method based on virtual platform internal memory shared mechanism for virtual application, described virtual application is the application program that operates in virtual machine inside, and described collaboration method is for sharing the same partial memory of read-write and communicate and synchronously based on a plurality of virtual machines.
2. the large capacity internal memory sharing method of a kind of virtual machine according to claim 1, is characterized in that, described method comprises the steps:
S1. on physical server, start virtual platform;
S2. on virtual platform, start some virtual machines as the carrier of application operation;
S3. on virtual platform, start a virtual machine as the carrier of shared drive;
S4. in shared drive carrier inside, move virtual driving, the interface that calling virtual platform provides carries out page Authorized operation, and the page of appointment is licensed to other virtual machine mappings;
S5. shared drive carrier inside is set up event channel, monitors the connection of application virtual machine;
S6. the virtual driving of application virtual machine internal operation, the interface that calling virtual platform provides shines upon the shared drive page, is mapped to kernel or user's state address space of appointment;
S7. many application virtual machines carry out read-write operation to the same partial memory page, thereby communicate and resource sharing fast;
S8. in map operation process, can be undertaken synchronously and communicate by letter by event channel mechanism.
3. application rights requires a kind of expanding system based on virtual machine shared drive mechanism of method described in 2, it is characterized in that, described system comprises the following stated module: virtual basic system expansion module, for expanding on virtual platform, thereby allow to support the internal memory that surpasses 1G capacity to share at bottom; The object of expansion module is to expand internal memory to share authorization list, and the ability of making it have is preserved a large amount of mandates and map information.
4. application rights requires a kind of expanding system based on virtual machine shared drive mechanism of method described in 2, it is characterized in that, described system comprises the following stated module: internal memory provides module;
Described module runs on provides the virtual machine of shared drive inner, will specify the internal memory of physical location to authorize, and licenses to designated virtual machine access; Licensing process can obtain corresponding grant number, and each Authorized operation can be authorized a page; The page after mandate is marked as and can allows designated virtual machine access according to authorization list; The grant number that Authorized operation produces must be passed to by transmitting module in the memory-mapped module of authorized virtual machine end to be used.
5. application rights requires a kind of expanding system based on virtual machine shared drive mechanism of method described in 2, it is characterized in that, described system comprises the following stated module: memory-mapped module; Described virtual machine is received and is shared after grant number, calls the interface of mapping, using grant number as parameter, is mapped to kernel spacing or the user's space address of appointment; Mapping block can be used the page-map of authorizing to kernel spacing or user's space; After mapping, returning to what obtain is the virtual address of designated space; After mapped page, virtual machine has the authority of read and write to this page, but cannot discharge this part resource, because the owner of this page remains, provides the virtual machine of sharing the page.
6. application rights requires a kind of expanding system based on virtual machine shared drive mechanism of method described in 2, it is characterized in that, described system comprises the following stated module: grant number is transmitted module; Described grant number transmission module plays is providing virtual machine and the use of sharing the page to share the effect of transmitting grant number between page virtual machine;
The mechanism of transmitting module employing is shared queue's mode;
The page that first is shared, transmits the effect that the page is used as transmitting other all shared grant number;
Each provides between the virtual machine of shared drive and the application virtual machine of use shared drive and exists to transmit a page;
Transmit the grant number of the page and use nonautomatic mode to transmit, for example transmit in manual or socket path;
The grant number of the shared page of rear generation all will be put into the transmission page, when the transmission page is piled, internal memory provides module to send out message to memory-mapped module by event channel, and memory-mapped module notifies internal memory to provide module can continue to put into new grant number by event channel after just taking out all grant number equally;
By a large amount of grant number of mode circulating transfer of similar table tennis, guarantee to realize large internal memory and share.
7. according to the system described in claim 3 to 5 any one, it is characterized in that, described system adopts virtual platform to select technology:
The virtual platform of selecting must provide basic page management mechanism, allows the page to be accessed by a plurality of virtual machines;
In page management, must there is pair mechanism that page access authority is controlled.
8. system according to claim 6, is characterized in that, described system adopts the shared virtual machine number control technology of the page: if authorization list put under, can share to numerous virtual machine and use;
Each project of authorization list comprises provides the virtual machine of mandate ID, target virtual machine ID, the physical page address of authorization page;
Support a plurality of virtual machines to access the same page, also support a virtual machine to shine upon the shared page of a plurality of virtual machine mandates.
9. system according to claim 7, is characterized in that, described system adopts page Share Permissions control technology:
In order to guarantee maximum dirigibility, system is not distinguished and is treated level of privilege virtual machine and non-level of privilege virtual machine, allows both sides' the page to share and access mutually;
Access limit acquiescence is given, and in Authorized operation, acquiescence is authorized the authority that other virtual machines are read and write this page;
Acquiescence support is mapped to kernel state and user's state address space; Described system adopts memory-mapped scale restriction technologies:
Under default situations, only allow virtual machine to share 64M internal memory;
Add after virtual basic system expansion, allow 4G memory headroom to share.
10. system according to claim 8, is characterized in that, described system adopts grant number to transmit designing technique:
Owing to using and must first be mapped to local address space sharing the page, and must conduct interviews to authorization list during mapping, so the index of authorization list is that grant number must pass to the virtual machine that need to carry out page-map;
Transfer mode has a variety of, such as socket Internet Transmission etc., but full blast or by sharing the page, transmit, saved between kernel state user state repeatedly data copy;
First shared page need to pass to virtual machine by manual or socket mode as transmitting the page;
Between every pair of share-mapping virtual machine, need to have one to transmit the page;
Transmit the event channel that the page need to be arranged in pairs or groups and be connected, to guarantee that when the page is filled with the side of collecting can receive that message appropriately processes;
The foundation of event channel makes the work of transmit leg and the side of collecting synchronous;
Described system adopts user's state Address space mappinD technology, must mapping API be exposed to user's state by creating corresponding character device, more conveniently adds the lib storehouse of new user's state again, the access of encapsulation to character device;
On Xen4.0 platform, the mapping of user's state address space by libxenctrl storehouse API and/dev/gntdev character device realizes;
Described system adopts DMA address Method with Space Mapping Technique:
DMA address spatial mappings is identical with the non-DMA address space use of kernel state application, all can be used to carry out page mandate and mapping.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310149915.4A CN104123233A (en) | 2013-04-26 | 2013-04-26 | Virtual machine high-capacity memory sharing method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310149915.4A CN104123233A (en) | 2013-04-26 | 2013-04-26 | Virtual machine high-capacity memory sharing method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN104123233A true CN104123233A (en) | 2014-10-29 |
Family
ID=51768653
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310149915.4A Pending CN104123233A (en) | 2013-04-26 | 2013-04-26 | Virtual machine high-capacity memory sharing method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104123233A (en) |
Cited By (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105511940A (en) * | 2015-11-30 | 2016-04-20 | 广州云宏信息科技股份有限公司 | Method and system authorizing virtual machine to access Xenstore in Xen virtualization |
| CN106557427A (en) * | 2015-09-25 | 2017-04-05 | 中兴通讯股份有限公司 | The EMS memory management process and device of shared drive data base |
| CN106874128A (en) * | 2017-01-22 | 2017-06-20 | 广州华多网络科技有限公司 | Data transmission method and device |
| CN107168894A (en) * | 2017-06-30 | 2017-09-15 | 联想(北京)有限公司 | Memory sharing access method and electronic equipment |
| CN107368353A (en) * | 2017-07-26 | 2017-11-21 | 郑州云海信息技术有限公司 | A kind of method and apparatus for realizing virutal machine memory heat addition |
| CN108984300A (en) * | 2018-07-03 | 2018-12-11 | 北京华大九天软件有限公司 | A kind of layout data checks method, system and storage medium |
| CN109358818A (en) * | 2018-10-30 | 2019-02-19 | 深圳润迅数据通信有限公司 | A kind of block device I/O Request processing method of data center |
| CN109783207A (en) * | 2017-11-13 | 2019-05-21 | 厦门雅迅网络股份有限公司 | Protect the method and system of dual system shared drive data safety |
| CN110007863A (en) * | 2019-04-10 | 2019-07-12 | 苏州浪潮智能科技有限公司 | A kind of storage service access method and relevant apparatus based on lib bank interface |
| CN110262875A (en) * | 2019-06-25 | 2019-09-20 | 苏州浪潮智能科技有限公司 | The communication means and system of Windows virtual machine and KVM host based on patch mechanism |
| CN110442425A (en) * | 2019-07-19 | 2019-11-12 | 南京芯驰半导体科技有限公司 | A kind of virtualization address space shielding system and method |
| CN111143851A (en) * | 2019-12-11 | 2020-05-12 | 上海交通大学 | Detection method and system suitable for leakage of kernel object address of operating system |
| CN111638968A (en) * | 2019-03-01 | 2020-09-08 | 维塔科技(北京)有限公司 | Shared memory access permission setting method and device, electronic equipment and storage medium |
| CN113342465A (en) * | 2021-06-18 | 2021-09-03 | 上海交通大学 | Giant virtual machine based on release consistency memory synchronization |
| CN113641466A (en) * | 2021-10-15 | 2021-11-12 | 云宏信息科技股份有限公司 | Memory allocation method and system for XEN platform and computer readable storage medium |
| CN114520825A (en) * | 2022-01-07 | 2022-05-20 | 中汽创智科技有限公司 | Distributed Hypervisor microkernel architecture, communication method and device |
| CN115061784A (en) * | 2022-08-17 | 2022-09-16 | 中诚华隆计算机技术有限公司 | Method and device for sharing memory by security container |
| CN115309511A (en) * | 2022-09-28 | 2022-11-08 | 亿咖通(湖北)技术有限公司 | Xen-based data interaction method and device, storage medium and electronic equipment |
| WO2023230766A1 (en) * | 2022-05-30 | 2023-12-07 | 华为技术有限公司 | Data transmission method and virtualization system |
| CN117573419A (en) * | 2024-01-16 | 2024-02-20 | 上海芯联芯智能科技有限公司 | Page exception handling method and device |
| CN117573419B (en) * | 2024-01-16 | 2024-04-26 | 上海芯联芯智能科技有限公司 | Page exception handling method and device |
-
2013
- 2013-04-26 CN CN201310149915.4A patent/CN104123233A/en active Pending
Cited By (32)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106557427A (en) * | 2015-09-25 | 2017-04-05 | 中兴通讯股份有限公司 | The EMS memory management process and device of shared drive data base |
| CN106557427B (en) * | 2015-09-25 | 2021-11-12 | 中兴通讯股份有限公司 | Memory management method and device for shared memory database |
| CN105511940B (en) * | 2015-11-30 | 2019-02-01 | 云宏信息科技股份有限公司 | The method and system of authorization virtual machine access Xenstore in a kind of Xen virtualization |
| CN105511940A (en) * | 2015-11-30 | 2016-04-20 | 广州云宏信息科技股份有限公司 | Method and system authorizing virtual machine to access Xenstore in Xen virtualization |
| CN106874128A (en) * | 2017-01-22 | 2017-06-20 | 广州华多网络科技有限公司 | Data transmission method and device |
| CN106874128B (en) * | 2017-01-22 | 2020-11-20 | 广州华多网络科技有限公司 | Data transmission method and device |
| CN107168894B (en) * | 2017-06-30 | 2020-08-25 | 联想(北京)有限公司 | Memory sharing access method and electronic equipment |
| CN107168894A (en) * | 2017-06-30 | 2017-09-15 | 联想(北京)有限公司 | Memory sharing access method and electronic equipment |
| CN107368353A (en) * | 2017-07-26 | 2017-11-21 | 郑州云海信息技术有限公司 | A kind of method and apparatus for realizing virutal machine memory heat addition |
| CN107368353B (en) * | 2017-07-26 | 2020-12-01 | 郑州云海信息技术有限公司 | Method and device for realizing hot addition of virtual machine memory |
| CN109783207A (en) * | 2017-11-13 | 2019-05-21 | 厦门雅迅网络股份有限公司 | Protect the method and system of dual system shared drive data safety |
| CN109783207B (en) * | 2017-11-13 | 2023-08-22 | 厦门雅迅网络股份有限公司 | Method and system for protecting dual-system shared memory data security |
| CN108984300A (en) * | 2018-07-03 | 2018-12-11 | 北京华大九天软件有限公司 | A kind of layout data checks method, system and storage medium |
| CN109358818A (en) * | 2018-10-30 | 2019-02-19 | 深圳润迅数据通信有限公司 | A kind of block device I/O Request processing method of data center |
| CN111638968A (en) * | 2019-03-01 | 2020-09-08 | 维塔科技(北京)有限公司 | Shared memory access permission setting method and device, electronic equipment and storage medium |
| CN111638968B (en) * | 2019-03-01 | 2024-03-15 | 维塔科技(北京)有限公司 | Shared memory access authority setting method and device, electronic equipment and storage medium |
| CN110007863A (en) * | 2019-04-10 | 2019-07-12 | 苏州浪潮智能科技有限公司 | A kind of storage service access method and relevant apparatus based on lib bank interface |
| CN110262875A (en) * | 2019-06-25 | 2019-09-20 | 苏州浪潮智能科技有限公司 | The communication means and system of Windows virtual machine and KVM host based on patch mechanism |
| CN110442425A (en) * | 2019-07-19 | 2019-11-12 | 南京芯驰半导体科技有限公司 | A kind of virtualization address space shielding system and method |
| CN111143851A (en) * | 2019-12-11 | 2020-05-12 | 上海交通大学 | Detection method and system suitable for leakage of kernel object address of operating system |
| CN111143851B (en) * | 2019-12-11 | 2023-08-08 | 上海交通大学 | Detection method and system suitable for kernel object address leakage of operating system |
| CN113342465B (en) * | 2021-06-18 | 2022-06-21 | 上海交通大学 | Giant virtual machine based on release consistency memory synchronization |
| CN113342465A (en) * | 2021-06-18 | 2021-09-03 | 上海交通大学 | Giant virtual machine based on release consistency memory synchronization |
| CN113641466A (en) * | 2021-10-15 | 2021-11-12 | 云宏信息科技股份有限公司 | Memory allocation method and system for XEN platform and computer readable storage medium |
| CN114520825A (en) * | 2022-01-07 | 2022-05-20 | 中汽创智科技有限公司 | Distributed Hypervisor microkernel architecture, communication method and device |
| CN114520825B (en) * | 2022-01-07 | 2023-12-26 | 中汽创智科技有限公司 | Hypervisor micro-kernel architecture based on distributed mode, communication method and equipment |
| WO2023230766A1 (en) * | 2022-05-30 | 2023-12-07 | 华为技术有限公司 | Data transmission method and virtualization system |
| CN115061784A (en) * | 2022-08-17 | 2022-09-16 | 中诚华隆计算机技术有限公司 | Method and device for sharing memory by security container |
| CN115061784B (en) * | 2022-08-17 | 2022-10-25 | 中诚华隆计算机技术有限公司 | Method and device for sharing memory by security container |
| CN115309511A (en) * | 2022-09-28 | 2022-11-08 | 亿咖通(湖北)技术有限公司 | Xen-based data interaction method and device, storage medium and electronic equipment |
| CN117573419A (en) * | 2024-01-16 | 2024-02-20 | 上海芯联芯智能科技有限公司 | Page exception handling method and device |
| CN117573419B (en) * | 2024-01-16 | 2024-04-26 | 上海芯联芯智能科技有限公司 | Page exception handling method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104123233A (en) | Virtual machine high-capacity memory sharing method | |
| US10691363B2 (en) | Virtual machine trigger | |
| Zhou et al. | A bare-metal and asymmetric partitioning approach to client virtualization | |
| US9619270B2 (en) | Remote-direct-memory-access-based virtual machine live migration | |
| CN110858185A (en) | Virtual device composition in an extensible input/output (I/O) virtualization (S-IOV) architecture | |
| EP3388934A1 (en) | Method and apparatus for processing read/write request in physical host | |
| CN101076782B (en) | Method and device for providing virtual blade server | |
| CN101398769B (en) | Processor resource integrating and utilizing method transparent to operating system | |
| WO2017121273A1 (en) | Method and device for processing i/o request under kvm virtualization | |
| CN102609298B (en) | Based on network interface card virtualization system and the method thereof of hardware queue expansion | |
| US20060206891A1 (en) | System and method of maintaining strict hardware affinity in a virtualized logical partitioned (LPAR) multiprocessor system while allowing one processor to donate excess processor cycles to other partitions when warranted | |
| CN103034524A (en) | Paravirtualized virtual GPU | |
| CN104714846A (en) | Resource processing method, operating system and equipment | |
| CN102262557A (en) | Method for constructing virtual machine monitor by bus architecture and performance service framework | |
| CN101876954B (en) | Virtual machine control system and working method thereof | |
| US10853259B2 (en) | Exitless extended page table switching for nested hypervisors | |
| US20120124186A1 (en) | Systems, devices, and methods for multiple host management | |
| Tu et al. | Secure I/O device sharing among virtual machines on multiple hosts | |
| CN105556473A (en) | I/O task processing method, device and system | |
| CN102799465A (en) | Virtual interrupt management method and device of distributed virtual system | |
| Chang et al. | On construction and performance evaluation of a virtual desktop infrastructure with GPU accelerated | |
| Rasmusson et al. | Performance overhead of KVM on Linux 3.9 on ARM Cortex-A15 | |
| Xu et al. | Exploiting analytics shipping with virtualized MapReduce on HPC backend storage servers | |
| CN113568734A (en) | Virtualization method and system based on multi-core processor, multi-core processor and electronic equipment | |
| CN104468307A (en) | Real-time communication system based on virtual machine |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20141029 |