CN102652305A

CN102652305A - Virtual storage target offload techniques

Info

Publication number: CN102652305A
Application number: CN2010800577574A
Authority: CN
Inventors: J.奥欣斯; D.L.格林
Original assignee: Microsoft Corp
Current assignee: Microsoft Technology Licensing LLC
Priority date: 2009-12-17
Filing date: 2010-11-23
Publication date: 2012-08-29
Anticipated expiration: 2030-11-23
Also published as: US20110154318A1; EP2513784A2; US10248334B2; RU2562436C2; JP2013514588A; EP2513784A4; CN102652305B; RU2012130056A; KR101782342B1; WO2011084257A3; US20170017422A1; KR20120115519A; WO2011084257A2; JP6055310B2; US9389895B2

Abstract

A virtual machine storage service can use a unique network identifier and a SR-IOV compliant device can be used to transport I/O between a virtual machine and the virtual machine storage service. The virtual machine storage service can be offloaded to a child partition or migrated to another physical machine along with the unique network identifier.

Description

Virtual store target Unloading Technology

Background technology

Virtual machine technique can be used for packing operating load and move it in data center.This have great benefit from the ability that a physical host moves to another physical host to the user with operating load, because it allows dynamic machine to merge, this causes much lower hardware and handling cost.Virtual machine typically visits storage through the module that processing is positioned at the Storage Virtualization of supervisory routine, management zone or its combination.In this model; Virtual machine typically via such as the internal zone dividing communication bus (as at the U. S. application No.11/128 that is entitled as " Partition Bus "; The example partition bus of describing in 647, the content of this application is herein incorporated by integral body by reference) and so on the software communication path send storage I/O request to said module.Between virtual machine and the supervisory routine (or management zone) communicate by letter owing to the operational communications path and when the message transfer contingent any context switch and cause (incur) CPU cycle cost (cycle cost).Therefore, it is desired being used for through reducing the original technology that increases processing I/O request efficiency of CPU one-tenth.

Summary of the invention

An example embodiment of the present disclosure has been described a kind of method.In this example; This method includes but not limited to: realize (effectuate) transportable stores service; Wherein this transportable stores service is configured to manage the virtual hard disk I/O request to child partition, and wherein this transportable stores service is allocated for the only network identifier of network; And this transportable stores service is configured to the storage target in the network.In addition to the foregoing, in the claim, accompanying drawing and the text that form a part of this disclosure, described aspect other.

An example embodiment of the present disclosure has been described a kind of method.In this example, this method includes but not limited to: the first only network identifier additional (attach) that will be used for network adapter is to being configured to manage the stores service to the virtual hard disk I/O request of child partition; And will append to child partition by the virtual functions (function) that network adapter realizes, wherein this virtual functions comprises the second only network identifier.Except aforementioned, in the claim, accompanying drawing and the text that form a part of this disclosure, described aspect other.

An example embodiment of the present disclosure has been described a kind of method.In this example; This method includes but not limited to: in child partition, carry out stores service; Wherein this stores service is configured to manage the virtual hard I/O request to second child partition, and wherein this stores service is assigned with the only network identifier in network.Except aforementioned, in the claim, accompanying drawing and the text that form a part of this disclosure, described aspect other.

Those skilled in the art can understand one or more different aspect of the present disclosure and can include but not limited to: the circuit and/or the programming of the each side of the present disclosure that is used to realize that this paper quotes; Said circuit in fact can be the combination in any that is configured to realize according to the design alternative of system designer hardware, software and/or the firmware of the each side that this paper quoted with programming.

Aforementioned is summary, and therefore must comprise simplification, summary and the omission of details.It only is illustrative that those skilled in the art will understand this summary, and does not plan to limit by any way.

Description of drawings

Fig. 1 has described example computer system, wherein can realize each side of the present disclosure.

Fig. 2 has described to be used to put into practice the operating environment of each side of the present disclosure.

Fig. 3 has described to be used to put into practice the operating environment of each side of the present disclosure.

Fig. 4 illustrates the computer system that comprises SR-IOV compatible network equipment.

Fig. 5 illustrates the relation between storer in the virtualized environment.

Fig. 6 has described an embodiment of the present disclosure.

Fig. 7 has described to be used to illustrate the operating environment of each side of the present disclosure.

Fig. 8 has described to be used to put into practice the operating process of each side of the present disclosure.

Fig. 9 has described the alternate embodiments of the operating process of Fig. 8.

Figure 10 has described to be used to put into practice the operating process of each side of the present disclosure.

Figure 11 has described the alternate embodiments of the operating process of Figure 10.

Figure 12 has described to be used to put into practice the operating process of each side of the present disclosure.

Figure 13 has described the alternate embodiments of the operating process of Figure 12.

Embodiment

All embodiment can carry out on one or more computer systems.Fig. 1 and following discussion aim to provide the brief general introduction of the suitable computing environment that can realize therein the disclosure.

Run through the firmware that terms circuit that the disclosure uses can comprise nextport hardware component NextPort (such as hardware interrupts controller, hard disk drive, network adapter, graphic process unit, hardware based video/audio codec) and be used to operate this hardware.Terms circuit can also comprise microprocessor, special IC and/or one or more logic processor, for example by the one or more kernels in many kernels (core) General Porcess Unit of firmware and/or software arrangements.(a plurality of) but (a plurality of) function that logic processor can be loaded from storer (for example RAM, ROM, firmware and/or high capacity storage) by the instruction arranged for executing that embodies operation logic.In an example embodiment, wherein circuit comprises the combination of hardware and software, and the implementor can write the source code that embodies logic, and it is compiled into subsequently can be by the machine readable code of logic processor execution.Developed into the stage that between the function that hard-wired function or software are realized, there are differences hardly because those skilled in the art can understand present technical merit, therefore the selection to the software and hardware of realizing function as herein described only is design alternative.In other words, can be converted into hardware configuration of equal value, and hardware configuration itself can be converted into software process of equal value, the selection of hardware implementation mode and software realization mode is left for the implementor because those skilled in the art can understand software process.

Referring now to Fig. 1, described exemplary computer system 100.Computer system 100 can comprise logic processor 102, for example carries out the hyperthread of kernel.Although illustrate a logic processor 102; But computer system 100 can have a plurality of logic processors in other embodiments, for example a plurality of execution kernels of having of each processor substrate and/or wherein each processor substrate can have a plurality of processor substrates of a plurality of execution kernels.As shown in the figure, various computer-readable recording mediums 110 can be through being coupled to various system components one or more interconnects of logic processor 102.System bus can be some types bus-structured any, the bus structure of said some types comprise any local bus of memory bus or Memory Controller, peripheral bus and the multiple bus architecture of use.In example embodiment, computer-readable recording medium 110 can comprise for example random-access memory (ram) 104, memory device 106 (for example dynamo-electric hard disk drive, solid-state hard drive etc.), firmware 108 (for example FLASH RAM or ROM) and movable memory equipment 118 (such as CD-ROM, floppy disk, DVD, FLASH driver, External memory equipment etc.).Those skilled in the art should understand the computer-readable medium that can use other types, such as magnetic tape cassette, flash card, digital video disc, Bernoulli Jacob (Bernoulli) box.

Computer-readable recording medium 110 can be provided for the non-volatile and volatile storage of processor executable 122, data structure, program module and other data of computing machine 100.Being included in the basic input/output (BIOS) 120 that transmits the basic routine of information between the element that helps between the starting period in computer system 100 can be stored in the firmware 108.A plurality of programs can be stored in firmware 108, memory device 106, RAM 104 and/or the movable memory equipment 118 and can be carried out by the logic processor that comprises operating system and/or application program 102.

Computing machine 100 can receive order and information through input equipment 116, and input equipment 116 can include but not limited to keyboard and pointing apparatus.Other input equipments can comprise microphone, operating rod, game paddle (pad), scanner or the like.These can be connected to logic processor 102 through the serial port interface that is coupled to system bus with other input equipments, and are often connected through other interfaces such as USB port (USB).The display device of display or other types also can be connected to system bus via the interface such as video adapter (can be the part of graphic process unit 112 or be connected to graphic process unit 112).Except display, computing machine typically comprises other peripheral output device (not shown), such as loudspeaker and printer.The example system of Fig. 1 can also comprise host adapter, small computer system interface (SCSI) bus and be connected to the External memory equipment of SCSI bus.

Computer system 100 can be operated in the networked environment that the logic that uses remote computer connects.This remote computer can be another computing machine, server, router, network PC, peer device or other common network nodes, and typically can comprise many or all elements that preceding text are described with respect to computer system 100.

When in LAN or WAN networked environment, using, computer system 100 can be connected to LAN or WAN through NIC 114.NIC 114 (it can be inner or outside) can be connected to logic processor.In networked environment, program module or its part described with respect to computer system 100 can be stored in the remote memory storage device.Will be appreciated that it is exemplary that network described herein connects, and can use other means of between computing machine, setting up communication linkage.And, being suitable for computerized system especially well although imagined a plurality of embodiment of the present disclosure, content does not plan the disclosure is limited to such embodiment in the document.

Referring now to Fig. 2 and 3, they have described to be configured to realize the computer system 200 of virtual machine and 300 high level block diagram.In example embodiment of the present disclosure, computer system 200 and 300 can comprise the element and the assembly that can operate with the realization virtual machine described in Fig. 1.Forward Fig. 2 to, such assembly is a supervisory routine (hypervisor) 202, and it also can be called as virtual machine monitor in the art.Supervisory routine 202 among the embodiment that is described can be configured to control and arbitrate the access hardware to computer system 100.Say that broadly supervisory routine 202 can generate the execution environment that is called as subregion (for example, virtual machine).In an embodiment, child partition can be considered to the elementary cell by the isolation (isolation) of supervisory routine 202 supports.Promptly; Each child partition (246 and 248) can be mapped to one group of hardware resource under supervisory routine 202 controls; For example storer, equipment, logic processor circulation or the like; And/or father's subregion and supervisory routine 202 can isolate a process (process) in the subregion in order to avoid visit the resource of another subregion, for example can client operating system in the subregion and the storer in another subregion be isolated.In an embodiment, supervisory routine 202 can be the software product that isolates, operating system a part, be embedded in the firmware of mainboard, special IC or its combination.

In the example of being described, computer system 100 comprises father's subregion 204, and it also can be counted as the territory 0 that is similar in the community that increases income.Father community 204 can be configured to through using the virtualization services that typically in the community that increases income, is called as back driver to provide device (VSP) to the client operating system of in child partition, carrying out resource to be provided.In this exemplary architecture, father's subregion 204 can be controlled (gate) visit to bottom hardware.Say that broadly VSP 228 can be used for through virtualization services client (VSC) (community typically is called as FED(front end driver) increasing income) interface duplex being arrived hardware resource.Each child partition can comprise one or more virtual processors, such as virtual processor 230 to 232, client operating system 220 to 222 can manage and scheduling thread to carry out above that.Usually, virtual processor 230 to 232 is executable instruction and associated state information, and it provides the expression of the concurrent physical processor with certain architectures.For example, a sub-partition can have the virtual processor of the characteristic that contains Intel x86 processor, and another virtual processor can have the characteristic of PowerPC processor.Virtual processor in this example can be mapped to the logic processor of computer system, and the virtual processor of call instruction is carried out by the logic processor support.Therefore, in these example embodiment, a plurality of virtual processors can be carried out simultaneously, and for example another logic processor executive supervisor instruction.The combination of virtual processor, various VSC and storer can be considered to virtual machine in the subregion.

Client operating system 220 to 222 can comprise any operating system, such as the operating system from Microsoft, Apple, the community that increases income etc.Client operating system can use user/kernel operations pattern, and can have the kernel that can comprise scheduler, memory manager etc.Each client operating system 220 to 222 can have file system and the client operating system self that is associated; The file system that is associated can have storage application above that, such as terminal server, e-commerce server, e-mail server etc.Client operating system 220-222 can scheduling thread on virtual processor 230-232, carrying out, and the instance of such application can be implemented.

Referring now to Fig. 3, it illustrates the alternative framework of the framework described in preceding text Fig. 2.Fig. 3 has described the assembly similar with Fig. 2; Yet in this example embodiment, supervisory routine 202 can comprise that virtualization services provides device 228 and device driver 224, and father's subregion 204 can comprise configuration tool 236.In this framework, supervisory routine 202 can be carried out the supervisory routine 202 same or analogous functions with Fig. 2.The supervisory routine 202 of Fig. 3 can be the software product that isolates, operating system a part, be embedded in the firmware of mainboard, perhaps the part of supervisory routine 202 can realize through special IC.In this example, father's subregion 204 can have the instruction that can be used for configuration manager 202, however the hardware access request can handle by supervisory routine 202, and be not sent to father's subregion 204.

In embodiment of the present disclosure, meet clearly the network adapter of quoting " Single Root Input/Output Virtualization specification (the virtual standard of single the I/O) " correction 1.0 that is herein incorporated through integral body and can be installed in such as in the computer system described in the accompanying drawing.The example adapter can be from Intel " Gigabit ET Dual Port Server Adapter (kilomegabit ET dual-ported server adapter) ".The network equipment that the SR-IOV ability is arranged be can through virtual interface be physical function and between for example virtual machine or any other process the hardware device of share I/O adapter.Each virtual interface (also being called as virtual functions (VF)) roughly shows as the independent NIC on the high-speed PCI bus of computer system.For example, each virtual functions can have the pci configuration space and the only network identifier of emulation, for example Media Access Control address (MAC Address), worldwide name etc.Therefore, each virtual functions can support to be used to visit the addressing uniquely of physical function and the independent path of strong subregion.

Forward Fig. 4 to, it illustrates computer system 400, and it comprises SR-IOV compatible adapter 402 (" adapter ").Be similar to mentioned abovely, computer system 400 can comprise and is similar to the assembly of preceding text about the assembly of Fig. 1-3.Adapter 402 can comprise physical function 410, and this physical function can be corresponding to the port that can be connected to network and internal router 412.Internal router 412 can be configured to from such as distributing to network identifier 420-424 route data

virtual functions

404 or 406, adapter 402, said adapter for example is the virtual adapter that all has virtual port.

In example embodiment, network adapter 402 can be an Ethernet Adaptation Unit, and virtual functions can be the virtual ethernet adapter.In this example, the unique identifier of virtual functions will be an ethernet mac address.In the fiber channel example, adapter 402 can be the fiber channel host bus adaptor, and virtual functions can be the virtual fiber channel host bus adaptor with worldwide name, and worldwide name comprises that global nodename and worldwide port name claim.In the infinite bandwidth example, virtual functions can be the virtual unlimited bandwidth end points with global identifier.

Network identifier 424 is shown in broken lines, and the network adapter of its indication such as fiber channel host bus adaptor or Ethernet Adaptation Unit can allow a plurality of unique identifiers to share the single physical port.In fiber channel; This ability is called as the virtual or NPIV of N_ port id; And adapter can comprise the embedded type virtual switch to be called as the pattern operation of promiscuous mode in Ethernet, or filtration and route to the data of particular mac address addressing with the separation storage buffer.

Each network identifier possibly be associated with software protocol stack (414-418), and the software protocol stack is configured to formatted message so that this information can be sent out through network.In specific TCP/IP example, process can be through the instance of application layer port binding to the application layer of TCP/IP stack.At last, can be handled by the one group of function that is present in the layer that is called as the medium key-course by the difference in functionality information processed of protocol stack, this media access control layer is responsible for assembling the frame (frame) of the data that can pass through the fabric transmission.This of protocol stack layer adds the Media Access Control address that is used for virtual functions on network, send out frame.Protocol stack is delivered to Physical layer with the frame of assembling then, and Physical layer is configured to the information translation in the frame is electric signal and said frame sent to network.

RAM is coupled in the I/O interconnection (such as the high-speed PCI interconnection) that I/O MMU 426 (I/O-MMU) can be used for carrying out direct memory access operation.In embodiment of the present disclosure, I/O-MMU 426 can comprise coming the page table of self-supervisor 202, and it will become the system physical address from client's physical address translations of subregion.I/O-MMU 426 is shown in broken lines, and it indicates him to may reside in a plurality of positions in the computer system 400.For example, I/O-MMU can be chip or the assembly of logic processor on the mainboard.

Fig. 5 illustrates relation between the client's physical address and system physical address in an embodiment of the present disclosure.Customer memory is the view of memory (view) by supervisory routine 202 controls.Customer memory can be assigned to client operating system and controlled by their memory manager.Client's physical address can be supported (back) by system physical address (SPA), for example by the storer of the physical computer system of supervisory routine 202 management.As shown in the figure, in an embodiment, GPA and SPA may be arranged to memory block, for example one or more pages of storer.Relation between GPA and the SPA can be kept by shadow page table; These page tables are such as the commonly assigned U.S. Patent application No11/128 that is entitled as " Enhanced Shadow Page Table Algorithms (strengthening the shadow page table algorithm) "; Page table described in 665, the content of the document is quoted through integral body and is herein incorporated.In operation, when client operating system was stored data in GPA piece 1, in fact these data can be stored among the different SPA, such as the piece in the system 6.In embodiment of the present disclosure, I/O-MMU 426 can I/O operating period carry out conversion with will store data directly from a GPA spatial movement to another GPA space.In this embodiment, owing to need in the supervisory routine instruction, operation not realize these conversions, thus can save the logic processor circulation.

Fig. 6 illustrates the advanced operation environment that is used to describe virtual store target Unloading Technology.Fig. 6 illustrates via SR-IOV network adapter 402 and virtual functions 406 thereof and the virtual machine stores service 602 of communicating by letter with Storage Virtualization client 604.As shown in the figure, in this embodiment of the present disclosure, SR-IOV network adapter 402 can be used between virtual machine and virtual machine stores service, transmitting I/O through walking around (bypassing) software communication path.This has reduced the CPU round-robin amount that is used to carry out the I/O that is used for virtual machine conversely, has increased the ability of migration stores service 602, and has reduced burden and/or the burden on the supervisory routine 202 on the host operating system of in father's subregion, carrying out potentially.

Virtual machine stores service 602 can be configured to communicate by letter with physical storage device; These physical storage devices for example are the logical unit number (LUN) that is provided by the SAN that represents child partition, and said numbering for example possibly be to have used the virtualized dirk of other storage virtualization technologies (dirk).In an example, this can comprise that configuring virtual machine stores service 602 is to receive the I/O request and they are routed to LUN from virtual machine.In another example, wherein LUN is sub-distribution, and virtual machine stores service 602 can be configured to: generate virtual hard; They are exposed to virtual machine; And with them as virtual hard disk driver (VHD) file storage on the LUN or on the phisical drive.The VHD representation of file can be encapsulated in the virtual hard disk in the single file.It is physical store that the dish that virtual machine stores service 602 can be resolved this document and can be exposed to client operating system 220 is embodied as like it.The virtual hard disk that is generated by virtual machine stores service 602 can be expressed to bus, and this bus can seem the mode access customer operating system in this locality with these virtual hard disks.

In an embodiment of the present disclosure; Can be configured to the storage target of data center through the storage target component that the only network identifier is appended to virtual machine stores service 602 and for example will be used for advertisement virtual machine stores service 602; And virtual machine stores service 602 is configured as the storage target in the network, such as fiber channel target or Internet Small Computer Systems Interface (iSCSI) target.In the iSCSI example context, virtual machine stores service 602 can be through realizing and can implementing iscsi target through the LUN of internet protocol access child partition.Virtual machine storage client 604 or client operating system can obtain the address of virtual machine stores service 602, and the connection of emulation to the connection of SCSI hard disk can be set.Virtual machine storage client 604 can be handled virtual machine stores service 602 with treatment S CSI or the identical mode of hard disk drive, and virtual machine stores service 602 can provide virtual hard (serve up) to give child partition.In this example, the file system that virtual machine storage client 604 can be created and manage on the virtual disk that is directly provided by virtual machine stores service 602, and needn't load (mount) remote directory, as will doing in the networking file system environment.According to the viewpoint of client OS 220, it has the network of network of being coupled to adapter, and this network coupled is to the one or more logical blocks that operate with the mode that is similar to hard disk drive.

Fig. 7 illustrates the exemplary operations environment that is used to put into practice each side of the present disclosure.Be similar to Fig. 6, one or more SR-IOV network adapter can be used between virtual machine and virtual machine stores service, transmitting I/O, thereby eliminate the needs that use the software communication path to send I/O.This has reduced and is used to carry out the CPU round-robin amount to the I/O of virtual machine, has increased the ability of migration stores service 602, and reduces burden and/or the burden on the supervisory routine 202 on the host operating system potentially.

In this example context; The data center that comprises two computer systems 700 and 702 is illustrated as and is connected to switch 704 (although show two computer systems, those skilled in the art can understand data center can have more computer system).Computer system 700 and 702 can have and is similar to the assembly described in Fig. 1-4, and switch 704 can be the switch of interconnection and the whole infrastructure of router.And computer system 700 and 702 is illustrated as and comprises some characteristic with the disclosed technology of herein interpreted more clearly, and the disclosure is not limited in the topological structure of being described, realize.

Computer system 700 can comprise manager 250; It is configured to according to the techniques described herein migrations stores service 602, thus virtual machine stores service 602 with the dotted line diagram to indicate its another subregion can be from a zoned migration to identical or different computer system.Virtual functions 706 and 708 shown in broken lines with indication: in certain embodiments virtual machine stores service 602 can be directly and SR-IOV adapter 402 get in touch, and need not visit it via virtual functions.In this example embodiment, father's subregion 204 and 712 can be controlled physical hardware, and will not need virtual functions.

Continue the general introduction of this figure, in embodiment of the present disclosure, virtual machine stores service 602 can be distributed to its unique identifier and this identifier is moved to different partition areas together with any essential status information and move through extraction.In an example, this process can comprise: extract unique identifier by logic processor operations manager 250; By logic processor operations manager 250 order adapters (402 or 718) this unique identifier is appended to the virtual functions in the different subregions; And it is appended to virtual functions itself by the instance of logic processor operations manager 250 order virtual machine stores service 602.In another example, this process can comprise: extract unique identifier by logic processor operations manager 250; By logic processor operations manager 250 order adapters (402 or 718) this unique identifier is appended to adapter (402 or 718); And use this unique identifier on fabric, to communicate by letter by the instance of logic processor operations manager 250 order virtual machine stores service 602 of instantiation in different subregions.

Hereinafter is a series of process flow diagrams of describing operating process.For easy to understand, these process flow diagrams are organized into and make that viewpoint presents implementation to initial process flow diagram via the overall situation " big figure ", and follow-up process flow diagram provides further condiment and/or details.And those skilled in the art can understand, and the operation that with dashed lines is described is considered to optional.

Referring now to Fig. 8, it illustrates the operating process that is used to put into practice each side of the present disclosure.As shown in the figure; Operation 800 begins operating process and operates 802 to show the transportable stores service of realization; Wherein this transportable stores service is configured to manage the virtual hard disk I/O request that is used for child partition, and wherein this transportable stores service is allocated for the unique identifier of network.For example, and forward Fig. 6 to, the transportable stores service such as virtual machine stores service 602 can be realized by computer system.That is, the instruction of indication virtual machine stores service 602 can be carried out by logic processor.Virtual machine stores service 602 is considered to transportable, because it is attached to the only network identifier and can moves to another subregion from a subregion alone, does not promptly move other administration modules.

In an example embodiment, virtual machine stores service 602 can exclusively be used unique identifier on network, and for example it can be to use unique process that unique network address is communicated by letter in data center.In this example, virtual machine stores service 602 can be configured to its state of serializing, makes status information can be sent to different partition areas and is used to another instance of configuring virtual machine stores service 602.In another example embodiment, virtual machine stores service 602 can be moved in being attached to the virtual machine of virtual functions.In this example, virtual machine stores service 602 can also be used unique identifier communication exclusively in network.Migration virtual machine stores service 602 can comprise serializing comprise virtual machine stores service 602 virtual machine state and send it to another subregion.

In particular example, and forward Fig. 7 to, virtual machine stores service 602 can be moved to child partition 246 by uncle's subregion 204.In this particular example, logic processor can operations manager 250, and promptly logic processor can move the instruction of instruct manager 250, and extracts the unique identifier that is used in data center, communicating by letter by virtual machine stores service 602.Then, this unique identifier instance that can be sent to child partition 246 and virtual machine stores service 602 can be activated.Routing table in the adapter 402 can be updated and the I/O request can be routed to child partition 246 but not father's subregion 204 by adapter 402.In this example, child partition 246 can be configured to use the unique identifier except already used any other unique identifier.

Continue the description of Fig. 8, operation 804 illustrates transportable stores service is configured to the storage target in the network.For example, in an example of the present disclosure, virtual machine stores service 602 can be configured to the storage target in the data center.Be similar to the description of preceding text, virtual machine stores service 602 can be attached to the unique identifier in the network and detected by client OS 220 is the storage target.Can open communication session between client OS 220 and virtual machine stores service 602, and client OS 220 can detect the virtual hard that is exposed by virtual machine stores service 602 and use this virtual disk, is local hard drive like them.In particular example, virtual machine stores service 602 can the aforesaid iscsi target of emulation.In this example, virtual machine stores service 602 can expose virtual disk but not physical disks and handle the I/O of self virtualizing machine through reading or write LUN or physical disks.

Forward Fig. 9 to, it illustrates the alternate embodiments of the operating process of Fig. 8.Operation 906 illustrates moves to remote computer system with transportable stores service.For example, and forward Fig. 6 to, in one embodiment, transportable stores service (for example the virtual machine stores service 602) can be moved to the remote computer system in the data center.For example, in one embodiment, remote computer system can have the more how available I/O bandwidth of computer system than current trustship (host) virtual machine stores service 602, and can make the decision of mobile storage service 602.In this example, logic processor can operations manager 250 and is extracted and distribute to the unique identifier of stores service 602 and send it to remote computer.Subsequently, the manager 250 of remote computer can append to unique identifier the instance of stores service 602.

In particular example, and forward Fig. 7 to, virtual machine stores service 602 can be moved to father's subregion 712 from child partition 246.In this particular example, the manager 250 of computer system 700 can extract the unique identifier that appends to virtual machine stores service 602 and send it to computer system 702.The manager 250 of computer system 702 can append to the instance of the virtual machine stores service 602 of operation in father's subregion 712 in operation on the logic processor and with unique identifier.In this example, virtual store service 602 can be used unique identifier through using or do not use virtual functions 708 from by client's transmission/reception I/O of 602 services of the virtual machine stores service the child partition 246 time.

In this particular example, the status information of virtual machine stores service 602 and protocol stack can be sent to computer system 702, make that the I/O service is continual.For example, can send enough information, with the manager 250 configuration protocol stacks that allow computer system 702 function equivalent state with the protocol stack that reflects computer system 700 at least to computer system 702.Status information can comprise that the next one that will be sent out divides into groups number, socket number of being used, maximum buffer size, server end slogan, client end slogan or the like.Status information can also comprise the information such as the higher level protocol information.Other examples can relate to the information of employed cryptographic protocol.

In this example embodiment, will operate incessantly to the service of client, because according to the viewpoint of client, connect and suspended but not go offline.For example; When virtual machine stores service 602 is moved; Protocol stack can be summed up (wrap-up) current operation that it is being carried out (for example through accomplish or cancel these operations), and in short time period, keeps the agreement transmission of the virtual machine storage client 604 of the information of not sending to step back (back off) message to being tied to request protocol alternatively.When the protocol stack on the computer system 702 during by instantiation, it can have with computer system 700 on the state of protocol stack equivalence, and can be on network communicate by letter with before according with computer system 700 associated unique identification.New configured protocols stack on the computer system 702 can be configured to send alternatively and restarts (resume) message, and the agreement of service virtual machine storage client 604 can restart to send I/O.Switch 704 can solve route, makes protocol message be sent to the virtual machine stores service 602 on the computer system 702.

Continue the description of Fig. 9, operation 908 illustrates configuration I/O MMU becoming the system physical address with client's physical address translations that the I/O request that is used for child partition is associated.For example, and with reference to Fig. 7, in an embodiment of the present disclosure, the I/O MMU 426 of computer system 700 can be used for client's physical address translations is become the system physical address.For example, when client operating system 220 was initiated I/O operation (for example reading or writing), client operating system 220 generated the order that relates to client's physical address that possibly be converted into the system physical address.In an example embodiment, these conversions can be at I/O-MMU 426 but not are taken place among the MMU.Through the storer conversion is unloaded to I/O-MMU 426, reduced the burden on supervisory routine 202 and/or the father's subregion 204.For example, client OS 220 can issue (issue) read operation, and it comprises the request of the dish side-play amount that reads the customer memory address.In this example; I/O MMU 426 can use table, and this table becomes the system address in the customer memory address of physically supporting (back) client to want to read with the customer memory map addresses of child partition 248 to system address and with the customer memory address translation.Virtual machine stores service 602 can receive request and obtain client just in information requested and the response message of the data that comprise previous request is provided.This response can provide in the buffer zone like the appointment of customer memory address; Adapter 402 can become the system physical address with the customer memory address translation that is provided with I/O-MMU 426 in the case, and adapter 402 can copy the request buffer response data to satisfy the request of client to then from the response buffering district.

When client was on the physical computer identical with virtual machine stores service 602, this technology type was similar to direct memory access (DMA) (DMA) operation of the memory to memory of being carried out by peripherals.In this example embodiment; I/O operation can be similar to the dma operation of memory to memory, because network adapter 402 retrieving information and represent virtual machine to store client 604 or virtual machine stores service 602 moves to another piece of system physical address with it from a piece of system physical address.A particular example can comprise the read operation by 604 issues of virtual machine storage client.In this example, virtual machine storage client 604 can be issued and specify it to want to read the read operation of the page of the storage data in the memory page of its control.In this example, the page of data is copied into the virtual machine stores service 602 employed pages to satisfy request and then this copying data to be stored in the locked memory pages of client 604 appointments to virtual machine.

Continue the description of Fig. 9, operation 910 illustrates from child partition and receives the I/O work request, and wherein child partition is affixed to the virtual functions that comprises the second only network identifier that is used for network.For example, as shown in Figure 6, in one embodiment, child partition 248 can comprise virtual functions 406.In this example, the I/O request got in touch and can be sent to child partition 248 can exclusively via virtual functions 406 and SR-IOV adapter 402.Adapter 402 can confirm that order is addressed to and virtual machine stores service 602 associated unique identification symbol, and can send and should order to it.In the case, order can be sent to virtual machine stores service 602 from the I/O of child partition 248, and need not send request to the child partition communication interface through supervisory routine 202 or through child partition.And adapter 402 can use the unique identifier of client 604 and virtual machine stores service 602 to confirm that which locked memory pages will be used as buffer zone and therefore confirm copies data between which address space.

In particular example, I/O request can be the write operation of the position on the position (in client's physical address) of specific data and the virtual hard that data should be written into.In this example, Storage Virtualization client 604 can place request one or more information block of the unique identifier that is addressed to virtual machine stores service 602.In this example, adapter 402 can receive request and send it to virtual machine stores service 602.Adapter 402 can move to the system physical address of distributing to virtual machine stores service 602 with data in addition from child partition client physical address.Promptly; Adapter can be configured to I/O-MMU 426 that the two becomes the system physical address from client's physical address translations with transmission and send buffer, and adapter 402 can send buffer zone according to the system physical address internally with data in inside then and copies send buffer to.Virtual machine stores service 602 then can be with data storage in the suitable position consistent with the implementation of its virtual hard.Can understand like those skilled in the art, this possibly relate to the use virtual hard drive file, and it possibly relate to data storage on LUN, and perhaps it possibly relate to other technologies and the position that is used to store data redundantly.

Continue the description of Fig. 9, operation 912 is illustrated in carries out transportable stores service in first subregion, in second subregion, carry out the management service that is configured to managing virtual machines, and wherein child partition is the 3rd subregion.For example, in one embodiment, virtual machine stores service 602 can be carried out in first subregion (such as child partition 246), and father's subregion 204 can the operational management service, and virtual machine storage client 604 can be moved in subregion 248.In this example embodiment, virtual machine stores service 602 is in the subregion that separates with managing process.In this configuration, child partition 246 can operate as the specific store subregion effectively, as the SAN target.This configuration can reduce the burden on supervisory routine 202 and the father's subregion.For example, through separating stores service and father's subregion, can reduce the locking in the operating system.And through allocating computer system in this way, the quantity of the message that can need send in the by stages through minimizing reduces the burden on the supervisory routine scheduler.

Continue the description of Fig. 9, operation 914 illustrates the virtual functions of transportable stores service with the network adapter that comprises the only network identifier is associated, and child partition is appended to second virtual functions of network adapter.For example, and forward Fig. 7 to, in one embodiment, virtual machine stores service 602 can be associated with the virtual functions such as virtual functions 404.In the instance that virtual machine stores service 602 is moved in child partition 246; Virtual functions 404 can be used; Therefore it is with controlled way visit adapter 402, promptly to guarantee that any process in the child partition 246 do not visit the mode of the data outside its subregion and visit adapter 402.In addition, the virtual machine snapshot operation can be used to move virtual machine stores service 602.

Continue the description of Fig. 9, operation 916 illustrates in response to receive the I/O work request from child partition and is carrying out transportable stores service to logic processor transmission notice and definite logic processor.For example, in one embodiment, when I/O need of work software processes, supervisory routine 202 can receive interrupts and operation.Supervisory routine 202 can be discerned the logic processor that is moving or be scheduled operation virtual machine stores service 602, and can notify this logic processor, promptly interrupts or the lightweight notice through sending.If virtual machine stores service 602 is arranged in child partition, then this interruption can be sent to logic processor, and need not wake management zone up with service message.Just carry out if virtual machine stores service 602 is current, then the context to virtual machine stores service 602 switches and needn't take place, and interrupts also needn't taking place, and uses the lightweight notice because can change into.

Continue the description of Fig. 9; Operation 918 illustrates confirms that this I/O flow meets (compliant) security strategy when I/O flow (traffic) transmits via network adapter between only network identifier and at least one other only network identifier.For example, in one embodiment, adapter 402 can comprise the security strategy that is used for network traffics.In this example embodiment, adapter 402 can be configured to confirm that the I/O flow that between virtual machine stores service 602 and another unique identifier (for example appending to an identifier of virtual machine), sends meets this security strategy.In particular example, security strategy possibly need to encrypt all I/O flows.In this example, adapter 402 can be configured to be determined to the expressly form or encrypted that writes of virtual hard.In another example, security strategy possibly need VLAN to keep independent fully, does not wherein allow data traffic between the end points of each in different virtual LAN.

Forward Figure 10 now to, it illustrates the operating process that is used to put into practice each side of the present disclosure, and it comprises operation 1000,1002 and 1004.Operation 1000 begins this operating process, and operates 1002 and illustrate the first only network identifier that is used for network adapter is appended to stores service, and this stores service is configured to the I/O request of managing virtual hard disk drive disk to child partition.For example, and forward Fig. 6 to, in an embodiment of the present disclosure, SR-IOV adapter 402 can be realized a plurality of network identifiers and distribute to virtual machine stores service 602 with one in them.In the fiber channel example, the fiber channel host bus adaptor can use N_PortID virtual or (NPIV) to allow on same port, using a plurality of unique identifiers.In this fiber channel example, virtual machine stores service 602 can exclusively use the NPIV address of being distributed on fabric, to communicate by letter.

Continue the description of Figure 10, operation 1004 illustrates the virtual functions of being realized by network adapter is appended to child partition, and wherein this virtual functions comprises the second only network identifier.For example, and once more with reference to Fig. 6, SR-IOV adapter 402 can instantiation comprises the virtual functions 406 of only network identifier, and it is appended to virtual machine.In this example embodiment, adapter 402 has been configured to the effect of switch, and it is routed to stores service 602 with the I/O request through adapter, walks around supervisory routine 202 or independent partition-to-partition communication mechanism.This has reduced on logic processor execution command conversely with notice and used time of change partitions.

Forward Figure 11 now to, it illustrates the alternate embodiments of the operating process of Figure 10, and it comprises

additional operations

1106,1108,1110,1112 and 1114.Operation 1106 illustrates the request of transmission to dispose second virtual functions to be included in the first only network identifier of the remote computer system that comprises second network adapter.For example; In one embodiment; Logic processor can execute instruction in manager 250 and can be created on the request of configuration virtual function in the remote computer system, and said remote computer system has another adapter to comprise the only network identifier that appends to virtual machine stores service 602.Forward Fig. 7 to, in particular example, the manager 250 on the computer system 700 can send to the computer system 702 with adapter 718 with the request that is generated.Request in this example can by the manager in the computer system 702 250 be used for order it, with instantiation virtual functions 710, and it is comprised and the instance associated unique identification of virtual machine stores service 602 symbol.

Continue the description of Figure 11, operation 1108 illustrates moves to second virtual functions that child partition and configuration distribute to this child partition to use the first only network identifier with stores service.For example, and forward Fig. 7 to, logic processor can operations manager 250 and virtual machine stores service 602 is moved to child partition 246 from for example father's subregion 204.In this example, logic processor can operations manager 250 and extract with virtual machine stores service 602 associated unique identification symbol and send it to adapter 402.Adapter 402 can instantiation virtual functions 404 and unique identifier is appended to it.Subsequently, manager 250 can append to unique identifier the instance of virtual machine stores service 602.In this example embodiment, virtual machine stores service 602 is in the subregion that separates with managing process and has become the specific store subregion of running as iscsi target effectively.

Forward operation 1110 now to, it illustrates client's physical address translations that will be associated with the I/O request from child partition by the I/O MMU and becomes the system physical address.For example, and with reference to Fig. 7, in an embodiment of the present disclosure, the I/O MMU 426 of computer system 700 can be used for client's physical address translations is become the system physical address.For example, when client operating system 220 was initiated I/O operation (for example reading or writing), client operating system 220 generated the order that relates to client's physical address.In this example, I/O MMU 426 can use the table that the customer memory map addresses of child partition 246 is arrived the system address that is used by father's subregion 204.Adapter 402 can be configured to IO-MMU 426 that the two becomes the system physical address from client's physical address translations with transmission and send buffer; And adapter 402 can send buffer zone internally with data then and copy inner send buffer to, or vice versa.

Forward operation 1112 now to, it illustrates the security strategy compliance (compliance) of configuration network adapter with monitoring this I/O flow when the I/O flow transmits between the only network identifier and the second only network identifier.For example, in one embodiment, adapter 402 can comprise the security strategy to network traffics.In this example embodiment, adapter 402 can be configured to confirm that the I/O flow that between virtual machine stores service 602 and another unique identifier (for example appending to an identifier of virtual machine), sends meets this security strategy.A particular example can comprise the security strategy that needs some virtual machine to use certain unique identifier in network, to send I/O.Adapter 402 in this example can be monitored the packets of information of self virtualizing machine and confirm whether they meet security strategy.

Forward operation 1114 now to, it illustrates to the remote computer system request of sending and surpasses the I/O request amount of predetermined threshold and distribute the first only network identifier to this stores service with the instantiation stores service and in response to confirming to have received from remote computer system.For example; In an embodiment of the present disclosure, logic processor can move the instruction of instruct manager 250 and send the request of the instance that guides remote computer system (such as computer system 702) instantiation virtual machine stores service 602 and it is appended to unique identifier.I/O request associated unique identification symbol of monitoring and arriving and definite the reception from computer system 702 above after a plurality of requests of threshold value, logic processor can generate this request.In particular example, manager 250 can be confirmed the I/O request that accorded with reception 60% in 30 minutes from current and computer system 702 associated unique identification in the past.In this example, manager 250 can be confirmed: carry out and move it if virtual machine stores service 602 is just local on computer system 702, then the performance of data center can be improved.

Forward Figure 12 now to, it illustrates the operating process that comprises operation 1200 and 1202.Operation 1200 begins this operating process and operates 1202 to be illustrated in and to carry out stores service in the child partition; Wherein this memory device is configured to manage the virtual hard disk driving-disc I/O request to second child partition, and wherein this stores service is assigned with the only network identifier in network.For example, in one embodiment, virtual machine stores service 602 can be implemented and in network, be assigned with unique identifier (for example worldwide name) in child partition (for example child partition 246).In this example, child partition 246 can be by supervisory routine 202 and/or 204 controls of father's subregion.In this configuration, child partition 246 can become the specific store subregion of running as iscsi target effectively.

Forward Figure 13 now to, it illustrates the alternate embodiments of the operating process of Figure 12, and it comprises

operation

1304,1306,1308,1310 and 1312.Forward operation 1304 to, it illustrates to the remote computer system request of sending distributes the first only network identifier with the instantiation stores service and in response to the I/O request amount of confirming to receive from remote computer system that surpasses predetermined threshold for this stores service.For example; In an embodiment of the present disclosure; Logic processor can move the instruction of instruct manager 250 and send the request of the instance of guide remote computer system (such as computer system 702) instantiation virtual machine stores service 602, and it is appended to unique identifier.I/O request associated unique identification symbol of monitoring and arriving and definite the reception from computer system 702 above after a plurality of requests of threshold value, logic processor can generate this request.In particular example, manager 250 possibly confirmed in the past 30 minutes to have received 60% I/O request from current with computer system 702 associated unique identification symbol.In this example, manager 250 can be confirmed: carry out and move it if virtual machine stores service 602 is just local on computer system 702, then the performance of data center can be improved.

Continue the description of Figure 13, operation 1306 illustrates moves to supervisory routine with stores service.For example, and forward Fig. 7 to, in one embodiment, virtual machine stores service 602 can be moved to supervisory routine 202.In this example embodiment, computer system 702 can have and is similar to framework depicted in figure 3, and can make the decision that stores service 602 is moved to supervisory routine 202 from child partition 246.In this example, logic processor can operations manager 250 also extract and virtual machine stores service 602 associated unique identification symbol, and supervisory routine 202 can append to this identifier the instance of virtual machine stores service 602.In an example embodiment, because supervisory routine 202 control hardwares, so it can be configured to visit the physical function of adapter 402.In the implementation of fiber channel example, fiber channel host bus controller can use NPIV to order via adapter 402 transmission/reception I/O to use unique identifier.

Continue the description of Figure 13, operation 1308 illustrates moves to father's subregion with stores service.For example, and forward Fig. 7 to, in one embodiment, virtual machine stores service 602 can be moved to father's subregion 204 or 712 from child partition 246.In this example, logic processor can operations manager 250 also extract and virtual machine stores service 602 associated unique identification symbol, and sends it to the father's subregion 204 on remote computer or the local computer system.Subsequently, this unique identifier can be affixed to the instance of stores service 602.

Continue the description of Figure 13, operation 1310 illustrates distributes to supervisory routine with stores service.In this example embodiment, virtual functions 406 can be affixed to child partition 248 and it can have the second only network identifier on network.As shown in the figure, in this example embodiment, child partition 246 with 248 the two can be affixed to identical SR-IOV adapter 402.Therefore, in this example embodiment, can be through SR-IOV adapter 402 but not via supervisory routine 202 or via the partition-to-partition communication mechanism and need not send I/O through switch 704 and transmit the I/O request.

Continue the description of Figure 13, operation 1312 illustrates configuration I/O MMU and becomes the system physical address with client's physical address translations that will be associated with the I/O request to child partition.For example, and with reference to Fig. 7, in an embodiment of the present disclosure, the I/O MMU 426 of computer system 700 can be used to client's physical address translations is become the system physical address.For example, when client operating system 220 was initiated I/O operation (for example reading or writing), client operating system 220 generated the order that relates to client's physical address.In this example, I/O MMU 426 can use the table that the customer memory map addresses of child partition 248 is arrived the system address that is used by father's subregion.Adapter 402 can be configured to IO MMU 426 that the two becomes the system physical address from client's physical address translations with transmission and send buffer; And adapter 402 can send buffer zone internally with data then and copy inner send buffer to, or vice versa.

Previous embodiment has been set forth the various embodiment of system and/or process via example and/or application drawing.Up to now; Because such block diagram and/or example comprises one or more functions and/or operation, it will be understood to those of skill in the art that each function and/or the operation in such block diagram or the example can realize through hardware, software, firmware or its almost any combination widely individually and/or jointly.

Although the particular aspects of described herein theme is shown and described; But those skilled in the art based on the instruction of this paper will know can do not break away from theme described herein and Qi Geng wide in range aspect situation under make and change and revise; Therefore and accompanying claims will be included in all such changes and modification in their scope, because all such changes and being modified in the true spirit and scope of theme described herein.

Claims

1. system comprises:

Be used to realize the circuit of transportable stores service, wherein this transportable stores service is configured to manage the virtual hard disk I/O request to child partition, and wherein this transportable stores service is allocated for the only network identifier of network; And

Be used for will this transportable stores service being configured to the circuit of the storage target of network.

2. the system of claim 1 further comprises:

Be used for transportable stores service is moved to the circuit of remote computer system.

3. the system of claim 1 further comprises:

Be used to dispose the I/O MMU becomes the system physical address with client's physical address translations that will be associated with the I/O request to child partition circuit.

4. the system of claim 1 further comprises:

Be used for receiving from child partition the circuit of I/O work request, wherein this child partition is affixed to the virtual functions that comprises the second only network identifier that is used for network.

5. the system of claim 1 further comprises:

The circuit that be used for carrying out said transportable stores service, execution is configured to the management service of managing virtual machines in second subregion at first subregion, and wherein this child partition is the 3rd subregion.

6. the system of claim 1 further comprises:

Be used for transportable stores service is associated with the virtual functions of the network adapter that comprises the only network identifier and said child partition is appended to the circuit of second virtual functions of this network adapter.

7. the system of claim 1 further comprises:

Be used in response to receiving the I/O work request sending the circuit that notice and definite logic processor are being carried out said transportable stores service to logic processor from child partition.

8. the system of claim 1 further comprises:

Be used for confirming that when the I/O flow transmits via network adapter this I/O flow meets the circuit of security strategy between said only network identifier and at least one other only network identifier.

9. computer approach comprises:

The first only network identifier that will be used for network adapter appends to and is configured to manage the stores service to the virtual hard disk driving-disc I/O request of child partition; And

To be appended to said child partition by the virtual functions that network adapter realizes, wherein this virtual functions comprises the second only network identifier.

10. the computer approach of claim 9 further comprises:

Send configuration second virtual functions with the request that comprises first network identifier to the remote computer system that comprises second network adapter.

11. the computer approach of claim 9 further comprises:

Stores service is moved to second virtual functions that child partition and configuration distribute to this child partition to use the first only network identifier.

12. the computer approach of claim 9 further comprises:

Client's physical address translations that will be associated with the I/O request from child partition by the I/O MMU becomes the system physical address.

13. the computer approach of claim 9 further comprises:

The configuration network adapter is to monitor the security strategy compliance of this I/O flow when the I/O flow transmits between the said only network identifier and the second only network identifier.

14. the computer approach of claim 9 further comprises: send the instantiation stores service and received the I/O request amount that surpasses predetermined threshold and request from the first only network identifier to this stores service that distribute from remote computer system in response to confirming to remote computer system.

15. the computer approach of claim 9 further comprises:

The migration memory allocated is served supervisory routine and is disposed second virtual functions to use the first only network identifier.