CN101436165B

CN101436165B - System and method for management of an IOV adapter

Info

Publication number: CN101436165B
Application number: CN2008101763014A
Authority: CN
Inventors: A·C·布朗; D·M·弗赖穆斯; J·A·帕夫米; R·J·雷西奥; S·M·瑟伯
Original assignee: International Business Machines Corp
Current assignee: International Business Machines Corp
Priority date: 2007-11-15
Filing date: 2008-11-14
Publication date: 2012-06-27
Anticipated expiration: 2028-11-14
Also published as: JP2009123217A; CN101436165A; JP5305848B2; US8141092B2; US20090133028A1

Abstract

A system and method which provide a mechanism for an I/O virtualization management partition (IMP) to control the shared functionality of an I/O virtualization (IOV) enabled I/O adapter (IOA) through a physical function (PF) of the IOA while the virtual functions (VFs) are assigned to client partitions for normal I/O operations directly. A hypervisor provides device-independent facilities to the code running in the IMP and client partitions. The IMP may include device specific code without the hypervisor needing to sacrifice its size, robustness, and upgradeability. The hypervisor provides the virtual intermediary functionally for the sharing and control of the IOA's control functions.

Description

The system and method that is used for the management of IOV adapter

Technical field

The present invention relates generally to improved data handling system and method.More specifically; The present invention is intended to be used for through managing the system and method for I/O virtual (IOV) adapter in the virtual intermediary of system administration manager (hypervisor) and through the supervisory routine in I/O virtual (IOV) management zone, and said I/O virtual (IOV) adapter is such as periphery component interconnection (PCI) IOV adapter.

Background technology

Most of modern computing equipment use some versions that utilized at first the periphery component interconnection standard of being created in the nineties in 20th century by Inte1 or I/O (I/O) adapter and the bus of realization.Periphery component interconnection (PCI) standard code is used for peripherals is appended to the computer bus on the computer motherboard.PCI Express or PCIe are to use the realization of the PCI computer bus of existing PCI programming concept, but make computer bus based on complete difference and serial physical layer communication agreement faster.This Physical layer is not to be made up of the bidirectional bus that can between a plurality of equipment, share, but is made up of the single one way link of two equipment of lucky connection.

Fig. 1 shows the exemplary diagram that is combined with according to the topological system of PCI Express (PCIe) framework (fabric) of PCI Express (PCIe) standard.As shown in Figure 1; System 100 comprises the primary processor (CPU) 110 and storer 120 that is coupled to root complex (root complex) 130; Root complex 130 then is coupled to one or more PCIe end points 140 (in the PCIe standard, use a technical term " end points " refers to the I/O adapter of supporting PCIe), PCI Express-PCI bridge 150 and one or more interconnect switch 160.Root complex 130 expressions are connected to the CPU/ storer root of the I/O level of I/O adapter.Root complex 130 comprises main bridge (host bridge), zero or the integrated end points of more root complexs, zero or a plurality of root complex Event Collector and one or more root port.The I/O level that each root port support is independent.The I/O level can comprise root complex 130, zero or more interconnect switches 160 and/or bridge 150 (it comprises interchanger or PCIe framework) and such as one or more end points of end points 140,170 and 182-188.About the more information of PCI and PCIe, referring to obtainable PCI and PCIe standard on the website of the special interest group of the periphery component interconnection of www.pcisig.com (PCI-SIG).

Except PCI and PCIe standard, PCI-SIG has defined also that be used for how designing can be by I/O virtual (IOV) standard of the shared I/O adapter (IOA) of some logical partitions (LPAR).LPAR is divided into a plurality of resource groups with the processor of computing machine, storer and memory device, can each resource group be operated with using independently by the operation system example of himself like this.The quantity of the logical partition that can create depends on the processor model of system and available resource.Typically, subregion is used for various objectives such as the operation of database manipulation, client/server so that isolation test environment and production environment etc.Each subregion can be communicated by letter with other subregions, is positioned on the independent machine as other subregions.In the modern system of supporting a plurality of LPAR, can between LPAR, share some resources.As mentioned above, in PCI and PCIe standard, a kind of sharable this type of resource is to use the I/O adapter of I/O virtualization mechanism.

Be used to define the standard that how to design the IOA that can be shared by some LPAR though PCI-SIG provides, how this standard not definition is connected to main system with IOA.And, the function of sharing of the virtualized IOA of I/O that how this standard has not stipulated management and utilization.This is because the PCI-SIG standard paid close attention to is the standard of setting up the operation be used for the PCIe framework under root complex.In other words, PCI-SIG do not provide to be used for root complex with and on any definition of standard, this be because, with root complex with and go up the field of considering to do system chambers (system houses).That is, for example each of Intel platform, IBM Power

platform and Sparc platform can have different system unlisted in the PCI-SIG standard and realize demand.

Summary of the invention

Illustrative embodiment provides and has been used for using the function management at the IOV management zone, manages the system and method for I/O virtual (IOV) adapter through the virtual intermediary in system administration manager.The mechanism of illustrative embodiment attempts to solve such situation, and wherein by sharing I/O (I/O) framework more than one logical partition (LPAR), and wherein each LPAR can be potentially and other a plurality of LPAR share I/O adapters (IOA).Shown embodiment has defined a kind of mechanism; Be used at I/O virtual management subregion (IMP); When a plurality of virtual functions (VF) directly being distributed to the client partitions that is used for conventional I/O operation, make a LPAR control the sharing functionality of supporting the virtualized IOA of I/O through physical function (PF).

Especially; Utilize the mechanism of illustrative embodiment; Distribute the separate addresses space of himself to visit the virtual functions (VF) of distributing to it to each LPAR, and directly receive interruption, make the perception of each LPAR be from the VF that is assigned to it; For the purpose of the I/O of routine operation, it has the independently IOA of himself.Can between a plurality of LPAR, share each VF such as the IOA of PCI Express (PCIe) I/O virtual (IOV) adapter.Through PF being distributed to I/O virtual management subregion (IMP), via the physical function (PF) of IOV adapter, the mechanism of illustrative embodiment provides the facility of the shared resource of managing I/O V adapter.In addition, for the accessing common VF functionality purpose of (for example, for network adapter, being used for the network address of VF), IMP shows as the interface of PF, is referred to as the management of adapter southern side here.

The code of system administration manager in running on IMP provides the facility that is independent of IOV.IMP can comprise the code of device-specific, sacrifices its size, robustness and upgradability and need not system administration manager.System administration manager provides virtual mediating function, is used for sharing and controlling the control function of IOV adapter.More particularly, run in the system administration manager code for manage the VF fault processing fully, VF resets and the purpose of configuration operation and serve as virtual intermediary (VI) to a plurality of VF.

In an illustrative embodiment, a kind of virtualized method of I/O (I/O) that is used to manage is provided.This method can be included in and create one or more client partitions in the data handling system; Each client partitions has the relevant portion of the resource of data handling system, and the one or more I/O end points that are associated with data handling system are carried out the operation of I/O virtual management.The operation of I/O virtual management can be operated to dispose at least one physical function or the virtual functions of one or more I/O end points.This method can further comprise uses the virtual intermediary that is separated with I/O virtual management subregion (IMP) to control the visit of I/O virtual management operation to the resource of one or more I/O end points.Virtual intermediary can prevent that the client partitions visit is not assigned to the resource of one or more I/O end points of this client partitions.

This method further is included in and creates I/O virtual management subregion (IMP) in the data handling system.IMP can carry out the operation of I/O virtual management to the one or more I/O end points that are associated with data handling system, so as to managing at least one physical function of one or more I/O end points.Do not allow one or more client partitions to carry out the I/O virtual management operation at least one physical function.

Can carry out by virtual intermediary and create one or more client partitions and create IMP.This virtual intermediary can be and data handling system system associated manager.Can provide being used for visit to system administration manager at the configuration space of a plurality of physical functions of one or more I/O end points and a plurality of virtual functions.Can only provide being used for configuration space to IMP at a plurality of physical functions that are assigned to IMP of one or more I/O end points, and the visit of configuration space that belongs to a plurality of virtual functions of a plurality of physical functions that are assigned to IMP.System administration manager can allow to visit the configuration space that in one or more I/O end points, is assigned to a plurality of virtual functions of client partitions by client partitions, and can stop the configuration space by a plurality of virtual functions that are assigned to client partitions of client partitions visit in one or more I/O end points.

This method comprises that further said HMC (HMC) guides the execution of I/O virtual management operation from HMC (HMC) reception input.Input from HMC can be the order of initiating to be used for new I/O end points is added to the hot-inserting type operation of data handling system.In the case; Carry out the operation of I/O virtual management and can comprise interpolation based on new I/O end points; And to one or more client partitions assigned resources; Dispose IMP based on the interpolation of new I/O end points, and dynamically at least one client partitions in one or more client partitions, add at least one virtual functions of new I/O end points.

From the input of HMC can be the order of initiating to be used for hot-inserting type operation that existing I/O end points is removed from data handling system.In the case; Carry out the operation of I/O virtual management and can comprise that sign has at least one client partitions in one or more client partitions of the resource that is associated with virtual functions with the I/O end points that is removed, and the resource that from the use by at least one client partitions, removes virtual functions and be associated with virtual functions.Carry out the operation of I/O virtual management and can further comprise the resource that from use, removes physical function and be associated, and make the I/O end points outage (power down) that is removed with physical function by IMP.

Input from HMC can be the order of disposal system that dynamically resets Configuration Data.In the case; Carry out the operation of I/O virtual management and can comprise the selection of reception the virtual functions of the I/O end points that will add the selected client partitions in one or more client partitions to; Be provided for the resource of the selected virtual functions in selected client partitions, and loading is used for the device drives at the selected virtual functions of selected client partitions.

Input from HMC can be the order of disposal system that dynamically resets Configuration Data.In the case; Carry out the operation of I/O virtual management and can comprise the selection of reception the virtual functions of the I/O end points that will from the client partitions of one or more client partitions, selecting, remove; From further use, remove selected virtual functions, and remove the resource that is used at the selected virtual functions of selected client partitions by selected client partitions.

Input from HMC can be the order that changes the network address that is associated with the selected virtual functions of I/O end points.In the case, carry out the operation of I/O virtual management and can comprise the physical function that sign is associated with selected virtual functions, and send management request to IMP.Management request can comprise the address information of the network address that is used to change selected virtual functions.Carrying out the operation of I/O virtual management can further comprise, is based on the address information in the management request, via driving with the physical function associated device, changes the network address of selected virtual functions.

The inquiry virtual functions error condition firmware that the operation of execution I/O virtual management can be included in the device drives in the client partitions that receives comfortable one or more client partitions in the virtual intermediary calls, and from virtual intermediary the error message of the error condition of the virtual functions that has identified the I/O end points is provided to device drives.And; Carrying out the operation of I/O virtual management can comprise: by the error message of device drives registration from the reception of virtual intermediary; Firmware to data handling system sends the virtual functions request that resets from device drives; The virtual functions that has error condition to reset, and after resetting, the configuration virtual function.

Carrying out the operation of I/O virtual management can comprise, detects client partitions by said virtual intermediary and loses efficacy; Determine whether to carry out client partitions pilot operationp again; If do not guide client partitions again, then remove the virtual functions resource that is associated with the client partitions that lost efficacy, and add the virtual functions resource that is removed to free virtual functions resource pool from one or more I/O end points.And carrying out the operation of I/O virtual management can comprise, if will guide client partitions again, then after the client partitions that lost efficacy guides again, redistributes the virtual functions resource by virtual intermediary to the client partitions that lost efficacy.

Further, carry out the operation of I/O virtual management and can comprise the inefficacy that detects IMP by virtual intermediary, and initiate guiding again IMP.Carrying out the operation of I/O virtual management can further comprise; Time-out is called by the firmware of making about one or more client partitions of a plurality of virtual functions of being associated with a plurality of physical functions that are assigned to IMP; Guiding again up to IMP is accomplished; And after the guiding of accomplishing IMP again, stop the time-out that the firmware of being made by one or more client partitions is called.

In another illustrative embodiment, a kind of computer program that comprises computer-readable medium is provided, it has record computer-readable program above that.When on computing equipment, carrying out, computer-readable program causes that computing equipment carries out multiple operation and combination thereof that preceding text are summarized about the illustrative embodiment of method.

In another illustrative embodiment, a kind of device is provided.This device can comprise processor and the storer that is coupled to processor.Storer can comprise instruction, and when being carried out by processor, it causes multiple operation and combination thereof that processor execution preceding text are summarized about the illustrative embodiment of method.

In the detailed description hereinafter to exemplary embodiment of the present invention; These and other feature and advantage of the present invention have been described; Perhaps consider the detailed description to exemplary embodiment of the present invention hereinafter, these and other feature and advantage of the present invention will become to those one skilled in the art and show and suggestion.

Description of drawings

When together with advantages, through with reference to the detailed description of hereinafter, will understand the present invention better and preferably use pattern and its further purpose and advantage illustrative embodiment, wherein accompanying drawing comprises:

Fig. 1 shows the exemplary plot of the system that is associated with the PCIe fabric topology known in affiliated field;

Fig. 2 shows the exemplary plot of system virtualizationization known in affiliated field;

Fig. 3 shows the exemplary plot of virtualized first mode of I/O that is used to use virtualization layer to make the PCI root complex;

Fig. 4 shows and is used to use the local exemplary plot that the PCII/O adapter makes virtualized second mode of I/O of PCI root complex of sharing;

Fig. 5 is an exemplary plot of supporting the virtual end points of PCIeI/O;

Fig. 6 is the exemplary plot of system architecture, has wherein utilized end points or the adapter of supporting IOV according to an illustrative embodiment;

Fig. 7 shows the exemplary plot of the definition of platform firmware being called according to one group of example logic subregion (LPAR) of an illustrative embodiment;

Fig. 8 shows according to one group of example hardware supervisor console (HMC) of illustrative embodiment exemplary plot to the definition of the request of platform and response;

Fig. 9 is the process flow diagram of having summarized according to an illustrative embodiment of exemplary " start (bring up) " operation of system that is used to have I/O virtual (IOV) adapter;

Figure 10 A-10B has summarized the process flow diagram that is used for the IOV adapter is added to the exemplary hot-inserting type operation of system according to an exemplary embodiment;

Figure 11 is a process flow diagram of having summarized the exemplary hot-inserting type operation that is used for the IOV adapter is removed from system according to an illustrative embodiment;

Figure 12 has summarized the process flow diagram that is used for dynamically virtual functions (VF) being added to the exemplary operation of logical partition (LPAR) according to an illustrative embodiment;

Figure 13 is a process flow diagram of having summarized the exemplary operation that is used for dynamically VF is removed from LPAR according to an illustrative embodiment;

Figure 14 has summarized according to an illustrative embodiment to be used for from carried out the process flow diagram of the wrong exemplary operation that recovers by the mistake that VF caused;

Figure 15 is a process flow diagram of having summarized the exemplary IOA southern side bookkeeping of the network address that is used to change on network adapter according to an illustrative embodiment;

Figure 16 is a process flow diagram of having summarized the exemplary operation that is used for the client partitions termination according to an illustrative embodiment; And

Figure 17 is a process flow diagram of having summarized the exemplary operation that is used for the termination of IMP subregion according to an illustrative embodiment.

Embodiment

Illustrative embodiment provides and has been used for through manage the mechanism of I/O virtual (IOV) adapter in the virtual intermediary of system administration manager and through the supervisory routine in the IOV management zone.Thereby, in order to understand the mechanism of illustrative embodiment, importantly, at first understand in the system that has utilized system administration manager or other virtual platforms how to realize that I/O is virtual.Should recognize, though about periphery component interconnection express delivery (PCIe, Peripheral Component Interconnet Express) adapter or end points and will describe illustrative embodiment,, the invention is not restricted to this.But, can the mechanism of illustrative embodiment be implemented within the virtualized any I/O framework of the I/O that is supported in the I/O adapter.And, should recognize, though according to the realization that has utilized system administration manager illustrative embodiment has been described,, the invention is not restricted to this.On the contrary, can use the virtual platform of the other types except that system administration manager, realize that this does not break away from the spirit and scope of the present invention and no matter it is current any combination with software, hardware or software and hardware known or exploitation after a while.

Fig. 2 shows the exemplary plot of system virtualizationization known in affiliated field.System virtualizationization is the division of processor, storer, I/O adapter, memory device and other resources to physical system, wherein himself system image instance and application and independent operation of every group of utilization of resources.In this system virtualizationization; Virtual resource is made up of physical resource; And as the agency of physical resource, physical resource is storer, disc apparatus and other nextport hardware component NextPorts with interface/function of frameworkization for example, and virtual resource has same external interface and function.System virtualizationization is typically utilized virtualization layer, and this virtualization layer is created virtual resource and it is mapped to physical resource, so as to the isolation between the virtual resource is provided.Typically virtualization layer is provided as one of software, firmware and hardware mechanisms, perhaps the combination of software, firmware and hardware mechanisms.

As shown in Figure 2, typically in virtualization system, application 2 10 is communicated by letter with system image (SI) 220, and system image 220 is the component softwares such as general or special purpose operating system, and utilizes it to distribute specific virtual and physical resource.System image 220 is associated with virtual system 230, and it is included as operation necessary physics of single SI instance or virtual resource, for example, and virtualization processor, storer, I/O adapter, memory device etc.

Through using virtual system 230, system image 220 is by means of virtualization layer 240 visit physical system resources 250.Virtualization layer 240 management is to the resources allocation of SI, and the resource that will distribute to SI is isolated with the visit from other SI.Usually carry out this distribution and isolation based on the mapping resources of carrying out by virtualization layer 240, and by one or more mapping resources data structures that virtualization layer 240 is safeguarded.

Can use this type of virtually to allow virtual to I/O operation and I/O resource.That is,, can virtualization layer 240 partially or even wholly be realized with system administration manager by the I/O unit that surpasses one the shared single physical of SI that uses virtualization layer 240 about I/O virtual (IOV).System administration manager can be that software, firmware or its are similar, uses it to come through interfering for example from configuration, I/O and the storage operation of SI and the one or more IOV of support that go to direct memory visit (DMA), completion and the interrupt operation of SI.

Fig. 3 shows the exemplary plot of virtualized first mode of I/O that is used to use virtualization layer to make the PCI root complex.As shown in Figure 3; It can be the primary processor group 310 of chip, mainboard, blade (blade) or its similar one or more processor; It can support a plurality of system image 320-330, uses (not shown) through the addressable system resource such as PCIe end points 370-390 of said a plurality of system image 320-330.System image is via virtualization layer 340, PCIe root complex 350 and one or more PCIe switch 360 and/or other PCIe architectural elements and communicate by letter with virtualized resource.

Utilize the mode shown in Fig. 3, the virtualization layer 340 that can partially or fully be embodied as the virtual platform of system administration manager or other types can relate to all I/O affairs, and carries out all I/O virtualization.For example, virtualization layer 340 will be multiplexed on the single queue in PCIe end points 370-390 from the I/O request of the I/O queue, IOQ of multiple SI.Thereby virtualization layer 340 serves as the agency between SI320-330 and physics PCIe end points 370-390.

Fig. 4 shows and is used to use the local exemplary plot that the PCII/O adapter makes virtualized second mode of I/O of PCI root complex of sharing.As shown in Figure 4; It can be the perhaps primary processor group 410 of similar one or more processors of chip, mainboard, blade; It can support a plurality of system image 420-430, uses (not shown) through the addressable system resource such as PCIeI/O virtual (IOV) end points 470-490 of a plurality of system image 420-430.System image 420-430 is via PCIe root complex 440 and one or more PCIe switch 460 and/or other PCIe architectural elements and communicate by letter with virtualized resource.

PCIe root complex 440 comprises the virtual enablers (RCVE of virtualized root complex that has promoted with the I/O operation of the end points 470-490 that supports IOV; Root complexvirtualization enabler) 442, it can comprise one or more address translation and protection list data structure, interrupt table data structure and similar.Can use address translation and protection list data structure by PCIe root complex 440; Carry out the address translation between virtual address and actual address, be used for for example virtualized resource, arrive access control and other virtualization operations of the mapping of a plurality of SI virtual resource based on virtualized resource.For example, these root complex interrupt table data structures are addressable through the PCIe memory address space, and for example are used for interrupt map to the appropriate interrupt handling routine that is associated with a plurality of SI.

As in the mode shown in Fig. 3, also in the virtual structure of Fig. 4, virtualization layer 450 is provided.Use virtualization layer 450 together with the PCIe end points of not supporting IOV that can be coupled to PCIe switch 460.Promptly; Because these PCIe end points do not have to this locality of I/O virtual (IOV) (promptly; Inner at end points) support; With the similar fashion of describing about Fig. 3 with previous preceding text, utilize the virtualization layer 450 that can partially or even wholly be embodied as system administration manager or other virtual platforms by the PCIe end points.

For the PCIe end points 470-490 that supports IOV; The main purpose of using virtualization layer 450 to be used for configuration transaction; And virtualization layer 450 does not relate to the memory address space operation; Such as memory mapped I/O (MMIO) operation of initiating from SI, direct memory visit (DMA) operation of perhaps initiating from PCIe end points 470-490.On the contrary, between SI420-430 and end points 470-490, directly carry out data transmission, and need not the interference of virtualization layer 450.By means of a plurality of RCVE442 and support IOV PCIe end points 470-490 in the for example physics and the I/O virtualizes logical of virtual functions built, making direct I/O between SI420-430 and end points 470-490 operate becomes possibility.The ability of carrying out direct I/O operation has increased the speed of carrying out the I/O operation significantly, but it needs PCIe end points 470-490 to support that I/O is virtual.

Fig. 5 is an exemplary plot of supporting PCIe I/O virtual (IOV) end points.As shown in Figure 5, PCIe IOV end points 500 comprises PCIe port 510, through PCIe port 510, can carry out with the PCIe interchanger of PCIe framework and similarly communicates by letter.520 pairs of configuration management functions of internal router 530 provide communication path with a plurality of virtual functions (VF) 540-560.Configuration management function 530 can be the physical function (PF) relative with virtual functions 540-560.As the term that in the PCI standard, uses, physics " function (function) " is the logical groups by single configuration space representative.In other words, physics " function " is circuit logic, and they can be based on the data of the configuration space that is associated that is stored in the function in the storer and are configured, and make it for example can in inseparable resource 570, provide.

Can use preparation management function 530 to come configuration virtual function 540-560.In supporting the virtualized end points of I/O, virtual functions is a function of sharing the physical endpoint resource of one or more for example links, and can in the shareable resource pond 580 of for example PCIe IOV end points 500, with another function this function be provided.Intervene when need not the operation from system administration manager; Virtual functions directly conduct is used for from the I/O of system image and the receiver of storage operation, and as direct memory visit (DMA), to the completion of system image (SI) and the source of interrupt operation.

About " function " supported by the PCIe end points, the PCIe end points can have many dissimilar configurations.For example, end points can be supported single physical function (PF), a plurality of independently PF, or even a plurality of PF that relies on mutually.In supporting the virtualized end points of local I/O, each PF that end points is supported can be associated with one or more virtual functions (VF), himself can be dependent on a plurality of VF that are associated with other PF.

Illustrative embodiment provides and has been used for using the function management at the IOV management zone, manages the system and method for I/O virtual (IOV) adapter of the PCIe end points 500 of all support IOV as shown in fig. 5 through the virtual intermediary in this system administration manager.The mechanism of illustrative embodiment is absorbed in following situation: by sharing I/O (I/O) framework can comprise one or more PCIe switches such as the PCIe switch 460 among Fig. 4 more than a logical partition (LPAR), and each LPAR can be potentially shares the I/O adapter (IOA) of the PCIe end points 470-490 among Fig. 4 for example with a plurality of other LPAR.Illustrative embodiment has defined a kind of mechanism; Be used for operating for common direct I/O; When distributing to a plurality of client partitions, make a LPAR (I/O virtual management subregion (IMP)) control the sharing functionality of supporting the virtualized IOA of I/O through physical function (PF) a plurality of virtual functions (VF).

Especially; Utilize the mechanism of illustrative embodiment; Distribute the separate addresses space of its institute self to visit the virtual functions (VF) of distributing to it to each LPAR, and receive direct interruption, make the perception of each LPAR be from the VF that is assigned to it; For the I/O operation of routine, it has the independently IOA of himself.Along with each LPAR obtains himself VF, can between a plurality of LPAR, be shared such as the VF group of the IOA of virtual (IOV) adapter of PCI Express (PCIe) I/O or I/O end points.Through PF being distributed to I/O virtual management subregion (IMP), via the physical function (PF) of IOV adapter/end points, the mechanism of illustrative embodiment is that the shared resource of managing I/O V adapter or I/O end points is provided convenience.In addition, for the purpose of accessing common VF functionality, IMP is referred to as the management of adapter southern side here as the interface (for example, for network adapter, being used for the network address of VF) to PF.

The code of system administration manager in running on IMP provides the facility that is independent of IOV.IMP can comprise the code of device-specific, sacrifices its size, robustness and upgradability and need not system administration manager.System administration manager provides virtual mediating function, is used for sharing and controlling the control function of IOV adapter.More particularly, run in the system administration manager code for manage the VF fault processing fully, VF resets and the purpose of configuration operation and show as virtual intermediary (VI) to a plurality of VF.

Fig. 6 is the exemplary plot of system architecture, has wherein utilized end points or the adapter of supporting IOV according to an illustrative embodiment.Can be combined in the mechanism shown in Fig. 4 and be implemented in the mechanism shown in Fig. 6.For example, capable of using and system image 1420 in Fig. 4 is provided at the IMP603 shown in Fig. 6 explicitly, and simultaneously, capable of using and system image 2430 in Fig. 4 is provided at the client partitions 604 among Fig. 6 explicitly.Similarly; The I/O framework 611 of Fig. 6 can be included in the PCIe switch 460 among Fig. 4; IOV end points 614 can be similar to any one of PCIe end points 470-490 in Fig. 4; And

end points

615 and 616 can or support the end points of IOV or do not support the end points of IOV, such as the end points 370-390 in Fig. 3.

As shown in Figure 6; System 601 comprises main system 626, I/O framework 611 and one or more I/O end points 614-616; Main system 626 can be for example server, client computing device or its similar any data processing equipment; I/O framework 611 can comprise one or more communication links and one or more switch; I/O end points 614-616 can be the PCIeI/O end points that has as the I/O end points 614 of the end points of supporting IOV in an illustrative embodiment, and other end points 615-616 can be the end points of supporting IOV or not supporting IOV.Main system 626 comprises platform hardware 610, system administration manager 625, the logical partition (LPAR) 603 and 604 as the bottom hardware of data processing equipment, and corresponding partition firmware (PFW) 623 and 624.Though described illustrative embodiment about using system manager 625 here, should recognize the virtual platform of other types capable of using and do not break away from the spirit and scope of the present invention.

In an illustrative embodiment, system administration manager 625 can be the code of on platform hardware 610, carrying out, and is the part of platform firmware.Similarly, partition firmware (PFW) 623-624 also can be the part of platform firmware, but owing to logically it is thought of as the part of the code of the LPAR that in LPAR, carries out, thereby is shown as with LPAR603 and 604 and is associated.

LPAR

603 and 604 has assigned operating system mirror image or instance and the resource of in LPAR, carrying out.In addition,

LPAR

603 and 604 can carry out other application, software, code or it is similar in LPAR.For example, about the particular importance of illustrative embodiment, for example one of a plurality of LPAR of LPAR603 carry out the code that causes that LPAR 603 is used as IOV management zone (IMP) 603 and operate.Other a plurality of LPAR 604 can be used as client partitions and operate.Though in Fig. 6, only show an IMP 603 and a client partitions 604, should recognize, in main system 626, can provide more than an IMP 603 and client partitions 604, and the spirit and the scope that do not break away from illustrative embodiment.

System administration manager 625 can be visited the configuration space 619,621 of IOV end points 614, and the configuration space 617 of visit I/O framework 611.Employed here term " configuration space " refers to the disjoint address space from memory mapped I/O (MMIO) address space, assigns the configuration data that this memory mapped I/O (MMIO) address space is used for being used in storage the specific components of system 601 and uses.Further; When a plurality of physical functions (PF) 612 are distributed to IMP 603; Operating system 631 and the device drives 605 of IMP can be visited the configuration space 619 that is used for a plurality of physical functions (PF) 612, and the configuration space 621 that can visit a plurality of virtual functions (VF) 613 that belong to the PF that is assigned to IMP 603.Any configuration space 621 that client partitions operating system 632 and device drives 606 can directly be visited a plurality of VF 613 that are assigned to them.System administration manager 625 serves as intermediary; Perhaps more specifically; Serve as virtual intermediary (VI) and guarantee that a plurality of client partitions can visit to they correctly operate its a plurality of VF 613 needed configuration spaces, but can not visit can cause the configuration space of the problem of relevant other client partitions.That is, client partitions 604 is only visited the configuration space of being assigned for its a plurality of VF 613.Client partitions 604 further can be visited memory mapped I/O (MMIO) space of being distributed for its a plurality of VF 613 622.

Management application 640 on HMC (HMC) 609 and HMC 609 self; Pass through to telemanagement order (RMC) interface 602 of IMP 603 and client partitions 604; And the interface 620 that passes through to the same type of system administration manager 625 communicates, HMC (HMC) 609 can reside on the main system 626 or (as directed) in independent data processing equipment (not shown).Management application 640 (hereinafter; It is referred to HMC609 together with HMC 609 in the lump simply) serve as cammander (orchestrator) and be controlled at the functional of multiple assembly in the system 601; And provide user interface 642 to check system configuration to people, and import relevant they hope which kind of resources allocation is given the information of which LPAR 603-604.Management application 640 can provide and can describe the many difference in functionalitys that call in more detail according to hereinafter by the user.Alternatively, can call these functions automatically in response to the incident or the input that have triggered the initiation of this type of function, and need not user intervention.

As stated, client partitions 604 can visit the configuration space 621 of a plurality of virtual functions (VF) 613 in the end points 614 of supporting IOV directly with 614 communications of the end points of supporting IOV.For this type of communication; System administration manager 625 serves as virtual intermediary (VI); And prevent that client partitions from influencing on the configuration space of other client partitions (for example, on the configuration space of a plurality of virtual functions (VF) that are not assigned to this client partitions) and carrying out configuration operation in meeting.

Fig. 7 shows the exemplary plot of calling the definition of (call) according to one group of example logic subregion (LPAR) of an illustrative embodiment to platform firmware.Especially, the exemplary platform firmware that Fig. 7 shows the many aspects that can be used for realizing illustrative embodiment calls 700, and shows these calling application in which subregion (client or IMP).For example, can carry out these calling by

subregion

603 or 604 to the platform firmware that is associated 623 in Fig. 6 or 624.Hereinafter is in a plurality of description to Fig. 9-17, with providing the example that platform firmware is called 700 use with regard to these LPAR.Make the description that these are called with continuing with reference to the unit among the figure 6, the unit among Fig. 6 is as carrying out the example that calls the unit of the operation that is associated with these.

First calls 702, and promptly read/write I/O configuration space calls 702 for 604 uses of client partitions, visits the configuration space 621 of its a plurality of VF 613 that are associated.System administration manager 625 serves as and is used for the VI that these call, and prevents that client partitions 604 execution from can influence the configuration operation of other client partitions.Also use this to call and utilize its a plurality of PF 619 of system administration manager 625 configuration once more by IMP 603, system administration manager 625 serves as VI and prevents that IMP 603 visits from not being a plurality of PF and the configuration space of VF of part in the territory of its control.In addition, as the VI in the IOV environment, system administration manager 625 also need be carried out by the specified action of PCI SIGIOV standard.For example, in VF, do not duplicate some configuration fields, and thereby, when visiting the configuration space 619 of PF through the configuration space 621 of VF, the VI that is provided by system administration manager 625 must obtain these fields from the configuration space 619 of PF.

It is one of to be called by a plurality of PF special uses that IMP 603 makes that the PF that resets calls 704.This calls and for example makes that the PF of PF 621 resets, and the result, and all VF of the for example VF 613 that is associated with this PF are resetted.When making this and call, IMP 603 must at first use forbidding VF to call 706 (describing hereinafter), forbids all VF that are associated with this PF, and this PF is resetted, and then, all VF configurations is stored as their appearance before PF resets again.

Forbidding VF calls 706 notification platform firmwares forbids any further MMIO operation to specific (a plurality of) VF, and prevents to arrive client partitions 604 from direct memory visit (DMA) and the interruption of (a plurality of) VF.There are a large amount of modes to realize this point, to guarantee data integrity issues not to take place.In an illustrative embodiment, design hardware like this, if make client partitions 604 attempt carrying out the MMIO load operation from its VF, then client partitions 604 can receive complete 1 the data of being returned, and does not exist as VF.When the data of not expecting complete 1, expectation device drives 606 is checked complete 1 data, and expectation device drives 606 uses the inquiry PF/VF error condition firmware that literary composition is described in the back to call 710, verifies whether complete 1 represent error condition.As above-described, and hereinafter further describes, and expectation mainly uses forbidding VF to call 706 in wrong restoration scenario, and thereby, forbidding VF calls 706 can cause that for example the VF of device drives 606 drives, get into the recovery of himself.

Client partitions 604 through use reset VF call 708 can reset them a plurality of VF613.System administration manager 625 serve as VI and send to VF 613 reset before the checking VF 613 that VF calls appointment in 708 that resetting be assigned to subregion.In this way, system administration manager 625 prevents reset (a plurality of) VF of another client partitions of a client partitions.As be described further below; Owing in these batch operations, relate to system administration manager; Thereby system administration manager follows the tracks of which subregion which VF is distributed to, and system administration manager can for example be followed the tracks of about being used for the information such as the verification operation of these operations through this information of storage in one of a plurality of system administration manager data structures.

It is that the PF that allows the for example device drives 605 in IMP 603 is driven that inquiry PF and VF error condition call 710; Perhaps the VF of the for example device drives 606 in client partitions 604 drives, and obtains being respectively applied for the calling platform firmware of the wrong status information of its PF 612 and VF 613.From the hardware status register platform hardware 610 for example, obtain this wrong status information by system administration manager.Can error condition be set through the for example appearance of the mistake in the operation of I/O end points 614, and can drive by the PF of the for example device drives 605 in IMP603, or the VF of the for example device drives 606 in client partitions 604 drive and use this error condition to allow the recovery from mistake.Call this wrong status information of removing from hardware through for example the reset PF704 or the VF706 that resets.

For obtain to be used for the platform errors data that mistake is recovered, provide to be used to load with storage operation call 712 with inquiry I/O framework error message call 714 PF that enables and VF.Call 710 when inquiry PF and VF error condition and return wrong status information; It points out that to appropriate device drives 605 or 606 PF612 or VF 613 that it is associated are in error condition; Device drives 605,606 can be used and be used to load and storage operation is called 712 PF that enables and VF; Be opened to the path of PF 612 or VF 613, and need not allow the DMA path to open.Then, the purpose that device drives 605,606 can use MMIO to load and storage operation confirms for problem and obtain status informations from I/O end points 614.Except load from MMIO with storage operation obtain the error message of device-specific, can also use the error message of inquiry I/O framework to call 714 error messages that obtain to the definite non-device-specific of problem.

Interrupt using inquiry interrupt source to call 716 to all and obtain interrupt source, so that virtual interruptable controller.Thereby, as shown in Fig. 7, can the two use inquiry interrupt source call 716 by client partitions 604 and IMP603.Use is set up DMA conversion and is called 718 and set up and be used for converting I/O address into system address change the mechanism.This address change the mechanism be for I/O virtual needed.Call for these, system administration manager 625 serves as VI, and prevents that client partitions 604 foundation from will influence the address translation of other client partitions.System administration manager 625 is responsible for to multiple subregion Resources allocation, and for example, storer, a plurality of processor and I/O resource, and afterwards remain on this information and be used for the for example authority of the subregion of this resource of particular memory address of authentication-access in its data structure.As shown in Figure 7, because PF need not carry out dma operations to IMP 603 usually, thereby can generally not that IMP603 uses and sets up the DMA conversion and call 718 by client partitions 604.

That finally, from the use of LPAR, adds/remove VF calls 720 and from the use of LPAR, add/remove calling the 722 IOA hot-inserting types that are used to will be described below and dynamically reconfiguring operation of PF.Through definition, owing to a plurality of VF are not distributed to IMP603, and the heat that the owner of VF controls VF inserts operation, thus by client partitions 604 but not IMP 603 use from the use of LPAR, add/remove VF call 720.Can be by IMP 603 but not client partitions 604 use from the use of LPAR, add/remove PF call 722.Do not allow client partitions 604 to come to change by any way PF; Thereby; Can't use from the use of LPAR add/remove PF call 722, reason is, allows client partitions 604 to do the safety problem that can cause a plurality of VF that are associated about other client partitions that are associated with PF like this.

Fig. 8 shows the exemplary plot that arrives the definition of platform request and response according to one group of example hardware supervisor console (HMC) of an illustrative embodiment.Especially, Fig. 8 shows exemplary HMC to platform firmware request 800, and has identified expectation and make the response that this request and assembly are expected to which assembly in system.For example, can make these and call via the one or more of

interface

602 and 620 through the HMC 609 in Fig. 6.For example, be calling of IMP603 or client partitions 604 if HMC 609 makes target, HMC 609 can make via RMC interface 602 and calling so.Be the calling of the system administration manager 625 of system administration manager for example if HMC 609 is just making target, so, can make via interface 620 and calling.Below, these HMC will be provided the more detailed example to the use of platform request/response 800 about a plurality of of Fig. 9-17.

As shown in Figure 8, through using inquiry I/O architecture configuration request 802, can be with regard to the I/O architecture configuration and queries system manager 625.For example, when initial system boot (bring up), can use this request 802 obtain all available I/O equipment and functions thereof, can it be presented the distribution of the multiple LPAR that is used to select and the user is hoped dispose like this to the user by HMC 609.

Set up initial LPAR I/O configuring request 804 by HMC 609 uses, come reporting system manager 625 what to determine to dispose the user of HMC 609.This configuration not only comprises the I/O configuration, and comprises the LPAR configuration feature, such as the quantity of the CPU that is assigned to the multiple LPAR in system and the quantity of storer.

Also can when power up (power up) to I/O adapter or end points and when cut off the power supply by HMC 609 reporting system managers 625.Can realize this point through powering up/cut off the power supply IOA request 806.

The request 808-812 of some HMC to platform arranged, use it to indicate dynamic interpolation to the PF or the VF of system (that is, when system can move, and after initial configuration) by HMC.By the dynamic interpolation request 808 that HMC 609 uses PF/VF, come reporting system manager 625 to expect to add specific resources to a LPAR or a plurality of LPAR.Before intention is added this type of resource, need reporting system manager 625, make that it can be to these LPAR assigned resources.In this way, when LPAR Request System manager 625 came to its LPAR interpolation VF or PF, system administration manager 625 had been ready to assign these resources.

In case system administration manager 625 has been accomplished the request 808 of dynamic interpolation PF/VF, can be with regard to any interpolation notice IMP 603 and client partitions 604.The request 810 of dynamically adding PF is notified IMP 603 with regard to the new PF that is added can be used.Discuss as hereinafter, when for example adding I/O adapter or end points hot-inserting type ground to system, this possibly take place.This request 810 notice IMP 603 add PF to its architecture configuration, initialization PF configuration space, and loading equipemtn drives, and the IOV ability that enables PF.IMP 603 through 722 " adding " ability called of from the use of LPAR, adding/removing PF in Fig. 7 to the completion of system administration manager 625 signalings operation.

The new VF that dynamic request 812 notice client partitions 604 of adding VF will be added can use.This for example possibly occur in when adding I/O adapter or end points hot-inserting type ground to system, perhaps hopes when client partitions 604 is added the additional ability of already present I/O adapter with untapped a plurality of VF or end points as the user.These request 812 notice client partitions 604 are added VF its architecture configuration, initialization VF configuration space, loading equipemtn driving to and are brought into use VF.Client partitions 604 through 720 " adding " ability called of from the use of LPAR, adding/removing VF to the completion of system administration manager 625 signalings operation.

The request 814-818 of some HMC to platform arranged, use it to indicate by HMC 609 and from system, dynamically remove PF or VF (that is, when system can move, and after initial configuration).Use the request 814 that dynamically removes PF/VF by HMC 609, come reporting system manager 625 to expect to remove specific resources from a LPAR or a plurality of LPAR.The system administration manager 625 of need giving advance notice makes after being removed by (a plurality of) LPAR signaling, and it can be cancelled and assigns the resource that is used for these LPAR.In case system administration manager 625 has been accomplished the request 814 that dynamically removes PF/VF, can be with regard to any notice IMP 603 and client partitions 604 of removing.

Then, use the request 816 that dynamically removes VF, notify client partitions 604 from the configuration of use and LPAR, to remove VF by HMC 609.For example when from system's hot-inserting type when removing I/O adapter or end points; Perhaps for example hope when client partitions 604 removes the ability of the I/O adapter that existed or end points as the user; This possibly take place, and makes can it be distributed to the use that another LPAR is used for this LPAR once more.This request notice client partitions 604 is stopped using VF, and request resets to VF's, removes device drives, and from the architecture configuration of OS, removes VF.Client partitions 604 through 720 " removing " ability called of from the use of LPAR, adding/removing VF to the completion of system administration manager 625 signalings operation.

The request that dynamically removes 818 notice IMP 603:PF to PF will be removed.As stated, before sending this request, HMC 609 must remove all VF that are associated with PF from the client partitions 604 using.For example when hot-inserting type from system when removing I/O adapter or end points, possibly take place removing of PF.This request 818 notice IMP 603 stop using PF, and 704 requests of calling through resetting remove device drives, and from the architecture configuration of OS, remove PF the resetting of PF.IMP 603 through 722 " removing " ability called of from the use of LPAR, adding/removing PF to the completion of system administration manager 625 signalings operation.

The critical function of IMP 603 provides the secure access to the resource of I/O adapter or end points; The resource of this I/O adapter or end points or shared between a plurality of LPAR that just use a plurality of VF that are associated with PF; Perhaps the resource of this I/O adapter or end points is unique for VF; But to the change of being undertaken by PF, VF should be an inaccessible.The network address that an example of this type of resource of network I/O adapter or end points is and is associated by the PF that a plurality of VF share, the perhaps unique network address for VF.To be called " southern side " management here to the management of these device-specific resources to I/O adapter or end points.Realize this point by I/O adapter (IOA) or end points through HMC 609, ensure southern side management request 820 and all these type of management through the Secure Application on HMC609 (for example, login is used or other authentication application).When IMP 603 received the IOA southern side management request 820 that is used for IOA, IMP 603 transmitted the request that is used for PF to device drives 605, and device drives 605 makes the equipment particular request through the PF interface to IOA, so that realize this request.

Through using the request 822 of previous order completion of inquiry and state, come to make the current state of all HMC to platform request 800 to the target of raw requests.This allows HMC 609 to put a series of HMC request in order, and points out the state of previous request to the user through the HMC interface.Because it is quite slow accomplishing certain operations, and hope to visit the current state of this type of request, thereby can make this type of request.

Now, current description turns to the more detailed explanation of preceding text in the multiple request shown in Fig. 7-8 and the use of calling.Fig. 9-the 17th has summarized the process flow diagram of the multiple operation of illustrative embodiment mechanism.For example, the operation that can be implemented in general introduction among Fig. 9-17 by for example HMC, system administration manager, IMP, client partitions, platform firmware/hardware, I/O port and similar number of mechanisms thereof has been described in Fig. 6.Thereby as an example that participates in the mechanism of the illustrative embodiment of the execution of the operation of general introduction among Fig. 9-17, the unit to Fig. 6 in the time of suitable carries out reference.

Should be appreciated that can be by the combination of illustrated each square of computer program instructions realization flow and a plurality of squares in the flow process diagram.Can provide these computer program instructions to produce a kind of machine to processor or other programmable data processing device, make the instruction on processor or other programmable data processing device, carried out create to be used for the device that is implemented in the function that one or more flow process square describes.Also can on computer-readable memory or storage medium, store these computer program instructions; It can guide processor or other programmable data processing device to come with ad hoc fashion work, makes the instruction that is stored in computer-readable memory or the storage medium produce the goods that comprise the command device that is implemented in the function of describing in one or more flow process square.

Thereby, the illustrated a plurality of square supports of flow process be used to carry out the device of specific function combination, be used to carry out combination and the program instruction means that is used to carry out specific function of the step of specific function.Should be appreciated that also that through carrying out the computer system based on specialized hardware of specific function or step perhaps the combination through specialized hardware and computer instruction can illustrated each square of realization flow and the combination of the square in flow process illustrates.

Further, provide process flow diagram to be illustrated in the operation of carrying out among the illustrative embodiment.Process flow diagram and do not mean that the statement or the hint about specific operation or the more specifically restriction of sequence of operation.The operation of adjustable whole process flow diagram adapts to specific implementation, and does not break away from the spirit and scope of the present invention.

Fig. 9 is the process flow diagram of having summarized according to an illustrative embodiment of exemplary " startup " (" bring-up ") operation of system that is used to have I/O virtual (IOV) adapter.As shown in Figure 9, initial I/O framework proposition operation 900 starts from carrying out the I/O framework and powers up (step 902).In response to powering up to the I/O framework; System administration manager 625 can be through finding to be coupled to I/O framework 611 such as the configuration space of detecting I/O framework 611 all I/O adapter 614-616 (step 904), by the function (step 906) of finding the I/O adapter 614-616 that operation is found, and such as being a plurality of PF612 (step 908) through detecting which function that the IOV configuration space found.

For a plurality of PF, system administration manager 625 can be detected configuration space, for example, such as through the initial VF field in the configuration space that reads in each PF, comes further to confirm the quantity (step 910) by a plurality of VF that each PF supported.Then, system administration manager 625 can be confirmed as necessary all system resources of I/O adapter (IOA) 614-616 in the system 601, and more specifically, is needed those resources (step 912) of a plurality of VF in the case.Resource comprises following each item: such as being used for loading and the quantity of the amount of the needed I/O address of the storage space space of storage operation, the quantity of needed requester identifier (RID), needed DMA window space and needed interruption and similar from the memory mapped I/O (MMIO) of the processor of system for IOA is mapped to.The DMA window space is the amount for the convenient needed table space of address translation in root complex (RC).

Confirm that in case made system administration manager 625 can determine whether to become the quantity (step 914) of the VF that enables owing to the resource limit in system 601 needs restriction according to resource needed.Then, system administration manager 625 can be showed to the user such as being used for to HMC 609 its all information (step 916) of from detect the I/O configuration space, having collected of report.HMC 609 can report this information to system administration manager, as the response of the for example inquiry I/O architecture configuration HMC in Fig. 8 being asked 802.Then, the user can be to a plurality of LPAR Resources allocation, perhaps to the available pool Resources allocation of free resource.The information of being reported can comprise the for example information of user's needs, so that make the decision-making about the content in I/O framework and end points, said content comprises the item such as seller ID, device id and function quantity.Information can comprise the quantity of the PF that is supported by I/O adapter or end points, the quantity of the supported VF of each PF further, can be by a plurality of initial VF of a plurality of PF and to because with mode mentioned above or the system constraint calculated by system administration manager 625 similarly and the quantity Calculation of supported a plurality of VF is confirmed such as it.

Now, the user at HMC 609 will confirm they hope how to assign all system resource.Note, also can be used for that logical partition to system 601 distributes or the automatized script of dispatching system resource or any other robotization mechanism and the unartificial this point of accomplishing among the HMC 609 through operating in.Thereby in exemplary embodiment, HMC609 has represented collected I/O framework and terminal point information (step 918) to the user, and the user describes how to assign these resources (step 920).This distribution not only comprises I/O, and comprises the other system resource such as processor and storer.For example, in case the user has made the distribution to resource, then via the initial LPARI/O configuration HMC request 804 that is provided with in Fig. 8, to system administration manager 625 report this point (step 922).

For a plurality of VF; If be less than the initial VF that is used for corresponding PF by the quantity of the VF of user's appointment (that be assigned with and free pod the two); The NumVFs configuration space parameter that system administration manager 625 can PF so is set to desired value; And can allow to be used for the I/O virtual (step 924) of a plurality of PF via according to the VF enable bit that in PCI IOV standard, defines.Then, system administration manager 625 can be set up the resource of each subregion, and this resource comprises MMIO addressing, DMA addressing, interruption and other resources (step 926), and can create (a plurality of) IMP 603 necessary subregions (step 928).

Then; System administration manager 625 guiding are used for the partition firmware 623 and 624 of (a plurality of) IMP 603 and (a plurality of) client partitions 604, and can be used for using (step 930) at the OS of those LPAR with (a plurality of) client partitions 604 transmission I/O configuration structures to (a plurality of) IMP 603.When (a plurality of) client partitions 604 is distributed (a plurality of) VF613; System administration manager 625 begins to regulate the I/O configuration access (step 932) that (moderate) (that is, beginning to serve as virtual intermediary) is used for all (a plurality of) VF 613 from (a plurality of) client partitions 604 to them.System administration manager 625 can guide IMP subregion (a plurality of) OS, and then, can wait for from the just initialization of (a plurality of) IMP 603 indication (step 934) of (a plurality of) PF 612.For example, call " interpolation " function of 722 via the firmware that from the use of LPAR, adds/remove PF in Fig. 7, (a plurality of) IMP603 can provide this indication.

In case be used for VF 613 IMP 603 initialization PF 612, then can load the OS (step 936) that is used for VF 613.(a plurality of) client partitions 604 can load a plurality of device drives 606 and be used for their (a plurality of) VF 613 (steps 938).In case loaded device drives 616 for VF 613, then needed for example to call 702 and dispose its VF 613 (step 940) via read/write I/O configuration space.Then, device drives 606 can be brought into use its VF 613, and operation stops then.

In case system start-up and can moving is then hoped and can be added new I/O adapter or end points to system dynamics ground.Figure 10 A-10B has summarized the process flow diagram that is used for the IOV adapter is added to the exemplary hot-inserting type operation of system according to an illustrative example.Shown in Figure 10 A-10B, the user that hot-inserting type interpolation operation 1000 starts from HMC control desk 609 begins hot-inserting type and uses (step 1004), and this hot-inserting type is used provides user interface hot-inserting type operating period guide user.From hot-inserting type was used, user or application can be chosen in the empty I/O groove in the I/O framework 611, insert new I/O adapter or end points, for example I/O end points 614 (step 1006) to this sky I/O groove.Then, the user can insert new I/O adapter (IOA) or end points (step 1008), and can be inserted into (step 1010) via HMC 609 report IOA.One of ordinary skill in the art recognize, through new IOA is inserted in the untapped groove, and make system automatically call hot-inserting type to use, and can automatically transfer the hot-inserting type that is used for such as the I/O bus of PCIe and use.Then; HMC 609 sends message to system administration manager 625 and comes through " powering up " ability of using for example powering up in Fig. 8/HMC that cuts off the power supply request 806 IOA to be powered up; And can obtain its configuration information (step 1012), and operation proceeds to the step 1014 in Figure 10 B.

Similar mode with system administration manager 625 acquisition of information in initial framework start-up operation 900; System administration manager 625 can be found IOA (step 1014); Can find the function (step 1016) on IOA, and can find that which function is a plurality of PF (step 1018).For a plurality of PF; The quantity (step 1020) of the VF that system administration manager 625 can be found to be supported by each PF; Can confirm essential system resource (step 1022), confirm system resource constraints (step 1024), and can be to HMC 609 report IOA configuration informations (step 1026).

HMC 609 can represent configuration information (step 1028) to the user.The user can specify which resources allocation to give which client partitions (step 1030) with; And HMC 609 can be to the assignment (step 1032) of system administration manager 625 these resources of report; And if necessary, then system administration manager 625 can be provided with NumVFs (step 1034) in configuration space.System administration manager 625 can be provided with the resource (step 1036) of each subregion, and can create (a plurality of) subregions (step 1038) that are used for (a plurality of) new IMP 603.System administration manager 625 can guide (a plurality of) IMP 603; And to these (a plurality of) IMP 603 transmission I/O configuration structures (step 1040); (a plurality of) IMP OS can be guided, and the signal (step 1042) that the relevant a plurality of PF 612 from (a plurality of) IMP OS have been initialised can be waited for.On this aspect, except VF 613 adds, the IOA interpolation is accomplished, and system administration manager 625 can add completion and signal (step 1044) with regard to PF 612 to HM C609.

HMC 609 can send the request 812 of dynamically adding VF to (a plurality of) client partitions 604 now, is somebody's turn to do the VF 613 that (a plurality of) client partitions 604 will receive the IOA 614 that makes a fresh start.This has begun client partitions 604 processes and has added (a plurality of) VF (step 1046).(a plurality of) OS in (a plurality of) client partitions 604 can detect (a plurality of) new VF 613, and can load (a plurality of) device drives 606 and be used for (a plurality of) new VF 613 (steps 1048).(a plurality of) device drives 606 that is used for (a plurality of) VF613 can be called 702 via for example read/write I/O configuration space and dispose their (a plurality of) VF 613 (steps 1050), and can bring into use (a plurality of) VF 613 that they newly are configured.Then operation stops.

Except dynamically adding I/O adapter or end points to the I/O framework 611 of the system 601 of illustrative embodiment, it is useful can allowing I/O adapter or end points are dynamically removed from I/O framework 611.Figure 11 has summarized the process flow diagram that is used for removing from system the exemplary hot-inserting type operation of I/O adapter according to an illustrative embodiment.Shown in figure 11, hot-inserting type removes operation 1100 and starts from user at HMC control desk 609 places and begin hot-inserting type and use (step 1104).The I/O adapter (IOA) that the user can select to be removed or such as the end points (step 1106) of I/O end points 614.

HMC 609 confirms that which subregion (if any) comprises from PF612 and VF 613 resources (step 1108) with the IOA that is removed.HMC 609 can just signal to system administration manager 625 to remove coming of resource through for example using the request 814 dynamically remove PF/VF.Then, HMC 609 can be via the request 816 that for example dynamically removes VF each client partitions 604 request of sending of the VF 613 on being included in IOA come from use, to remove VF (step 1112).(a plurality of) client partitions 604 can be stopped using their (a plurality of) VF 613 (steps 1114) that are associated with the request that dynamically removes VF 816.

(a plurality of) client partitions 604 can be called 720 " removing " ability and signal to system administration manager 625 via what for example from LPAR uses, add/remove, from the use by (a plurality of) client partitions 604, removes their (a plurality of) VF 613 (steps 1118).System administration manager 625 wait for all client partitions 604 in using with regard to them to the release of their (a plurality of) VF and signal (step 1120), and then, can remove (step 1122) to HMC 609 report (a plurality of) VF.

Be accompanied by VF and remove completion, HMC 609 can begin to remove the process of (a plurality of) PF 612 that is associated with IOA 614 now.HMC 609, comes from use, to remove (a plurality of) PF612 and realizes this point through sending message to each IMP 603 that is included in the PF612 on the IOA 614 for example via the request that dynamically removes PF 818.Then; (a plurality of) IMP 603 can stop using specified (a plurality of) PF612 (step 1126) according to indication, and signals from their subregion uses to system administration manager 625 through 722 " removing " ability called of from LPAR uses, adding/removing PF and remove PF612.Then (a plurality of) IMP 603 can remove the PF612 resource (step 1128) that is associated from their subregion.

System administration manager 625 wait for all (a plurality of) IMP 603 in using with regard to them to the release of (a plurality of) PF 612 and signal (step 1130), and can remove (step 1132) to HMC 609 report (a plurality of) PF.Do not re-use IOA 614 now, and HMC 609 can be to system administration manager 625 transmission message, so that for example ask 806 " outage " ability and close the power supply (step 1134) to IOA 614 via powering up/cut off the power supply IOA.Then, system administration manager 625 can make IOA 614 outages, can from system administration manager 625, remove IOA 614 resources, and accomplishes (step 1136) to HMC 609 reports.HMC609 can accomplish (step 1138) to user report, and the user can extract IOA 614 (if expectation) then thus, and operation is accomplished.

A useful characteristic of the realization of the virtual and a plurality of VF of I/O is when for being used for load balancing when required, in system, has the ability that enough resources dynamically replace those related resources.Through having an available resources pond; Can from available resources, select a plurality of VF and be used to add to LPAR; Perhaps remove VF among the LPAR of the resource through never need utilizing VF well, and in the LPAR that can better use the VF resource, add VF and accomplish this point.Figure 12 has summarized the process flow diagram that is used for dynamically virtual functions (VF) being added to the exemplary operation of logical partition (LPAR) according to an illustrative example.For example when needing the operation of optimization system 601, can use the operation of in Figure 12, summarizing to come from a LPAR to another replacement VF resource.

Shown in figure 12, the user that VF interpolation operation 1200 starts from HMC 609 places begins dynamically to reconfigure (DR) application (step 1204).The user can select to add from the freelist of a plurality of VF 613, and can specify to which client partitions and add VF613 (step 1206).HMC 609 can be for example via to the dynamic interpolation request 808 of PF/VF and represent this information (step 1208) to system administration manager 625.System administration manager 625 can be provided with the client partitions resource, comprises MMIO addressing, DMA addressing, interruption and similar (step 1210) thereof.

System administration manager 625 can be ready to proceed to notify HMC609 with regard to operation through sending the completed signal of institute's requested operation.Next, HMC 609 can ask client partitions 604 dynamically to add VF 613 (step 1214) via the request 812 of dynamic interpolation VF.Client partitions 604 can detect new VF 613, and can load the device drives 606 (step 1216) that is used for new VF 613.Be used for the device drives 606 of VF 613 and can be for example call 702 and dispose its VF 613 (step 1218) via read/write I/O configuration space.At last, HMC 609 can notify the user with regard to the completion of operation, and (step 1220) accomplished in operation.

Figure 13 is a process flow diagram of having summarized the exemplary operation that is used for dynamically VF is removed from LPAR according to an illustrative example.It is other parts with the ability of the removal of the I/O resource between a plurality of LPAR of system 601 that dynamic VF in Figure 13 removes operation 1300.That is, can use the operation shown in Figure 13 with VF from its with a LPAR related remove, and use the VF that in Figure 12 of preceding text, summarizes out to add operation and add VF to another LPAR then.Can according in shown specific example through the manual, user process accomplish VF from a subregion remove and to the interpolation of another subregion, perhaps through confirming by program that VF is not made full use of by a subregion and can being used and automatically accomplish VF is removed and adds to another subregion from a subregion by another subregion.For the resource that is similar to storer, this type of load balancing program is known in affiliated field.Can use herein dynamically the removing and add the load balancing program is expanded so that also balanced I/O resource VF of the present invention of describing.

Shown in figure 13, VF removes operation 1300 and starts from user at HMC 609 places and begin DR and use (step 1304).The user can select the VF that is removed, and such as VF 613 (step 1306), and HMC 609 can dynamically remove PF/VF request 818 (steps 1308) to for example system administration manager 625 transmissions.For example, HMC 609 can send the request 816 that dynamically removes VF to client partitions 604, begins VF and removes process (step 1310).According to indicated, client partitions 604 can receive this request, and stops using VF 613 (step 1312).

Client partitions 604 can remove VF613 (step 1314) to system administration manager 625 signallings via for example from LPAR uses, adding/remove the calling 720 " removing " function of VF from the use of a plurality of subregions.Then, system administration manager 625 can remove VF 613 resources (step 1316) that are associated from client partitions 604.System administration manager 625 can be to remove (step 1318) of HMC 609 report VF.HMC 609 can notify the user (step 1320) with regard to the completion of operation, and operation stops.

Except dynamically adding and remove I/O adapter or end points, and the resource that can between a plurality of LPAR, replace a plurality of VF and be associated, in virtualization system, the robustness that can from mistake, recover for fault processing is useful.Illustrative embodiment provides a kind of mechanism, and being used for can be through fetching error message, registration error information, reset and dispose a plurality of VF and attempting restarting uncompleted I/O operation then and recover from mistake.Describe as back literary composition, in Figure 14, summarized this operation.

Figure 14 has summarized according to an illustrative example to be used for from carried out the process flow diagram of the wrong exemplary operation that recovers by the mistake that VF caused.Having manipulated of in Figure 14, summarizing stops can to recognize the wrong prior art that exists to all DMA of VF and MMIO up to device drives, and can moderately recover.Yet the operation of Figure 14 utilizes the mechanism of illustrative example to replenish this prior art, controls the error recovery procedure (ERP) that is used for a plurality of VF.

Shown in figure 14, after the mistake that makes VF get into MMIO halted state and DMA halted state takes place, 1400 beginnings of VF error recovery operation.At this state, the dma operation from VF will not be done, and when being in the MMIO halted state, will abandon the MMIO storage operation, and MMIO load operation complete 1 the data that will obtain being returned.Error recovery procedure (ERP) for example starts from, and the device drives of device drives 606 receives complete 1 data (step 1404) that load operation is returned from MMIO.Do not expect to return (whether perhaps uncertain its is expected) device drives 606 of complete 1 data, the inquiry VF error condition firmware that for example is used for VF613 calls 710, obtains VF status information (step 1406).Since this time VF 613 be in error condition, system administration manager 625 can respond with regard to VF 613 and be in the inquiry (step 1408) that MMIO stops to stop with DMA error condition.Device drives 606 can be used and for example inquire about I/O framework error message firmware and call 714, obtains the information about the error register in I/O framework 611, and the problem that is used for the back is confirmed (step 1410).System administration manager 625 error message response inquiries capable of using (step 1412), and device drives 606 can be used for information registering the analysis (step 1414) of back in error log.

Now, device drives 606 its VF that is ready to reset, and attempt restarting operation.Device drives 606 can be used the VF firmware request 708 that for example resets, the VF 613 (step 1416) that resets, and system administration manager 625 completion capable of using respond response request (step 1418).Then, device drives 606 for example can use that read/write I/O configuration space calls 702, disposes its VF 613 (step 1420).Device drives 606 attempts restarting uncompleted I/O operation (step 1422), and operation stops.

As previously mentioned; The critical function of IMP 603 provides the secure access to the resource of IOA or end points; Wherein the resource of this IOA or end points is perhaps shared between a plurality of LPAR that just use a plurality of VF that are associated with PF; Perhaps the resource of this IOA or end points is unique for VF, but to the change of being undertaken by PF, VF should be an inaccessible.The network address that an example of this type of resource of network I/O adapter (IOA) or end points is and is associated by the PF that a plurality of VF share, the perhaps unique network address for VF.To be called IOA " southern side " management of I/O adapter or end points to the management of these device-specific resources.Figure 15 is a process flow diagram of having summarized the exemplary IOA southern side bookkeeping of the network address that is used to change on network adapter according to an illustrative example.Should recognize,, be used for sharing between other a plurality of LPAR that use a plurality of VF that are associated with PF, perhaps be unique resource as far as VF can carry out similar operation though Figure 15 has indicated the network address that changes on network adapter particularly.

Shown in figure 15, the user that operation starts from HMC control desk 609 places begins should be used for changing the network address (step 1504) that is used for VF 613.The user can select VF 613 and address (perhaps, application can provide the address based on the context of operating) (step 1506).HMC 609 can confirm the IMP 603 of PF612 and control VF 613, and can send IOA southern side management request 820 to IMP 603, and in IOA southern side management request 820, transmits VF613 and address information (step 1508).IMP 603 can be to PF device drives 605 transmission information, and PF device drives 605 continues to be sent to PF 612 with it and is used for handling (step 1510).HMC 609 can accomplish and inquiry IMP 603 with regard to order, and when accomplishing, can notify the user to operate completion (step 1512), and operation stops.

Should recognize, when system 601 can operate, but client partitions 604 terminating operations in system 601.The different reasons that have many client partitions 604 to stop such as the irrecoverable error in the software in client partitions, perhaps only influence the mistake of expendable this hardware of the subregion that has distributed hardware.Figure 16 is a process flow diagram of having summarized the exemplary operation that is used for the client partitions termination according to an illustrative example.A kind of situation has been guided in the operation of in Figure 16, summarizing 1600, and wherein client partitions stops, and or guides again or permanent the termination.

Shown in figure 16, when client partitions 604 is operated and when stopping for some reason, the operation beginning.Can detect the system administration manager 625 when client partitions stops; It for example can detect client partitions 604 in panic the calling from the subregion to the system administration manager and stop (step 1604); And said panic calling is because expendable software error, perhaps owing to when irrecoverable hard error, caused from the hardware check interruption of hardware.Then, for example determined according to restarting in the strategy of the user who from the part of initially setting up subregion, is provided with, system administration manager 625 can confirm whether client partitions 604 is set to guide again (step 1606).

If client partitions 604 is not set to guide again, then system administration manager 625 can remove the VF resource and add said (a plurality of) VF to system administration manager VF free pod (step 1608) from the subregion that is terminated.In fact, the client partitions 604 of inefficacy can be not in office why not for a long time after by guiding again, and thereby operation stop, be that a kind of terminations is machine-processed.

Confirm that client partitions 604 will be by guiding (step 1606) again if make, so because it will be by guiding again, system administration manager 625 to the subregion of correspondence redistribute with the subregion that is being terminated in the resource (step 1610) that is associated of (a plurality of) VF.Then, operation stops.

Should recognize that in the operating period of system 601, IMP 603 also may lose efficacy.If IMP 603 lost efficacy, then expectation setting always guides IMP 603 again, because this IMP603 serves as the controller that is used in the resource of potential many VF 813 of many client partitions 604.Figure 17 is a process flow diagram of having summarized the exemplary operation 1700 that is used for the termination of IMP subregion according to an illustrative embodiment.Shown in figure 17, along with the IMP subregion stops, operation 1700 beginnings.System administration manager 625 for example detects the termination of IMP 603 subregions in panic the calling from the subregion to the system administration manager; And initiation is to the guiding again (step 1704) of IMP 603; Wherein said panic calling is because expendable software error, perhaps owing to when expendable hard error, caused from the hardware check interruption of hardware.

Only if do not suspend inquiry interrupt source firmware call 716 with DMA conversion firmware be set call 718; So that routine operation can continue; System administration manager 625 can suspend for calling 700 with any and all firmwares at all (a plurality of) VF that are associated by (a plurality of) PF among the IMP 603 of guiding again, accomplishes (step 1706) up to guiding again.Have with client partitions 604 and continue routine operations at (a plurality of) VF that is associated by (a plurality of) PF among the IMP 603 that restarts; Only if they need be made the firmware that has suspended and call; In the case, system administration manager 625 time-out firmwares call up to the guiding again of IMP and accomplish (step 1708).System administration manager 625 is given IMP 603 with identical PF resources allocation, and is to guide again at IMP 603 detecting operations of guiding again, and store status again carefully, and making not to influence the operation (step 1710) of VF.Then, operation stops.

Thereby; Illustrative embodiment provides mechanism; This mechanism allows in independent management zone rather than in system administration manager, to carry out the management software that is used for southern side adapter dedicated management, and is independently operated by the adapter of system administration manager execution picture configuration and mismanage.The code that hope is relevant with the I/O adapter remains on outside the system administration manager, prevents the performance impact to system administration manager.And; The code that hope is relevant with the I/O adapter is placed in the independent management zone; Make and to minimize owing to any instability of the system administration manager code that frequent updating caused; For example, frequent updating is owing to upgrade the necessity of the code that is used for the unique southern side management software of different adapters.Thereby, the mechanism of illustrative example provide minimize instable simultaneously to the more efficient operation of system administration manager.

Should recognize that exemplary embodiment can adopt the embodiment of the embodiment of whole hardware, whole softwares or comprise the form of both embodiment of hardware and software unit.In the exemplary embodiment, realize the mechanism of illustrative embodiment with software, software includes but not limited to firmware, resident software, microcode etc.

Further; Illustrative embodiment can take can use or the form of the addressable computer program of computing machine recordable media from computing machine, computing machine can with or the computing machine recordable media provide and be used for using or combining computing machine or any instruction execution system and the program code that uses by computing machine or any instruction execution system.For purpose of description; Computing machine can with or the computing machine recordable media can be can comprise, any device of storage, communication, propagation or convey program; Wherein said program is used for using through instruction execution system, device or equipment, perhaps combined command executive system, device or equipment and use.

Medium can be for example electricity, magnetic, light, electromagnetism, infrared or semiconductor system (or device or equipment).The example of computing machine recordable media comprises semiconductor or solid-state memory, tape, removable computer disks, random access storage device (RAM), ROM (read-only memory) (ROM), hard disc and CD.The present example of CD comprises compact disk-ROM (read-only memory) (CD-ROM), compact disk-read/write (CD-R/W) and DVD.

Be suitable for storing and/or the data handling system of executive routine code will comprise at least one processor that is coupled to memory cell through system bus directly or indirectly.Local storage, mass-memory unit and cache memory that memory cell is utilized the term of execution of can being included in program code actual; It provides the interim storage at least some program codes, so that must obtain the quantity of the number of times of code be reduced in the term of execution from mass-memory unit again.

Can be directly or the I/O controller through between two parties and system is coupled in I/O or I/O equipment (including but not limited to keyboard, display, indicating equipment etc.).Also can network adapter be coupled to system, allow the privately owned or public network of data handling system through between two parties to be coupled to other data handling systems or remote printer or memory device.Modulator-demodular unit, cable modem and Ethernet card are merely some current available types of network adapters.

Represented the description of this invention, and be not intended to disclosed form limit or restriction the present invention in order to illustrate with purpose of description.To one of ordinary skill in the art, many adjustment and variation are conspicuous.Select and description embodiment, so that explain principle of the present invention, practical application best, and the field others skilled in the art understand the present invention under allowing, and are used to have the various embodiments of the multiple adjustment that is suitable for specific use expection.

Claims

1. one kind is used for the virtualized method at data handling system management I/O I/O, comprising:

In said data handling system, create one or more client partitions, each client partitions has the part that is associated of the resource of said data handling system;

The one or more I/O end points that are associated with said data handling system are carried out the operation of I/O virtual management, and wherein said I/O virtual management operation moves with the physical function that disposes said one or more I/O end points or at least one of virtual functions; And

Use the virtual intermediary that is separated with I/O virtual management subregion IMP to control the visit of said I/O virtual management operation to the resource of said one or more I/O end points; Wherein said virtual intermediary prevents that the client partitions visit is not assigned to the resource of said one or more I/O end points of said client partitions, and is wherein carried out by said virtual intermediary and create one or more client partitions and create said IMP; And

Wherein in said data handling system, create I/O virtual management subregion IMP; The wherein said IMP pair of one or more I/O end points that are associated with said data handling system are carried out the operation of I/O virtual management; So as to managing at least one physical function of said one or more I/O end points; And wherein, do not allow said one or more client partitions to come said at least one physical function is carried out the operation of I/O virtual management.

2. method according to claim 1, wherein, said virtual intermediary is and said data handling system system associated manager.

3. method according to claim 2; Wherein provide being used for visit at the configuration space of the physical function of said one or more I/O end points and virtual functions to system administration manager; And wherein; Only provide to the configuration space of the physical function that in said one or more I/O end points, is used for being assigned to said IMP with in the visit of said one or more I/O end points to the configuration space that is used for virtual functions to IMP, the wherein said configuration space that is used for virtual functions belongs to the physical function that is assigned to said IMP.

4. method according to claim 3; Wherein said system administration manager allows to visit the configuration space that in said one or more I/O end points, is assigned to the virtual functions of said client partitions by client partitions, and stops the configuration space by the virtual functions that not be assigned to said client partitions of said client partitions visit in said one or more I/O end points.

5. method according to claim 1 further comprises: receive input from HMC HMC, said HMC HMC guides the execution of said I/O virtual management operation.

6. method according to claim 5 is the order of initiating to be used for new I/O end points is added to the hot-inserting type operation of said data handling system from the input of said HMC wherein, and wherein carries out said I/O virtual management operation and comprise:

Based on interpolation, to said one or more client partitions assigned resources to said new I/O end points;

Based on said interpolation, dispose said IMP to said new I/O end points; And

Dynamically add at least one virtual functions of said new I/O end points at least one client partitions in said one or more client partitions.

7. method according to claim 5 is the order of initiating to be used for removing from said data handling system the hot-inserting type operation of existing I/O end points from the said input of said HMC wherein, and wherein carries out said I/O virtual management operation and comprise:

At least one client partitions of sign in said one or more client partitions with the resource that is associated with the virtual functions of the said I/O end points that will be removed;

The resource that from use, removes said virtual functions and be associated with said virtual functions by said at least one client partitions;

From use by said IMP, the resource that removes physical function and be associated with said physical function; And

Make the I/O end points outage that is removed.

8. method according to claim 5, wherein the said input from said HMC is the order that dynamically reconfigures said data handling system, and wherein carries out the operation of said I/O virtual management and comprise:

The selection of the virtual functions of the I/O end points that reception is added selected client partitions that will be in said one or more client partitions;

Be provided for the resource of the selected virtual functions in said selected client partitions; And

Loading is used for the device drives at the said selected virtual functions of said selected client partitions.

9. method according to claim 5, wherein the said input from said HMC is the order that dynamically reconfigures said data handling system, and wherein carries out the operation of said I/O virtual management and comprise:

Reception is to the selection of the virtual functions of the I/O end points that will remove from the selected client partitions said one or more client partitions;

From further use, remove said selected virtual functions by said selected client partitions; And

Remove the resource that is used at the selected virtual functions of said selected client partitions.

10. method according to claim 5 is the order that changes the network address that is associated with the selected virtual functions of I/O end points from the said input of said HMC wherein, and wherein carries out said I/O virtual management and operate and comprise:

The physical function that sign is associated with said selected virtual functions;

Send management request to said IMP, wherein said management request comprises the address information of the said network address that is used to change said selected virtual functions; And

Be based on the said address information in the said management request,, change the said network address of said selected virtual functions via driving with said physical function associated device.

11. method according to claim 1 is wherein carried out said I/O virtual management operation and is comprised:

In said virtual intermediary, the inquiry virtual functions error condition firmware of the device drives in the client partitions of the next comfortable said one or more client partitions of reception calls;

From said virtual intermediary, the error message of the error condition of the virtual functions that has identified the I/O end points is provided to said device drives;

By the said error message of said device drives registration from the reception of said virtual intermediary;

Firmware to said data handling system sends the request from the virtual functions that resets of said device drives, and the said virtual functions with said error condition is resetted; And

After resetting, dispose said virtual functions.

12. method according to claim 1 is wherein carried out said I/O virtual management operation and is comprised:

Detecting client partitions by said virtual intermediary lost efficacy;

Determine whether to carry out client partitions pilot operationp again;

If do not guide said client partitions again, then remove the virtual functions resource that is associated with the client partitions that lost efficacy, and add the virtual functions resource that is removed to free virtual functions resource pool from said one or more I/O end points; And

If will guide said client partitions again, then after the client partitions of said inefficacy guides again, redistribute said virtual functions resource to the client partitions of said inefficacy by said virtual intermediary.

13. method according to claim 1 is wherein carried out said I/O virtual management operation and is comprised:

Detect the inefficacy of said IMP;

Initiate guiding again by said virtual intermediary to said IMP;

The firmware that is suspended about the virtual functions that is associated with the physical function that is assigned to said IMP by said one or more client partitions calls, and accomplishes up to the said guiding again to said IMP; And

The said completion of guiding again along with to said IMP is stopped the time-out that firmware is called by said one or more client partitions.

14. one kind is used for the virtualized device at data handling system management I/O I/O, comprises:

Be used for creating in said data handling system the device of one or more client partitions, each client partitions has the part that is associated of the resource of said data handling system;

Be used for the one or more I/O end points that are associated with said data handling system are carried out the device of I/O virtual management operation, wherein said I/O virtual management operation moves with the physical function that disposes said one or more I/O end points or at least one of virtual functions; And

Be used to use the virtual intermediary that is separated with I/O virtual management subregion IMP to control the device of said I/O virtual management operation to the visit of the resource of said one or more I/O end points; Wherein said virtual intermediary prevents that the client partitions visit is not assigned to the resource of said one or more I/O end points of said client partitions; Wherein carry out and create one or more client partitions, and create said IMP by said virtual intermediary; And

Be used for creating the device of I/O virtual management subregion IMP in said data handling system; The wherein said IMP pair of one or more I/O end points that are associated with said data handling system are carried out the operation of I/O virtual management; So as to managing at least one physical function of said one or more I/O end points; And wherein, do not allow said one or more client partitions that said at least one physical function is carried out the operation of I/O virtual management.

15. device according to claim 14, wherein, said virtual intermediary is and said data handling system system associated manager.

16. device according to claim 15 also comprises:

Be used for providing being used for device in the visit of the configuration space of the physical function of said one or more I/O end points and virtual functions to said system administration manager, and

Be used for only providing to the configuration space of the physical function that is used for being assigned to said IMP at said one or more I/O end points with at the device of said one or more I/O end points to the visit of the configuration space that is used for virtual functions to IMP, the wherein said configuration space that is used for virtual functions belongs to the physical function that is assigned to said IMP.

17. device according to claim 16; Wherein said system administration manager allows to visit the configuration space that in said one or more I/O end points, is assigned to the virtual functions of said client partitions by client partitions, and stops the configuration space by the virtual functions that not be assigned to said client partitions of said client partitions visit in said one or more I/O end points.

18. device according to claim 14 wherein receives input from HMC HMC, said HMC HMC guides the execution of said I/O virtual management operation.

19. device according to claim 18, wherein the said input from said HMC is the order of initiating to be used for new I/O end points is added to the hot-inserting type operation of said data handling system, and wherein also comprises:

Be used for based on interpolation, to the device of said one or more client partitions assigned resources said new I/O end points;

Be used for disposing the device of said IMP based on interpolation to said new I/O end points; And

Be used for device at least one virtual functions that dynamically adds said new I/O end points at least one client partitions of said one or more client partitions.

20. device according to claim 14 also comprises:

Be used in said virtual intermediary, from the device drives the client partitions of said one or more client partitions, receive the device that inquiry virtual functions error condition firmware calls;

Be used for providing to said device drives the device of error message of the error condition of the virtual functions that has identified the I/O end points from said virtual intermediary;

Be used for by the device of said device drives registration from the said error message of said virtual intermediary reception;

Be used for sending request, so that have the device that the said virtual functions of said error condition resets from the virtual functions that resets of said device drives to the firmware of said data handling system; And

Be used for after resetting, dispose the device of said virtual functions.

21. device according to claim 14 also comprises:

Be used for detecting the device that client partitions lost efficacy by said virtual intermediary;

Be used to determine whether to carry out the client partitions device of pilot operationp again;

Need not guide said client partitions again if be used for; Then from said one or more I/O end points, remove the resource of the virtual functions that is associated with the client partitions that lost efficacy, and add the device of the said virtual functions resource that is removed to free virtual functions resource pool; And

If be used for to guide said client partitions again, then after the client partitions of said inefficacy guides again, redistribute the device of said virtual functions resource to the client partitions of said inefficacy by said virtual intermediary.

22. device according to claim 14 also comprises:

Be used to detect the device of the inefficacy of said IMP;

Be used for initiating the device of guiding again to said IMP by said virtual intermediary;

Be used for calling, up to the device that the said guiding again of said IMP is accomplished by the firmware that said one or more client partitions are suspended about the virtual functions that is associated with the physical function that is assigned to said IMP; And

Be used for along with the said completion of guiding again the device of the time-out that stops by said one or more client partitions firmware is called to said IMP.