CN102722414B - Input/output (I/O) resource management method for multi-root I/O virtualization sharing system - Google Patents
Input/output (I/O) resource management method for multi-root I/O virtualization sharing system Download PDFInfo
- Publication number
- CN102722414B CN102722414B CN201210160593.9A CN201210160593A CN102722414B CN 102722414 B CN102722414 B CN 102722414B CN 201210160593 A CN201210160593 A CN 201210160593A CN 102722414 B CN102722414 B CN 102722414B
- Authority
- CN
- China
- Prior art keywords
- root node
- pcie
- equipments
- functions
- resource
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Abstract
The invention discloses an input/output (I/O) resource management method for a multi-root I/O virtualization sharing system. The method for implementing centralized initialization configuration and allocation management on all I/O equipment in the multi-root I/O virtualization sharing system by using a master control root node comprises the following steps of: 1, performing enumeration discovery and initialization configuration on I/O resources in the system by the master control root node; 2, assisting a management user in managing and controlling I/O equipment resources in the system through a peripheral component interconnect express (PCIe) management module by the master control root node, and executing an I/O resource allocation strategy formulated by the management user; and 3, establishing a mapping relation between equipment functions and root nodes thereof according to the PCIe management module and an I/O resource allocation table by the master control root node to realize dynamic allocation of t he I/O resources.
Description
Technical field
The present invention relates to I/O Intel Virtualization Technology, particularly relate to a kind of I/O method for managing resource for the many virtual shared systems of I/O.
Background technology
In conventional data centers or cloud computing center, each blade server has independently I/O resource, as network interface card, HBA adapter etc.Yet calculate the close-coupled with I/O resource, make the expansion of two class resources be subject to the restriction of server space simultaneously.In addition, high-performance I/O equipment (as 10GigE network interface card) only depends on a station server, has reduced undoubtedly its utilization ratio, has improved system synthesis originally.For addressing the above problem, will remove the close-coupled of calculating with I/O resource, its key is exactly to realize I/O equipment sharing between a plurality of servers.
In recent years, the technology of sharing of I/O equipment has obtained the development of the property changed, and industry standard organizes the special interest group PCI-SIG of PCI successively to formulate PCI-SIG Single Root I/O Virtualization (SR-IOV: single IO is virtual) and PCI-SIG Multi-Root I/O Virtualization (MR-IOV: many IO are virtual) standard.Possess the different virtual machine (VM/Guest) that the PCI Express equipment of SR-IOV ability can be running on same host and share, and provide the I/O performance that approaches the machine for virtual machine.SR-IOV has obtained the extensive support of I/O equipment vendors at present, as the gigabit ethernet card of support SR-IOV, 10G Ethernet card, HBA adapter etc.
Possess some virtual machines (VM/Guest) that the PCI Express equipment of MR-IOV ability can be moved on a plurality of hosts and share, MR-IOV technology can remove completely calculate and I/O between close-coupled.But because the MR-IOV agreement of PCI-SIG is to the larger change of original system (comprising the modification to PCIe basic agreement, PCIe device structure and PCIe switch architecture), be difficult to be accepted by industry member.Yet, for itself possessing as SR-IOV and multifunctional equipment etc. to have, supply directly shared ability of a plurality of virtual machines simultaneously, if provide a series of additional supports to it, it is feasible realizing I/O resource shared between a plurality of root nodes.
When an I/O equipment is shared by a plurality of root nodes by PCIe switch architecture, each root node is enumerated this equipment by trial after system starts.This will cause system competition, and the modification to I/O equipment in topological structure of root node, can affect the operation of other root nodes, finally causes system cisco unity malfunction.
Summary of the invention
For addressing the above problem, the invention provides the I/O method for managing resource in the virtual shared system of a kind of many I/O, for solving the technical matters of resource management and the distribution of the many virtual shared systems of IO, avoid conflict and the competition of each root node in the many virtual shared systems of I/O.
The present invention discloses a kind of I/O method for managing resource for the many virtual shared systems of I/O, comprising:
Step 2, main control root node is by PCIe administration module, and the I/O device resource in auxiliary management user management and control system, carries out the I/O resource allocation policy that leading subscriber is formulated;
For the I/O method for managing resource of the many virtual shared systems of I/O, described step 2 comprises:
Step 21, all PCIe module informations present to leading subscriber in PCIe administration module collection system, auxiliary leading subscriber initialization or revise resource allocation policy;
Step 22, PCIe administration module is according to user's I/O resource allocation policy, and transmitting apparatus distribution instruction realizes the allocation manager of I/O resource.
For the method for managing resource of the many virtual shared systems of I/O, described PCIe module information comprises:
Root node information, described root node information, comprises the number of root node, I/O resource that each root node has and the load condition of each root node;
I/O facility information, described I/O facility information, No. ID, functions of the equipments descriptor and functions of the equipments take situation to comprise functions of the equipments.
For the I/O method for managing resource of the many virtual shared systems of I/O, described I/O resource allocation table comprises:
Described resource allocation table is arranged in each PCIe downstream port of the many virtual shared control units of I/O, the corresponding I/O resource allocation table of each I/O equipment; Described resource allocation table has recorded the mapping relations of functions of the equipments and the root node under it; Wherein, table index is in main control root node PCIe territory No. ID of functions of the equipments, table clause content description root node sign ID under functions of the equipments and No. ID in affiliated root node PCIe territory thereof.
For the I/O method for managing resource of the many virtual shared systems of I/O, the I/O resource dynamic in described step 3 is distributed and is comprised:
I/O initializing resource distributes, and described I/O initializing resource distributes and refers to when system initialization, and main control root node is each subordinate root node distributing equipment function, sets up the process of initial PCIe topological structure;
I/O resource reclaims, and described I/O resource reclaims and refers in the situation that not affecting other root node work, and main control root node is deleted the process of I/O functions of the equipments at subordinate root node;
I/O redistribute resources, described I/O redistribute resources refers in the situation that not affecting other root node work, main control root node increases the process of I/O functions of the equipments to subordinate root node.
For the I/O method for managing resource of the many virtual shared systems of I/O, described I/O initializing resource distributes and comprises:
Step 61, leading subscriber is formulated initial allocation strategy according to the I/O resource of the demand of root node and system, and by PCIe administration module transmitting apparatus distribution instruction;
Step 62, PCIe administration module distributes transaction packet by transmitting apparatus, carries out the resource allocation policy of leading subscriber; Described devices allocation transaction packet, that PCIe internal memory is write transaction packet, described PCIe internal memory is write transaction packet and has been indicated the functions of the equipments that will operate No. ID, the sign of the root node under No. ID in main control root node PCIe territory, these functions of the equipments ID with and in affiliated root node PCIe territory No. ID; Wherein, affiliated root node is for No. ID No. ID, subordinate root node, indicates a devices allocation instruction;
Step 63, PCIe downstream port receives the transaction packet of devices allocation described in step 62, resolve transaction packet obtain No. ID, the functions of the equipments that will operate, the sign of the root node under it ID with and in affiliated root node PCIe territory No. ID; And by root node sign ID and its ID component content in affiliated root node PCIe territory by under functions of the equipments, write in the corresponding list item of I/O resource allocation table of No. ID indication of functions of the equipments that will operate, and then be that each subordinate root node has been specified the I/O functions of the equipments of distributing;
Step 64, after main control root node completes the I/O device resource initialization of subordinate root node is distributed, the directed rescanning of PCIe equipment, the functions of the equipments that discovery and initial configuration distribution obtain of carrying out of subordinate root node.
For the I/O method for managing resource of the many virtual shared systems of I/O, described I/O resource reclaims and comprises:
Step 71, leading subscriber determines to delete certain functions of the equipments from certain root slave node according to demand, by PCIe management software transmitting apparatus recovery command;
Step 72, PCIe administration module distributes transaction packet by transmitting apparatus, carries out the resource recovery command of leading subscriber; Described devices allocation transaction packet, is that PCIe internal memory is write transaction packet, described PCIe internal memory write things bag indicated No. ID, root node under No. ID, the I/O functions of the equipments that will operate, these functions of the equipments with and in affiliated root node PCIe territory No. ID; Wherein, the root node sign ID under function is main control root node sign ID, indicates an equipment recovery command;
Step 73, PCIe downstream port receiving equipment distributes transaction packet, resolve transaction packet obtain No. ID, the functions of the equipments that will operate, the sign of the root node under it ID with and in affiliated root node PCIe territory No. ID;
Step 74, affiliated root node sign ID judges; If the affiliated root node sign ID indicating in configuration transaction bag is main control root node sign ID, indicate an equipment recovery command;
Step 75, reads the list item corresponding to I/O resource allocation table of No. ID indication of functions of the equipments that transaction packet will operate, and to the subordinate root node of list item indication, sends a virtual hot plug event, and notice has I/O functions of the equipments to be pulled out;
Step 76, the corresponding list item of I/O resource allocation table of No. ID indication of functions of the equipments of reset operation; By being complete zero functional identity component content by main control root node sign ID and value, write in the corresponding list item of I/O resource allocation table of No. ID indication of functions of the equipments that will operate;
Step 77, target subordinate root node receives this hot plug event, this equipment is removed from PCIe topological structure, and unload corresponding driver.
For the I/O method for managing resource of the many virtual shared systems of I/O, described I/O redistribute resources comprises:
Step 81, leading subscriber determines according to demand and indicates that to certain subordinate root node, to increase certain equipment merit standby, by PCIe management software transmitting apparatus reallocation instruction;
Step 82, PCIe administration module distributes transaction packet by transmitting apparatus, carries out the redistribute resources instruction of leading subscriber; Described devices allocation transaction packet, that PCIe internal memory is write transaction packet, described PCIe internal memory is write transaction packet and has been indicated the I/O functions of the equipments that will operate No. ID, the sign of the root node under No. ID in main control root node PCIe territory, these functions of the equipments ID with and in affiliated root node PCIe territory No. ID; Wherein, the root node sign ID under function is subordinate root node sign ID, indicates a devices allocation instruction;
Step 83, PCIe downstream port receiving equipment distributes transaction packet, resolve transaction packet obtain No. ID, the functions of the equipments that will operate, the sign of the root node under it ID with and in affiliated root node PCIe territory No. ID;
Step 84, affiliated root node sign ID judges; If the affiliated root node ID indicating in configuration transaction bag is non-main control root node ID, indicate a devices allocation instruction;
Step 85, to a virtual hot plug event of subordinate root node transmission of transaction packet indication, notifies corresponding subordinate root node to have I/O equipment to insert its PCIe topological structure; Simultaneously, by root node sign ID and its ID component content in affiliated root node PCIe territory by under functions of the equipments, write in the corresponding list item of I/O resource allocation table of No. ID indication of functions of the equipments that will operate, and then be that this subordinate root node increases I/O functions of the equipments of distribution;
Step 86, target subordinate root node receives hot plug event, this equipment is scanned and initial configuration, and load corresponding driver.
Beneficial effect of the present invention is: by I/O method for managing resource of the present invention, solved system competition and the conflict of being shared to cause by a plurality of root nodes because of an I/O equipment in the many virtual shared systems of I/O, and can realize I/O resource according to the demand of system and between a plurality of root nodes, carry out dynamic assignment, the platform of an operability is provided for leading subscriber simultaneously.
Accompanying drawing explanation
Fig. 1 is the structural representation of the many virtual shared systems of I/O;
Fig. 2 is that many of the present invention share a SR-IOV equipment schematic diagram;
Fig. 3 is I/O resource allocation table schematic diagram of the present invention;
Fig. 4 is I/O initializing resource allocation list schematic diagram of the present invention;
Fig. 5 is I/O resource dynamic recovery process figure of the present invention;
Fig. 6 is I/O resource dynamic reallocation process flow diagram of the present invention;
Fig. 7 is I/O method for managing resource schematic diagram of the present invention.
Embodiment
Provide the specific embodiment of the present invention below, by reference to the accompanying drawings the present invention has been made to detailed description.
Cause system crash and competition when avoiding a plurality of root nodes to share an I/O equipment, the invention provides a kind of I/O method for managing resource for the many virtual shared systems of I/O.As shown in Figure 7, described I/O method for managing resource, refers to and utilizes a main control root node to carry out centralized initial configuration and allocation manager to all I/O equipment in the many virtual shared systems of IO.It comprises following characteristics:
A) only having main control root node to enumerate finds and initial configuration I/O device resource.System powers on, first the root node starting utilizes the system initialization software of ready-made standard to search for discovery to all PCIe equipment in system, only have main control root node to enumerate and find each I/O equipment, and configure the resources such as corresponding I/O address space, memory address space and interruption for each I/O device initialize.
B) main control root node is by PCIe administration module, and the I/O device resource in auxiliary management user management and control system, carries out the I/O resource allocation policy that leading subscriber is formulated.
C) main control root node is by PCIe administration module and I/O resource allocation table, and the mapping relations of apparatus for establishing function and the root node under it, realize I/O resource dynamic and distribute.
Described PCIe administration module, is characterized in that: all PCIe module informations present to leading subscriber in its collection system, resource allocation policy is formulated or revised to auxiliary leading subscriber.Meanwhile, PCIe administration module is according to user's resource allocation policy, and transmitting apparatus distribution instruction, rewrites I/O resource allocation table, realizes the allocation manager of I/O resource.
Described PCIe assembly includes but not limited to root nodes all in system and all I/O equipment.Described root node information includes but not limited to: the I/O resource that the number of root node, each root node have and the load condition of each root node.Described I/O facility information includes but not limited to: functions of the equipments ID identifier (as B/D/F), functions of the equipments descriptor (comprising vendor id, device id, revision ID etc.) and functions of the equipments take situation etc.
Described I/O resource allocation table, is characterized in that: described resource allocation table is arranged in each PCIe downstream port of the many virtual shared control units of I/O, the corresponding I/O resource allocation table of each I/O equipment; Described resource allocation table has recorded the mapping relations of functions of the equipments and the root node under it.Wherein, table index is No. ID (as the functions number) of functions of the equipments in main control root node PCIe territory, table clause content description root node sign ID and No. ID (as the Bus number Bus/ device number Device/ function Function) in affiliated root node PCIe territory thereof under functions of the equipments.
Fig. 1 has described the structural representation of the many virtual shared systems of I/O.The many virtual shared systems of I/O mainly comprise three parts: root node subsystem, I/O equipment subsystem and the virtual shared control unit of Duo Gen I/O.The many virtual shared control units of IO are coupled root node subsystem and I/O equipment subsystem by PCIe interface protocol, realize an I/O device resource and are directly shared by a plurality of root nodes.
Root node subsystem in the many virtual shared systems of I/O comprises a plurality of root nodes, and each root node is comprised of CPU group (CPU set) and the internal memory (Mem) of root complex (Root Complex, RC) and connection thereof.Wherein, operation has the root node of PCIe management related software to be referred to as main control root node (Master Root Node, mRN), and main control root node only has one (below using mRN0 sign), comprises one group of PCIe management software; It has absolute management control right to all I/O device resources in system, and the resource allocation policy of formulating according to user, and the hardware resource of I/O equipment in system be take to functions of the equipments as unit, dynamically distributes to other root nodes.Other root nodes are called subordinate root node (Slave Root Node, sRN), subordinate root node can have and a plurality ofly (below uses sRN1, sRN2,, sRNn sign), and according to the distribution of main control root node, the independent right to use that has certain I/O resource, can and only can be used master control root node to distribute to its I/O device resource.On root node, can move a plurality of isomorphisms or isomery virtual machine (Virtual Machine, VM), the I/O resource that virtual machine management program (Virtual Machine Manager, VMM) is responsible for the distribution of scheduling main control root node is that each VM is used.
I/O equipment subsystem in the many virtual shared systems of I/O comprises a plurality of I/O equipment, they possess the ability that service is provided for a plurality of virtual machines simultaneously, can be to comprise a physical function (Physical Function, PF) and corresponding a plurality of virtual functions (Virtual Function, VF) SR-IOV equipment or comprise a plurality of PF and the SR-IOV equipment of corresponding a plurality of VF can be also multi-functional I/O equipment etc.
The virtual shared control unit bag of many I/O in the many virtual shared systems of I/O is comprised of several PCIe upstream ports (PCIe Upstream Port), the many root switch of PCIe and three parts of several PCIe downstream ports (PCIe Downstream Port).Wherein, the PCIe controller that PCIe upstream port comprises the PCIe port type definition meeting in PCI-Express fundamental norms, it is responsible for the interconnection with root node subsystem, and the conversion of transaction packet between the single environment of PCIe and many environment of PCIe; The many root switch of PCIe, essence is by the switch of N+M port of a plurality of PCI bridges foundation, by set up the Virtual PC Ie switch of a 1+M port for each root node, realizes each root node and is connected with the logic of M I/O equipment; PCIe downstream port, the PCIe controller that comprises the PCIe port type definition meeting in PCI-Express fundamental norms and directly two funtion parts of the virtual interfacing equipment of I/O, be responsible for the interconnection with I/O equipment subsystem, and the conversion of transaction packet between the single environment of PCIe and many environment of PCIe, realize each root node and directly access physical I/O functions of the equipments.
Fig. 2 has described the schematic diagram that a plurality of root nodes are shared a SR-IOV equipment.A plurality of root nodes pass through many root switch of PCIe Share interlinkage at the SR-IOV of PCIe downstream port device resource.When system starts, each root node enumerates by attempting the I/O equipment connecting under the many root switch of PCIe, and essence is to enumerate the same I/O equipment connecting under PCIe downstream port.This will cause system competition, and the modification to SR-IOV equipment of root node, can affect the operation of other root nodes, finally causes system cisco unity malfunction.Therefore, conflict and competition for fear of each system, realizing a SR-IOV device resource can be shared by a plurality of root node simultaneously, the present invention utilizes a main control root node mRN0 to carry out control and management to SR-IOV equipment, and is subordinate root node distributing equipment by equipment I/O resource allocation table.I/O resource allocation table is arranged in each PCIe downstream port of the many virtual shared control units of I/O, the corresponding I/O resource allocation table of each I/O equipment.The functions of the equipments of distributing to it that record in equipment I O resource allocation table can only be found and use to subordinate root node.
As shown in Figure 3, list item has recorded the mapping relations of functions of the equipments and the root node under it to I/O resource allocation table.The corresponding I/O resource allocation table of each equipment of the many virtual shared systems of I/O, wherein table index be functions of the equipments at the identifier (as function number) in main control root node PCIe territory, table clause content description root node sign ID and the signs in subordinate root node PCIe territory thereof under functions of the equipments.By configuration I/O resource allocation table, the I/O resource of an equipment be take to functions of the equipments and be mounted to each root node as unit, and then set up accordingly independently PCIe topological structure for each root node, realized shared to equipment I/O resource of a plurality of root nodes.
As shown in Figure 2, when root node subsystem powers on, root node starts and utilizes the system initialization software of ready-made standard that SR-IOV equipment is searched for and found and initial configuration.Yet, because equipment I O resource allocation table now does not record the assignment information of any functions of the equipments, except main control root node mRN0, all subordinate root nodes (sRN1, sRN2 ..., sRNn) all can not find function in SR-IOV equipment.After system has started, main control root node mRN0 is according to user resources allocation strategy, by configuration device IO resource allocation table recording unit function assignment information, SR-IOV device hardware resource be take to functions of the equipments and distribute to subordinate root node as unit (virtual functions).After device initialize is assigned, subordinate root node is directed carries out the rescanning of PCIe equipment.By equipment I/O resource allocation table, subordinate root node can find to distribute the virtual functions in the SR-IOV equipment obtaining.When subordinate root node completes the rescanning of PCIe equipment, load corresponding virtual functions VF and drive, can use specific virtual functions, and then realize many of SR-IOV device resource and share.
A kind of I/O method for managing resource for the many virtual shared systems of I/O provided by the invention, the resource allocation policy that can formulate according to user, realizes the dynamic assignment of I/O resource between a plurality of root nodes.Described I/O resource dynamic is distributed and is comprised that I/O initializing resource distributes, and I/O resource reclaims and I/O redistribute resources.
Fig. 4 has described and has utilized the process flow diagram of main control root node to the configuration of I/O initializing resource.Described I/O initializing resource configuration, finger is when system starts, main control root node is enumerated all I/O equipment in discovery and initial configuration system, and the I/O resource allocation policy of formulating according to leading subscriber, for each subordinate root node distributing equipment function, the process of setting up initial PCIe topological structure, it comprises the following steps:
First, root node subsystem powers on, and first the root node of startup utilizes the system initialization software of ready-made standard that all PCIe equipment in system are searched for and found and initial configuration.During device initialize, the configuration space of each function in system configuration softward interview equipment, in system No. ID of distribution function, distributes the resources such as corresponding I/O address space, memory address space and interruption simultaneously for it according to its resource requirement.At the system electrifying startup initial stage, only have main control root node can find with initial configuration I/O subsystem in each I/O equipment, and be the I/O resource allocation table allocate memory address that each I/O equipment is corresponding.
Main control root node initializers finishing equipment find and configuration after, all PCIe module informations present to leading subscriber in the PCIe management software collection system of moving it on, auxiliary leading subscriber carries out the formulation of resource allocation policy.The PCIe assembly of described collection includes but not limited to root nodes all in system and all I/O equipment.Described root node information includes but not limited to: the I/O resource that the number of root node, each root node have and the load condition of each root node.Described I/O facility information includes but not limited to: functions of the equipments ID identifier (as B/D/F), functions of the equipments descriptor (comprise vendor id, device id, revision ID etc.), attribute (as multifunctional equipment or SR-IOV equipment etc.) and the functions of the equipments of equipment take situation etc.
Then, leading subscriber is formulated initial allocation strategy according to the demand of each root node and existing I/O resource, and by PCIe management software transmitting apparatus distribution instruction.
PCIe management software receives after instruction, by transmitting apparatus, distributes transaction packet to carry out the resource allocation policy of leading subscriber.Described devices allocation transaction packet, that PCIe internal memory is write transaction packet, No. ID, root node under it has indicated the functions of the equipments that will operate No. ID (in main control root node PCIe territory No. ID), these functions of the equipments with and in affiliated root node PCIe territory No. ID.Wherein, affiliated root node is for No. ID No. ID, subordinate root node, indicates a devices allocation instruction.
PCIe downstream port receives the said equipment and distributes transaction packet, resolve transaction packet obtain No. ID, No. ID, functions of the equipments, the root node under it that will operate with and in affiliated root node PCIe territory No. ID; And by No. ID, the root node by under functions of the equipments and its ID component content in affiliated root node PCIe territory, write in the corresponding list item of I/O resource allocation table of No. ID indication of functions of the equipments that will distribute, and then be that each subordinate root node has been specified the I/O functions of the equipments of distributing.
After main control root node completes the I/O device resource initialization of subordinate root node is distributed, each subordinate root node is directed carries out the rescanning of PCIe equipment.Each subordinate root node is according to I/O resource allocation table, can find to distribute the functions of the equipments that obtain, and for it, is distributed in the resources such as in local PCIe territory No. ID, IO address space, memory address space and interruption according to its resource requirement.
The many virtual shared systems of I/O in the course of the work, can, in the situation that not affecting other root node work, dynamically be carried out system equipment and upgrade upgrading, and existing I/O resource is redistributed.No matter be redistributing of renewal of the equipment upgrading or existing equipments resources, essence is all to delete relevant device function and to root node, increase the process of relevant device function from root node.The invention provides a kind of I/O resource dynamic distributing method, can, in the situation that not affecting other root node work, from subordinate root node, delete I/O functions of the equipments or increase I/O functions of the equipments to subordinate root node.From subordinate root node, delete I/O functions of the equipments and be referred to as the recovery of I/O resource, the process that increases I/O functions of the equipments to subordinate root node is referred to as I/O redistribute resources.
Fig. 5 has described the process flow diagram that I/O resource reclaims, and its process is as described below:
First, it is standby that leading subscriber determines to delete certain equipment merit from certain subordinate root node according to demand, and by PCIe management software transmitting apparatus recovery command.
Then PCIe management software distributes transaction packet by transmitting apparatus, carries out the resource recovery command of leading subscriber.Described devices allocation transaction packet, that PCIe internal memory is write transaction packet, No. ID, root node under it has indicated the I/O functions of the equipments that will operate No. ID (in main control root node PCIe territory No. ID), these functions of the equipments with and in affiliated root node PCIe territory No. ID.Wherein, the root node under function is No. ID, main control root node No. ID, indicates an equipment recovery command.
When PCIe downstream port receiving equipment distributes transaction packet, resolve transaction packet obtain No. ID, No. ID, functions of the equipments, the root node under it that will operate with and in affiliated root node PCIe territory No. ID.And affiliated root node ID is judged, if the affiliated root node ID indicating in configuration transaction bag is main control root node ID, indicate an equipment recovery command.Then, read the list item corresponding to I/O resource allocation table of No. ID indication of functions of the equipments that transaction packet will operate, and send a virtual hot plug event to the subordinate root node of list item indication, notice has I/O functions of the equipments to be pulled out.Meanwhile, by being complete zero functional identity component content by No. ID, main control root node and value, write in the corresponding list item of I/O resource allocation table of No. ID indication of functions of the equipments that will operate.
When target subordinate root node receives after above-mentioned virtual hot plug event, according to system running state, determine the entry-into-force time of this request, then these functions of the equipments are removed from system, and unload relevant driver, the affairs that stop all uncompleted affairs and will generate.
Fig. 6 has described the process flow diagram of I/O redistribute resources, and its process is as described below:
First leading subscriber determines to increase certain functions of the equipments to certain subordinate root node according to demand, and by PCIe management software transmitting apparatus recovery command
Then PCIe management software distributes transaction packet by transmitting apparatus, carries out the redistribute resources instruction of leading subscriber.Described devices allocation transaction packet, that PCIe internal memory is write transaction packet, No. ID, root node under it has indicated the I/O functions of the equipments that will operate No. ID (in main control root node PCIe territory No. ID), these functions of the equipments with and in affiliated root node PCIe territory No. ID.Wherein, the root node under function is for No. ID No. ID, subordinate root node, indicates a devices allocation instruction.
When PCIe downstream port receiving equipment distributes transaction packet, resolve transaction packet obtain No. ID, No. ID, functions of the equipments, the root node under it that will operate with and in affiliated root node PCIe territory No. ID.And affiliated root node ID is judged, if the affiliated root node ID indicating in configuration transaction bag is not main control root node ID, indicate an equipment recovery command.Then, to a virtual hot plug event of subordinate root node transmission of transaction packet indication, notify corresponding subordinate root node to have I/O equipment to insert its PCIe topological structure.Simultaneously, by No. ID, the root node by under functions of the equipments and its ID component content in affiliated root node PCIe territory, write in the corresponding list item of I/O resource allocation table of No. ID indication of functions of the equipments that will operate, and then be that this subordinate root node increases I/O functions of the equipments of distribution.
When target subordinate root node receives after above-mentioned virtual hot plug event, according to system running state, determine the entry-into-force time of this request, then this equipment is scanned and initial configuration, and load corresponding driver.
Those skilled in the art is not departing under the condition of the definite the spirit and scope of the present invention of claims, can also carry out various modifications to above content.Therefore scope of the present invention is not limited in above explanation, but determined by the scope of claims.
Claims (7)
1. for an I/O method for managing resource for the many virtual shared systems of I/O, it is characterized in that, comprising:
Step 1, main control root node is enumerated and is found and initial configuration the I/O resource in system;
Step 2, main control root node is by PCIe administration module, and the I/O device resource in auxiliary management user management and control system, carries out the I/O resource allocation policy that leading subscriber is formulated;
Step 3, main control root node is by PCIe administration module and I/O resource allocation table, and the mapping relations of apparatus for establishing function and the root node under it, realize I/O resource dynamic and distribute; Wherein, I/O resource dynamic is distributed and is comprised: I/O initializing resource distributes, and described I/O initializing resource distributes and refers to when system initialization, and main control root node is each subordinate root node distributing equipment function, sets up the process of initial PCIe topological structure; I/O resource reclaims, and described I/O resource reclaims and refers in the situation that not affecting other root node work, and main control root node is deleted the process of I/O functions of the equipments at subordinate root node; I/O redistribute resources, described I/O redistribute resources refers in the situation that not affecting other root node work, main control root node increases the process of I/O functions of the equipments to subordinate root node.
2. the I/O method for managing resource for the many virtual shared systems of I/O as claimed in claim 1, is characterized in that, described step 2 comprises:
Step 21, all PCIe module informations present to leading subscriber in PCIe administration module collection system, auxiliary leading subscriber initialization or revise resource allocation policy; Wherein, described PCIe assembly comprises root nodes all in system and all I/O equipment;
Step 22, PCIe administration module is according to user's resource allocation policy, and transmitting apparatus distribution instruction realizes the allocation manager of I/O resource.
3. the I/O method for managing resource for the many virtual shared systems of I/O as claimed in claim 2, is characterized in that, described PCIe module information comprises:
Root node information, described root node information, comprises the number of root node, I/O resource that each root node has and the load condition of each root node;
I/O facility information, described I/O facility information, No. ID, functions of the equipments descriptor and functions of the equipments take situation to comprise functions of the equipments.
4. the I/O method for managing resource for the many virtual shared systems of I/O as claimed in claim 1, is characterized in that, described I/O resource allocation table comprises:
Described resource allocation table is arranged in each PCIe downstream port of the many virtual shared control units of I/O, the corresponding I/O resource allocation table of each I/O equipment; Described resource allocation table has recorded the mapping relations of functions of the equipments and the root node under it; Wherein, table index is in main control root node PCIe territory No. ID of functions of the equipments, table clause content description root node sign ID under functions of the equipments and No. ID in affiliated root node PCIe territory thereof.
5. the I/O method for managing resource for the many virtual shared systems of I/O as claimed in claim 1, is characterized in that, described I/O initializing resource distributes and comprises:
Step 61, leading subscriber is formulated initial allocation strategy according to the I/O resource of the demand of root node and system, and by PCIe administration module transmitting apparatus distribution instruction;
Step 62, PCIe administration module distributes transaction packet by transmitting apparatus, carries out the resource allocation policy of leading subscriber; Described devices allocation transaction packet, that PCIe internal memory is write transaction packet, described PCIe internal memory is write transaction packet and has been indicated the functions of the equipments that will operate No. ID, the sign of the root node under No. ID in main control root node PCIe territory, these functions of the equipments ID with and in affiliated root node PCIe territory No. ID; Wherein, affiliated root node is for No. ID No. ID, subordinate root node, indicates a devices allocation instruction;
Step 63, PCIe downstream port receives the transaction packet of devices allocation described in step 62, resolve transaction packet obtain No. ID, the functions of the equipments that will operate, the sign of the root node under it ID with and in affiliated root node PCIe territory No. ID; And by root node sign ID and its ID component content in affiliated root node PCIe territory by under functions of the equipments, write in the corresponding list item of I/O resource allocation table of No. ID indication of functions of the equipments that will operate, and then be that each subordinate root node has been specified the I/O functions of the equipments of distributing;
Step 64, after main control root node completes the I/O device resource initialization of subordinate root node is distributed, the directed rescanning of PCIe equipment, the functions of the equipments that discovery and initial configuration distribution obtain of carrying out of subordinate root node.
6. the I/O method for managing resource for the many virtual shared systems of I/O as claimed in claim 1, is characterized in that, described I/O resource reclaims and comprises:
Step 71, leading subscriber determines to delete certain functions of the equipments from certain root slave node according to demand, by PCIe management software transmitting apparatus recovery command;
Step 72, PCIe administration module distributes transaction packet by transmitting apparatus, carries out the resource recovery command of leading subscriber; Described devices allocation transaction packet, is that PCIe internal memory is write transaction packet, described PCIe internal memory write things bag indicated No. ID, root node under No. ID, the I/O functions of the equipments that will operate, these functions of the equipments with and in affiliated root node PCIe territory No. ID; Wherein, the root node sign ID under function is main control root node sign ID, indicates an equipment recovery command;
Step 73, PCIe downstream port receiving equipment distributes transaction packet, resolve transaction packet obtain No. ID, the functions of the equipments that will operate, the sign of the root node under it ID with and in affiliated root node PCIe territory No. ID;
Step 74, affiliated root node sign ID judges; If the affiliated root node sign ID indicating in configuration transaction bag is main control root node sign ID, indicate an equipment recovery command;
Step 75, reads the list item corresponding to I/O resource allocation table of No. ID indication of functions of the equipments that transaction packet will operate, and to the subordinate root node of list item indication, sends a virtual hot plug event, and notice has I/O functions of the equipments to be pulled out;
Step 76, the corresponding list item of I/O resource allocation table of No. ID indication of functions of the equipments of reset operation; By being complete zero functional identity component content by main control root node sign ID and value, write in the corresponding list item of I/O resource allocation table of No. ID indication of functions of the equipments that will operate;
Step 77, target subordinate root node receives this hot plug event, this equipment is removed from PCIe topological structure, and unload corresponding driver.
7. the I/O method for managing resource for the many virtual shared systems of I/O as claimed in claim 1, is characterized in that, described I/O redistribute resources comprises:
Step 81, leading subscriber determines according to demand and indicates that to certain subordinate root node, to increase certain equipment merit standby, by PCIe management software transmitting apparatus reallocation instruction;
Step 82, PCIe administration module distributes transaction packet by transmitting apparatus, carries out the redistribute resources instruction of leading subscriber; Described devices allocation transaction packet, that PCIe internal memory is write transaction packet, described PCIe internal memory is write transaction packet and has been indicated the I/O functions of the equipments that will operate No. ID, the sign of the root node under No. ID in main control root node PCIe territory, these functions of the equipments ID with and in affiliated root node PCIe territory No. ID; Wherein, the root node sign ID under function is subordinate root node sign ID, indicates a devices allocation instruction;
Step 83, PCIe downstream port receiving equipment distributes transaction packet, resolve transaction packet obtain No. ID, the functions of the equipments that will operate, the sign of the root node under it ID with and in affiliated root node PCIe territory No. ID;
Step 84, affiliated root node sign ID judges; If the affiliated root node ID indicating in configuration transaction bag is non-main control root node ID, indicate a devices allocation instruction;
Step 85, to a virtual hot plug event of subordinate root node transmission of transaction packet indication, notifies corresponding subordinate root node to have I/O equipment to insert its PCIe topological structure; Simultaneously, by root node sign ID and its ID component content in affiliated root node PCIe territory by under functions of the equipments, write in the corresponding list item of I/O resource allocation table of No. ID indication of functions of the equipments that will operate, and then be that this subordinate root node increases I/O functions of the equipments of distribution;
Step 86, target subordinate root node receives hot plug event, this equipment is scanned and initial configuration, and load corresponding driver.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210160593.9A CN102722414B (en) | 2012-05-22 | 2012-05-22 | Input/output (I/O) resource management method for multi-root I/O virtualization sharing system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210160593.9A CN102722414B (en) | 2012-05-22 | 2012-05-22 | Input/output (I/O) resource management method for multi-root I/O virtualization sharing system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102722414A CN102722414A (en) | 2012-10-10 |
| CN102722414B true CN102722414B (en) | 2014-04-02 |
Family
ID=46948191
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210160593.9A Active CN102722414B (en) | 2012-05-22 | 2012-05-22 | Input/output (I/O) resource management method for multi-root I/O virtualization sharing system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102722414B (en) |
Families Citing this family (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140258577A1 (en) * | 2013-03-11 | 2014-09-11 | Futurewei Technologies, Inc. | Wire Level Virtualization Over PCI-Express |
| US9734096B2 (en) | 2013-05-06 | 2017-08-15 | Industrial Technology Research Institute | Method and system for single root input/output virtualization virtual functions sharing on multi-hosts |
| US9672167B2 (en) * | 2013-07-22 | 2017-06-06 | Futurewei Technologies, Inc. | Resource management for peripheral component interconnect-express domains |
| CN103701881B (en) * | 2013-12-18 | 2017-03-22 | 中国科学院计算技术研究所 | Virtual hotplug system for supporting input/output (I/O) function dynamic distribution and working method thereof |
| KR102209525B1 (en) * | 2014-01-06 | 2021-01-29 | 삼성전자주식회사 | Server system, method for mac address allocation and computer-readable recording medium |
| TWI556174B (en) * | 2014-03-05 | 2016-11-01 | 威盛電子股份有限公司 | System and method for assigning virtual functions and management host thereof |
| CN104123173B (en) * | 2014-07-22 | 2017-08-25 | 华为技术有限公司 | A kind of method and device for realizing inter-virtual machine communication |
| CN105446433A (en) * | 2014-08-22 | 2016-03-30 | 北京计算机技术及应用研究所 | PCI-E board card sharing box and implementation method thereof |
| CN104601684A (en) * | 2014-12-31 | 2015-05-06 | 曙光云计算技术有限公司 | Cloud server system |
| CN106033399B (en) * | 2015-03-20 | 2020-05-15 | 中兴通讯股份有限公司 | PCIe equipment and PCIe bus management method and device |
| CN106302322B (en) * | 2015-05-19 | 2020-05-26 | 腾讯科技（深圳）有限公司 | Virtual machine data flow management method and system |
| CN105721357B (en) * | 2016-01-13 | 2019-09-03 | 华为技术有限公司 | Switching equipment, peripheral parts interconnected High Speed System and its initial method |
| CN107145378A (en) * | 2016-03-01 | 2017-09-08 | 中兴通讯股份有限公司 | A kind of method for managing resource and device |
| TWI616759B (en) * | 2016-08-10 | 2018-03-01 | 創義達科技股份有限公司 | Apparatus assigning controller and apparatus assigning method |
| CN108958884B (en) * | 2018-06-22 | 2022-02-18 | 郑州云海信息技术有限公司 | Virtual machine management method and related device |
| CN109190420B (en) * | 2018-09-11 | 2020-08-25 | 网御安全技术（深圳）有限公司 | Server encryption and decryption blade, system and encryption and decryption method |
| CN111158849A (en) * | 2019-12-09 | 2020-05-15 | 中国船舶重工集团公司第七一六研究所 | Multi-computer computing device supporting virtual machines to share IO equipment and IO equipment sharing method thereof |
| CN113297124B (en) * | 2020-04-08 | 2022-05-06 | 阿里巴巴集团控股有限公司 | Device enumeration method, device, system and storage medium |
| CN111522692B (en) * | 2020-04-20 | 2023-05-30 | 浙江大学 | Multi-operating-system input and output equipment redundancy guarantee system based on virtual machine |
| CN113296884B (en) * | 2021-02-26 | 2022-04-22 | 阿里巴巴集团控股有限公司 | Virtualization method, virtualization device, electronic equipment, virtualization medium and resource virtualization system |
| CN117236965A (en) * | 2021-03-09 | 2023-12-15 | 支付宝（中国）网络技术有限公司 | Payment processing method and system |
| CN114428757A (en) * | 2021-12-06 | 2022-05-03 | 中国船舶重工集团公司第七一六研究所 | Computing device with reconfigurable architecture and reconfiguration method thereof |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN100502370C (en) * | 2006-07-25 | 2009-06-17 | 中山大学 | Optimizing system and method for medium transmission on different transmission channels |
| CN101419561A (en) * | 2007-10-26 | 2009-04-29 | 中兴通讯股份有限公司 | Resource management method and system in isomerization multicore system |
-
2012
- 2012-05-22 CN CN201210160593.9A patent/CN102722414B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN102722414A (en) | 2012-10-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102722414B (en) | Input/output (I/O) resource management method for multi-root I/O virtualization sharing system | |
| CN102707991B (en) | The many virtual shared method and systems of I/O | |
| CN102819447B (en) | Direct I/O virtualization method and device used for multi-root sharing system | |
| US8281303B2 (en) | Dynamic ejection of virtual devices on ejection request from virtual device resource object within the virtual firmware to virtual resource driver executing in virtual machine | |
| JP5266590B2 (en) | Computer system management method, computer system, and program | |
| US8386654B2 (en) | System and method for transforming PCIe SR-IOV functions to appear as legacy functions | |
| US8341327B2 (en) | Computer system and method for sharing PCI devices thereof | |
| CN100573481C (en) | Configurable PCI Express switch and control method thereof | |
| US8683110B2 (en) | I/O system and I/O control method | |
| US20180189109A1 (en) | Management system and management method for computer system | |
| CN104285218A (en) | Method and device for expanding PCIe bus region | |
| CN104767838B (en) | Micro server, method of assigning MAC address, and computer-readable recording medium | |
| US20110029693A1 (en) | Reserving pci memory space for pci devices | |
| CN103353861B (en) | Realize method and the device of distributed I/O resource pool | |
| CN104615572B (en) | Hot plug processing system and method | |
| KR20070003954A (en) | Simulation circuit of pci express endpoint and downstream port for a pci express switch | |
| CN105320628A (en) | Adaptation device, system and method for enabling single I/O device to be shared by multiple root nodes | |
| CN102722458A (en) | I/ O (input/output) remapping method and device for plurality of shared systems | |
| CN105302248A (en) | Chip set and server system | |
| CN104021047A (en) | Virtual function distribution system, method and management host thereof | |
| CN104298535A (en) | Extended read-only memory management system, method and management host thereof | |
| US10318460B2 (en) | UMA-aware root bus selection | |
| TWI616759B (en) | Apparatus assigning controller and apparatus assigning method | |
| CN109656675A (en) | Bus apparatus, computer equipment and the method for realizing physical host cloud storage | |
| CN112148467A (en) | Dynamic allocation of computing resources |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |