CN104123173B

CN104123173B - A kind of method and device for realizing inter-virtual machine communication

Info

Publication number: CN104123173B
Application number: CN201410351456.2A
Authority: CN
Inventors: 谭锐
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2014-07-22
Filing date: 2014-07-22
Publication date: 2017-08-25
Anticipated expiration: 2034-07-22
Also published as: CN104123173A

Abstract

The embodiment of the invention discloses a kind of method and system for realizing inter-virtual machine communication, the hardware resource consumption for reducing PCIe device, the efficiency high communicated between VM.Present invention method includes：The DMA engine reads data and by manager described in the data Cun Chudao after the reception for receiving the PF processors is instructed from described receive on the corresponding VM of instruction；The DMA engine is after the transmission for receiving the PF processors is instructed, from the manager reads data and the corresponding VM of instruction will be sent described in the data Cun Chudao.

Description

A kind of method and device for realizing inter-virtual machine communication

Technical field

The present invention relates to communication technical field, and in particular to a kind of method and device for realizing inter-virtual machine communication.

Background technology

In physical machine, typically pass through void in CPU (Central Processing Unit, abbreviation CPU) Intend monitor unit (Virtual Machine Manager, abbreviation VMM) and generate multiple virtual machines (Virtual Machine, letter Claim VM), each VM is operated in a completely isolated environment, it can be understood as multiple virtual meters are simulated in physical machine Calculation machine, virtual machine can be operated such as virtual computer as real computer.

PCIe (can also be called PCI-Express, full name is Peripheral Component Interface Express) it is newest bus and interface standard, main advantage is that message transmission rate is high, and it has increased single node newly Input/output virtualization specification (Single Root I/O Virtualization, abbreviation SR-IOV) technology, it is possible to achieve Multiple such as VM share network interface card PCIe device and CPU connection.PCIe device is CPU ancillary equipment, and SR-IOV specifications allow One PCIe device obtains PCIe for possessing its all physical function by virtualization technology, by PCIe device division and set Standby, referred to as physical function (Physical Function, abbreviation PF) generates multiple virtual functions by virtualization technology (Virtual Function, abbreviation VF), each VF is considered as a virtual PCIe device, wherein, PF is except possessing The institute of PCIe device is functional, is also equipped with the configuration and administration feature of SR-IOV extensions；VF depends on some PF, possesses PCIe The lightweight function of equipment, including resource necessary to data transfer and a small amount of configuration resource.Can be by one or more VF A VM is distributed to, and then each VM is it can be seen that an independent virtual PCIe device, i.e. VF.That is on CPU One VM of operation can correspond to a VF in PCIe device.

For example, connecting CPU and PCIe device by PCIe buses, one or more of PCIe device VF is distributed to A VM in CPU, now multiple VM can share PCIe device.When needing data interaction between multiple VM, it is necessary to pass through Communicate and realize between VF, i.e., one VM data send into PCIe device VF corresponding with the VM from CPU via PCIe interface, And purpose VF is issued by VF, then the purpose VM into CPU is sent via PCIe interface by purpose VF.Because VF only possesses data biography Resource and a small amount of configuration resource necessary to defeated, therefore, VF communication need to carry out configuration management by PF, such as, judge The ability with communication is needed between two VF or regulation of overall importance etc. is carried out.Existing SR-IOV specifications are not provided What PF communicated with VF implements mechanism.As shown in figure 1, N number of VM is generated by VMM in physical machine, and including one PCIe managers and PCIe compound interfaces, VMM are connected with VM and PCIe managers respectively, and the VMM is connected to that PCIe is compound to be connect On mouth；Network interface card includes a PF, N number of VF and PCIe terminal interfaces, and PCIe buses are connected with PF and VF respectively, and CPU PCIe compound interfaces are connected on PCIe terminal interfaces by PCIe buses.Distribute a set of hard-wired for each VF Mailbox (mailbox) is cached and Door Bell (doorbell) registers realize the communication between PF and VF, i.e., be every in network interface card Individual VF configures a Mailbox caching and a Door Bell register, realizes the mechanism of a kind of similar " bell of knocking at the door ".If VF There are data to need transmission, then data to be transmitted directly into PF, then again by the mechanism of " bell of knocking at the door " between PF and VF, by number According to being transmitted to purpose VF.

In above-mentioned embodiment illustrated in fig. 1, a PF correspond to N number of VF, when VF needs to send data, just directly to PF Data are sent, are likely to result in while there are multiple VF to give PF transmission data, and PF limited memories are likely to result in loss of data, in addition, Because the Mailbox memory sizes cached are limited to network interface card memory size, if the length communicated between PF and VF is more than 64bytes, and network interface card internal memory is less than 64bytes, it is necessary to which data are split into repeatedly, reduces systematic function.

The content of the invention

, can the embodiments of the invention provide a kind of method and device for realizing inter-virtual machine communication for drawbacks described above The hardware resource consumption of PCIe device is reduced, the transmission of big data between VM is supported, the efficiency and reliability communicated between VM is improved.

First aspect present invention provides a kind of method for realizing inter-virtual machine communication, applied to bus and interface standard In PCIe device, the PCIe device is connected with physical machine by PCIe buses, and the physical machine includes multiple virtual machine VM With a manager, the multiple VM and manager are realized in the physical machine by virtualization technology；The PCIe device Include at direct memory access engine, a physical function PF processors and multiple virtual functions VF processors, the PF Device and multiple VF processors is managed to realize in the PCIe device by virtualization technology；Wherein, a VM associates one Or multiple VF processors, the manager association PF processors；Methods described includes：

The DMA engine is read after the reception for receiving the PF processors is instructed from described receive on the corresponding VM of instruction Access according to and by manager described in the data Cun Chudao；

The DMA engine reads data and by institute after the transmission for receiving the PF processors is instructed from the manager State and sent described in data Cun Chudao in the corresponding VM of instruction.

With reference in a first aspect, in the first possible implementation, the DMA engine is receiving the PF processors Include before receiving instruction：The PF processors receive the first announcement information that the DMA engine is sent, and described first notifies letter Cease is used to notify the PF processors to have VM to need to send data for the DMA engine；The PF processors detect the management Whether device is ready to receive the data that the corresponding VM of the VF processors is sent；If so, the PF processors are to the DMA engine Send and receive instruction, it is described to receive the VM that instruction includes needing to send data.

With reference to the first possible implementation of first aspect, in second of possible implementation, at the PF The data whether the reason device detection manager gets out receive the corresponding VM transmissions of the VF processors include：The PF processing Device detects in the manager whether allocated order caching is idle to determine whether the manager is ready to receive described Allocated order caching is used for data storage in the data that the corresponding VM of VF processors is sent, the manager.

With reference in a first aspect, or first aspect the first possible implementation, or second of first aspect may Implementation, in the third possible implementation, the transmission caching from the manager reads data and by institute Include after stating in the order caching that the corresponding VM of instruction is sent described in data Cun Chudao：The DMA engine is to the PF processing Device sends the first response message, and first response message is used to notify the PF processors, successfully instructs described receive In manager described in the data Cun Chudao that corresponding VM is sent.

With reference in a first aspect, or first aspect the first possible implementation, or second of first aspect may Implementation, or the third possible implementation, in the 4th kind of possible implementation, the DMA engine is being received Include after the transmission instruction of the PF processors：The DMA engine detection is described to send whether the corresponding VM of instruction is ready to Receive the data；Read data if so, performing from the manager and instruction correspondence will be sent described in the data Cun Chudao VM in step.

With reference to the 4th kind of possible implementation, in the 5th kind of possible implementation, the DMA engine detection is described Whether send the corresponding VM of instruction and be ready to receive the data includes：The DMA engine detection transmission instruction is corresponding Whether allocated order caching is idle to determine that described send instructs whether corresponding VM is ready to receive the data in VM, Described send instructs allocated order caching in corresponding VM to be used for data storage.

With reference to the 4th kind of possible implementation, or the 5th kind of possible implementation, the 6th kind of possible realization side In formula, it is described from the manager read data and in the corresponding VM of instruction being sent described in the data Cun Chudao after wrap Include：The DMA engine sends the second response message to the PF processors, and second response message is used to notify at the PF Reason device has succeeded to read data from the manager and store to described send to be instructed in corresponding VM；The DMA engine to The associated VF processors of the corresponding VM of instruction that send send the second announcement information, and second announcement information is used to notify It is described to send the associated VF processors of the corresponding VM of instruction, it will successfully send and referred to described in the data Cun Chudao of the manager Make in corresponding VM.

Second aspect of the embodiment of the present invention provides a kind of direct memory access engine, applied to bus and interface In standard PCIe device, the PCIe device is connected with physical machine by PCIe buses, and the physical machine includes multiple virtual Machine VM and a manager, the multiple VM and manager are realized in the physical machine by virtualization technology；Described PCIe device includes the DMA engine, a physical function PF processors and multiple virtual functions VF processors, the PF processing Device and multiple VF processors realized in the PCIe device by virtualization technology, wherein, a VM associate one or Multiple VF processors, the manager associates the PF processors, and the DMA engine includes：

First processing module, it is corresponding from the reception instruction after being instructed in the reception for receiving the PF processors Data are read on VM and by manager described in the data Cun Chudao；

Second processing module, after being instructed in the transmission for receiving the PF processors, data are read from the manager And instructed being sent described in the data Cun Chudao in corresponding VM.

With reference to second aspect, in the first possible implementation, the DMA engine also includes：First respond module, For to the PF processors send the first response message, first response message be used for notify the PF processors, into Work(is by manager described in the data Cun Chudao for receiving and instructing corresponding VM to send.

With reference to the first possible implementation of second aspect, or second aspect, in second of possible implementation In, the DMA engine also includes：Detection module, for detecting that described send instructs whether corresponding VM is ready to receive described Data.

With reference to second of possible implementation of second aspect, in the third possible implementation, the DMA draws Holding up also includes：Second respond module, for sending the second response message to the PF processors, second response message is used for Notify the PF processors have succeeded to read from the manager to be sent described in data Cun Chudao in the corresponding VM of instruction；With, To VF processors second announcement information of transmission for sending and instructing corresponding VM associated, second announcement information is used to lead to Know the VF processors for sending and instructing corresponding VM associated, will successfully be sent described in the data Cun Chudao of the manager Instruct in corresponding VM.

The third aspect of the embodiment of the present invention additionally provides a kind of system for realizing inter-virtual machine communication, it may include：Bus and Interface standard PCIe device and physical machine, the PCIe device are connected with physical machine by PCIe buses, are wrapped in the physical machine Multiple virtual machine VM and a manager are included, the multiple VM and manager are realized in the physical machine by virtualization technology In；The PCIe device includes a physical function PF processors, multiple virtual functions VF processors and direct memory access (DMA) DMA engine, the PF processors and multiple VF processors are realized in the PCIe device by virtualization technology；Wherein, one The individual VM associates one or more VF processors, and the manager associates the PF processors；

The DMA engine includes：

Second processing module, after being instructed in the transmission for receiving the PF processors, data are read from the manager And instructed being sent described in the data Cun Chudao in corresponding VM；

The PF processors are used to send the reception instruction to the DMA engine and described send instructs.

With reference to the third aspect, in the first possible implementation, the PF processors are additionally operable to：Receive the DMA The first announcement information that engine is sent, first announcement information is that the DMA engine is used to notify the PF processors to have VM Need to send data；Detect whether the manager is ready to receive the data that the corresponding VM of the VF processors is sent；If so, Then perform to the DMA engine and send the step of reception instruction and the transmission instruction, the reception, which is instructed, includes needs Send the VM of data.

With reference to the first possible implementation of the third aspect, or the third aspect, in second of possible implementation In, the PF processors are additionally operable to：Detect in the manager whether allocated order caching is idle to determine the management Whether device is ready to receive the data that the corresponding VM of the VF processors is sent, and the transmission instructs allocated in corresponding VM Order caching is used for data storage.

With reference to second of possible implementation of the third aspect, in the third possible implementation, the DMA draws Holding up also includes：First respond module, for sending the first response message to the PF processors, first response message is used for The PF processors are notified, successfully by manager described in the data Cun Chudao for receiving and instructing corresponding VM to send.

With reference to the third aspect, in the 4th kind of possible implementation, the DMA engine also includes：Detection module, is used for Detection is described to send whether the corresponding VM of instruction is ready to receive the data.

With reference to the 4th kind of possible implementation of the third aspect, in the 5th kind of possible implementation, the DMA draws Holding up also includes：Second respond module, for sending the second response message to the PF processors, second response message is used for Notify the PF processors have succeeded to read from the manager to be sent described in data Cun Chudao in the corresponding VM of instruction；With, To VF processors second announcement information of transmission for sending and instructing corresponding VM associated, second announcement information is used to lead to Know the VF processors for sending and instructing corresponding VM associated, will successfully be sent described in the data Cun Chudao of the manager Instruct in corresponding VM.

As can be seen from the above technical solutions, a kind of method for realizing inter-virtual machine communication provided in an embodiment of the present invention and Device has advantages below：By direct memory access (DMA) (Direct Memory Access, abbreviation DMA) engine in PF processing Directly reading and storing for data is realized between the manager of device association and the VM of VF relational processors, so that between realizing VM Communication, compared with prior art, without that, to PF processors and the corresponding caching of VF processors distribution, can subtract in PCIe device The consumption of few PCIe device hardware resource, and big data can be directly transmitted, effectively improve the efficiency communicated between VM.

Brief description of the drawings

In order to illustrate the technical solution of the embodiments of the present invention more clearly, will make below to required in the embodiment of the present invention Accompanying drawing is briefly described, it should be apparent that, drawings in the following description are only some embodiments of the present invention, for For those of ordinary skill in the art, on the premise of not paying creative work, other can also be obtained according to these accompanying drawings Accompanying drawing.

The virtual machine communications applications schematic diagram that Fig. 1 provides for prior art of the present invention；

Fig. 2 a are the method flow schematic diagram provided in an embodiment of the present invention for realizing inter-virtual machine communication；

Fig. 2 b realize schematic diagram for the method provided in an embodiment of the present invention for realizing inter-virtual machine communication；

Fig. 3 is virtual machine communications applications schematic diagram provided in an embodiment of the present invention；

The method flow schematic diagram for realizing inter-virtual machine communication that Fig. 4 provides for another embodiment of the present invention；

The method flow schematic diagram for realizing inter-virtual machine communication that Fig. 5 provides for another embodiment of the present invention；

Fig. 6 is DMA engine structural representation provided in an embodiment of the present invention；

Fig. 7 a are the structural representation of the system provided in an embodiment of the present invention for realizing inter-virtual machine communication；

Fig. 7 b are the systematic difference schematic diagram provided in an embodiment of the present invention for realizing inter-virtual machine communication.

Embodiment

The embodiments of the invention provide a kind of method and device for realizing inter-virtual machine communication, for reducing PCIe device Hardware resource consumption, improves the communication efficiency between VM.

Below in conjunction with the accompanying drawing of the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out clear, complete Ground is described, it is clear that described embodiment is only a part of embodiment of the invention, rather than whole embodiments.Based on this Embodiment in invention, the every other reality that those of ordinary skill in the art are obtained under the premise of creative work is not made Example is applied, the scope of protection of the invention is belonged to.

Term " first ", " second ", " the 3rd " " in description and claims of this specification and above-mentioned accompanying drawing The (if present)s such as four " are for distinguishing similar object, rather than for describing specific order or precedence.It should manage The data that solution is so used can be exchanged in the appropriate case, so as to embodiments of the invention described herein for example can with except Order beyond those for illustrating or describing herein is implemented.In addition, term " comprising " and " having " and theirs is any Deformation, it is intended that covering is non-exclusive to be included, for example, containing the process of series of steps or unit, method, system, production Product or equipment are not necessarily limited to those steps clearly listed or unit, but may include not list clearly or for this A little processes, method, product or the intrinsic other steps of equipment or unit.

The present invention will be described in detail with specific embodiment below：

Fig. 2 a are referred to, Fig. 2 a are the method flow schematic diagram provided in an embodiment of the present invention for realizing inter-virtual machine communication； And combination Fig. 2 b, Fig. 2 b realize schematic diagram for the method provided in an embodiment of the present invention for realizing inter-virtual machine communication；Such as Fig. 2 a It is shown, a kind of method for realizing inter-virtual machine communication, applied in bus as shown in Figure 2 b and interface standard PCIe device, institute State PCIe device to be connected by PCIe buses with physical machine, the physical machine includes N number of VM and manager, N number of VM and Manager is realized in physical machine by virtualization technology；The PCIe device includes DMA engine, and in the PCIe device One physical function PF processors of middle configuration and M virtual functions VF processor, PF processors and VF processors are by virtualizing skill Art realized in PCIe device, wherein, N and M are the positive integer more than or equal to 1；One VM association is one or more VF processors, the manager associates the PF processors.

As shown in Figure 2 a, a kind of method for realizing inter-virtual machine communication specifically may include：

S201, the DMA engine receive the corresponding VM of instruction after the reception for receiving the PF processors is instructed from described It is upper to read data and by manager described in the data Cun Chudao；

Connected between PCIe device and physical machine by PCIe buses, DMA engine is connected with PF processors and VF processors Connect, control data between carrying out VM to be directly accessed by DMA engine.

The VM at least including needing to send data in instruction is received, illustrates that the VM has been already prepared to the number of needs transmission According to while manager is ready for receiving the preparation of data, DMA engine can read data storage from the corresponding VM of instruction is received Into manager.

Therefore, before step S201, PF processors make following processing：

A1, the PF processors receive the first announcement information that the DMA engine is sent, and first announcement information is institute Stating DMA engine is used to notify the PF processors to have VM to need to send data；

A2, the PF processors detect whether the manager is ready to receive what the corresponding VM of the VF processors was sent Data；

A3, if so, the PF processors to the DMA engine send receive instruction, it is described receive instruction include need hair Send the VM of data.

The data that receive on the corresponding VM of instruction are successfully being read, and are being stored to after manager, DMA engine is to PF processing Device sends the first response message, and first response message is used to notify PF processors, and it successfully instructs the reception corresponding In the data Cun Chudao managers that VM is sent.

S202, the DMA engine read the number after the transmission for receiving the PF processors is instructed from the manager According to and by the corresponding VM of transmission instruction described in the data Cun Chudao.

The executive agent of the embodiment of the present invention is DMA engine, and DMA engine will read number from the corresponding VM of instruction is received According to, and store data into manager；If DMA engine receives the transmission instruction of PF processors transmission, then will be from pipe Data are read in reason device, and are stored data into the corresponding VM of transmission instruction.The embodiment of the present invention is used as transfer using manager Point, is directly accessed by DMA engine between VM and manager, and then realizes the communication between VM, and will not be limited In the hardware resource of PCIe device, the processing of big data is supported, efficiency of transmission is higher.

After DMA engine receives the transmission instruction of the PF processors, DMA engine will also detect transmission instruction correspondence VM whether be ready to receive data, if so, then from manager read data and by the data Cun Chudao send instruction correspondence VM in.

Data in manager is successfully read, and storing to sending after the corresponding VM of instruction, DMA engine will also be to PF at Manage device send the second response message, for notify PF processors its succeed from manager read data and store refer to transmission Make in corresponding VM；The second announcement information is sent to the associated VF processors of the VM for receiving data again, for notifying to receive number According to the associated VF processors of VM, it is successfully by the data Cun Chudao of manager its associated VM.

In the preferred embodiment of the invention, each VM distributes two block address continuous memory headroom conduct in its internal memory VF is cached, and such as order caching and transmission are cached.Wherein, order caching is used to receive the data from manager, sends caching The data of manager are sent to for storing.Correspondingly, also distribute two block address continuous memory headroom conduct in manager PF is cached, and is equally respectively designated as order caching and sends to cache.VM order caching and the size of transmission caching can be more than 64Bytes, correspondingly, order caching in manager and sends the size of caching also greater than 64Bytes, is received in practical application The size of caching and transmission caching depends on the size of physical machine internal memory.Due to manager as VM communicate between the transfer that communicates Point, therefore, it can the order caching of first allocation manager and sends cache, slow according to order caching in manager and transmission afterwards The size deposited caches to distribute the order caching in all VM and send.

Meanwhile, it is the PF processors register group related to each VF processors configuration.Wherein, PF processors are configured Register group and its basic function are as follows：

A1、PF_Mailbox_Size：PF processors can be written and read operation, can be supported most by the write-in of PF processors Big cache size.

A2、PF_RxMailbox_Addr：PF processors can be written and read operation, be that PF processors are distributed in its internal memory Initial address (initial address of data storage in order caching i.e. in manager) for storing the data from VF.

A3、PF_TxMailbox_Addr：PF processors can be written and read operation, be that PF processors are distributed in its internal memory, The initial address of VF data (initial address that caching is sent in manager) is sent to for storing.

A4、PF_TxLength：PF processors can be written and read operation, be that PF processors are put into data in transmission caching Effective length value.

A5、Dest_VF：PF processors can be written and read operation, be write by PF processors, be to send the data hair in caching Past target VF.

A6、VF_Req_Sts：PF can be read, and can also write 0 clearing, be the status register of a multidigit, Each one VF processor of correspondence.When VF processors set some register of corresponding VF processors in VF request registers After being set to 1, an interruption can be triggered and issue PF.PF processors are by reading this register it is known which VF processor prepares to send out Data are sent to oneself.

A7、PF_Rx_En：PF can be read and write 1 set operation, be the control register of a multidigit, each Position one VF processor of correspondence.When it is the free time that PF processors, which determine the order caching of manager, it is determined that from some VF processor Receive data, it is necessary to which certain corresponding bit register is set to 1 (can be while also post correspondence in PF_Rx_Done registers Storage is reset), to notify DMA engine to start to read data to PF processing since the VM of VF relational processors transmission caching In the order caching of the manager of device association.Completed once DMA engine is read, DMA engine needs the corresponding position this register Reset, (show to have had been placed in number while following PF is finished receiving certain corresponding position of register PF_Rx_Done and is set to 1 According to needing PF processors to be handled).

It is worth noting that, one in PF_Rx_En can only be once set to 1, i.e. DMA engine once by PF processors Data can only be read from a VF processor.

A8、PF_Rx_Done：PF processors can be read and can write 0 clear operation, be the shape of a multidigit State register, each one VF processor of correspondence.When number of the DMA engine the transmission caching from some VF relational processor After having been finished according to reading, one can be sent interrupt and give PF processors, while the corresponding positions of this register are set to 1.PF processing Device can read this register to know whether the data in its order caching are effective.PF processors take the data in order caching Walk or using finishing after, the corresponding positions of this register can be set to 0, so as to prepare to receive new data.

A9、PF_Rx_Length：PF processors can be read, when reception of the DMA engine storage to manager The effective length value of data in caching is written to wherein, and the value in this register is only effective when PF_Rx_Done is 1.

A10、PF_Req：PF processors can be read with writable 1 set, but can not reset.Once PF processing Device has intactly been put into data to be sent after transmission caching, target VF is written in Dest_VF, transmission data Length is written to after PF_TxLength, this register can be set to 1, notifies DMA engine PF processors to have data to need hair Send.When DMA engine finds that corresponding target VF PF_Done registers are 0, the reading of data can be proceeded by.Once read Take into, DMA engine resets PF_Req, while the PF_Done of correspondence VF processors is set to the PF_ of 1, VF processors The actual data length value read of TxLength write-ins., can be to connecing after PF processors find that this PF_Req register is cleared Receive in caching and be put into new data.

Correspondingly, it is that register group and its basic function that VF is configured are as follows：

B1、PF_Mailbox_Size：VF processors can be read, and be the PF_Mailbox_ in PF processors Size mirror image, a simply read-only register, for informing the cache size that VF processor PF processors are supported；

After the manager of PF relational processors distributes cache size, its sizes values is written to PF_Mailbox_ In Size, each VM obtains manager by the mirror image PF_Mailbox_Size for reading to go in the VF processors of each auto correlation and set Cache size, so as to the correct size of data for setting VM to be sent to PF, it is to avoid more than the cache size that PF processors are distributed There is loss of data.

B2、VF_RxMailbox_Addr：VF processors can be written and read operation, be that VF processors are distributed in its internal memory, Store the caching initial address (i.e. the initial address of order caching) from PF processor datas.

B3、VF_TxMailbox_Addr：VF processors can be written and read operation, be that VF processors are distributed in its internal memory, The caching initial address (initial address for sending caching) of PF processor datas is sent to for depositing.

B4、VF_TxLength：VF processors can be written and read operation, be that VF processors are put into transmission caching The effective length value of data.

B5、VF_Req：VF processors can be read with writeable 1 set, but can not reset.Once VF processors Data to be sent have intactly been put into after transmission caching, this register 1 can be set to, hair is interrupted to trigger one PF processors are sent to, to notify PF processors to receive data.This register will receive data in PF processors, to it PF_Rx_Done registers are removed after resetting.After VF processors find that this register is cleared, show that sending caching can be put into New data.

B6、PF_Done：VF processors can be read to reset with writeable 0, but VF processors can not write 1 set.When DMA engine is the digital independent in the transmission caching in the manager from PF relational processors into the order caching of the VF Afterwards, this register can be put 1, VF processors is sent to trigger an interruption, to notify VF processors to receive data；Or Whether VF processors by timing to be 1 judge whether to need to receive data if inquiring about this register.In VF processors reception After data in caching have been used, it can be reset to this register write 0, so as to receive new data.

B7、PF_TxLength：VF processors, which can only be read, to be write, when DMA engine caches the transmission for carrying out manager In digital independent into the order caching of the VF after, the length value specifically write can be written in this register, so as to VF Processor is taken.

It is worth noting that, the value in PF_TxLength registers is simply effective when PF_Done is 1.

Below by taking X86 platforms as an example, the present invention is described further.Specifically as shown in figure 3, CPU leads in X86 platforms Cross PCIe buses to be connected with PCIe device, N number of VM, and configuration manager, respectively each VM and pipe are generated by VMM in CPU Manage device and distribute continuous two pieces of internal memories, respectively order caching and sending is cached, each VM and one in PCIe device or Multiple VF processors are associated, and manager is associated with PF, specifically, there is its operating system in each VM, accordingly, also The VF drivings of the VF processor associated with the VM are installed.Similarly, except installing its operating system in manager, also corresponding peace PF drivings equipped with the PF processor associated with the manager.

Wherein, (the English of PCIe root nodes as defined in PCIe protocol is also included in physical machine：PCIe Root Complex, Can abbreviation RC), the PCIe root nodes pass through the (English of root port as defined in PCIe protocol：Root Port, can be referred to as RP) It is connected in PCIe buses.And also include the (English of PCIe terminal notes as defined in PCIe protocol in PCIe device：PCIe Endpoint), the PCIe terminal notes are connected in PCIe buses, and are connected with PF processors, each VF processor.

Also include register group in PF processors, wherein, register group includes the register of above-mentioned introduction；Equally, exist Each VF processor includes a register group, and its register group includes the register of the VF processors of above-mentioned introduction.

Based on above-mentioned framework, it is to be understood that can pass through being communicated between original VF processors and PF processors Communicate with realizing between VM and VM.The data Cun Chudao PF processors for the VM for being associated VF processors, which are explained below, to be associated Manager in implementation process, it is specific that referring to Fig. 4, Fig. 4 is another embodiment of the present invention, realizing of providing is logical between virtual machine The schematic flow sheet of the method for letter, it may include：

S401, VF processor determine whether register VF_Req value is eliminated；

Wherein, whether register VF_Req value is eliminated, that is, whether register VF_Req value is equal to 1 ' b0. When the associated VM of VF processors transmission caching is placed into data, register VF_Req writes 1 set, in VM transmission caching Data be read after, be eliminated.When not being eliminated, send and perform step S401；When being eliminated, step is performed S402。

Data write-in to be sent is sent caching by S402, VF processor, and data length is write into register VF_ TxLength, and 1 set is write to register VF_Req；

Wherein, VF processors can write 1 set to VF_Req, but can not reset.VF_Req writes 1 set, is exactly to deposit Device VF_Req writes 1 ' b1, illustrates that data to be sent are put into transmission caching by VF processors, can notify PF processors To receive data.

S403, DMA engine are detected after register VF_Req set, by the register VF_Req_Sts correspondences of PF processors VF processors status register set, and to PF processors send the first announcement information；

DMA engine by the status register of the multidigit in PF processors to should VF processors register set, so The first announcement information is sent to PF processors afterwards, so that PF processors know that the associated VM of which VF processor needs to send number According to.

Specifically, the first announcement information is DMA engine by the state of the corresponding VF processors of register VF_Req_Sts After register set, an interrupt signal of generation.

Whether the order caching in S404, PF processor detection management device is idle；

, can be for receiving the data from VM when the order caching of manager is idle；The order caching of manager is not When idle, then continue waiting for, until the order caching of manager is idle.Therefore, when manager order caching is idle, step is turned to Rapid S405.

S405, PF processor are according to register VF_Req_Sts value, by the corresponding positions 1 of register PF_Rx_En, so Sent afterwards to DMA engine and receive instruction；

PF processors by be set in VF_Req_Sts 1 position, it is known that be that the associated VM of which VF send data, and then The corresponding positions of PF_Rx_En are also put 1, illustrate that order caching has carried out reception data in the associated manager of PF processors Prepare, DMA engine can go to read data.

S406, DMA engine read data from the transmission caching for receiving the corresponding VM of instruction, and the data read are deposited In the order caching for storing up manager；

S407, DMA engine write register PF_Rx_ by register PF_Rx_Done set, and by data length Length, resets to register PF_Rx_En.

Data are successfully read in DMA engine, and after storing data into the order caching of manager, by register PF_ Rx_Done writes 1 set, and PF processors are by reading register PF_Rx_Done value, it is known that the number in manager order caching According to effective.Meanwhile, DMA engine also sends the first response message to PF processors, that is, sends an interrupt signal, notifies PF Processor successfully by digital independent and is stored.

And the length of the data read will write register PF_Rx_Length, so that PF processors are deposited by reading Device PF_Rx_Length value knows the length of data.

Afterwards, PF processors know that the data in the order caching of manager are effective according to register PF_Rx_Done value Afterwards, if the data interaction between VM by the data in manager order caching, it is necessary to be sent to target VM, then PF processing Data in manager order caching are put into transmission caching by device, are ready for sending.If the data in manager order caching It is the information that the associated VF processors of VM ask resource to PF processors, then PF processors are according to manager order caching Data, the data that VF processors are asked be put into transmission caching, while the data in manager order caching be used up it is complete.

Then PF processors remove register PF_Rx_Done values, illustrate that the data in manager order caching are read Or using finishing, can prepare to receive new data.0 clearing is write into the corresponding positions of register VF_Req_Sts, illustrates the current position Corresponding VF processors do not have data transmission.

In addition, DMA engine also removes the register VF_Req of VF processors value, VF processors are by reading register VF_Req value, after register VF_Req value is eliminated, can be put into new data to associated VM transmission caching.

In another embodiment, the present invention will introduce the implementation process that data Cun Chudao VM are read from manager, and attached Fig. 4 flows are similar, referring to Fig. 5, the flow for the method for realizing inter-virtual machine communication that Fig. 5 provides for another embodiment of the present invention Schematic diagram, it may include：

S501, PF processor determine whether register PF_Req value is eliminated；

Wherein, whether register PF_Req value is eliminated, that is, whether register PF_Req value is equal to 1 ' b0. After register PF_Req value is eliminated, data are put into being cached toward the transmission of manager.Therefore, it be not eliminated When, send and perform step S501, until being eliminated, when being eliminated, turn to and perform step S502.

Data write-in to be sent is sent caching by S502, PF processor, and target VF processors are written into register The Dest_VF and length write-in PF_Tx_Length by data；

Send the transmission caching that caching refers to be distributed in the manager of the PF relational processors.

PF processors write the identity number for the target VF processors for needing to be sent in register Dest_VF.

S503, PF processor send instruction to register PF_Req set to DMA engine；

Wherein, PF processors can be to register PF_Req set, but can not reset.PF_Req is set, and is exactly by 1 ' B1 writes register PF_Req, illustrates that the transmission that data to be sent have been put into manager by PF processors is cached, Ran Houtong Know DMA engine to read.

Whether S504, the PF_Done of DMA engine detection target VF processors reset；

Wherein, when register PF_Done is cleared, it is idle to illustrate the order caching in the VM of VF relational processors , data can be put into order caching.

When register PF_Done is not cleared also, continue waiting for；After register PF_Done is cleared, steering is held Row step S505.

S505, DMA engine read data from the transmission caching of manager, and by the data Cun Chudao targets VF read In the order caching for the VM for managing device association；

DMA engine is it is determined that the transmission caching of manager has data to be sent, and target VF relational processors When VM order caching is idle, the reception read data from the transmission caching of manager and stored to the target VF VM associated is delayed In depositing.

The length of data is write register PF_TxLength and right by S506, DMA engine to register PF_Done set Register PF_Req writes 1 ' b0 and removed；

After DMA engine completes digital independent, the register PF_Done in target VF processors is write into 1 set, target VF Processor judges whether the data in VM order cachings are effective by reading register PF_Done.

S507, DMA engine send the second announcement information to target VF processors.

Second announcement information be used for notify the target VF processors, successfully by the data Cun Chudao of the manager its In corresponding VM order caching.

Specifically, the second announcement information is the interrupt signal that DMA engine is produced.

Target VF processors are after the second announcement information is received, according to actual business requirement, to the number in VM order cachings According to making respective handling.Afterwards, PF_Done is write into 0 clearing, so that VM order caching can prepare to receive new data.

Referring to Fig. 6, the embodiment of the present invention additionally provides a kind of DMA based on the above-mentioned method for realizing inter-virtual machine communication Engine, the DMA engine is arranged in the PCIe device shown in accompanying drawing 3, is connected respectively with PF processors and VF processors；Such as Fig. 6 Shown, a kind of DMA engine 600 may include：

First processing module 610, after being instructed in the reception for receiving the PF processors, instruction correspondence is received from described VM on read data and by manager described in the data Cun Chudao；

Second processing module 620, after being instructed in the transmission for receiving the PF processors, number is read from the manager According to and by the corresponding VM of transmission instruction described in the data Cun Chudao.

Wherein, the first processing module in DMA engine receives instruction after the reception for receiving PF processors is instructed from described Read on the VM of corresponding VF relational processors in manager described in data Cun Chudao；And Second processing module is receiving the PF After the transmission instruction of processor, from the corresponding VM of instruction is sent described in the manager reads the data Cun Chudao, without The other storage allocation in PCIe device, reduces the resource consumption of PCIe device, reads data by DMA, effectively increases VM Between communication efficiency.

In an embodiment of the invention, above-mentioned DMA engine 600 also includes：

First respond module, for sending the first response message to the PF processors, first response message is used for The PF processors are notified, successfully by manager described in the data Cun Chudao for receiving and instructing corresponding VM to send.

In another embodiment of the invention, above-mentioned DMA engine 600 also includes：

Detection module, for detecting that described send instructs whether corresponding VM is ready to receive the data；

Data are read if so, then being performed by above-mentioned Second processing module from the manager and by the data Cun Chudao institutes State the step sent in the corresponding VM of instruction.

Second respond module, for sending the second response message to the PF processors, second response message is used for Notify the PF processors have succeeded to read from the manager to be sent described in data Cun Chudao in the corresponding VM of instruction；With, To VF processors second announcement information of transmission for sending and instructing corresponding VM associated, second announcement information is used to lead to Know the VF processors for sending and instructing corresponding VM associated, will successfully be sent described in the data Cun Chudao of the manager Instruct in corresponding VM.

Fig. 7 a are refer to, Fig. 7 a are that a kind of structure for the system for realizing inter-virtual machine communication provided in an embodiment of the present invention is shown It is intended to；As shown in Figure 7a, within the system including physical machine and PCIe device, the PCIe device is total by PCIe with physical machine Line is connected, and the physical machine includes multiple virtual machine VM and a manager, and the multiple VM and manager pass through virtualization Technology is realized in the physical machine；The PCIe device includes a physical function PF processors, at multiple virtual functions VF Device and direct memory access engine are managed, the PF processors and multiple VF processors are realized in institute by virtualization technology State in PCIe device；Wherein, a VM associates one or more VF processors, and the manager is associated at the PF Manage device；

Above-mentioned DMA engine such as accompanying drawing 6, be referred to it is above-mentioned be discussed in detail, will not be repeated here.

In addition, above-mentioned PF processors are used to send the reception instruction to the DMA engine and described send instructs.

In an enforceable mode, the PF processors are additionally operable to：Receive the first notice that the DMA engine is sent Information, first announcement information is that the DMA engine is used to notify the PF processors to have VM to need to send data；Detection institute State whether manager is ready to receive the data that the corresponding VM of the VF processors is sent；If so, then performing to the DMA engine The step of reception instruction and transmission instruction is sent, the reception instructs the VM for including needing to send data.

In an enforceable mode, the PF processors are additionally operable to：Detect allocated reception in the manager Whether caching is idle to determine whether the manager is ready to receive the data that the corresponding VM of the VF processors is sent, described Sending allocated order caching in the corresponding VM of instruction is used for data storage.

Fig. 7 b are referred to, Fig. 7 b show for the systematic difference for realizing inter-virtual machine communication that another embodiment of the present invention is provided It is intended to；In above-mentioned manager and each VM, one is distributed respectively and sends caching and order caching, meanwhile, in PF processors One group of register is assigned, one group of register is assigned in each VF processor, is specifically introduced as described above, is no longer gone to live in the household of one's in-laws on getting married herein State.

In the above-described embodiments, the description to each embodiment all emphasizes particularly on different fields, and does not have the portion being described in detail in some embodiment Point, it may refer to the associated description of other embodiment.

It is apparent to those skilled in the art that, for convenience and simplicity of description, the system of foregoing description, The specific work process of device and unit, may be referred to the corresponding process in preceding method embodiment, will not be repeated here.

In several embodiments provided herein, it should be understood that disclosed system, apparatus and method can be with Realize by another way.For example, device embodiment described above is only schematical, for example, the unit Divide, only a kind of division of logic function there can be other dividing mode when actually realizing, such as multiple units or component Another system can be combined or be desirably integrated into, or some features can be ignored, or do not perform.It is another, it is shown or The coupling each other discussed or direct-coupling or communication connection can be the indirect couplings of device or unit by some interfaces Close or communicate to connect, can be electrical, machinery or other forms.

The unit illustrated as separating component can be or may not be it is physically separate, it is aobvious as unit The part shown can be or may not be physical location, you can with positioned at a place, or can also be distributed to multiple On NE.Some or all of unit therein can be selected to realize the mesh of this embodiment scheme according to the actual needs Mark.

In addition, each functional unit in each embodiment of the invention can be integrated in a processing unit, can also That unit is individually physically present, can also two or more units it is integrated in a unit.Above-mentioned integrated list Member can both be realized in the form of hardware, it would however also be possible to employ the form of SFU software functional unit is realized.

If the integrated unit is realized using in the form of SFU software functional unit and as independent production marketing or used When, it can be stored in a computer read/write memory medium.Understood based on such, technical scheme is substantially The part contributed in other words to prior art or all or part of the technical scheme can be in the form of software products Embody, the computer software product is stored in a storage medium, including some instructions are to cause a computer Equipment (can be personal computer, server, or network equipment etc.) performs the complete of each embodiment methods described of the invention Portion or part steps.And foregoing storage medium includes：USB flash disk, mobile hard disk, read-only storage (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disc or CD etc. are various can store journey The medium of sequence code.

A kind of method and system for realizing inter-virtual machine communication provided by the present invention are described in detail above, it is right In those of ordinary skill in the art, according to the thought of the embodiment of the present invention, can in specific embodiments and applications There is change part, in summary, this specification content should not be construed as limiting the invention.

Claims

1. a kind of method for realizing inter-virtual machine communication, applied in bus and interface standard PCIe device, the PCIe device It is connected with physical machine by PCIe buses, the physical machine includes multiple virtual machine VM and a manager, multiple VM Realized with manager by virtualization technology in the physical machine；The PCIe device includes direct memory access Engine, a physical function PF processors and multiple virtual functions VF processors, the PF processors and multiple VF processors pass through Virtualization technology is realized in the PCIe device；Wherein, a VM associates one or more VF processors, described Manager associates the PF processors；Characterized in that, methods described includes：

The DMA engine reads number from described receive after the reception for receiving the PF processors is instructed on the corresponding VM of instruction According to and by manager described in the data Cun Chudao；

The DMA engine reads data and by the number after the transmission for receiving the PF processors is instructed from the manager Instructed according to storage to described send in corresponding VM.

2. according to the method described in claim 1, it is characterised in that the DMA engine is receiving the reception of the PF processors Include before instruction：

The PF processors receive the first announcement information that the DMA engine is sent, and first announcement information is that the DMA draws Hold up for notifying the PF processors to there is VM to need to send data；

The PF processors detect whether the manager is ready to receive the data that the corresponding VM of the VF processors is sent；

If so, the PF processors are sent to the DMA engine receives instruction, the reception instruction includes needing to send data VM。

3. method according to claim 2, it is characterised in that the PF processors detect whether the manager is ready to Receiving the data of the corresponding VM transmissions of the VF processors includes：

The PF processors detect in the manager whether allocated order caching is idle whether to determine the manager Getting out receive allocated order caching in the data that the corresponding VM of the VF processors is sent, the manager is used to store Data.

4. the method according to any one of claims 1 to 3, it is characterised in that the transmission from the manager is cached Read data and include after being sent described in the data Cun Chudao in the order caching for instructing corresponding VM：

The DMA engine sends the first response message to the PF processors, and first response message is used to notify the PF Processor, successfully by manager described in the data Cun Chudao for receiving and instructing corresponding VM to send.

5. the method according to any one of claims 1 to 3, it is characterised in that the DMA engine is receiving the PF processing Include after the transmission instruction of device：

The DMA engine detection is described to send whether the corresponding VM of instruction is ready to receive the data；

If so, performing data and the step that will be sent described in the data Cun Chudao in instructing corresponding VM are read from the manager Suddenly.

6. method according to claim 5, it is characterised in that the DMA engine detection is described to send the corresponding VM of instruction Whether be ready to receive the data includes：

Whether DMA engine detection described to send allocated order caching in the corresponding VM of instruction idle to determine the hair if being Send whether the corresponding VM of instruction is ready to receive the data, the transmission instructs allocated order caching in corresponding VM to use In data storage.

7. method according to claim 5, it is characterised in that from the manager read data and by the number described Include afterwards to described send in the corresponding VM of instruction according to storage：

The DMA engine sends the second response message to the PF processors, and second response message is used to notify the PF Processor has succeeded to read data from the manager and store to described to be sent in the corresponding VM of instruction；

The DMA engine sends the second announcement information to the VF processors that instruct corresponding VM associated of sending, and described the Two announcement informations are used to notify the VF processors for sending and instructing corresponding VM associated, successfully by the number of the manager Instructed according to storage to described send in corresponding VM.

8. a kind of direct memory access engine, applied in bus and interface standard PCIe device, the PCIe device with Physical machine is connected by PCIe buses, and the physical machine includes multiple virtual machine VM and a manager, multiple VM and Manager is realized in the physical machine by virtualization technology；Include the DMA engine, a physics in the PCIe device Function PF processors and multiple virtual functions VF processors, the PF processors and multiple VF processors are real by virtualization technology In the present PCIe device, wherein, a VM associates one or more VF processors, and the manager associates institute State PF processors, it is characterised in that the DMA engine includes：

First processing module, after being instructed in the reception for receiving the PF processors, is received on the corresponding VM of instruction from described Read data and by manager described in the data Cun Chudao；

Second processing module, after being instructed in the transmission for receiving the PF processors, reads data from the manager and incites somebody to action Sent described in the data Cun Chudao in the corresponding VM of instruction.

9. DMA engine according to claim 8, it is characterised in that the DMA engine also includes：

First respond module, for sending the first response message to the PF processors, first response message is used to notify The PF processors, successfully by manager described in the data Cun Chudao for receiving and instructing corresponding VM to send.

10. DMA engine according to claim 8 or claim 9, it is characterised in that the DMA engine also includes：

Detection module, for detecting that described send instructs whether corresponding VM is ready to receive the data.

11. DMA engine according to claim 10, it is characterised in that the DMA engine also includes：

Second respond module, for sending the second response message to the PF processors, second response message is used to notify The PF processors have succeeded to read from the manager to be sent in the corresponding VM of instruction described in data Cun Chudao；With, to institute State and send associated VF processors the second announcement informations of transmission of the corresponding VM of instruction, second announcement information is used to notify institute State and send the associated VF processors of the corresponding VM of instruction, will successfully send and instructed described in the data Cun Chudao of the manager In corresponding VM.

12. a kind of system for realizing inter-virtual machine communication, it is characterised in that including：Bus and interface standard PCIe device and thing Reason machine, the PCIe device is connected with physical machine by PCIe buses, and the physical machine includes multiple virtual machine VM and one Manager, the multiple VM and manager are realized in the physical machine by virtualization technology；The PCIe device includes One physical function PF processors, multiple virtual functions VF processors and direct memory access engine, the PF processors and Multiple VF processors are realized in the PCIe device by virtualization technology；Wherein, a VM association is one or more The VF processors, the manager associates the PF processors；

The DMA engine includes：

Second processing module, after being instructed in the transmission for receiving the PF processors, reads data from the manager and incites somebody to action Sent described in the data Cun Chudao in the corresponding VM of instruction；

13. system according to claim 12, it is characterised in that

The PF processors are additionally operable to：The first announcement information that the DMA engine is sent is received, first announcement information is institute Stating DMA engine is used to notify the PF processors to have VM to need to send data；Detect whether the manager is ready to receive institute State the data that the corresponding VM of VF processors is sent；Instructed and the hair if so, then performing and sending described receive to the DMA engine The step of sending instruction, the reception instruction includes the VM for needing to send data.

14. the system according to claim 12 or 13, it is characterised in that

The PF processors are additionally operable to：Detect in the manager whether allocated order caching is idle to determine the management Whether device is ready to receive the data that the corresponding VM of the VF processors is sent, and the transmission instructs allocated in corresponding VM Order caching is used for data storage.

15. system according to claim 14, it is characterised in that

The DMA engine also includes：

16. system according to claim 12, it is characterised in that

The DMA engine also includes：

17. system according to claim 16, it is characterised in that

The DMA engine also includes：