CN103384551B

CN103384551B - A kind of virtual machine communication method of Based PC IE network, server and system

Info

Publication number: CN103384551B
Application number: CN201380000042.9A
Authority: CN
Inventors: 李思聪; 邱军
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2013-01-04
Filing date: 2013-01-04
Publication date: 2015-12-09
Anticipated expiration: 2033-01-04
Also published as: WO2014106321A1; CN103384551A

Abstract

The invention provides a kind of virtual machine communication system and method for Based PC IE network, this system comprises: source host, and source host comprises monitor of virtual machine and at least one virtual machine; Monitor of virtual machine on source host communicates with the virtual machine on source host and is connected; Wherein, the network request tackled on ICP/IP protocol stack carries out processing and the monitor of virtual machine sent on source host as source virtual machine by a virtual machine on source host; Do are the data comprised in network request, according to the ID of the object virtual machine of network request, sent to the PCIE corresponding to source host in PCIENTB address space corresponding to destination host belonging to object virtual machine by the monitor of virtual machine on source host? NTB address space.PCIE network as the communication mechanism between virtual machine, is achieved the high-speed traffic between virtual machine by the present invention; And also break through the restriction of PCIE network, does the subnet management of Based PC IE network share the PCIE in subnet management server? NTB address space, improves the communication speed between virtual machine.

Description

A kind of virtual machine communication method of Based PC IE network, server and system

Technical field

The invention belongs to computer internet communication technical field, relate to a kind of virtual machine communication method, particularly relate to a kind of virtual machine communication method based on peripheral component interconnection (PeripheralComponentInterconnectExpress, PCIE) network, server and system.

Background technology

Along with the deployment of more multiple virtualization server, Intel Virtualization Technology development, network virtualization occupies more and more consequence, and the communication between virtual machine (VirtualMachine is called for short VM) is also more and more frequent.

At present, inter-virtual machine communication is mainly based on Ethernet.During two communication between virtual machines, first the User space application program in virtual machine can call Socket interface, by transmission control protocol (TransmissionControlProtocol, be called for short TCP)/Internet Protocol (InternetProtocol, be called for short IP) storehouse, actual forwarding and process is carried out in monitor of virtual machine (VirtualMachineMonitor is called for short VMM).And ICP/IP protocol is more complicated, cause operating system (OperatingSystem, be called for short OS) protocol processes occupy the very most of of network delay, in addition, in data transmission procedure, data Replica and interrupt processing cause host cpu (CentralProcessingUnit is called for short CPU) resource cannot be absorbed in process customer service frequently.

Industry it is also proposed to be walked around ICP/IP protocol stack in kernel by the mode of shared drive and realizes the method for high-speed communication between server internal virtual machine, but the communication between the virtual machine that can only be applied to same physical machine inside at present.

Summary of the invention

The shortcoming of prior art in view of the above, the object of the present invention is to provide a kind of virtual machine communication method of Based PC IE network, server and system, for solving the problem that between the virtual machine in prior art in different physical machine, communication speed is slow.

First aspect, embodiments provides a kind of virtual machine communication method based on peripheral component interconnection PCIE network, comprising:

Source virtual machine on source host tackles network request on ICP/IP protocol stack, and the data comprised in described network request are sent to the monitor of virtual machine on described source host;

Monitor of virtual machine on described source host is according to the destination address of described network request, determine the PCIENTB address space of the destination host that on PCIE switch, described source host is corresponding, wherein, the main frame of described destination host belonging to the object virtual machine of described network request, on described PCIE switch, the PCIENTB address space of the destination host that described source host is corresponding is the micro-space, PCIENTB address corresponding to described source host in the PCIENTB address sub-spaces described PCIE switch being distributed to described destination host, wherein, PCIENTB address space is divided into the PCIENTB address sub-spaces of the main frame number equal parts be connected with PCIE switch, it is micro-space, each other host assignment address in the address sub-spaces that each main frame is corresponding, corresponding other main frames in micro-space, each described address,

The PCIE message comprising described data is sent to the PCIENTB address space of the destination host that described source host is corresponding on described PCIE switch by the monitor of virtual machine on described source host, so as the monitor of virtual machine of described destination host according to the ID of the object virtual machine of the described network request in described PCIE message by the object virtual machine of described Data dissemination to described network request.

In the first possible implementation of first aspect, the monitor of virtual machine on described source host, according to the destination address of described network request, is determined the PCIENTB address space of the destination host that on PCIE switch, described source host is corresponding, is specially:

Monitor of virtual machine on described source host is according to the destination address of described network request, determine the destination host of described network request, and according to the PCIENTB address space information of described destination host corresponding to the described source host of local cache, determine the PCIENTB address space of the described destination host that described source host is corresponding on described PCIE switch;

Wherein, the PCIENTB address space information of the described destination host that described source host is corresponding is that monitor of virtual machine on the described source host source virtual machine on described source host sends buffer memory when arp request carries out address resolution.

In conjunction with the first possible implementation of first aspect, in the implementation that the second is possible, source virtual machine on described source host sends buffer memory when arp request carries out address resolution, comprising:

Described source virtual machine watch-dog initiates the request message of the PCIENTB address space information of described destination host corresponding to the described source virtual machine of inquiry to subnet management server after intercepting the described arp request that described source virtual machine sends to the object virtual machine of described network request;

Monitor of virtual machine on described source host receives the response message that described subnet management server returns, the PCIENTB address space information of described destination host corresponding for the described source virtual machine comprised in described response message is buffered in this locality, wherein, the PCIENTB address space of the described destination host that the PCIENTB address space of the described destination host that described source virtual machine is corresponding is corresponding with described source host is identical.

In conjunction with first aspect or the first possible implementation of first aspect or the possible implementation of the second of first aspect, in the implementation that the third is possible, the PCIE message of the data comprised in described network request is sent to the PCIENTB address space of the destination host that described source host is corresponding on described PCIE switch by the monitor of virtual machine on described source host, comprising:

The ID of the data that described network request comprises by the monitor of virtual machine on described source host, described source virtual machine and source NTB port id put into the payload of described PCIE message, using the destination address of the ID of the object virtual machine of described network request as described PCIE message, send described PCIE message to described PCIE switch, make described PCIE switch will correspond to the PCIENTB address space of described source host in the PCIENTB address space of described PCIE message write destination host;

Wherein, described source NTB port id is the ID of the NTB port be connected with the monitor of virtual machine on described source host in described PCIE switch.

The third possible implementation of the implementation possible in conjunction with the second of first aspect or first aspect, in the 4th kind of possible implementation, also comprises:

Whole PCIENTB address space carries out maintenance management by described subnet management server, and described whole PCIENTB address space is divided into the PCIENTB address sub-spaces of the number identical with the main frame number that described PCIE switch connects;

The corresponding main frame of each described address sub-spaces, for receiving the data sending to described respective hosts;

It is micro-space, each other host assignment address in each described address sub-spaces, other main frames described in corresponding one of micro-space, each described address, the data that other main frames described send to described address sub-spaces respective hosts are come from for receiving, wherein, other main frames described are except described address sub-spaces respective hosts, the main frame be connected with described PCIE switch.

In conjunction with the 4th kind of possible implementation of first aspect, in the 5th kind of possible implementation, also comprise:

Described subnet management server distributes described whole PCIENTB address space when described PCIE netinit, and by the PCIENTB address space information of self notice All hosts, the PCIENTB address space information of the described subnet management server corresponding to each host caches, so that the request message that can send described inquiry destination host PCIENTB address space to described subnet management server.

In conjunction with second to the 5th kind of arbitrary possible implementation of first aspect, in the 6th kind of possible implementation, also comprise:

Described subnet management server is safeguarded there is the corresponding relation of main frame and virtual machine, and according to the establishment of virtual machine, change, destruction real-time update corresponding relation.

In conjunction with first to the 6th kind of arbitrary possible implementation of first aspect or first aspect, in the 7th kind of possible implementation, the monitor of virtual machine of described destination host, comprising the object virtual machine of described Data dissemination to described network request according to the ID of the object virtual machine of the described network request in described PCIE message:

The monitor of virtual machine of described destination host reads the described PCIE message in the PCIENTB address space corresponding to described source host in the PCIENTB address space of described destination host, according to the ID of the object virtual machine of the described network request in described PCIE message by the object virtual machine of described Data dissemination to described network request.

Second aspect, embodiments provides a kind of virtual machine communication system based on peripheral component interference networks, comprising: at least two main frames, PCIE switches, and described at least two main frames are connected by described PCIE switch communication;

Each main frame in described at least two main frames, for running virtual machine and virtual machine monitoring module, wherein, source virtual machine on source host in described at least two main frames tackles network request on ICP/IP protocol stack, and the data comprised in described network request are sent to the monitor of virtual machine of described source host, the destination address of the described network request that the monitor of virtual machine of described source host sends according to described source virtual machine, determine the PCIENTB address space of the destination host that described source host is corresponding on described PCIE switch, the PCIE message of the data comprised in described network request is sent to the PCIENTB address space of the described destination host that described source host is corresponding on described PCIE switch, so that the monitor of virtual machine of described destination host according to the ID of the object virtual machine of the described network request in described PCIE message by the object virtual machine of described Data dissemination to described network request,

Wherein, the main frame of described destination host belonging to the object virtual machine of described network request, on described PCIE switch, the PCIENTB address space of the destination host that described source host is corresponding is the micro-space, PCIENTB address corresponding to described source host in the PCIENTB address sub-spaces described PCIE switch being distributed to described destination host, wherein, PCIENTB address space is divided into the PCIENTB address sub-spaces of the main frame number equal parts be connected with PCIE switch, it is micro-space, each other host assignment address in the address sub-spaces that each main frame is corresponding, corresponding other main frames in micro-space, each described address.

In the first possible implementation of second aspect, described each main frame comprises:

Virtual machine runs module, for running described virtual machine;

Virtual machine monitoring module, for running described virtual machine monitor.

In the implementation that the second of second aspect is possible, described each main frame comprises: at least one processor, memory;

Described memory, for storing execution instruction;

Described processor, for when described main frame runs, communicates with between described memory, performs described execution instruction and make described main frame run described virtual machine and described monitor of virtual machine.

In conjunction with the implementation that the second of the first or second aspect of second aspect or second aspect is possible, in the implementation that the third is possible, the destination address of the described network request that the monitor of virtual machine of described source host sends according to described source virtual machine, determine the PCIENTB address space of the destination host that described source host is corresponding on described PCIE switch, be specially:

The destination address of the described network request that the monitor of virtual machine on described source host sends according to source virtual machine, determine the destination host of described network request, and according to the PCIENTB address space information of described destination host corresponding to the described source host of local cache, determine the PCIENTB address space of the described destination host that described source host is corresponding on described PCIE switch;

In conjunction with the third possible implementation of second aspect, in the 4th kind of possible implementation, also comprise subnet management server;

Then, the source virtual machine of monitor of virtual machine on described source host on described source host sends buffer memory when arp request carries out address resolution, be specially: when described source virtual machine there is no the mac address information of object virtual machine of described network request, described source virtual machine sends described arp request to the object virtual machine of described network request, after monitor of virtual machine on described source host intercepts the described arp request of described virtual machine transmission, the solicited message of PCIENTB address space corresponding to the described object virtual machine of inquiry is initiated to described subnet management server, and after receiving the response message that described subnet management server returns, the PCIENTB address space information of destination host corresponding for the described source virtual machine comprised in described response message is buffered in this locality, wherein, the PCIENTB address space of the described destination host that the PCIENTB address space of the described destination host that described source virtual machine is corresponding is corresponding with described source host is identical,

Described subnet management server, after the request message of PCIENTB address space corresponding to inquiry described object virtual machine sent for the monitor of virtual machine receiving described source host, the destination host belonging to described object virtual machine is determined according to the corresponding relation of main frame and virtual machine, obtain the PCIENTB address space of described destination host corresponding to described source host, send described response message to the monitor of virtual machine of described source host.

In conjunction with first to fourth kind of arbitrary possible implementation of second aspect or second aspect, in the 5th kind of possible implementation, the PCIE message of the data comprised in described network request is sent to the PCIENTB address space of the described destination host that described source host is corresponding on described PCIE switch by the monitor of virtual machine on described source host, comprising:

The data that described network request comprises by the monitor of virtual machine on described source host, the ID of described source virtual machine and source NTB port id put into the payload of described PCIE message, using the destination address of the ID of the object virtual machine of described network request as described PCIE message, send described PCIE message to described PCIE switch, make described PCIE switch will correspond to the PCIENTB address space of described source host in the PCIENTB address space of described PCIE message write destination host, wherein, described source NTB port id is the ID of the NTB port be connected with the monitor of virtual machine on described source host in described PCIE switch,

Then, described PCIE switch, also for the PCIENTB address space by corresponding to described source host in the PCIENTB address space of destination host described in said write.

In conjunction with the 4th kind or the 5th kind of possible implementation of second aspect, in the 6th kind of possible implementation:

Described subnet management server, also for the whole PCIENTB address space of maintenance management, described whole PCIENTB address space is divided into the PCIENTB address sub-spaces of the number identical with the main frame number that described PCIE switch connects, the corresponding main frame of each described address sub-spaces, for receiving the data sending to described respective hosts, it is micro-space, each other host assignment address in each described address sub-spaces, other main frames described in corresponding one of micro-space, each described address, the data that other main frames described send to described address sub-spaces respective hosts are come from for receiving, wherein, other main frames described are except described address sub-spaces respective hosts, the main frame be connected with described PCIE switch.

In conjunction with the 6th kind of possible implementation of second aspect, in the 7th kind of possible implementation:

Described subnet management server, also for distributing described whole PCIENTB address space when described PCIE netinit, and by the PCIENTB address space information of self notice All hosts, so that the PCIENTB address space information of the described subnet management server corresponding to each host caches thus the request message of described inquiry destination host PCIENTB address space can be sent to described subnet management server.

In conjunction with the 4th kind to the 7th kind arbitrary possible implementation of second aspect, in the 8th kind of possible implementation:

Described subnet management server, also for the corresponding relation of maintenance host and virtual machine, and according to the establishment of virtual machine, change, destruction real-time update corresponding relation.

In conjunction with the first of second aspect or second aspect to the 8th kind of arbitrary possible implementation, in the 9th kind of possible implementation, the monitor of virtual machine of described destination host, comprising the object virtual machine of described Data dissemination to described network request according to the ID of the object virtual machine of the described network request in described PCIE message:

The third aspect, embodiments provides a kind of server, comprising:

Virtual machine runs module, for running virtual machine;

Virtual machine monitoring module, states virtual machine monitor for running;

The data comprised in described network request for tackling network request on ICP/IP protocol stack, and are sent to monitor of virtual machine by described virtual machine;

Described monitor of virtual machine, for the destination address of described network request sent according to described virtual machine, determine the PCIENTB address space of the destination host that on PCIE switch, described server is corresponding, the PCIE message of the data comprised in described network request is sent to the PCIENTB address space of the described destination host that described server is corresponding on described PCIE switch, so that the monitor of virtual machine of described destination host according to the ID of the object virtual machine of the described network request in described PCIE message by the object virtual machine of described Data dissemination to described network request,

Wherein, the main frame of described destination host belonging to the object virtual machine of described network request, on described PCIE switch, the PCIENTB address space of the destination host that described server is corresponding is the micro-space, PCIENTB address corresponding to described server in the PCIENTB address sub-spaces described PCIE switch being distributed to described destination host, wherein, PCIENTB address space is divided into the PCIENTB address sub-spaces of the main frame number equal parts be connected with PCIE switch, it is micro-space, each other host assignment address in the address sub-spaces that each main frame is corresponding, corresponding other main frames in micro-space, each described address.

In the first possible implementation of the third aspect, described monitor of virtual machine is used for the destination address of the described network request sent according to described virtual machine, determine the PCIENTB address space of the destination host that on PCIE switch, described server is corresponding, be specially:

Described monitor of virtual machine, for the destination address of described network request sent according to described virtual machine, determine the destination host of described network request, and according to the PCIENTB address space information of described destination host corresponding to the described server of local cache, determine the PCIENTB address space of the described destination host that described server is corresponding on described PCIE switch;

Wherein, to be described monitor of virtual machine send buffer memory when arp request carries out address resolution at described virtual machine to the PCIENTB address space information of the described destination host that described server is corresponding.

In conjunction with the first possible implementation of the third aspect, in the implementation that the second is possible, described monitor of virtual machine is buffer memory when described virtual machine transmission arp request carries out address resolution, is specially:

Described virtual machine, also for, when described virtual machine not having the mac address information of object virtual machine of described network request, the object virtual machine to described network request sends described arp request;

Described monitor of virtual machine, also for intercept described virtual machine send described arp request after, the solicited message of PCIENTB address space corresponding to the described object virtual machine of inquiry is initiated to subnet management server, and after receiving the response message that described subnet management server returns, the PCIENTB address space information of destination host corresponding for the described virtual machine comprised in described response message is buffered in this locality, wherein, the PCIENTB address space of the described destination host that the PCIENTB address space of the described destination host that described virtual machine is corresponding is corresponding with described server is identical.

In conjunction with the third aspect or the first possible implementation of the third aspect or the possible implementation of the second of the third aspect, in the implementation that the third is possible, the PCIE message of the data comprised in described network request is sent to the PCIENTB address space of the described destination host that described server is corresponding on described PCIE switch by the monitor of virtual machine on described source host, comprising:

Described monitor of virtual machine, the payload of described PCIE message is put into for the ID of the data comprised by described network request, described virtual machine and source NTB port id, using the destination address of the ID of the object virtual machine of described network request as described PCIE message, send described PCIE message to described PCIE switch, make described PCIE switch will correspond to the PCIENTB address space of described server in the PCIENTB address space of described PCIE message write destination host;

Wherein, described source NTB port id is the ID of the NTB port be connected with described monitor of virtual machine in described PCIE switch.

In conjunction with second or the third possible implementation of the third aspect, in the 4th kind of possible implementation:

The PCIENTB address space of the described destination host that described server is corresponding is managed by described subnet management server maintenance, comes from for receiving the data that described server sends to described destination host;

Wherein, whole PCIENTB address space is divided into the PCIENTB address sub-spaces of the number identical with the main frame number that described PCIE switch connects, the corresponding main frame of each described address sub-spaces, for receiving the data sending to described respective hosts, it is micro-space, each other host assignment address in each described address sub-spaces, other main frames described in corresponding one of micro-space, each described address, the data that other main frames described send to described address sub-spaces respective hosts are come from for receiving, other main frames described are except described address sub-spaces respective hosts, the main frame be connected with described PCIE switch.

In conjunction with the 4th kind of possible implementation of the third aspect, in the 5th kind of possible implementation:

Described monitor of virtual machine, also for receiving and the PCIENTB address space information of the subnet management server that server is corresponding described in buffer memory.

In conjunction with first to the 5th kind of arbitrary possible implementation of the third aspect or the third aspect, in the 6th kind of possible implementation, the monitor of virtual machine of described destination host, comprising the object virtual machine of described Data dissemination to described network request according to the ID of the object virtual machine in described PCIE message:

The monitor of virtual machine of described destination host reads the described PCIE message in the PCIENTB address space corresponding to described server in the PCIENTB address space of described destination host, according to the ID of the object virtual machine in described PCIE message by the object virtual machine of described Data dissemination to described network request.

As mentioned above, the virtual machine communication method of Based PC IE network of the present invention, server and system, have following beneficial effect:

The present invention using PCIE network as the communication mechanism between virtual machine, utilize the high bandwidth of PCIE network, the advantage of low delay, and pass through the subnet management of Based PC IE network, the virtual machine on the All hosts in network is made to walk around ICP/IP protocol stack in OS, share the PCIE non-transparent bridge (Non-TransparentBridge of subnet management server maintenance management, be called for short NTB) address space, further improve the communication speed between virtual machine.

Accompanying drawing explanation

Fig. 1 is the primary structure schematic diagram of the virtual machine communication system of Based PC IE network of the present invention.

Fig. 2 is the structural representation of main frame in the virtual machine communication system of Based PC IE network of the present invention.

Fig. 3 is the structural representation of the virtual machine in the virtual machine communication system of Based PC IE network of the present invention, main frame run.

Fig. 4 is the structural representation of the monitor of virtual machine in the virtual machine communication system of Based PC IE network of the present invention, main frame run.

Fig. 5 is the structural representation of PCIE switch in the virtual machine communication system of Based PC IE network of the present invention.

Fig. 6 is the structural representation of subnet management server in the virtual machine communication system of Based PC IE network of the present invention.

Fig. 7 is the schematic flow sheet of the virtual machine communication method of Based PC IE network of the present invention.

Fig. 8 is the detailed process schematic diagram of step S1 in the virtual machine communication method of Based PC IE network of the present invention.

Fig. 9 is the detailed process schematic diagram of address resolution in the virtual machine communication method of Based PC IE network of the present invention.

Figure 10 is the server architecture schematic diagram of another embodiment of the present invention.

Embodiment

Below by way of specific instantiation, embodiments of the present invention are described, those skilled in the art the content disclosed by this specification can understand other advantages of the present invention and effect easily.The present invention can also be implemented or be applied by embodiments different in addition, and the every details in this specification also can based on different viewpoints and application, carries out various modification or change not deviating under spirit of the present invention.

For the accompanying drawing that this embodiment part relates to, it should be noted that, the diagram provided in the present embodiment only illustrates basic conception of the present invention in a schematic way, then only the assembly relevant with the present invention is shown in diagram but not component count, shape and size when implementing according to reality is drawn, it is actual when implementing, and the form of each assembly, quantity and ratio can be a kind of change arbitrarily, and its assembly layout form also may be more complicated.

Along with the development of PCIE technology, it is data communication bus between two main frames that PCIE bus is expanded further.PCIE transmits packet and carrys out addressing by address realm, namely can read and write the address space distributing to ancillary equipment during processor access ancillary equipment, and final PCIE bus passes through this address data message route to ancillary equipment.This routing mechanism requires that the address space distributing to ancillary equipment in a host computer system can not repeat, but cannot ensure that address space does not repeat when carrying out interaction data by PCIE between two hosts, now PCIE bus introduces non-transparent bridge NTB mechanism.NTB mechanism utilizes the address translation rule pre-set to be changed the address that a main frame mails to the data message of another main frame, is converted to the address realm that can not conflict in destination host, thus realizes the data route of PCIE bus between main frame.NTB is generally integrated in the middle of PCIE exchange chip, provides the inter-host communication link of high broadband and low delay, and has the function of PCIE device, such as interruption and error handle etc.In addition, PCIE switch mainly plays following two aspect effects for the realization that Host2Host (host-to-host) communicates: 1) for PIO (ProgrammingInputOutput, process channel) read and write carry out identifying (Identity is called for short ID) route and address route by NTB; 2) for DMA (Directmemoryaccess, direct memory access) transmission and receive by ID route.

Because the time of delay of PCIE bus is 1 delicate, the delay of Ethernet is then close to 10 microseconds, ICP/IP protocol is extremely complicated simultaneously, the protocol processes of OS is caused to occupy the very most of of network delay, in addition, in data transmission procedure, the data Replica frequently of ICP/IP protocol stack process and interrupt processing cause host CPU resources cannot be absorbed in process customer service, so the present invention proposes a kind of virtual machine communication method of Based PC IE network, the method constructs the PCIE network of the data exchange service that is all VM, the VMM of main frame uses the address window of PCIENTB to communicate with interrupt mechanism, thus the communication accelerated between VM.

Below in conjunction with embodiment and accompanying drawing, the present invention is described in detail.

Embodiment

The present embodiment provides a kind of virtual machine communication system of Based PC IE network, as shown in Figure 1, comprises at least 2 main frames 10 (Host), a PCIE switch 20 (PCIESwitch) and a subnet management server 30; Each main frame 10 runs and has a monitor of virtual machine (VMM) 111 and at least one virtual machine (VM) 101; PCIE switch 20 is connected with All hosts, realizes the data retransmission between main frame.Described subnet management server 30 is connected with described PCIE switch 20, for the corresponding relation of main frame in maintenance system and virtual machine.The main frame of the system provided in the embodiment of the present invention is specifically as follows common computer, mobile terminal, work station or server, private server etc., subnet management server can be a normal hosts in system, can be a private host, the present invention do concrete restriction yet.

Below main frame, PCIE switch and subnet management server are described in further detail.

As shown in Figure 2, each described main frame 10 comprises: virtual machine runs module 100, virtual machine monitoring module 110 and hardware platform 120.Virtual machine operation module 100 and virtual machine monitoring module 110 each provide hardware platform and the software platform of virtual machine, virtual machine monitoring module 110 contains virtual machine monitor 111, be the intermediate layer between physical hardware and virtual machine, be responsible for coordinating each virtual machine to the access of hardware platform.Virtual machine runs in module 100 and has run multiple virtual machine 101.Monitor of virtual machine 111 on every platform main frame communicates with all virtual machines 101 on this main frame and is connected, and the data sent by virtual machine 101 send to described PCIE switch 20 with PCIE agreement.

In addition, the hardware platform 120 of main frame can comprise the low-speed device such as the high speed I/O equipment such as CPU, internal memory, network interface card, storage and InterWorking Equipment and basic input and output.Virtual machine operates on the hardware platform of main frame, and perception is completed by PCIE network less than the communication of network cluster inside.

The data comprised in described network request for tackling network request on ICP/IP protocol stack, and are sent to monitor of virtual machine 111 by virtual machine 101.Preferably, virtual machine 101, for tackling described network request on ICP/IP protocol stack, obtains the data comprised in described network request, the data comprised in described network request are put into the virtual-machine data bag buffer queue of oneself correspondence, and notify monitor of virtual machine 111.Preferably, virtual machine 101 can only carry out tackling for the network request of data type message and carry out subsequent treatment, and the network request of non-data type message also carries out subsequent treatment according to original ICP/IP protocol stack mode.

Monitor of virtual machine 111, for after the described network request receiving virtual machine 101 transmission, according to the destination address of described network request, determine the PCIENTB address space of the destination host that on PCIE switch, source host is corresponding, the PCIE message of the data comprised in described network request is sent to the PCIENTB address space of the described destination host that described source host is corresponding on described PCIE switch, so that the monitor of virtual machine of described destination host according to the ID of the object virtual machine in described PCIE message by the object virtual machine of described Data dissemination to described network request.Wherein, the main frame of described destination host belonging to the object virtual machine of described network request, on described PCIE switch, the PCIENTB address space of the destination host that described source host is corresponding is the PCIENTB address space corresponding to described source host in the PCIENTB address space described PCIE switch being distributed to described destination host.

Preferably, VMM111, for receive virtual machine 101 send network request after, according to the destination address (i.e. the IP address of the object virtual machine of network request) in network request, determine the PCIENTB address space of the destination host that source host is corresponding, namely determine that this source host communicates with destination host (be source VM communicate with the object VM on destination host) on source host herein time, need the PCIENTB address space of the destination host of write.For convenience of description, in this article, the main frame at the virtual machine place sending this network request is called source host, the main frame at the object VM place of this network request is called destination host.

Preferably, when VMM111 being cached with the PCIENTB address space of the destination host corresponding to this network request (i.e. physical host corresponding to the IP address of the object virtual machine of network request), then VMM111 can determine the PCIENTB address space that destination host that described network request is corresponding is corresponding, i.e. this source host PCIENTB address space of destination host that should write.Preferably, virtual machine 101 send address resolution protocol (AddressResolutionProtocol, be called for short ARP) request carry out address resolution flow process time, VMM111 can the PCIENTB address space of destination host corresponding to this source host of buffer memory.Further, when virtual machine 101 will send TCP/IP network request, if virtual machine 101 there is no the mac address information of the object virtual machine of described network request, then virtual machine 111 can according to ICP/IP protocol send ARP request carry out address resolution.Be specially:

Virtual machine 101, also for when virtual machine 101 not having the mac address information of object virtual machine of network request, the object virtual machine to network request sends Ethernet ARP request.The IP address of object virtual machine is comprised in ARP request.Then, VMM111, after the ARP also sent for receiving described virtual machine 101 asks, the request message of PCIENTB address space corresponding to inquiry object virtual machine is initiated to subnet management server 30, this request message is with the IP address of object virtual machine, and receive that described subnet management server 30 returns comprise the response message of the PCIENTB address space information of destination host corresponding to described source VM after, the PCIENTB address space information of destination host corresponding for described source VM is buffered in this locality.

Subnet management server 30, for receive PCIENTB address space corresponding to inquiry object virtual machine that VMM initiates request message after, the destination host belonging to described object virtual machine is determined according to the corresponding relation of main frame and virtual machine, know the PCIENTB address space of the destination host that source host is corresponding, send response message to monitor of virtual machine 111.

Preferably, subnet management server 30, for all PCIENTB address spaces of maintenance management, PCIENTB address space is configured on the PCIE switch that to communicate with described source host and destination host and be connected, described PCIENTB address space is divided into the PCIENTB address sub-spaces of the main frame number equal parts be connected with PCIE switch, the corresponding main frame of each described address sub-spaces, for receiving the data sending to this main frame, it is micro-space, each other host assignment address in the address sub-spaces that each main frame is corresponding, corresponding other main frames in micro-space, each described address, the data that these other main frames send to this main frame that described address sub-spaces is corresponding are come from for receiving, other main frames mentioned here are except main frame corresponding to described address sub-spaces, the main frame be connected with PCIE switch.Such as, PCIE switch is maximum can access N number of main frame (main frame 1-main frame N), so the PCIENTB address space of subnet management server maintenance also will be divided into N number of PCIENTB address sub-spaces, and a PCIENTB address sub-spaces correspond to a main frame.Further, a PCIENTB address sub-spaces can separate again micro-space, N-1 PCIENTB address, corresponding other main frames in micro-space, each PCIENTB address, mode by writing the PCIENTB address space of specifying between main frame and main frame just can realize mutual communication, such as, whole PCIENTB address space is divided into N number of PCIENTB address sub-spaces, the PCIENTB address sub-spaces of main frame 1 correspondence is divided into micro-space, N-1 address, be designated H2 respectively, H3, HN, when main frame 2 communicates with main frame 1, namely main frame 2 writes data to the H2 region (micro-space, address) in the PCIENTB address space (address sub-spaces) of main frame 1 correspondence can complete data by the transmission of main frame 2 to main frame 1.The address space all Hosts oneself corresponding to of subnet management server 30 meeting when PCIE netinit in informing network cluster.The embodiment of the present invention can realize multiple main frame by this mechanism of address space and communicate with main frame is concurrent

Further, subnet management server 30, also for the corresponding relation of the virtual machine of safeguarding each main frame and it runs, and according to this corresponding relation of real-time update such as establishment, change, destruction of virtual machine.The corresponding relation of described subnet management server maintenance can be the mapping relations of IP address or the mapping relations of virtual machine ID, or the mapping relations of other types, such as, can be the corresponding relation between the IP address of main frame and the ID of virtual machine, or the corresponding relation between PCIENTB address space corresponding to main frame and the ID of virtual machine.The establishment of each VM, destruction, change can notify that subnet management server refreshes its mapping relations.Described subnet management server can be a physical host, also can be a virtual machine.

Concrete, subnet management server 30, for receiving the request message that source VMM sends, according to the IP address of the object virtual machine carried in message, and the destination host preserved of local maintenance and the corresponding relation of object virtual machine determine the destination host belonging to described object virtual machine, and determine the PCIENTB address space of the destination host that source VM is corresponding further, the PCIENTB address space of the object VM that namely source VM is corresponding, send response message to source virtual machine watch-dog, the PCIENTB address space information of destination host corresponding for source VM is returned to source VMM.

Then, described monitor of virtual machine 111 is after the PCIENTB address space message obtaining destination host corresponding to described source VM, also for proceeding the ARP flow process of source VM, follow-up ARP flow process and existing ARP flow process similar: the ARP of Ethernet is asked the object virtual machine sending to described network request by monitor of virtual machine, namely, ARP request is sent to the VMM of the main frame at object virtual machine place by source VMM, after monitor of virtual machine on described destination host receives described ARP request, described ARP request is sent to described object virtual machine; Object virtual machine returns arp response, wherein comprises the MAC Address of object virtual machine; Described source virtual machine receives the arp response that described object virtual machine sends, and so far completes the ARP address resolution procedure of object virtual machine.

Preferably, monitor of virtual machine 111, for described network request being sent to the PCIENTB address space corresponding to described source host in the PCIENTB address space of destination host, be specially: monitor of virtual machine 111, for the data that described network request is comprised, source virtual machine ID and source NTB port id put into the payload of PCIE message, using the destination address of the ID of the object virtual machine of described network request as described PCIE message, send described PCIE message to PCIE switch 20, make described PCIE switch 20 will correspond to the PCIENTB address space of described source host in the PCIENTB address space of described data write destination host.Wherein, described source virtual machine ID is the ID of the virtual machine sending described data, and described source NTB port id is the ID of the NTB port be connected with the monitor of virtual machine on described source host in described PCIE switch.

Further, the monitor of virtual machine of destination host, also for according to the ID of object virtual machine by described Data dissemination to object virtual machine.Particularly, the monitor of virtual machine of destination host reads the data in the PCIENTB address space corresponding to described source host in PCIENTB address space corresponding to described destination host, according to the ID of the object virtual machine of described data by Data dissemination in the buffer queue of the object virtual machine of described data, and notify that the object virtual machine of described data receives data from described buffer queue.

Further, as shown in Figure 3, described virtual machine 101 comprises: network request blocking module 1011, core protocol module 1012, virtual equipment driver module 1013;

Described network request blocking module 1011, for tackling described network request on ICP/IP protocol stack.

Described core protocol module 1012, communicates with described network request blocking module 1011 and is connected, and processes for the described network request tackled described network interceptor module 1011.Concrete processing procedure is: core protocol module 1012 receives described network request, resolves described network request, obtains the data that described network request comprises.

Described virtual equipment driver module 1013, communicate with described core protocol module 1012 and be connected, data for being comprised by described network request put into packet buffering queue corresponding to described virtual machine, and notice and described virtual machine operate in the monitor of virtual machine on same main frame.Particularly, the data that described network request comprises are put into and to be supervised by monitor of virtual machine and to belong to the privately owned packet buffering queue (hereinafter referred to as buffer queue) of described virtual machine by the virtual equipment driver module 1013 in virtual machine, notify that monitor of virtual machine has new request to send simultaneously.Wherein, all privately owned buffer queue of each virtual machine, due to each main frame can be provided with multiple virtual machine, therefore the monitor of virtual machine on each main frame supervises the privately owned buffer queue of whole virtual machines on this main frame.

Further, as shown in Figure 4, described monitor of virtual machine 111, comprises to communicate with virtual machine 101 the PCIE packet delivery device 1111, the PCIE communication module 1112 that are connected.

Described PCIE packet delivery device 1111, for the data comprised in described network request being added source virtual machine ID and source NTB port id put into the payload of PCIE message, using the destination address of the ID of the object virtual machine of described network request as described PCIE message.Wherein, the virtual machine of interception network request is called source virtual machine, virtual machine for the purpose of the virtual machine of reception network request, main frame belonging to source virtual machine is source host, main frame for the purpose of main frame belonging to object virtual machine, described source virtual machine ID for described in send the ID of the virtual machine of data, described source NTB port id is the ID of the NTB port be connected with the monitor of virtual machine on described source host.Due to a main frame may run multiple stage VM, therefore the function of PCIE packet delivery device 1111 network request of certain VM is added source virtual machine ID and source NTB port id, distinguishes the data flow of each VM.

Described PCIE communication module 1112, described PCIE message for being generated by PCIE packet delivery device 1111 sends to described PCIE switch 20, makes described PCIE switch 20 will correspond to the PCIENTB address space of described source host in PCIENTB address space corresponding for described data write destination host.

PCIE communication module 1112 on monitor of virtual machine 111 on destination host, also for receiving PCIE message from described PCIE switch 20, then described PCIE packet delivery device 1111, also for PCIE message that PCIE communication module 1112 is received according to the object virtual machine ID in message by the buffer queue of the object virtual machine corresponding to the Data dissemination in PCIE message to message.

As shown in Figure 5, described PCIE switch 20 mainly comprises the non-transparent bridge NTB be connected with main frame Host, and NTB provides address window mechanism to be used for transceiving data between main frame.In the virtual machine communication system of Based PC IE network of the present invention, described PCIE switch 20 comprises at least 2 NTB ports, and NTB port is connected with main frame and subnet management server one_to_one corresponding.Such as in Figure 5, the address window at the NTB place of main frame 2 write by main frame 1, and so data can be sent to the NTB of main frame 2 from the NTB of main frame 1 with the form of data message by PCIE switch 20, are finally sent in the internal memory of main frame 2; Main frame 1 also can write the doorbell register (doorbell) of main frame 2 correspondence, and the NTB that main frame 2 is connected sends to main frame 2 and interrupts; Finally, PCIE switch 20 also has more senior remote direct data access (Remotedirectmemoryaccess is called for short RDMA) function, utilizes this function, and PCIE switch 20 can realize utilizing between the connected main frame of NTB and copy internal memory.

As shown in Figure 6, described subnet management server 30, comprises main frame and virtual machine corresponding relation administration module 301, PCIENTB address space menagement module 302, PCIE communication module 303.

Main frame and virtual machine corresponding relation administration module 301, for the corresponding relation of main frame in maintenance system and virtual machine.Particularly, described main frame and virtual machine corresponding relation administration module 301 maintain the corresponding relation of each Host and operation VM thereon in current system.The corresponding relation that described main frame and virtual machine corresponding relation administration module 301 are safeguarded can be IP mapping relations or ID mapping relations, or the mapping relations of other types, such as, attaching relation between the IP address of main frame and the ID of virtual machine or the corresponding relation between PCIENTB address space corresponding to main frame and the ID of virtual machine.Main frame and virtual machine corresponding relation administration module 301 can according to the real-time update main frame such as establishment, change, destruction of virtual machine and the corresponding relations of virtual machine.

Described PCIENTB address space menagement module 302 is the pre-configured PCIENTB address space of each main frame for maintenance management.All PCIENTB address spaces are divided into the PCIENTB address sub-spaces of the number identical with the main frame number that described PCIE switch 20 connects, the corresponding main frame of each described address sub-spaces, for receiving the data sending to main frame that described address sub-spaces is corresponding, further, it is micro-space, other host assignment address in each described address sub-spaces, other main frames described in corresponding one of micro-space, each described address, the data that other main frames described send to the main frame that described address sub-spaces is corresponding are come from for receiving, wherein, other main frames described are except main frame corresponding to described address sub-spaces, the main frame be connected with described PCIE switch.Such as, PCIE switch is maximum can access N number of main frame, and main frame 1-main frame N, so the PCIENTB address space of subnet management server maintenance also will be divided into N number of PCIENTB address sub-spaces, and a PCIENTB address sub-spaces correspond to a main frame.Further, a PCIENTB address sub-spaces can separate again micro-space, N-1 PCIENTB address, corresponding other main frames in micro-space, each PCIENTB address, mode by writing the PCIENTB address space of specifying between main frame and main frame just can realize mutual communication, such as, whole PCIENTB address space is divided into N number of PCIENTB address sub-spaces, the PCIENTB address sub-spaces of main frame 1 correspondence is divided into micro-space, N-1 address, be designated H2 respectively, H3, HN, when main frame 2 communicates with main frame 1, namely main frame 2 writes data to the H2 region (micro-space, address) in the PCIENTB address space (address sub-spaces) of main frame 1 correspondence can complete data by the transmission of main frame 2 to main frame 1.PCIENTB address space menagement module 302, the address space all Host subnet management servers 30 corresponding to of meeting when PCIE netinit in informing network cluster, the address space information of the subnet management server corresponding to this main frame of VMM meeting buffer memory on each main frame, so that VMM correctly can send the request message of the PCIENTB address space of destination host corresponding to query source VM to subnet management server when receiving the address resolution request of source VM.

Described PCIE communication module 303, for communicating with described PCIE switch 20, transmitting-receiving PCIE message.

Further, PCIE communication module 303, for receiving the request message that source VMM is sent by PCIE switch; Described main frame and virtual machine corresponding relation administration module 301, for the IP address according to the object virtual machine carried in this request message, and safeguard that the corresponding relation of destination host and the object virtual machine preserved determines the destination host belonging to described object virtual machine; PCIENTB address space menagement module 302, for the information according to described destination host and source host, determines the PCIENTB address space of the destination host that source VM is corresponding, the PCIENTB address space of the object VM that namely source VM is corresponding; Send response message to source virtual machine watch-dog by described PCIE communication module 303, the PCIENTB address space information of destination host corresponding for source VM is returned to source VMM.

PCIE network as the communication mechanism between virtual machine, has been skipped the transmission of Ethernet by the present invention, and the high bandwidth of efficiency utilization PCIE network, the advantage of low delay, considerably reduce propagation delay time, achieve the high-speed traffic between virtual machine.And, the present invention also breaches the restriction of PCIE network, achieve the subnet management of Based PC IE network, the virtual machine on the All hosts in network is made to walk around ICP/IP protocol stack in OS, maintain the corresponding relation of main frame and virtual machine in system, the unified management achieving whole system is safeguarded, further improves the integrality of the virtual machine communication system of Based PC IE network.In addition, the present invention achieves virtual machine sharing PCIENTB address space by subnet management server, solves the concurrent reading and writing problem of PCIE network, further improves the speed of data receiver between virtual machine.

The present embodiment also provides a kind of virtual machine communication method of Based PC IE network, and the method can be realized by the virtual machine communication system of described Based PC IE network, but its implementation structure is not limited to the system described in embodiment.

As shown in Figure 7, the virtual machine communication method of described Based PC IE network comprises:

S1, the source virtual machine on source host tackles network request on ICP/IP protocol stack, and the data comprised in described network request are sent to the monitor of virtual machine on described source host.

Particularly, can as shown in Figure 8, described step S1 comprises:

S11, source virtual machine tackles network request on ICP/IP protocol stack;

Preferably, can only carry out tackling for the network request of data type message and carry out subsequent treatment in the embodiment of the present invention, the network request of non-data type message also carries out subsequent treatment according to original ICP/IP protocol stack mode.Because the impact of the network request of non-data type message on host performance is little, and realize logic complexity, the method in the embodiment of the present invention is adopted to there is no clear superiority, namely, preferably, control message is still all handed to existing ICP/IP protocol stack and is carried out subsequent treatment by the embodiment of the present invention, and data interception type message carries out the subsequent treatment in the embodiment of the present invention.

S12, described source virtual machine resolves described network request, obtains the data that described network request comprises;

S13, described data are put into packet buffering queue corresponding to described source virtual machine by described source virtual machine, and notify the monitor of virtual machine on described source host.

S2, monitor of virtual machine on described source host is after receiving described network request, according to the destination address of described network request, determine the PCIENTB address space of the destination host that on PCIE switch, described source host is corresponding, wherein, the PCIENTB address space of destination host is PCIENTB address space PCIE switch being distributed to described destination host, the PCIENTB address space of the destination host that on PCIE switch, described source host is corresponding is the PCIENTB address space corresponding to described source host in the PCIENTB address space described PCIE switch being distributed to described destination host.After VMM on source host receives described network request, the destination host (physical host belonging to object virtual machine) that described network request is corresponding is determined according to the destination address (i.e. the IP address of the object virtual machine of network request) in network request, and determine the PCIENTB address space of the destination host that described source host is corresponding further, namely determine that this source host communicates with destination host (be also source VM communicate with the object VM on destination host) on source host herein time, need the PCIENTB address space of the destination host of write.PCIENTB address space is configured on a PCIE switch communicating with described source host and destination host and be connected.

Preferably, when the VMM on source host being cached with the PCIENTB address space of the destination host corresponding to this network request (i.e. physical host corresponding to the IP address of the object virtual machine of network request), VMM then on source host can determine the PCIENTB address space that destination host that described network request is corresponding is corresponding, i.e. this source host correspondence PCIENTB address space of destination host that should write.When source virtual machine transmission arp request carries out address resolution flow process, the PCIENTB address space of the destination host corresponding to this source host of VMM meeting buffer memory on source host, described destination host is exactly the physical host corresponding to IP address of the object virtual machine of this network request.Concrete, when source virtual machine will send TCP/IP network request, if source virtual machine there is no the mac address information of the object virtual machine of described network request, then source virtual machine can according to ICP/IP protocol send ARP request carry out address resolution, can see Fig. 9, wherein source VMM refers to the monitor of virtual machine on source host, and object VMM refers to the monitor of virtual machine on destination host.The specific implementation process of address resolution is:

S21, source virtual machine is by the object virtual machine transmission Ethernet ARP request of the monitor of virtual machine on source host to network request, and ARP request is here the standard request in ICP/IP protocol, and the inside can comprise the IP address of object virtual machine.

After S22, source VMM receive the ARP request with object virtual machine IP address, initiate the request message of PCIENTB address space corresponding to inquiry object virtual machine to subnet management server, this request message is with the IP address of object virtual machine.Source VMM preserves the PCIENTB address space information of subnet management server, send the request message of the PCIENTB address space of destination host corresponding to query source VM.

S23, described subnet management server determines the destination host belonging to described object virtual machine according to the main frame of its maintenance management and the corresponding relation of virtual machine, knows the PCIENTB address space of the destination host that source host is corresponding.

Preferably, subnet management server maintenance manages all PCIENTB address spaces, all PCIENTB address spaces are divided into the PCIENTB address sub-spaces of the main frame number equal parts be connected with PCIE switch, the corresponding main frame of each described address sub-spaces, for receiving the data sending to this main frame, it is micro-space, each other host assignment address in the address sub-spaces that each main frame is corresponding, corresponding other main frames in micro-space, each described address, the data that these other main frames send to this main frame that described address sub-spaces is corresponding are come from for receiving, other main frames mentioned here are except main frame corresponding to described address sub-spaces, the main frame be connected with PCIE switch.

Such as, PCIE switch is maximum can access N number of main frame (main frame 1-main frame N), so the PCIENTB address space of subnet management server maintenance also will be divided into N number of PCIENTB address sub-spaces, and a PCIENTB address sub-spaces correspond to a main frame.Further, a PCIENTB address sub-spaces can separate again micro-space, N-1 PCIENTB address, corresponding other main frames in micro-space, each PCIENTB address, mode by writing the PCIENTB address space of specifying between main frame and main frame just can realize mutual communication, such as, whole PCIENTB address space is divided into N number of PCIENTB address sub-spaces, the PCIENTB address sub-spaces of main frame 1 correspondence is divided into micro-space, N-1 address, be designated H2 respectively, H3, HN, when main frame 2 communicates with main frame 1, namely main frame 2 writes data to the H2 region (micro-space, address) in the PCIENTB address space (address sub-spaces) of main frame 1 correspondence can complete data by the transmission of main frame 2 to main frame 1.The embodiment of the present invention can realize multiple main frame by this mechanism of address space and communicate with main frame is concurrent.

Preferably, the corresponding relation of virtual machine that subnet management server is also safeguarded each main frame and it is run, and according to this corresponding relation of real-time update such as establishment, change, destruction of virtual machine.The corresponding relation of described subnet management server maintenance can be the mapping relations of IP address or the mapping relations of virtual machine ID, or the mapping relations of other types, such as, can be the corresponding relation between the IP address of main frame and the ID of virtual machine, or the corresponding relation between PCIENTB address space corresponding to main frame and the ID of virtual machine.

Concrete, subnet management server receives the request message that source VMM sends, subnet management server is according to the IP address of the object virtual machine carried in message, and the destination host preserved of local maintenance and the corresponding relation of object virtual machine determine the destination host belonging to described object virtual machine, and determine the PCIENTB address space of the destination host that source VM is corresponding further, the PCIENTB address space of the object VM that namely source VM is corresponding.

S24, subnet management server sends response message to source virtual machine watch-dog, and the PCIENTB address space information of destination host corresponding for source VM is returned to source VMM.

S25, after the monitor of virtual machine on described source host obtains the PCIENTB address space information of destination host corresponding to described source VM, is buffered in this locality by the PCIENTB address space information of destination host corresponding for described source VM.The namely PCIENTB address space information of the destination host that buffer memory source host is corresponding.

Monitor of virtual machine on described source host proceeds the ARP flow process of source VM after obtaining the PCIENTB address space message of destination host corresponding to described source VM, follow-up ARP flow process and existing ARP flow process similar:

S26, the ARP of Ethernet request is sent to the object virtual machine of described network request by the monitor of virtual machine on described source host, namely, ARP request is sent to the VMM of the main frame at object virtual machine place by source VMM, after monitor of virtual machine on described destination host receives described ARP request, described ARP request is sent to described object virtual machine;

S27, object virtual machine returns arp response, and described in the MAC Address wherein comprising object virtual machine, source virtual machine receives the arp response that described object virtual machine sends, and so far completes the ARP address resolution procedure of object virtual machine.

Preferably, subnet management server can when PCIE netinit pre-configured good PCIENTB address space, and by the All hosts in the address space information notice subnet corresponding to subnet management server, after each main frame receives the information of subnet management server, the address space information of the subnet management server of meeting corresponding to buffer memory oneself, so that the request message that correctly can send PCIENTB address space corresponding to inquiry to subnet management server.

S3, the PCIE message of the data comprised in described network request is sent to the PCIENTB address space of the destination host that described source host is corresponding on described PCIE switch by the monitor of virtual machine on described source host, so as the monitor of virtual machine of described destination host according to the ID of the object virtual machine in described PCIE message by the object virtual machine of described Data dissemination to this network request.

Particularly, described network request is sent to the PCIENTB address space corresponding to described source host in the PCIENTB address space of destination host by the monitor of virtual machine on described source host, detailed implementation is: the data that described network request comprises by the monitor of virtual machine on described source host, the ID of described source virtual machine and source NTB port id put into the payload of PCIE message, using the destination address of the ID of the object virtual machine of described network request as described PCIE message, send described PCIE message to described PCIE switch, make described PCIE switch will correspond to the PCIENTB address space of described source host in the PCIENTB address space of described data write destination host.Wherein, described source NTB port id is the ID of the NTB port be connected with the monitor of virtual machine on described source host in described PCIE switch.

Further, the virtual machine communication method of described Based PC IE network also comprises step S4, the monitor of virtual machine of described destination host according to the ID of described object virtual machine by described Data dissemination to object virtual machine.

Particularly, the detailed implementation of this step is: the monitor of virtual machine of described destination host reads the data in the PCIENTB address space corresponding to described source host in PCIENTB address space corresponding to described destination host, according to the ID of the object virtual machine of described data by Data dissemination in the buffer queue of the object virtual machine of described data, and notify that the object virtual machine of described data receives data from described buffer queue.Namely, have no progeny receiving in doorbell, the VMM of destination host can resolve rapidly the data of buffering area, and according to region (domain) ID of target VM by Data dissemination in the buffer queue of target VM, target VMM can inject interrupt notification target VM simultaneously has new data to arrive.Target VM finally can receive described data by the virtual equipment driver module in its kernel from buffer queue.The all privately owned buffer queue of each virtual machine, due to each main frame can be provided with multiple virtual machine, therefore the monitor of virtual machine in each main frame supervises the privately owned buffer queue of whole virtual machines on this main frame.

The embodiment of the present invention utilizes the corresponding relation of main frame and virtual machine in subnet management server maintenance PCIE network, achieve the address resolution function of target VM, the unified management further achieving whole PCIE network is safeguarded, improves the integration of PCIE network.

Figure 10 is the server architecture figure of the embodiment of the present invention, and as shown in the figure, the server 40 of the present embodiment comprises at least one processor 401, memory 405, at least one network interface unit 404, and at least one communication bus 402.This server 40 optionally comprises user interface 403, comprises display, keyboard or pointing device.Memory 405 stores and performs instruction, when server 40 runs, communicate between processor 401 with memory 405, processor 401 performs instruction and makes server 40 perform the method for the host implementation described in preceding embodiment of the present invention, be specially: can be as shown in Figure 7, source virtual machine on server tackles network request on ICP/IP protocol stack, and the data comprised in described network request are sent to the monitor of virtual machine on server; Monitor of virtual machine on server, after receiving described network request, according to the destination address of described network request, determines the PCIENTB address space of the destination host that on PCIE switch, described server is corresponding; The PCIE message of the data comprised in described network request is sent to the PCIENTB address space of the destination host that described server is corresponding on described PCIE switch by the monitor of virtual machine on server, so as the monitor of virtual machine of described destination host according to the ID of the object virtual machine in described PCIE message by the object virtual machine of described Data dissemination to this network request.Realize principle and technique effect and the method described in preceding embodiment similar, repeat no more herein.Operating system 406, comprises various program, for realizing various basic business and processing hardware based task.This server 40 also comprises display interface card.In addition, also provide a kind of computer-readable medium in the embodiment of the present invention, comprise computer executed instructions, computer executed instructions can make server 40 perform the method for the host implementation described in preceding embodiment of the present invention, it realizes principle and technique effect is similar, repeats no more herein.

Further, server 40 in the present embodiment, the main frame 10 in the virtual machine communication system of PCIE network in preceding embodiment can also be substituted, realize the function of main frame 10 in above-described embodiment, as the part in the virtual machine communication system of composition PCIE network, communicate with other parts in the virtual machine communication system of PCIE network and coordinate, make described PCIE network virtual machine communication system realize using PCIE network as the communication mechanism between virtual machine, realize the high-speed traffic between virtual machine.It is similar that it realizes principle and technique effect and the communication system described in preceding embodiment and communication means, repeats no more herein.

The present invention has evaded the time delay and host performance loss that ICP/IP protocol stack causes, and make use of the low delay of PCIE network, the advantage of high bandwidth, considerably reduces propagation delay time, achieves the high-speed traffic between virtual machine; Also measured PCIE realizes simultaneously, solves the concurrent reading and writing problem of PCIE network, further improves the speed of data receiver between virtual machine.

In sum, the present invention effectively overcomes various shortcoming of the prior art and tool high industrial utilization.

Through the above description of the embodiments, those skilled in the art can be well understood to the present invention can use hardware implementing, or firmware realizes, or their compound mode realizes.When implemented in software, above-mentioned functions can be stored in computer-readable medium or as the one or more instruction on computer-readable medium or code and transmit.Computer-readable medium comprises computer-readable storage medium and communication media, and wherein communication media comprises any medium being convenient to transmit computer program from a place to another place.Storage medium can be any usable medium that computer can access.Be not limited to as example: computer-readable medium can comprise RAM, ROM, EEPROM, CD-ROM or other optical disc storage, magnetic disk storage medium or other magnetic storage apparatus or can be used in carrying or storing the expectation with instruction or data structure form program code and can by any other medium of computer access.In addition.Any connection can be suitable become computer-readable medium.Such as, if software be use coaxial cable, optical fiber cable, twisted-pair feeder, Digital Subscriber Line (DSL) or such as infrared ray, radio and microwave and so on wireless technology from website, server or other remote source, so the wireless technology of coaxial cable, optical fiber cable, twisted-pair feeder, DSL or such as infrared ray, wireless and microwave and so on be included in affiliated medium fixing in.As used in the present invention, dish (Disk) and dish (disc) comprise compression laser disc (CD), laser dish, laser disc, Digital Versatile Disc (DVD), floppy disk and Blu-ray Disc, the copy data of the usual magnetic of its mid-game, dish then carrys out the copy data of optics with laser.Combination above also should be included within the protection range of computer-readable medium.

Above-described embodiment is illustrative principle of the present invention and effect thereof only, but not for limiting the present invention.Any person skilled in the art scholar all without prejudice under spirit of the present invention and category, can modify above-described embodiment or changes.Therefore, such as have in art usually know the knowledgeable do not depart from complete under disclosed spirit and technological thought all equivalence modify or change, must be contained by claim of the present invention.

Claims

1., based on a virtual machine communication method for peripheral component interconnection PCIE network, it is characterized in that, comprising:

2. method according to claim 1, is characterized in that, the monitor of virtual machine on described source host, according to the destination address of described network request, is determined the PCIENTB address space of the destination host that on PCIE switch, described source host is corresponding, is specially:

3. method according to claim 2, is characterized in that, source virtual machine on described source host sends buffer memory when arp request carries out address resolution, comprising:

4. the method according to claims 1 to 3 any one, it is characterized in that, the PCIE message of the data comprised in described network request is sent to the PCIENTB address space of the destination host that described source host is corresponding on described PCIE switch by the monitor of virtual machine on described source host, comprising:

5. method according to claim 3, is characterized in that, also comprises:

6. method according to claim 5, is characterized in that, also comprises:

7. method according to claim 3, is characterized in that, also comprises:

8. method according to claim 1, is characterized in that, the monitor of virtual machine of described destination host, comprising the object virtual machine of described Data dissemination to described network request according to the ID of the object virtual machine of the described network request in described PCIE message:

9. based on a virtual machine communication system for peripheral component interference networks, comprising: at least two main frames, PCIE switches, described at least two main frames are connected by described PCIE switch communication;

Wherein, the main frame of described destination host belonging to the object virtual machine of described network request, on described PCIE switch, the PCIENTB address space of the destination host that described source host is corresponding is the micro-space, PCIENTB address corresponding to described source host in the space, PCIENTB subaddressing described PCIE switch being distributed to described destination host, wherein, PCIENTB address space is divided into the PCIENTB address sub-spaces of the main frame number equal parts be connected with PCIE switch, it is micro-space, each other host assignment address in the address sub-spaces that each main frame is corresponding, corresponding other main frames in micro-space, each described address.

10. system according to claim 9, is characterized in that, described each main frame comprises:

Virtual machine runs module, for running described virtual machine;

Virtual machine monitoring module, for running described monitor of virtual machine.

11. systems according to claim 9, is characterized in that, described each main frame comprises: at least one processor, memory;

Described memory, for storing execution instruction;

12. according to the arbitrary described system of claim 9 to 11, it is characterized in that, the destination address of the described network request that the monitor of virtual machine of described source host sends according to described source virtual machine, determine the PCIENTB address space of the destination host that described source host is corresponding on described PCIE switch, be specially:

13. systems according to claim 12, is characterized in that, also comprise subnet management server;

14. systems according to claim 9, it is characterized in that, the PCIE message of the data comprised in described network request is sent to the PCIENTB address space of the described destination host that described source host is corresponding on described PCIE switch by the monitor of virtual machine on described source host, comprising:

15. systems according to claim 13, is characterized in that:

16. systems according to claim 15, is characterized in that:

17. systems according to claim 13, is characterized in that:

18. systems according to claim 9, is characterized in that, the monitor of virtual machine of described destination host, comprising the object virtual machine of described Data dissemination to described network request according to the ID of the object virtual machine of the described network request in described PCIE message:

19. 1 kinds of servers, comprising:

Virtual machine runs module, for running virtual machine;

Virtual machine monitoring module, states monitor of virtual machine for running;

20. servers according to claim 19, it is characterized in that, described monitor of virtual machine is used for the destination address of the described network request sent according to described virtual machine, determines the PCIENTB address space of the destination host that on PCIE switch, described server is corresponding, is specially:

21. servers according to claim 20, is characterized in that, described monitor of virtual machine is buffer memory when described virtual machine transmission arp request carries out address resolution, is specially:

22. according to claim 19 to the server described in 21 any one, it is characterized in that, the PCIE message of the data comprised in described network request is sent to the PCIENTB address space of the described destination host that described server is corresponding on described PCIE switch by described monitor of virtual machine, comprising:

23. servers according to claim 21, is characterized in that:

24. servers according to claim 23, is characterized in that:

25. servers according to claim 19, is characterized in that:

The corresponding relation of described server and described virtual machine by subnet management server maintenance, and according to the establishment of described virtual machine, change, destruction real-time update corresponding relation.

26. servers according to claim 19, is characterized in that, the monitor of virtual machine of described destination host, comprising the object virtual machine of described Data dissemination to described network request according to the ID of the object virtual machine in described PCIE message: