CN112905304A

CN112905304A - Communication method and device between virtual machines, physical host and medium

Info

Publication number: CN112905304A
Application number: CN202110250891.6A
Authority: CN
Inventors: 李波; 朱民
Original assignee: Sangfor Technologies Co Ltd
Current assignee: Sangfor Technologies Co Ltd
Priority date: 2021-03-08
Filing date: 2021-03-08
Publication date: 2021-06-04

Abstract

The application discloses a method, a device, a physical host and a medium for communication among virtual machines, wherein the method comprises the following steps: reading an original data packet to be transmitted in a target shared memory corresponding to a first virtual machine when it is monitored that a device driver corresponding to the first virtual machine performs write operation on an I/O notification register in PCIe equipment corresponding to the first virtual machine; determining the IP address of a second physical host of a receiver based on a second virtual machine identifier in the original data packet to be transmitted and a preset virtual machine identifier and server IP comparison table; and sending the original data packet to be transmitted to the second physical host based on the IP address of the second physical host, wherein the original data packet to be transmitted is used for being transmitted to a second virtual machine corresponding to the second virtual machine identifier. Therefore, the adaptability is strong, the IP address section does not need to be reserved, and the use is flexible.

Description

Communication method and device between virtual machines, physical host and medium

Technical Field

The present application relates to the field of communications technologies, and in particular, to a method, an apparatus, a physical host, and a medium for communication between virtual machines.

Background

With the improvement of hardware performance, the computing capacity of the physical host is greatly improved. However, the resource usage by the operating system is uneven, so that the hardware performance is not fully used, resulting in serious waste of computing resources. In this case, a virtualization technology is developed, so that a plurality of mutually independent virtual machines can be simultaneously run on the same physical host, and each virtual machine has complete hardware devices, including a BIOS (Basic Input Output System), a CPU, a memory, a disk, a network card, and the like. The cloud computing takes a virtualization technology as a core, provides elastic resources for users, greatly improves the resource utilization rate of a physical host, and reduces the user cost. In the current cloud computing platform, each virtual machine is generally equipped with only one network card for the user to perform network communication. However, in reality, many scenarios require data transmission between virtual machines on different physical hosts, and it is necessary to ensure that user traffic on the virtual machines is not affected. For example, the management platform of the service provider needs to perform management operations on the business Virtual Machine of the customer, and even needs to transmit some Tools into the Virtual Machine, such as VMtools (Virtual Machine Tools), and security Tools.

To meet the requirements in the above example, in the current KVM (Kernel-based Virtual Machine) platform, there are two communication methods between Virtual machines on different physical hosts: one is serial-port-based data communication, which requires direct specification of physical host IP (Internet Protocol) addresses of the other party in virtual machines of both communication parties, and once a virtual machine migrates, the physical host IP addresses configured in each virtual machine communicating with the virtual machine needs to be directly modified, so that the cloud computing platform has poor adaptability to large-scale, high-flexibility and dynamic automatic configuration. And secondly, the data communication based on the network needs to configure a special network card and reserve an IP section for each virtual machine, the management is complex, the participation of a user is needed, and the use is not flexible.

Disclosure of Invention

In view of this, an object of the present application is to provide a virtual machine communication method, apparatus, device, and medium, which can adapt to a large-scale cloud computing platform with strong flexibility and dynamic automatic configuration, and have strong adaptability, and do not need to reserve an IP address segment, and are flexible to use. The specific scheme is as follows:

in a first aspect, the present application discloses an inter-virtual machine communication method, which is applied to a first physical host running a first virtual machine, and includes:

when it is monitored that a device driver corresponding to the first virtual machine performs write operation on an I/O notification register in PCIe equipment corresponding to the first virtual machine, reading an original data packet to be transmitted in a target shared memory corresponding to the first virtual machine;

determining the IP address of a second physical host of a receiver based on the second virtual machine identifier in the original data packet to be transmitted and a preset virtual machine identifier and server IP comparison table;

and sending the original data packet to be transmitted to the second physical host based on the IP address of the second physical host, wherein the original data packet to be transmitted is used for being transmitted to a second virtual machine corresponding to the second virtual machine identifier.

Optionally, the sending the original data packet to be transmitted to the second physical host based on the IP address of the second physical host includes:

packaging the original data packet to be transmitted into a network data packet to be transmitted based on the IP address of the second physical host;

and sending the network data packet to be transmitted to the second physical host, wherein the network data packet to be transmitted is used for being analyzed and then transmitted to a second virtual machine corresponding to the second virtual machine identifier, and the second virtual machine runs on the second physical host.

Optionally, before the reading of the original data packet to be transmitted in the target shared memory corresponding to the first virtual machine when it is monitored that the device driver corresponding to the first virtual machine performs a write operation on the I/O notification register in the PCIe device corresponding to the first virtual machine, the method further includes:

when the first virtual machine is created on the first physical host, determining the target shared memory from a local preset shared memory body, and allocating the target shared memory to the first virtual machine;

and distributing an event notification file for the first virtual machine, and storing a first virtual machine identifier of the first virtual machine, the memory use information of the target shared memory and a file identifier of the event notification file into a preset memory use information table.

Optionally, in the process of determining the target shared memory from the local preset shared memory, the method further includes:

and when the residual memory space of the preset shared memory is smaller than the memory size required by the first virtual machine, dynamically expanding the preset shared memory.

Optionally, after the storing the first virtual machine identifier of the first virtual machine, the memory usage information of the target shared memory, and the file identifier of the event notification file in a preset memory usage information table, the method further includes:

and sending the memory use information of the target shared memory and the event notification file to PCIe equipment corresponding to the first virtual machine.

receiving a virtual machine identifier sent by a cloud platform management system to which the first virtual machine belongs and a physical host IP address corresponding to the virtual machine identifier;

and constructing a comparison table of the virtual machine identification and the server IP based on the virtual machine identification and the physical host IP address corresponding to the virtual machine identification.

Optionally, after the building the virtual machine identifier and server IP comparison table based on the virtual machine identifier and the physical host IP address corresponding to the virtual machine identifier, the method further includes:

creating a local cache table corresponding to the virtual machine identifier and the server IP comparison table;

and storing the virtual machine identifier and the list item with the use times exceeding the preset times in the past preset time length in the server IP comparison table into the local cache table.

Optionally, after the sending the original data packet to be transmitted to the second physical host based on the IP address of the second physical host, the method further includes:

receiving a confirmation network data packet sent by the second physical host, wherein the confirmation network data packet is obtained by packaging the confirmation original data packet by the second physical host based on a first virtual machine identifier in a confirmation original data packet, and the confirmation original data packet is used for confirming whether the second virtual machine receives the original data packet to be transmitted or not;

analyzing the confirmed network data packet to obtain the confirmed original data packet;

and transmitting the original confirmation data packet to the first virtual machine based on the first virtual machine identifier in the original confirmation data packet.

Optionally, the method further comprises:

receiving network data packets to be processed sent by other physical hosts except the first physical host;

analyzing the network data packet to be processed to obtain an original data packet to be processed corresponding to the network data packet to be processed;

storing the original data packet to be processed into a shared memory corresponding to the target virtual machine identifier according to the target virtual machine identifier in the original data packet to be processed;

and inserting corresponding destination virtual interrupt into the destination virtual machine, wherein the destination virtual interrupt is used for triggering a destination device driver corresponding to the destination virtual machine to receive the original data packet to be processed.

Optionally, the storing, according to the destination virtual machine identifier in the original data packet to be processed, the original data packet to be processed into a shared memory corresponding to the destination virtual machine identifier includes:

determining a shared memory corresponding to a target virtual machine identifier according to the target virtual machine identifier in the original data packet to be processed;

and storing the original data packet to be processed into a write ring buffer queue in the shared memory corresponding to the target virtual machine identifier.

In a second aspect, the present application discloses an inter-virtual machine communication apparatus, which is applied to a first physical host running a first virtual machine, and includes:

the data reading module is used for reading an original data packet to be transmitted in a target shared memory corresponding to the first virtual machine when it is monitored that a device driver corresponding to the first virtual machine performs write operation on an I/O notification register in PCIe equipment corresponding to the first virtual machine;

the IP address determining module is used for determining the IP address of the second physical host of the receiver based on the second virtual machine identifier in the original data packet to be transmitted and a preset virtual machine identifier and server IP comparison table;

and the data packet sending module is used for sending the original data packet to be transmitted to the second physical host based on the IP address of the second physical host, wherein the original data packet to be transmitted is used for being transmitted to a second virtual machine corresponding to the second virtual machine identifier.

In a third aspect, the present application discloses a physical host, comprising:

a memory and a processor;

wherein the memory is used for storing a computer program;

the processor is configured to execute the computer program to implement the inter-virtual machine communication method disclosed above.

In a fourth aspect, the present application discloses a computer readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the inter-virtual machine communication method disclosed in the foregoing.

It can be seen that, when it is monitored that the device driver corresponding to the first virtual machine performs write operation on the I/O notification register in the PCIe device corresponding to the first virtual machine, the application reads the original data packet to be transmitted in the target shared memory corresponding to the first virtual machine, then determines the IP address of the second physical host of the receiving party based on the second virtual machine identifier in the original data packet to be transmitted and the preset virtual machine identifier and server IP comparison table, and then sends the original data packet to be transmitted to the second physical host based on the IP address of the second physical host, where the original data packet to be transmitted is used for being transmitted to the second virtual machine corresponding to the second virtual machine identifier. Therefore, in the application, when it is monitored that the device driver corresponding to the first virtual machine performs write operation on the I/O notification register in the PCIe device corresponding to the first virtual machine, the original data packet to be transmitted may be read from the shared memory corresponding to the first virtual machine, and the PI address of the second physical host of the receiving party is determined based on the first virtual machine identifier in the original data packet to be transmitted and the preset virtual machine identifier and server IP address comparison table, and the original data packet to be transmitted is sent to the second physical host based on the IP address of the second physical host, and the second physical host may transmit the original data packet to be transmitted to the second virtual machine corresponding to the second virtual machine identifier, so that the physical host IP addresses of both communication parties do not need to be directly configured in the virtual machines, and only the preset virtual machine identifier and the server IP address comparison table need to be maintained, the physical host IP address of the communication receiver can be found according to the virtual identification, the virtual machine identification and the server IP address comparison table, the cloud computing platform is suitable for large-scale, high in flexibility and dynamic automatic configuration, the adaptability is strong, the virtual machine is provided with the corresponding virtual machine identification, the virtual machine identification corresponds to the physical host IP address through the virtual machine identification and the server IP address comparison table, an IP address section does not need to be reserved, and only the virtual machine identification and the server IP address comparison table need to be maintained when the virtual machine is migrated, closed or opened, so that the use is flexible.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.

FIG. 1 is a flow chart of a method for communication between virtual machines as disclosed herein;

FIG. 2 is a partial flow diagram of a particular method for inter-virtual machine communication as disclosed herein;

FIG. 3 is a partial flow diagram of a particular method for inter-virtual machine communication as disclosed herein;

FIG. 4 is a partial flow diagram of a particular method for inter-virtual machine communication as disclosed herein;

FIG. 5 is a flowchart of a particular method for inter-virtual machine communication as disclosed herein;

FIG. 6 is a schematic diagram of an inter-virtual machine communication apparatus according to the present disclosure;

fig. 7 is an architecture diagram of a method for inter-virtual machine inter-process high-speed communication across physical machines in a cloud virtualization platform according to the present disclosure;

FIG. 8 is a diagram illustrating shared memory management disclosed herein;

FIG. 9 is a schematic diagram of a PCIe device initialization disclosed herein;

fig. 10 is a schematic diagram of a physical host structure disclosed in the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

Referring to fig. 1, an embodiment of the present application discloses an inter-virtual machine communication method, which is applied to a first physical host running a first virtual machine, and the method includes:

step S11: when it is monitored that the device driver corresponding to the first virtual machine performs write operation on the I/O notification register in the PCIe device corresponding to the first virtual machine, reading an original data packet to be transmitted in a target shared memory corresponding to the first virtual machine.

In a specific implementation process, when it is monitored that a device driver corresponding to the first virtual machine performs a write operation on an I/O notification register in a PCIe (peripheral component interconnect express) device corresponding to the first virtual machine, reading an original data packet to be transmitted in a target shared memory corresponding to the first virtual machine, where the original data packet to be transmitted is a data packet written to a PCIe memory in the PCIe device after the first virtual machine establishes a connection with the PCIe device through a virtual socket, and the PCIe memory is a mapping existing in the PCIe device in the target shared memory.

That is, when a first virtual machine on the first physical host needs to transmit data to a second virtual machine on a second physical host other than the first physical host, the first virtual machine first establishes a connection with a PCIe device corresponding to the first virtual machine through a virtual socket (vsocket), writes an original data packet to be transmitted to a PCIe memory in the PCIe device, and performs a write operation on an I/O (Input/Output) notification register in the PCIe device through a device driver corresponding to the first virtual machine, at this time, the original data packet to be transmitted can be read from a target shared memory corresponding to the first virtual machine by the first physical host, where the PCIe memory is a mapping existing in the target shared memory in the PCIe device. The vsocket is a virtual socket, and an anonymous file handle is provided for the application program to use.

Therefore, the virtual machine which needs to send data only needs to write the original data packet to be transmitted into the PCIe memory in the corresponding PCIe equipment, and the physical host can read the original data packet to be transmitted from the shared memory mapped to the PCIe memory, so that zero-copy data transmission is realized between the virtual machine and the physical host through the shared memory, and the data transmission efficiency is improved.

When the cloud platform creates the virtual machine, the cloud platform management system assigns a unique identifier (VMID, virtual machine identifier) to the virtual machine, and the identifier is recovered only when the virtual machine is destroyed along with the whole life cycle of the virtual machine. The IP addresses of the physical hosts are fixed, and the information synchronization is realized by the network communication between the physical hosts.

In order to enable the user layer application program to use the device driver, the device driver packages the PCIe device into a character device and provides the vsocket to realize multi-process concurrent read-write data. The device driver also provides a set of kernel mode interfaces for the direct calling of the kernel module.

Specifically, an application program inside the first virtual machine may obtain a first virtual machine identifier VMID of the application program through a register on the PCIe device), and may set a second virtual machine identifier VMID of the communication peer through a program parameter. And establishing a vsocket through the first virtual machine identification VMID, the sender port, the second virtual machine identification VMID and the receiver port, and directly writing the original data packet to be transmitted into a PCIe memory of PCIe equipment corresponding to the first virtual machine. By the design, the virtual socket is only related to the virtual machine identifier of the sender, the port of the sender, the virtual machine identifier of the receiver and the port of the receiver, and once the application program is started, the application program cannot be changed due to the migration of the virtual machine.

When the first virtual machine writes the original data packet to be transmitted into the PCIe memory, a device driver corresponding to the first virtual machine triggers an interrupt request to write an I/O notification register in the PCIe device, and at the moment, the first physical host is triggered to read the original data packet to be transmitted from a target shared memory corresponding to the first virtual machine. Since the PCIe memory in the PCIe device is a mapping of the target shared memory corresponding to the first virtual machine in the PCIe device, the first physical host may directly read the original data packet to be transmitted from the target shared memory, and zero-copy data transmission between the first virtual machine and the first physical host is realized through the shared memory, so that the data transmission efficiency is improved.

The target shared memory comprises a read ring queue and a write ring queue. Each circular queue has two pointers, front and end. The sender operates end, and the receiver operates front, so that the read-write circular queue is realized in a lock-free mode, and the read-write speed is more efficient than that of a lock mechanism.

Step S12: and determining the IP address of the second physical host of the receiver based on the second virtual machine identifier in the original data packet to be transmitted, the preset virtual machine identifier and the server IP comparison table.

It can be understood that after the original data packet to be transmitted is read, the IP address of the second physical host of the receiving party needs to be determined based on the second virtual machine identifier in the original data packet to be transmitted and the preset comparison table of the virtual machine identifier and the server IP address, and the second virtual machine corresponding to the second virtual machine identifier is the virtual machine of the receiving party running on the second physical host.

Specifically, after the original data packet to be transmitted is read, the original data packet to be transmitted is analyzed, and according to a second virtual machine identifier in a packet header of the original data packet to be transmitted, a preset virtual machine identifier and server IP address comparison table is searched to obtain an IP address of a second physical host of a receiver corresponding to the second virtual machine identifier.

In practical application, when it is monitored that the device driver corresponding to the first virtual machine performs a write operation on the I/O notification register in the PCIe device corresponding to the first virtual machine, before reading an original data packet to be transmitted in the target shared memory corresponding to the first virtual machine, the method further includes: receiving a virtual machine identifier sent by a cloud platform management system to which the first virtual machine belongs and a physical host IP address corresponding to the virtual machine identifier; and constructing a comparison table of the virtual machine identification and the server IP based on the virtual machine identification and the physical host IP address corresponding to the virtual machine identification.

That is, the cloud platform management system to which the first virtual machine belongs sends each virtual machine identifier in the entire cloud platform and the physical host IP address of the host corresponding to each virtual machine identifier to each physical host on the cloud platform, and the first physical host needs to acquire the virtual machine identifier sent by the cloud platform management system and the physical host IP address corresponding to the virtual machine identifier, and constructs the virtual machine identifier and server IP comparison table based on the virtual machine identifier and the physical host IP address corresponding to the virtual machine identifier.

In practical application, in the cloud platform management system, the virtualization system of the cloud platform needs to be expanded to dynamically intercept operations such as starting, migration, closing and the like of the virtual machine, so that each time a new virtual machine is created, the VMID and the IP address of the physical host where the virtual machine is located are synchronized to each physical host of the cloud platform, so as to update the comparison table of the virtual machine identifier and the server IP. When the virtual machine is migrated, all the physical hosts are notified to update the virtual machine identifier and server IP comparison table so as to ensure timeliness of information.

After the comparison table of the virtual machine identifier and the server IP is constructed, a local cache table corresponding to the virtual machine identifier and the server IP comparison table can be created; and storing the virtual machine identifier and the list item with the use times exceeding the preset times in the past preset time length in the server IP comparison table into the local cache table.

That is, a local cache table may be created for the virtual machine identifier and server IP comparison table, and entries in the virtual machine identifier and server IP comparison table, which have been used more than a preset number of times within a preset time period in the past, are stored in the local cache table. Therefore, the frequently used table entries in the virtual machine identification and server IP comparison table can be stored in the local cache table, so that quick query can be realized, and the improvement of data transmission efficiency is facilitated.

Step S13: and sending the original data packet to be transmitted to the second physical host based on the IP address of the second physical host, wherein the original data packet to be transmitted is used for being transmitted to a second virtual machine corresponding to the second virtual machine identifier.

After the IP address of the second physical host is obtained, the original data packet to be transmitted needs to be sent to the second physical host based on the IP address of the second physical host, and after the second physical host receives the original data packet to be transmitted, the original data packet to be transmitted is transmitted to a second virtual machine corresponding to the second virtual machine identifier.

Sending the original data packet to be transmitted to the second physical host based on the IP address of the second physical host may specifically include: packaging the original data packet to be transmitted into a network data packet to be transmitted based on the IP address of the second physical host; and sending the network data packet to be transmitted to the second physical host, wherein the network data packet to be transmitted is used for being analyzed and then transmitted to a second virtual machine corresponding to the second virtual machine identifier, and the second virtual machine runs on the second physical host.

That is, the original data packet to be transmitted is encapsulated into a network data packet to be transmitted based on the IP address of the second physical host, then the network data packet to be transmitted is sent to the second physical host, the second physical host analyzes the network data packet to be transmitted after receiving the network data packet to be transmitted, so as to obtain the original data packet to be transmitted, and the original data packet to be transmitted is transmitted to the second virtual machine according to the identifier of the second virtual machine in the original data packet to be transmitted.

Referring to fig. 2, when it is monitored that the device driver corresponding to the first virtual machine performs a write operation on the I/O notification register in the PCIe device corresponding to the first virtual machine, before reading the original data packet to be transmitted in the target shared memory corresponding to the first virtual machine, the method further includes:

step S21: when the first virtual machine is created on the first physical host, the target shared memory is determined from a local preset shared memory body, and the target shared memory is allocated to the first virtual machine.

When it is monitored that the device driver corresponding to the first virtual machine performs write operation on an I/O notification register in PCIe devices corresponding to the first virtual machine, before reading an original data packet to be transmitted in a target shared memory corresponding to the first virtual machine, the first virtual machine is created on the first physical host, and related configuration is performed, so that the first virtual machine and the first physical host can more implement the virtual machine communication method.

Specifically, when the first virtual machine needs to be created on the first physical host, the target shared memory is determined from a local preset shared memory, and the target shared memory is allocated to the first virtual machine.

In the process of determining the target shared memory from the local preset shared memory, the method further includes: and when the residual memory space of the preset shared memory is smaller than the memory size required by the first virtual machine, dynamically expanding the preset shared memory. That is, when the remaining memory space in the preset shared memory is insufficient, the memory size of the preset shared memory may be dynamically enlarged, so that the subsequently started virtual machine has sufficient resources to create a channel based on the shared memory with the first physical host.

When the first virtual machine is created on the first physical host, the first physical host also receives the first virtual machine identifier and the IP address of the first physical host sent by the cloud platform management system, and updates the virtual machine identifier and the IP address of the first physical host into the virtual machine identifier and server IP comparison table.

Step S22: allocating an event notification file to the first virtual machine, and storing a first virtual machine identifier of the first virtual machine, memory usage information of the target shared memory, and a file identifier of the event notification file into a preset memory usage information table, where the memory usage information includes a start address and a memory size of the target shared memory, and the memory usage information table is used for resource recovery when the first virtual machine is closed or migrated from the first physical host.

In addition, an event notification file (eventfd) needs to be allocated to the first virtual machine, and a first virtual machine identifier of the first virtual machine, the memory usage information of the target shared memory, and a file identifier of the event notification file are stored in a preset memory usage information table. The memory usage information includes a start address and a memory size of the target shared memory. The memory usage information table is used for performing resource recovery of the shared memory when the first virtual machine is closed or migrated from the first physical host to another physical host.

Step S23: and sending the memory use information of the target shared memory and the event notification file to PCIe equipment corresponding to the first virtual machine, wherein the memory use information is used for mapping the target shared memory to the PCIe memory in the PCIe equipment, and the event notification file is used for registering virtual interrupt for the first virtual machine.

And further sending the memory usage information of the target shared memory and the event notification file to a PCIe device corresponding to the first virtual machine, where the memory usage information is used to map the target shared memory to the PCIe memory in the PCIe device, and the event notification file is used to register a virtual interrupt for the first virtual machine.

When the first virtual machine is started, a Unix socket file is designated, connection is established with the part, which is in charge of target shared memory allocation, of the first physical host, and data are transmitted. And after the connection is established, sending the memory use information of the target shared memory, the event notification file and other information to the PCIe equipment. And the PCIe equipment maps the target shared memory to the PCIe memory according to the memory use information of the target shared memory, and registers the event notification file eventfd as a virtual interrupt of the first virtual machine for synchronizing the information of the virtual machines. The virtual interrupt and the first virtual machine identifier may also be stored in an event information table, so that the virtual machine internal driver may be triggered to receive the data packet by injecting the corresponding virtual interrupt to the virtual machine according to the event information table.

In practical application, the PCIe device provides configuration information and parameter information of the PCIe device for the inside of the first virtual machine in a register manner, such as the memory size of the target shared memory, the first virtual machine identifier of the first virtual machine, INTx interrupt, the number and type of interrupts, and the like. A part of the information provided by the PCIe device is used when the device driver initializes, and a part of the information is used when an upper application connection is established, such as the first virtual machine identifier.

In order to realize efficient communication of a user layer, the device driver is constructed by utilizing a UIO (user space I/O, I/O technology running in a user space) framework, wherein the device driver comprises a user state driver and a kernel state driver, the user state driver encapsulates the use of the device driver by the user layer, and the kernel state driver is responsible for the registration and the interrupt processing of the device driver. That is, the device driver may be implemented using a UIO framework, and the device driver is implemented in a separate manner: user mode and kernel mode. The kernel state driver is responsible for registering the device driver and interrupting processing. The user state driver encapsulates the use of the device driver.

Referring to fig. 3, after sending the original data packet to be transmitted to the second physical host based on the IP address of the second physical host, the method further includes:

step S31: and receiving a confirmation network data packet sent by the second physical host, wherein the confirmation network data packet is obtained by packaging the confirmation original data packet by the second physical host based on a first virtual machine identifier in the confirmation original data packet, and the confirmation original data packet is used for confirming whether the second virtual machine receives the original data packet to be transmitted.

Step S32: and analyzing the confirmed network data packet to obtain the confirmed original data packet.

Step S33: and transmitting the original confirmation data packet to the first virtual machine based on the first virtual machine identifier in the original confirmation data packet.

After the first physical host sends the network data packet to be transmitted to the second physical host, the second virtual machine on the second physical host also returns an acknowledgement original data packet to the first virtual machine on the first physical host, so as to determine whether the second virtual machine receives the original data packet to be transmitted.

Specifically, the second virtual machine writes an original confirmation data packet into a PCIe memory of PCIe devices corresponding to the second virtual machine, the second physical host parses the original confirmation data packet to obtain a first virtual machine identifier in a packet header of the original confirmation data packet, finds an IP address of the first physical host corresponding to the first virtual machine identifier based on the first virtual machine identifier and a virtual machine identifier and server IP lookup table preset on the second physical host, encapsulates the original confirmation data packet into a confirmation network data packet by using the IP address of the first physical host, and sends the confirmation network data packet to the first physical host through a network, where the first physical host needs to receive the confirmation network data packet.

After receiving the confirmed network data packet, analyzing the confirmed network data packet to obtain the original confirmed data packet, and then sending the original confirmed data packet to the first virtual machine based on the first virtual machine identifier in the packet header of the original confirmed data packet, so that the first virtual machine determines whether the second virtual machine receives the original data packet to be transmitted. And if the original data packet to be transmitted is not received, the original data packet to be transmitted needs to be sent to the second virtual machine again.

Referring to fig. 4, the inter-virtual machine communication method further includes:

step S41: and receiving network data packets to be processed sent by other physical hosts except the first physical host.

Step S42: and analyzing the network data packet to be processed to obtain an original data packet to be processed corresponding to the network data packet to be processed.

Step S43: and storing the original data packet to be processed into a shared memory corresponding to the target virtual machine identifier according to the target virtual machine identifier in the original data packet to be processed.

Step S44: and inserting corresponding destination virtual interrupt into the destination virtual machine, wherein the destination virtual interrupt is used for triggering a destination device driver corresponding to the destination virtual machine to receive the original data packet to be processed.

The first physical host further needs to receive a to-be-processed network data packet sent by other physical hosts except the first physical host, after receiving the to-be-processed network data packet, analyzes the to-be-processed network data packet to obtain a to-be-processed original data packet, stores the to-be-processed original data packet into a shared memory corresponding to a target virtual machine identifier according to the target virtual machine identifier in the to-be-processed original data packet, and inserts a corresponding target virtual interrupt into the target virtual machine, wherein the target virtual interrupt is used for triggering a target device driver corresponding to the target virtual machine to receive the to-be-processed original data packet.

Specifically, after the original data packet to be processed is obtained, a shared memory corresponding to a target virtual machine identifier is determined according to the target virtual machine identifier in the original data packet to be processed; and storing the original data packet to be processed into a write ring buffer queue in the shared memory corresponding to the target virtual machine identifier. Determining a shared memory corresponding to a target virtual machine corresponding to the target virtual machine identifier according to the target virtual machine identifier in the original data packet to be processed and the memory use information table, and writes the original data packet to be processed into a write ring queue of a shared memory corresponding to the target virtual machine, then determining a virtual interrupt corresponding to the target virtual machine based on the event information table, inserting the virtual interrupt corresponding to the target virtual machine into the target virtual machine, triggering a device driver corresponding to the target virtual machine to receive the original data packet to be processed, wherein, after being read from the write ring queue, the original data packet to be processed may be buffered in a receive buffer queue corresponding to the virtual socket corresponding to the destination virtual machine, so that the write ring queue can also receive the original data packets to be processed corresponding to other virtual sockets in the destination virtual machine.

Referring to fig. 5, a flowchart of communication between virtual machines is shown. Calling a lib library by a virtual machine sender APP (Application) to create a virtual machine identifier and a port where the virtual machine sender APP is located, writing a to-be-transmitted original data packet into the virtual machine identifier and the port where the virtual machine receiver APP is located through a virtual socket link, acquiring the virtual machine identifier where the virtual machine receiver APP is located by a data packet forwarding module of a physical host A, inquiring a comparison table of the virtual machine identifier and a server IP to obtain an IP address of a physical host B of a receiver, packaging the to-be-transmitted network data packet into the to-be-transmitted network data packet and sending the to-be-transmitted network data packet to a data forwarding module of the physical host B, analyzing the packet header of the to-be-transmitted network data packet by the data forwarding module of the physical host B, acquiring the virtual machine identifier where the virtual machine receiver APP is located, inquiring the event information table to obtain a virtual interrupt iffd corresponding to the virtual machine identifier where the virtual machine receiver APP is located, writing the analyzed original data packet to be transmitted into a write ring queue in a shared memory corresponding to a virtual machine identifier where a virtual machine receiver APP is located, triggering a virtual interrupt ioeventfd corresponding to the virtual machine identifier where the virtual machine receiver APP is located, processing interrupt by a virtual machine where the virtual machine receiver APP is located, reading the original data packet to be transmitted from the write ring queue, performing connection confirmation by the virtual machine where the virtual machine receiver APP is located, and transmitting a confirmation original data packet, wherein the transmission process of the confirmation original data packet is the same as the processing process of the transmission of the original data packet to be transmitted by a forwarding module of the physical host A, and therefore, the repeated description is not performed here. And thirdly, when the virtual machine sender APP sends data to the virtual machine receiver APP again, virtual socket connection does not need to be established again, and when the virtual machine receiver APP processes the data packet in the write ring queue, the original data packet which cannot be processed in time can be discarded, and the virtual machine sender APP is required to retransmit.

Referring to fig. 6, an embodiment of the present application discloses an inter-virtual machine communication apparatus, which is applied to a first physical host running a first virtual machine, and includes:

the data reading module 11 is configured to read an original data packet to be transmitted in a target shared memory corresponding to the first virtual machine when it is monitored that a device driver corresponding to the first virtual machine performs a write operation on an I/O notification register in PCIe devices corresponding to the first virtual machine;

the IP address determining module 12 is configured to determine an IP address of the second physical host of the receiving party based on the second virtual machine identifier in the original data packet to be transmitted and a preset virtual machine identifier and server IP comparison table;

and a data packet sending module 13, configured to send the original data packet to be transmitted to the second physical host based on the IP address of the second physical host, where the original data packet to be transmitted is used to be transmitted to a second virtual machine corresponding to the second virtual machine identifier.

In some specific embodiments, the data packet sending module 13 is configured to:

In some specific embodiments, the virtual machine communication device further includes:

the channel resource management module is used for determining the target shared memory from a local preset shared memory body and distributing the target shared memory to the first virtual machine when the first virtual machine is created on the first physical host; and distributing an event notification file for the first virtual machine, and storing a first virtual machine identifier of the first virtual machine, the memory use information of the target shared memory and a file identifier of the event notification file into a preset memory use information table.

In some specific embodiments, the channel resource management module is configured to:

the IP address receiving module is used for receiving a virtual machine identifier sent by a cloud platform management system to which the first virtual machine belongs and a physical host IP address corresponding to the virtual machine identifier;

and the comparison table creating module is used for constructing a comparison table of the virtual machine identifier and the server IP based on the virtual machine identifier and the physical host IP address corresponding to the virtual machine identifier.

In some specific embodiments, the lookup table creating module is configured to:

a data packet receiving module, configured to receive a confirmation network data packet sent by the second physical host, where the confirmation network data packet is obtained by the second physical host by encapsulating an original confirmation data packet based on a first virtual machine identifier in the original confirmation data packet, and the original confirmation data packet is used to confirm whether the second virtual machine receives the original data packet to be transmitted; analyzing the confirmed network data packet to obtain the confirmed original data packet; and transmitting the original confirmation data packet to the first virtual machine based on the first virtual machine identifier in the original confirmation data packet.

In some specific embodiments, the packet receiving module is further configured to:

In some specific embodiments, the packet receiving module is configured to:

See fig. 7 for an architecture diagram of a method for inter-virtual machine inter-process high-speed communication across physical machines in a cloud virtualization platform. The aforementioned first physical Host includes a Host operating system (Host OS) and a physical machine in the figure. The data reading module 11, the IP address determining module 12, the data packet transmitting module 13, the IP address receiving module, the lookup table creating module, and the data packet receiving module in the inter-virtual machine communication device, which correspond to the data packet forwarding module in fig. 7. The channel resource management in the figure is also the channel resource management module in the inter-virtual machine communication device. A Guest virtual machine (Guest VM) runs on Qemu, including a Guest operating system and an APP thereon. The PCIe device emulation and resource mapping in the figure, i.e., the PCIe devices described previously. The KVM extension in the figure is used to extend the operations of starting, migrating, and shutting down a virtual machine. The virtual machine management in the cloud platform management system is used for managing the virtual machines in the cloud platform, distributing virtual machine identifications to the virtual machines, and sending the virtual machine identifications and the IP addresses of the corresponding physical hosts to the physical hosts in the cloud platform.

See fig. 8 for a diagram of shared memory management. The channel management module in fig. 7 includes a shared memory management module and a memory usage information table, and the channel management module updates the memory usage information table according to the virtual machine update operation of the KVM extension. The channel management module allocates the shared memory in the preset shared memory body to the corresponding virtual machine, and recovers the corresponding shared memory when the virtual machine is closed, and the shared memory corresponding to each virtual machine is mapped to the PCIe memory in the memory resources of the PCIe device.

Referring to FIG. 9, a schematic diagram of PCIe device initialization is shown. An event information table is maintained in a Host operating system (Host OS) of the physical Host, and the event information table comprises Virtual Machine Identifications (VMIDs) of all virtual machines and corresponding virtual interrupts (ioeventfd). And a virtual machine circular queue structure list, which comprises a write circular queue information structure (W-Rque information structure) and a read circular queue information structure (R-Rque information structure) corresponding to each virtual machine. The PCIe device comprises a register, a virtual interrupt (ioeventfd) registration module and a memory resource of the device, and is used for mapping a corresponding shared memory, and the device driver corresponding to the virtual machine comprises an interrupt application and registration module, and a write ring queue information structure (W-Rque information structure) and a read ring queue information structure (R-Rque information structure) which are used for managing a corresponding write ring queue and a read ring queue.

Referring to fig. 10, a schematic structural diagram of a physical host 20 provided in the embodiment of the present application is shown, where the physical host 20 may implement the steps of the virtual machine communication method disclosed in the foregoing embodiment.

In general, the physical host 20 in the present embodiment includes: a processor 21 and a memory 22.

The processor 21 may include one or more processing cores, such as a four-core processor, an eight-core processor, and so on. The processor 21 may be implemented by at least one hardware of a DSP (digital signal processing), an FPGA (field-programmable gate array), and a PLA (programmable logic array). The processor 21 may also include a main processor and a coprocessor, where the main processor is a processor for processing data in an awake state, and is also called a Central Processing Unit (CPU); a coprocessor is a low power processor for processing data in a standby state. In some embodiments, the processor 21 may be integrated with a GPU (graphics processing unit) which is responsible for rendering and drawing images to be displayed on the display screen. In some embodiments, the processor 21 may include an AI (artificial intelligence) processor for processing computing operations related to machine learning.

Memory 22 may include one or more computer-readable storage media, which may be non-transitory. Memory 22 may also include high speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In this embodiment, the memory 22 is at least used for storing the following computer program 221, wherein after being loaded and executed by the processor 21, the steps of the inter-virtual machine communication method disclosed in any one of the foregoing embodiments can be implemented.

In some embodiments, the physical host 20 may further include a display 23, an input/output interface 24, a communication interface 25, a sensor 26, a power supply 27, and a communication bus 28.

Those skilled in the art will appreciate that the architecture shown in FIG. 10 is not limiting of physical host 20 and may include more or fewer components than those shown.

Further, an embodiment of the present application also discloses a computer readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the inter-virtual machine communication method disclosed in any of the foregoing embodiments.

For a specific process of the inter-virtual machine communication method, reference may be made to corresponding contents disclosed in the foregoing embodiments, and details are not described herein again.

The embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.

The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

Finally, it is further noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of other elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

The method, the device, the physical host and the medium for communication between virtual machines provided by the present application are introduced in detail above, and a specific example is applied in the present application to illustrate the principle and the implementation of the present application, and the description of the above embodiment is only used to help understanding the method and the core idea of the present application; meanwhile, for a person skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.

Claims

1. A communication method between virtual machines is applied to a first physical host running a first virtual machine, and comprises the following steps:

2. The inter-virtual machine communication method according to claim 1, wherein the sending the original data packet to be transmitted to the second physical host based on the IP address of the second physical host comprises:

3. The inter-virtual-machine communication method according to claim 1, wherein before reading the original data packet to be transmitted in the target shared memory corresponding to the first virtual machine when it is monitored that the device driver corresponding to the first virtual machine performs a write operation on an I/O notification register in the PCIe device corresponding to the first virtual machine, the method further includes:

4. The inter-virtual-machine communication method according to claim 3, wherein in the process of determining the target shared memory from the local preset shared memory, the method further comprises:

5. The inter-virtual-machine communication method according to claim 3, wherein after storing the first virtual machine identifier of the first virtual machine, the memory usage information of the target shared memory, and the file identifier of the event notification file in a preset memory usage information table, the method further comprises:

6. The inter-virtual-machine communication method according to claim 1, wherein before reading the original data packet to be transmitted in the target shared memory corresponding to the first virtual machine when it is monitored that the device driver corresponding to the first virtual machine performs a write operation on an I/O notification register in the PCIe device corresponding to the first virtual machine, the method further includes:

7. The inter-virtual machine communication method according to claim 6, wherein after the building the virtual machine identifier and server IP lookup table based on the virtual machine identifier and the physical host IP address corresponding to the virtual machine identifier, the method further comprises:

8. The inter-virtual machine communication method according to claim 1, wherein after sending the original data packet to be transmitted to the second physical host based on the IP address of the second physical host, the method further comprises:

9. The inter-virtual-machine communication method according to any one of claims 1 to 8, further comprising:

10. The inter-virtual-machine communication method according to claim 9, wherein the storing the to-be-processed raw data packet into a shared memory corresponding to the destination virtual machine identifier according to the destination virtual machine identifier in the to-be-processed raw data packet includes:

11. An inter-virtual machine communication apparatus applied to a first physical host running a first virtual machine, comprising:

12. A physical host, comprising:

a memory and a processor;

wherein the memory is used for storing a computer program;

the processor is configured to execute the computer program to implement the inter-virtual-machine communication method according to any one of claims 1 to 10.

13. A computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the inter-virtual-machine communication method according to any one of claims 1 to 10.