CN115202826A

CN115202826A - Method for accessing virtual machine desktop under direct connection of display card

Info

Publication number: CN115202826A
Application number: CN202210931374.XA
Authority: CN
Inventors: 卓洪强; 刘毅; 肖飞; 叶梓枫; 马凯杰; 张瑶
Original assignee: Shenzhen Vclusters Information Technology Co ltd
Current assignee: Shenzhen Vclusters Information Technology Co ltd
Priority date: 2022-08-04
Filing date: 2022-08-04
Publication date: 2022-10-18

Abstract

The invention provides a method for accessing a virtual machine desktop under the direct connection of a display card, which comprises the steps of running agent software on a virtual machine and capturing desktop image data of the virtual machine; the agent software captures desktop image data and writes the desktop image data into a buffer queue based on front-end driving; the back end driver actively pulls the desktop image data in the buffer queue and transmits the pulled desktop data to the server of the host; and the server side of the host machine pushes the acquired desktop image data to the client side of the host machine. A buffer area of vi rt i o drive and vr i ng is arranged between a virtual machine and a host machine and is used for capturing desktop image data of physical video memory in the virtual machine and pushing the desktop image data to a VNC client of the host machine, so that a user can access the desktop of the virtual machine through the host machine to operate in a scene where a video card is directly communicated. The user does not need to connect the virtual machine desktop through the network, and the client can be normally used without being influenced by the network connection condition.

Description

Method for accessing virtual machine desktop under direct connection of display card

Technical Field

The invention relates to the field of virtual machine management platforms, in particular to a method for accessing a virtual machine desktop under the direct connection of a display card.

Background

The virtual machine is a computer system which can simulate one or more virtual computers on one physical computer through virtual machine software and runs an operating system. The virtual machine functions like a physical computer. However, a virtual machine is equipped with its own set of virtual hardware resources, which by default run in the Windows background and are used to test and run applications, such as installing an operating system on the virtual machine, installing applications, accessing network resources, etc., which are commonly used in the existing computer industry.

In a commonly used computer, a host (an entity machine) is connected to a physical video card, the host calls data of a physical video memory, and a virtual machine has a virtual video memory for calling the data of the virtual video memory. Because the virtual machine is only one application program essentially running on the physical computer and mainly depends on the CPU to perform calculation, if a worker needs to perform work with large GPU usage such as 3D rendering on the virtual machine, the efficiency is extremely low due to the CPU-dependent calculation mode. Therefore, a mode of direct connection of the display card is provided, namely, the physical display card is directly supplied to the virtual machine for use, and the virtual machine software can directly call the physical display memory of the physical display card.

However, in the virtualization management platform based on the qemu + kvm architecture, the function of accessing the virtual machine desktop is essential, and is generally used during system installation and software configuration, and under a normal condition, because the hardware system of the virtual machine is simulated by the host, the host can obtain the complete control capability of hardware, including obtaining image data in the virtual video memory, data in the virtual sound card, and the like. However, in a scene of direct display card communication, because the display card of the virtual machine is not simulated by software any more, but the rendering capability is provided by hardware, at this time, the qemu process of the host machine cannot directly access data in the display memory, and the user cannot operate the desktop of the virtual machine through the vnc client.

Disclosure of Invention

The invention provides a method for accessing a virtual machine desktop under the direct connection of a display card, which aims to solve the problem that a host machine cannot acquire data in a physical display memory under the direct connection scene of the existing display card.

The invention provides a method for accessing a virtual machine desktop under the direct connection of a display card, which comprises the following steps: step S1: running agent software on the virtual machine, wherein the agent software is used for capturing desktop image data of the virtual machine; step S2: the agent software captures desktop image data and writes the desktop image data into a buffer queue based on front-end driving; and step S3: the back end driver actively pulls the desktop image data in the buffer queue and transmits the pulled desktop data to the server of the host; and step S4: and the server side of the host machine pushes the acquired desktop image data to the client side of the host machine.

Preferably, the agent software is agent software, the front-end driver is virtio front-end driver, the back-end driver is virtio back-end driver, the buffer queue is vring queue, the server of the host is a VNC server, and the client is a VNC client.

Preferably, step S2 specifically includes: step S21: initializing the environment according to the initialization flow of the DXGI interface; step S22: an idxgioutputdupling object instance is obtained, and desktop image data is obtained by the AcquireNextFrame function.

Preferably, step S22 is followed by: step S23: and drawing the shape of the mouse into a page according to the position of the mouse obtained by query.

Preferably, in step S21, checking whether the desktop image data is updated based on the DXGI interface function, and if not, performing hibernation in the code level; if yes, encoding the image data.

Preferably, in step S21, based on the pixel format of the host server, the DXGI interface selects the same pixel format for environment initialization.

Compared with the prior art, the method for accessing the desktop of the virtual machine by the display card directly has the following advantages that:

by arranging a virtio driver and vring buffer area between the virtual machine and the host machine, the virtual machine is used for capturing desktop image data of physical display memory in the virtual machine and pushing the desktop image data to a VNC client of the host machine, so that a user can also access the desktop of the virtual machine through the host machine to operate in a scene of direct communication of a display card. Meanwhile, the user does not need to connect the virtual machine desktop through a network, and even if the virtual machine network is not configured or a network fault occurs, the client can be normally used without being influenced by the network connection condition, so that the access stability is ensured.

Drawings

Fig. 1 is a flowchart of a method for a graphics card to directly access a desktop of a virtual machine according to a first embodiment of the present invention.

Fig. 2 is a block diagram of a method for directly accessing a virtual machine desktop by a graphics card according to a first embodiment of the present invention.

Fig. 3 is a flowchart of step S2 in a method for directly accessing a virtual machine desktop by a graphics card according to a first embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail below with reference to the accompanying drawings and implementation examples. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the invention.

Referring to fig. 1, a first embodiment of the present invention provides a method for a graphics card to directly access a desktop of a virtual machine, including the following steps:

step S1: and running agent software on the virtual machine for capturing desktop image data of the virtual machine.

Step S2: the agent software captures desktop image data and writes the desktop image data into a buffer queue based on the front-end driver.

And step S3: and the back end driver actively pulls the desktop image data in the buffer queue and transmits the pulled desktop data to the server of the host.

And step S4: and the server side of the host machine pushes the acquired desktop image data to the client side of the host machine.

It can be understood that, in step S1, the agent software is agent software installed on the virtual machine, and may be used to capture image data of a physical video memory in the virtual machine. An agent program agent is run in the virtual machine, and desktop image data is obtained through an API (application program interface) of an operating system of the virtual machine, such as DXGI (extensible markup language) or EGL (extended markup language).

It is understood that, in step S2, the agent captures desktop image data, for example, in Linux, there are various ways to capture screen data, and according to the type of the system gui, the acquisition of the image data needs to be completed through a system interface such as x11, EGL, or wayland. After the agent captures the desktop image data, the desktop image data is written into a vring buffer queue based on virtio front-end driving, and the vring buffer queue is used for the host machine and the virtual machine to transmit data. The specific process is shown in fig. 2.

It can be understood that in step S3, the virtio backend driver actively pulls data in the vring queue, and transmits the pulled desktop image data to the server of the host, and the host receives the data by using the VNC server.

It can be understood that, in step S4, after receiving the data, the VNC server pushes the data to the VNC client, so as to achieve the effect of accessing the virtual machine desktop.

It can be understood that the virtiol front-end driver is divided into a linux end and a window end, and after the system is installed, a driver needs to be loaded, and a bottom-layer interface is mainly provided for the upper-layer application agent to call. And the virtio back-end driver is mainly used for processing data sent by the virtual machine. And adding the backend user-defined equipment, namely recompiling qemu codes, and adding a virtio ID, a device ID and corresponding processing codes according to a virtio specification. On the other hand, since the virtio device belongs to a PCI device, and the real device discovery and configuration operations depend on the PCI protocol, the virtio device needs to be mounted on the virtio bus and also needs to be added to the PCI so as to be perceived by the outer layer. Finally, when the virtual machine is started, a command line parameter-qemu-device needs to be added to enable the user-defined device.

Referring to fig. 3, step S2 specifically includes:

step S21: and initializing the environment according to the initialization flow of the DXGI interface.

Step S22: an idxgioutputdupling object instance is obtained, and desktop image data is obtained by the AcquireNextFrame function.

It is understood that in step S21, implemented by the agent service, in window, the environment needs to be initialized according to the initialization procedure of the DXGI interface.

Optionally, as an embodiment, in step S21, it is checked whether the desktop image data is updated based on the DXGI interface function, and if not, the sleep is performed at the code level; if so, the image data is encoded, which can reduce the performance overhead of the system.

Optionally, as another embodiment, in step S21, based on the pixel format of the host VNC server, the DXGI interface selects the same pixel format for environment initialization. Specifically, the VNC server and the VNC client of the host use RFB as a transmission protocol, and the DXGI interface needs to convert image data and convert and package frame data into a pixel format accepted by the RFB protocol.

It is understood that in step S22, the idxgioutputdupling object instance is obtained based on the DXGI (DirectX Graphics Infrastructure) interface of the operating system, and the AcquireNextFrame function is a common desktop image data acquisition function.

It can be understood that the initialization of the VNC Server and the interaction process with the VNC client need to be adjusted, a part of the adjustment needs to be concentrated in a displaychangelisteneropsdclops structure object, the object member is a pointer function, which defines an operation set for monitoring changes of the display device, another related object is structVncJobQueue, which is used for the graphics card device to provide data to the VNC Server, at this time, the desktop image data source in the queue is no longer from the virtual display memory but from the vring queue, the data in the queue comes from the virtio-defined device, and when the backend driver receives the data transmitted from the front end, the data is cached in the queue first.

After step S22, the method further includes:

step S23: and drawing the shape of the mouse into a page according to the position of the mouse obtained by query.

It can be understood that, in step S23, the desktop image data obtained through the AcquireNextFrame function does not contain a mouse graphic, so that after the image frame is obtained, a drawing operation needs to be performed once, and the shape of the mouse is drawn into the page according to the mouse position obtained through the query.

Compared with the prior art, the method for directly accessing the desktop of the virtual machine by the display card has the following advantages that:

In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart.

Which when executed by a processor performs the above-described functions defined in the method of the present application. It should be noted that the computer memory described herein can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer memory may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof.

More specific examples of computer memory may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, a computer readable signal medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In this application, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

Computer program code for carrying out operations for aspects of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The present invention is not limited to the above preferred embodiments, and any modifications, equivalent alterations and improvements made within the spirit of the present invention should be included in the scope of the present invention.

Claims

1. A method for accessing a virtual machine desktop under the direct connection of a display card is characterized by comprising the following steps: the method comprises the following steps:

step S1: running agent software on the virtual machine, wherein the agent software is used for capturing desktop image data of the virtual machine;

step S2: the agent software captures desktop image data and writes the desktop image data into a buffer queue based on front-end drive;

and step S3: the back end driver actively pulls the desktop image data in the buffer queue and transmits the pulled desktop image data to the server of the host machine;

2. The method for the graphics card to access the desktop of the virtual machine directly as claimed in claim 1, wherein: the agent software is agent software, the front-end driver is virtio front-end driver, the back-end driver is virtio back-end driver, the buffer queue is a vring queue, the server of the host is a VNC server, and the client is a VNC client.

3. The method for the graphics card to access the desktop of the virtual machine directly as claimed in claim 1, wherein: the step S2 specifically includes:

step S21: environment initialization is required to be carried out according to the initialization flow of the DXGI interface;

4. The method for video card to access the desktop of a virtual machine as claimed in claim 3, wherein:

step S22 is followed by:

5. The method for the graphics card to access the desktop of the virtual machine directly as claimed in claim 3, wherein: in step S21, checking whether the desktop image data is updated based on the DXGI interface function, and if not, performing hibernation in the code level; if yes, encoding the image data.

6. The method for the graphics card to access the desktop of the virtual machine directly as claimed in claim 3, wherein: in step S21, based on the pixel format of the host server, the DXGI interface selects the same pixel format for environment initialization.