CN117992157A - Image display method and device of virtual machine - Google Patents
Image display method and device of virtual machine Download PDFInfo
- Publication number
- CN117992157A CN117992157A CN202211366888.1A CN202211366888A CN117992157A CN 117992157 A CN117992157 A CN 117992157A CN 202211366888 A CN202211366888 A CN 202211366888A CN 117992157 A CN117992157 A CN 117992157A
- Authority
- CN
- China
- Prior art keywords
- virtual
- virtual machine
- layers
- image
- hardware
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 52
- 230000002194 synthesizing effect Effects 0.000 claims abstract description 35
- 238000009877 rendering Methods 0.000 claims abstract description 23
- 238000012545 processing Methods 0.000 claims description 39
- 238000004590 computer program Methods 0.000 claims description 16
- 238000005516 engineering process Methods 0.000 abstract description 2
- 230000015572 biosynthetic process Effects 0.000 description 15
- 238000003786 synthesis reaction Methods 0.000 description 15
- 238000010586 diagram Methods 0.000 description 11
- 230000006870 function Effects 0.000 description 8
- 230000005540 biological transmission Effects 0.000 description 5
- 238000004891 communication Methods 0.000 description 5
- 238000007726 management method Methods 0.000 description 5
- 230000007246 mechanism Effects 0.000 description 2
- 239000000203 mixture Substances 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000010438 heat treatment Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000013468 resource allocation Methods 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/451—Execution arrangements for user interfaces
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Processing Or Creating Images (AREA)
Abstract
The embodiment of the application provides an image display method and device of a virtual machine, wherein the method comprises the following steps: acquiring a plurality of first layers generated by an application program from a virtual machine; requesting, by the host virtual machine, use of virtual hardware resources; and respectively rendering the plurality of first layers according to the virtual hardware resources to obtain a plurality of second layers, synthesizing the plurality of second layers to obtain a target image, and displaying the target image. The embodiment of the application can solve the problem that different operating systems cannot share one set of display equipment in the related technology, and by virtualizing hardware resources, different operating systems can share one set of display equipment, thereby saving hardware cost.
Description
Technical Field
The application relates to the field of image display, in particular to an image display method and device of a virtual machine.
Background
The multi-display screen is used in the life and is quite common, the multi-screen display function is provided under the scenes of an automobile cabin, an intelligent home and the like, some of the screens provide display interfaces of instrument and meter data, some of the screens provide interfaces of information entertainment, and a user can perform interactive operation on the display screens at any time. The images displayed by the multi-screen are derived from the application of the upper layer, and the images of the upper layer are displayed on the screen through a series of operations of rendering, synthesizing, displaying by a display and the like of the images, wherein the operations are all realized by related hardware support.
In the use scenario of multiple screens, the display content may originate from different operating systems, where different display driving frameworks are used, and different driving frameworks mean that the operating systems cannot share a set of display devices, so that the operating systems hope to be able to display hardware (such as a graphics processor and a display screen) exclusively, but this reduces the utilization rate of the hardware and increases the cost of the hardware.
The existing virtualized hardware method of the multi-screen virtualization technical scheme mostly adopts a virtualized graphics card method, the graphics card is used as the only display hardware, the graphics card is responsible for image rendering and synthesizing work, the Hypervisor is responsible for specific task scheduling, the hardware is delivered to different operating systems for use, but the graphics card is not a special device for image synthesizing, the synthesizing efficiency is low, and the graphics card itself already bears a large amount of computing and rendering work and participates in image synthesizing work, the graphics card support with larger calculation force is needed, and the deployment cost of the system is improved.
Therefore, there is no reasonable solution to the problem that different operating systems cannot share a set of display devices in the related art.
Disclosure of Invention
The embodiment of the application provides an image display method and device of a virtual machine, which at least solve the problem that different operating systems in the related art cannot share one set of display equipment.
According to an embodiment of the present application, there is provided an image display method of a virtual machine, applied to a slave virtual machine, the method including:
acquiring a plurality of first layers generated by an application program;
Requesting, by the host virtual machine, use of virtual hardware resources;
And respectively rendering the first layers according to the virtual hardware resources to obtain a plurality of second layers, synthesizing the second layers to obtain a target image, and displaying the target image.
According to an embodiment of the present application, there is provided an image display method of a virtual machine, applied to a host virtual machine, the method including:
receiving a use application requesting use of virtual hardware resources from a virtual machine;
and calling the virtual hardware resource, providing the virtual hardware resource for the slave virtual machine to instruct the slave virtual machine to respectively render a plurality of first layers generated by an application program according to the virtual hardware resource to obtain a plurality of second layers, synthesizing the plurality of second layers to obtain a target image, and displaying the target image.
According to another embodiment of the present application, there is provided an image display apparatus of a virtual machine, applied to a slave virtual machine, the apparatus including:
the first acquisition module is used for acquiring a plurality of first layers generated by the application program;
the second acquisition module is used for acquiring virtual hardware resources through the main virtual machine;
and the image processing module is used for respectively rendering the plurality of first image layers according to the virtual hardware resources to obtain a plurality of second image layers, synthesizing the plurality of second image layers to obtain a target image, and displaying the target image.
According to another embodiment of the present application, there is provided an image display apparatus of a virtual machine, applied to a main virtual machine, the apparatus including:
the first receiving module is used for receiving a use application for acquiring virtual hardware resources from the virtual machine;
the calling module is used for calling the virtual hardware resources, feeding the virtual hardware resources back to the slave virtual machine, indicating the slave virtual machine to respectively render a plurality of first image layers generated by an application program according to the virtual hardware resources to obtain a plurality of second image layers, synthesizing the plurality of second image layers to obtain a target image, and displaying the target image.
According to another embodiment of the present application, there is provided an image display system of a virtual machine, the system including:
the host virtual machine is used for receiving a use application of the virtual hardware resources acquired by the slave virtual machine; invoking the virtual hardware resource and feeding back the virtual hardware resource to the slave virtual machine;
One or more slave virtual machines for obtaining a plurality of first layers generated by the application program; respectively rendering the first layers according to the virtual hardware resources to obtain a plurality of second layers, synthesizing the second layers to obtain a target image, and displaying the target image;
And the Hardware library Hardware is used for integrating Hardware resources required by image display, wherein the Hardware resources comprise: the display control unit is used for displaying the display interface and the display equipment;
The hypervisor Hypersivor is configured to virtualize the hardware resource to obtain the virtual hardware resource, where the virtual hardware resource includes: virtual graphics processor, virtual display control unit, virtual display interface and virtual display device.
According to a further embodiment of the present application, there is also provided a computer-readable storage medium having stored therein a computer program, wherein the computer program, when executed by a processor, performs the steps of any of the method embodiments described above.
According to a further embodiment of the application, there is also provided an electronic device comprising a memory having stored therein a computer program and a processor arranged to run the computer program to perform the steps of any of the method embodiments described above.
The embodiment of the application provides an image display method and device of a virtual machine, which are characterized in that an integrated set of hardware resources is subjected to virtualization processing, then a host virtual machine manages the call of each virtual hardware resource, virtual hardware resources are allocated according to the use requirement of a slave virtual machine, and the rendering, synthesis and display of images are completed, so that a plurality of virtual machines of different operating systems can share one set of image display equipment, and the hardware cost is saved.
Drawings
Fig. 1 is a hardware configuration block diagram of an image display method of a virtual machine according to an embodiment of the present application;
FIG. 2 is a flow chart of a method of displaying images of a virtual machine according to an embodiment of the present application;
FIG. 3 is a flowchart of an image display method of a virtual machine according to an embodiment of the present application;
FIG. 4 is an architecture diagram of a virtual machine image display system according to another embodiment of the present application;
FIG. 5 is a workflow diagram of a composite portion of an image display method of a virtual machine in an embodiment of the application;
FIG. 6 is an overall workflow diagram of an image display method of a virtual machine in an embodiment of the application;
fig. 7 is a block diagram of an image display apparatus of a slave virtual machine according to an embodiment of the present application;
fig. 8 is a block diagram of an image display apparatus of a host virtual machine according to an embodiment of the present application.
Detailed Description
Embodiments of the present application will be described in detail below with reference to the accompanying drawings in conjunction with the embodiments.
It should be noted that the terms "first," "second," and the like in the description and the claims of the present application and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order.
The method embodiments provided in the embodiments of the present application may be performed in a mobile terminal, a computer terminal, a cloud server, or similar computing device. Taking a computer terminal as an example, fig. 1 is a block diagram of a hardware structure of an image display method of a virtual machine according to an embodiment of the present application, as shown in fig. 1, a hardware board may include one or more (only one is shown in fig. 1) processors 102 (the processors 102 may include, but are not limited to, a microprocessor (Microcontroller Unit, abbreviated as MCU) or a processing device such as a programmable logic device) and a memory 104 for storing data, where the mobile terminal may further include a transmission device 106 for a communication function and an input/output device 108. It will be appreciated by those skilled in the art that the structure shown in fig. 1 is merely illustrative and not limiting of the structure of the mobile terminal described above. For example, the mobile terminal may also include more or fewer components than shown in fig. 1, or have a different configuration than shown in fig. 1.
The memory 104 may be used to store computer programs, such as software programs of application software and modules, such as computer programs corresponding to the image display method of the virtual machine in the embodiment of the present application, and the processor 102 executes the computer programs stored in the memory 104, thereby executing various functional applications and virtual desktop connections, that is, implementing the method described above. Memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include memory remotely located relative to the processor 102, which may be connected to the mobile terminal via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The transmission means 106 is arranged to receive or transmit data via a network. Specific examples of the network described above may include a wireless network provided by a communications provider. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, simply referred to as a NIC) that can connect to other network devices through a base station to communicate with the internet. In one example, the transmission device 106 may be a Radio Frequency (RF) module, which is used to communicate with the internet wirelessly.
In this embodiment, an image display method of a virtual machine is provided, which is applied to a slave virtual machine, and fig. 2 is a flowchart of an image display method of a virtual machine according to an embodiment of the present application, as shown in fig. 2, and the flowchart includes the following steps:
step S202, a plurality of first layers generated by an application program are obtained;
step S204, requesting to use virtual hardware resources through the main virtual machine;
Step S206, respectively rendering the first layers according to the virtual hardware resources to obtain a plurality of second layers, synthesizing the second layers to obtain a target image, and displaying the target image.
In this embodiment, step S204 may specifically include:
step S2042 is to send an application for using the virtual hardware resource to the host virtual machine, where the application for using is used to instruct the host virtual machine to call the virtual hardware resource, and the virtual hardware resource includes: a virtual graphics processor, a virtual display control unit, a virtual display interface, and a virtual display device;
step S2044, receiving the virtual hardware resource provided by the host virtual machine.
In this embodiment, step S2042 may specifically include:
generating an interrupt signal for applying for the virtual hardware resource;
sending the interrupt signal to the main virtual machine through an event channel;
in this embodiment, the system architecture of the master virtual machine and the slave virtual machine may be established using the open source virtualization technique Xen.
Further, the master Virtual machine communicates with the slave Virtual machine by using an event channel, which is an asynchronous event notification mechanism, and physical interrupts (pIRQ), virtual interrupts (vIRQ), inter-Virtual Inter-Processor Interrupt (vIPI) and Inter-Virtual Inter-domain communication (IDC) on Xen architecture are all implemented through the event channel.
In this embodiment, the virtual Hardware resources in step S2042 are obtained by the hypervisor Hypersivor performing virtualization processing on the Hardware resources in the Hardware library Hardware, where the Hardware resources include: graphics processor (Graphics Processing Unit, GPU for short), display control unit, display interface, and display device.
Further, hypersivor can virtualize all the Hardware resources in Hardware and have useful virtualized Hardware interfaces and virtual Hardware driver scheduling functions. Specifically, hypersivor performs virtualization processing on the graphics processor in Hardware to obtain a virtual graphics processor; hypersivor carrying out virtualization processing on the display control unit in Hardware to obtain a virtual display control unit; hypersivor carrying out virtualization processing on the display interface in Hardware to obtain a virtual display interface; hypersivor carrying out virtualization processing on the display device in Hardware to obtain a virtual display device.
In this embodiment, step S206 may specifically include:
Step S2062, rendering the plurality of first layers by the virtual graphics processor, respectively, to obtain the plurality of second layers;
Step S2064, synthesizing the plurality of second image layers by the virtual graphics processor or the virtual display control unit to obtain the target image;
step S2066, transmitting the target image to the virtual display device for display via the virtual display interface.
In this embodiment, step S2064 may specifically include:
Carrying out layer aliasing processing on the plurality of second layers to obtain a layer aliasing result;
determining the image layer aliasing result as the target image under the condition that the image layer aliasing result does not need small window display or split screen display;
and under the condition that the image layer aliasing result needs small window display or split screen display, carrying out small window processing or split screen processing on the image layer aliasing result to obtain the target image.
Further, the virtual display control unit in step S2064 may display images of different sources by implementing screen separation or widget display on the same display using the virtualized PIP/PIP (Picture in Picture/Picture out Picture, abbreviated as PIP-POP) technology.
In this embodiment, the virtual Hardware resource further includes a virtual memory, and the corresponding Hardware resource in the Hardware library Hardware further includes a memory device. The image processing result of each step in the above embodiment may be stored through a virtual memory, and the plurality of first image layers, the plurality of second image layers, and the target image are stored into the virtual memory; the virtual memory is obtained by carrying out virtualization processing on memory devices in the Hardware library Hardware by the hypervisor Hypersivor.
Further, in the above embodiment, the first layers, the second layers and the target image may be obtained from a virtual memory. The layer aliasing result in step S2064 may also be stored in the virtual memory.
In this embodiment, through the steps S202 to S206, the problem that different operating systems cannot share a set of display device in the related art can be solved, and virtual hardware resources are called from a virtual machine through a main virtual machine, so that rendering, synthesizing and displaying of images are realized, the basic requirements on image processing are met, and the cost required for deploying and installing hardware resources is reduced.
In an embodiment of the present application, an image display method of a virtual machine is provided and applied to a host virtual machine, and fig. 3 is a flowchart of the image display method of the virtual machine according to an embodiment of the present application, as shown in fig. 3, the flowchart includes the following steps:
step S302, receiving a use application requesting to use virtual hardware resources from a virtual machine;
step S304, calling the virtual hardware resources, providing the virtual hardware resources for the slave virtual machine to instruct the slave virtual machine to respectively render a plurality of first image layers generated by an application program according to the virtual hardware resources to obtain a plurality of second image layers, synthesizing the plurality of second image layers to obtain a target image, and displaying the target image.
In this embodiment, the virtual hardware resources in step S304 include: virtual graphics processor, virtual display control unit, virtual display interface and virtual display device.
Further, step S304 may specifically include:
Step S3042, calling the virtual graphics processor and providing the virtual graphics processor to the slave virtual machine, where the virtual graphics processor is configured to instruct the slave virtual machine to render the plurality of first layers to obtain the plurality of second layers;
step S3044, calling the virtual display control unit and providing the virtual display control unit to the slave virtual machine, where the virtual display control unit is configured to instruct the slave virtual machine to synthesize the plurality of second image layers, so as to obtain the target image;
Step S3046, calling the virtual display interface and the virtual display device, and providing the virtual display interface and the virtual display device to the slave virtual machine, where the virtual display interface is configured to instruct the slave virtual machine to send the target image to the virtual display device, and the virtual display device is configured to display the target image.
In another embodiment, the host virtual machine is also called a privileged virtual machine, and only the privileged virtual machine can directly apply for virtual hardware resources to the hypervisor Hypersivor, and other common virtual machines can only indirectly apply through the privileged virtual machine. And further, the damage to the system caused by the wrong call of the common virtual machine is avoided.
In this embodiment, step S302 may specifically include:
receiving an interrupt signal sent from a virtual machine through an event channel, wherein the interrupt signal is used for requesting to use the virtual hardware resource;
The virtual Hardware resource is obtained by carrying out virtualization processing on a Hardware resource in a Hardware library Hardware by using a hypervisor Hypersivor, and the Hardware resource comprises: the display device comprises a graphic processor, a display control unit, a display interface and a display device.
In this embodiment, the virtual hardware resource in the above step further includes a virtual memory; and calling the virtual memory and providing the virtual memory to the slave virtual machine, wherein the virtual memory is used for the slave virtual machine to store the first image layers, the second image layers and the target image, and the virtual memory is obtained by carrying out virtualization processing on memory devices in a Hardware library Hardware by a hypervisor Hypersivor.
In this embodiment, through the steps S302 to S304, the problem that different operating systems cannot share one set of display device in the related art can be solved, the master virtual machine invokes virtual hardware resources and provides the virtual hardware resources for the slave virtual machines to use, so as to implement rendering, synthesizing and displaying of images, implement statistical management and scheduling of the virtual hardware resources, meet the use requirements of multiple virtual machines by using one set of hardware resources, and reduce the cost required for deploying and installing the hardware resources.
In another embodiment of the present application, a virtual machine image display system is provided, which adopts a virtualization scheme of Xen architecture, but the present application may also adopt other virtualization schemes to implement the same management of hardware.
Fig. 4 is a schematic diagram of a virtual machine image display system according to another embodiment of the present application, as shown in fig. 4, the system including: hardware layer 42, virtualization layer 44, privileged virtual machine 46, and normal virtual machine 48.
In this embodiment, the hardware layer 42 is used to integrate display related hardware resources. Specific hardware resources may include: the device comprises a Display screen Display, a Display interface, a memory allocation and storage unit, a graphic processor, an image processing unit, a layer aliasing unit and a PIP-POP unit.
In this embodiment, the hardware layer 42 may simulate hardware by using a simulated device model, and simulate a physical device that can be identified by an operating system by using software, so as to realize sharing of display hardware devices.
Further, each type of hardware resource in the hardware layer 42 may have a plurality of hardware units with the same function, and each unit may independently provide a corresponding service for the upper virtual machine, but each hardware unit may be allocated to only one common virtual machine 48 at the same time.
In this embodiment, the virtualization layer 44 has a virtualized hardware interface and a virtualized hardware driver scheduling function.
In this embodiment, the virtualization layer 44 may perform virtualization processing on all hardware resources in the hardware layer 42, and the virtualization layer 44 determines the order in which the virtual machine system accesses the hardware.
Further, the virtualized hardware interface in virtualization layer 44 may provide a virtualized hardware interface for an upper layer virtual machine to invoke virtual display hardware by privileged virtual machine 46.
In this embodiment, virtualization layer 44 is implemented by a hypervisor, also known as a virtual machine monitor (Virtual Machine Monitor, referred to as a VMM).
In the present embodiment, virtual machine systems are divided into two types, a privileged virtual machine 46 and a normal virtual machine 48, and one privileged virtual machine 46 can manage a plurality of normal virtual machines 48.
In this embodiment, the privileged virtual machine 46 is a virtual machine with the highest authority, and the extension of the monitor program and the management service of the system manage the virtual hardware interface abstracted by the virtualization layer, the virtualized Input/Output (I/O) and the virtualized direct memory access device (Direct Memory Access, DMA) and the virtualized display device, and the privileged virtual machine 46 can call the native drivers of these physical devices and provide the backend device drivers to the other general virtual machines 48.
In this embodiment, the common virtual machine 48 is a virtual machine for interaction with a user, and the Guest virtual machine system Guest OS is installed on the common virtual machine 48 and can directly run on hardware, where the specific number of virtual machines and the specific operating system depend on requirements.
In this embodiment, the common virtual machine 48 does not have a physical hardware device, and the access to the device needs to be achieved by the backend device driver of the privileged virtual machine 46 by applying for services from the native backend device driver to the backend device driver of the privileged virtual machine 46.
In this embodiment, all of the display image content sources are available with the application programs in the common virtual machine 48.
In this embodiment, the normal virtual machine 48 generates an interrupt when the hardware device needs to be invoked, and the interrupt signal is captured by the virtualization layer 44, but the virtualization layer 44 cannot directly access the hardware device, and needs to access the specific hardware device through the driver of the privileged virtual machine 46.
Further, when there are multiple tasks that need to apply for hardware resource services at the same time, the virtualization layer 44 may determine the order in which each task uses hardware resources. The particular resource allocation policy may depend on the temporal order or priority of the tasks.
By the embodiment of the application, the unified management of hardware resources can be realized, a set of hardware resources can be provided for different operating systems for use, the application range of the hardware resources is improved, and the deployment cost of system hardware is reduced.
Fig. 5 is a flowchart of an image display method synthesizing section of a virtual machine in the embodiment of the present application. The method can be applied to the virtual machine image display system in the above embodiment, as shown in fig. 5, in the embodiment of the present application, two image synthesis modes are adopted: graphics processor synthesis, and display control unit synthesis.
In the present embodiment, the image synthesizing method mainly uses a display control unit (Display Controller, abbreviated as DPC) for synthesizing, and the image synthesizing method can use a graphics processor for synthesizing only the function that the present application supports synthesizing an image with the graphics processor.
Specifically, the display control unit is used for reading the image or video data from the storage unit, processing the image or video data by the image processing unit, mixing the overlapped layers, and sending the final data to the display interface for display. The main hardware unit in the display control unit is provided with an image processing, display interface and image layer aliasing.
In this embodiment, the graphics processor and the display control unit use hardware to layer multiple graphics into one picture, except that the graphics processor GPU is a general purpose computing hardware, and cannot optimize the composition function of the graphics layer, for example: the GPU does not know the stacking sequence of the layers, so that the efficiency in synthesis is lower; the GPU needs to access the memory frequently during synthesis, the display controller can realize online synthesis, the memory does not need to be accessed frequently, and the efficiency is higher; graphics processors cannot convert the format of an image, requiring more post-processing support. And the display control unit is optimized for the above composition functions.
In this embodiment, the common Virtual machine and the privileged Virtual machine communicate with each other by using an event channel, which is an asynchronous event notification mechanism, and physical interrupts (pIRQ), virtual interrupts (vIRQ), virtual Inter-machine process interrupts (Processor Interrupt, vIPI) and Virtual Inter-domain communications (Inter-Domain Communication, IDC) on Xen architecture are all implemented by using the event channel.
In this embodiment, a guest virtual operating system on a common virtual machine cannot directly access physical resources other than the present virtual machine, and only applies resources to a privileged virtual machine through a virtualization layer.
In this embodiment, when the virtualization layer is implemented using a hypervisor, the privileged virtual machine itself may apply for hardware resource services to the hypervisor by invoking the hypercall hypercalls. When there are multiple tasks that require application of the hardware resource service, hypersivor determines the order in which the tasks can use the hardware resources.
Further, hypercalls only allows privileged virtual machine calls with higher priority to avoid common virtual machine false calls hypercalls, resulting in possible disruption to the system.
In this embodiment, a driver on a guest virtual operating system on a common virtual machine is abstracted into a front-end device driver, and applies for resources to a back-end device driver of a privileged virtual machine, and the privileged virtual machine invokes underlying hardware (hardware layer) by using the back-end device driver.
In this embodiment, an application program on the general virtual machine generates image data, and when rendering is required, the general virtual machine generates an interrupt, and the virtualization layer captures and notifies the privileged virtual machine to call the virtual graphics processor GPU to render the image data, and the rendering result is stored in the virtual storage space. The synthesis mode is specified by an upper layer application, and if the graphics processor synthesis is needed, a privileged virtual machine calls a virtual Graphics Processor (GPU) to synthesize image data and displays the image data through a virtual display interface; and if the display control unit is needed to synthesize, calling the virtual display control unit to synthesize the image data by the privileged virtual machine and displaying the image data through the virtual display interface.
Fig. 6 is an overall workflow diagram of an image display method of a virtual machine in an embodiment of the present application. The method can be applied to the virtual machine image display system in the above embodiment, as shown in fig. 6, and specifically includes the following steps:
in step S601, the slave virtual machine Guest OS generates image data, the image rendering needs to be completed by calling the graphics processor, the Guest OS generates an interrupt, and the master virtual machine Domain0 takes over and calls the virtual graphics processor to render the image.
In step S602, the host virtual machine invokes the virtual graphics processor to render the image data, and stores the rendered result in the virtual storage space FrameBuffer. Step S603a, if the graphics processor is designated as the synthesized image hardware from the Guest OS end of the virtual machine, the host virtual machine calls the virtual graphics processor to extract the image data from the virtual storage space frame buffer and synthesize an image, and the synthesized result is stored in the shared Memory;
In step S603, if the use of the hardware synthesized image is designated from the Guest OS end of the virtual machine, the host virtual machine invokes the display hardware, and the display control unit DPC extracts the image data from the virtual Memory frame buffer to synthesize the image, and the synthesized result is also stored in the shared Memory.
In step S604, if the display screen has a requirement for displaying a small window or displaying a split screen, and the display content cannot be overlapped with the image synthesized by the hardware, after the image synthesis is completed, using the virtualized PIP-POP hardware to open up a separate window to display the small window or the split screen content on the synthesized image.
In step S605, the display hardware reads the image data in the virtual memory space FrameBuffer and transmits the image data to the display screen for display. In this embodiment, the images rendered in step S602 are in a layered state and cannot be directly displayed on the screen, and the images are synthesized into one image according to the given screen arrangement sequence through the image synthesis operation in step S603.
In this embodiment, the virtual machine may also specify the use of a graphics processor to synthesize an image. The main virtual machine calls display hardware, and uses the graphic processor GPU to extract image data from the virtual memory space frame buffer to synthesize the images.
Further, if the virtual machine needs to synthesize images quickly, a graphics processor can be selected for synthesizing, and if high efficiency is needed, a display control unit can be selected for synthesizing images. Both schemes can help the virtual machine find the greatest balance in the speed and efficiency of synthesizing images.
By the embodiment of the application, unified management of all display hardware can be realized, so that the utilization rate of hardware resources is improved. Meanwhile, the problems of low image synthesis efficiency and large heating value caused by image synthesis only through a graphics processor are solved, and the synthesis efficiency is improved by utilizing a display control unit with higher synthesized image performance.
According to another embodiment of the present application, there is also provided an image display apparatus of a virtual machine, which is applied to a slave virtual machine.
Fig. 7 is a block diagram of an image display apparatus of a slave virtual machine according to an embodiment of the present application, which includes the following modules as illustrated in fig. 7:
a first obtaining module 72, configured to obtain a plurality of first layers generated by an application program;
A second obtaining module 74, configured to obtain virtual hardware resources through the host virtual machine;
The image processing module 76 is configured to render the plurality of first image layers according to the virtual hardware resources, obtain a plurality of second image layers, synthesize the plurality of second image layers, obtain a target image, and display the target image.
In the present embodiment, the second acquisition module 74 includes:
The first sending unit is configured to send an application for using the virtual hardware resource to the host virtual machine, where the application for using is used to instruct the host virtual machine to call the virtual hardware resource, and the virtual hardware resource includes: a virtual graphics processor, a virtual display control unit, a virtual display interface, and a virtual display device;
The first receiving unit is used for receiving the virtual hardware resources provided by the main virtual machine.
In this embodiment, the first sending unit is further configured to generate an interrupt signal for applying for the virtual hardware resource; sending the interrupt signal to the main virtual machine through an event channel; the virtual Hardware resource is obtained by carrying out virtualization processing on a Hardware resource in a Hardware library Hardware by using a hypervisor Hypersivor, and the Hardware resource comprises: the display device comprises a graphic processor, a display control unit, a display interface and a display device.
In the present embodiment, the image processing module 76 includes:
The rendering unit is used for respectively rendering the plurality of first layers through the virtual graphic processor to obtain the plurality of second layers;
A synthesizing unit, configured to synthesize the plurality of second layers through the virtual graphics processor or the virtual display control unit, to obtain the target image;
And the second sending unit is used for sending the target image to the virtual display device for display through the virtual display interface.
In this embodiment, the synthesis unit is further configured to perform layer aliasing processing on the plurality of second layers to obtain a layer aliasing result; determining the image layer aliasing result as the target image under the condition that the image layer aliasing result does not need small window display or split screen display; and under the condition that the image layer aliasing result needs small window display or split screen display, carrying out small window processing or split screen processing on the image layer aliasing result to obtain the target image.
In this embodiment, the apparatus further includes a first storage module, configured to store the plurality of first layers, the plurality of second layers, and the target image into the virtual memory; the virtual memory is obtained by carrying out virtualization processing on memory devices in the Hardware library Hardware by the hypervisor Hypersivor.
According to another embodiment of the present application, there is also provided an image display apparatus of a virtual machine, which is applied to a host virtual machine.
Fig. 8 is a block diagram of an image display apparatus of a host virtual machine according to an embodiment of the present application, and as illustrated in fig. 8, the apparatus includes the following modules:
a first receiving module 82, configured to receive a use application for obtaining virtual hardware resources from a virtual machine;
And the calling module 84 is configured to call the virtual hardware resource, feed back the virtual hardware resource to the slave virtual machine, instruct the slave virtual machine to render multiple first layers generated by the application program according to the virtual hardware resource, obtain multiple second layers, synthesize the multiple second layers, obtain a target image, and display the target image.
In this embodiment, the virtual hardware resources called by the calling module 84 include: virtual graphics processor, virtual display control unit, virtual display interface and virtual display device.
In this embodiment, the calling module 84 further includes:
The display card calling unit is used for calling the virtual graphic processor and providing the virtual graphic processor for the slave virtual machine, wherein the virtual graphic processor is used for indicating the slave virtual machine to render the plurality of first layers to obtain the plurality of second layers;
a display control calling unit for calling the virtual display control unit and providing the virtual display control unit to the slave virtual machine, wherein the virtual display control unit is used for indicating the slave virtual machine to synthesize the plurality of second image layers to obtain the target image
The interface calling unit is used for calling the virtual display interface and the virtual display device and providing the virtual display interface and the virtual display device for the slave virtual machine, wherein the virtual display interface is used for indicating the slave virtual machine to send the target image to the virtual display device, and the virtual display device is used for displaying the target image.
In this embodiment, the first receiving module 82 is configured to receive, through an event channel, the interrupt signal sent from the virtual machine, where the interrupt signal is used to request to use the virtual hardware resource; the virtual Hardware resource is obtained by carrying out virtualization processing on a Hardware resource in a Hardware library Hardware by using a hypervisor Hypersivor, and the Hardware resource comprises: the display device comprises a graphic processor, a display control unit, a display interface and a display device.
In this embodiment, the calling module 84 further includes a memory calling unit, configured to call the virtual memory and provide the virtual memory to the slave virtual machine, where the virtual memory is used for the slave virtual machine to store the plurality of first image layers, the plurality of second image layers, and the target image, and the virtual memory is obtained by performing virtualization processing on a memory device in the Hardware library Hardware by the hypervisor Hypersivor.
According to still another embodiment of the present application, there is also provided an image display system of a virtual machine, including:
the host virtual machine is used for receiving a use application of the virtual hardware resources acquired by the slave virtual machine; invoking the virtual hardware resource and feeding back the virtual hardware resource to the slave virtual machine;
One or more slave virtual machines for obtaining a plurality of first layers generated by the application program; respectively rendering the first layers according to the virtual hardware resources to obtain a plurality of second layers, synthesizing the second layers to obtain a target image, and displaying the target image;
And the Hardware library Hardware is used for integrating Hardware resources required by image display, wherein the Hardware resources comprise: the display control unit is used for displaying the display interface and the display equipment;
The hypervisor Hypersivor is configured to virtualize the hardware resource to obtain the virtual hardware resource, where the virtual hardware resource includes: virtual graphics processor, virtual display control unit, virtual display interface and virtual display device.
In this embodiment, the master virtual machine corresponds to the privileged virtual machine 46 in the above embodiment, the slave virtual machine corresponds to the normal virtual machine 48 in the above embodiment, the Hardware library Hardware corresponds to the Hardware layer 42 in the above embodiment, and the hypervisor Hypersivor corresponds to the virtualization layer 44 in the above embodiment.
Embodiments of the present application also provide a computer readable storage medium having a computer program stored therein, wherein the computer program when executed by a processor performs the steps of any of the method embodiments described above.
In one exemplary embodiment, the computer readable storage medium may include, but is not limited to: a usb disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory RAM), a removable hard disk, a magnetic disk, or an optical disk, or other various media capable of storing a computer program.
An embodiment of the application also provides an electronic device comprising a memory having stored therein a computer program and a processor arranged to run the computer program to perform the steps of any of the method embodiments described above.
In an exemplary embodiment, the electronic apparatus may further include a transmission device connected to the processor, and an input/output device connected to the processor.
Specific examples in this embodiment may refer to the examples described in the foregoing embodiments and the exemplary implementation, and this embodiment is not described herein.
It will be appreciated by those skilled in the art that the modules or steps of the application described above may be implemented in a general purpose computing device, they may be concentrated on a single computing device, or distributed across a network of computing devices, they may be implemented in program code executable by computing devices, so that they may be stored in a storage device for execution by computing devices, and in some cases, the steps shown or described may be performed in a different order than that shown or described herein, or they may be separately fabricated into individual integrated circuit modules, or multiple modules or steps of them may be fabricated into a single integrated circuit module. Thus, the present application is not limited to any specific combination of hardware and software.
The above description is only of the preferred embodiments of the present application and is not intended to limit the present application, but various modifications and variations can be made to the present application by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the principle of the present application should be included in the protection scope of the present application.
Claims (15)
1. An image display method of a virtual machine, applied to a slave virtual machine, characterized in that the method comprises:
acquiring a plurality of first layers generated by an application program;
Requesting, by the host virtual machine, use of virtual hardware resources;
And respectively rendering the first layers according to the virtual hardware resources to obtain a plurality of second layers, synthesizing the second layers to obtain a target image, and displaying the target image.
2. The method of claim 1, wherein requesting, by the host virtual machine, use of the virtual hardware resource comprises:
Sending a use application of the virtual hardware resource to the main virtual machine, wherein the use application is used for indicating the main virtual machine to call the virtual hardware resource, and the virtual hardware resource comprises: a virtual graphics processor, a virtual display control unit, a virtual display interface, and a virtual display device;
and receiving the virtual hardware resource provided by the main virtual machine.
3. The method of claim 2, wherein sending the application for use of the virtual hardware resource to the host virtual machine comprises:
generating an interrupt signal for applying for the virtual hardware resource;
sending the interrupt signal to the main virtual machine through an event channel;
The virtual Hardware resource is obtained by carrying out virtualization processing on a Hardware resource in a Hardware library Hardware by using a hypervisor Hypersivor, and the Hardware resource comprises: the display device comprises a graphic processor, a display control unit, a display interface and a display device.
4. The method of claim 2, wherein rendering the plurality of first layers according to the virtual hardware resource, respectively, to obtain a plurality of second layers, synthesizing the plurality of second layers, to obtain a target image, and displaying the target image comprises:
Rendering the plurality of first layers through the virtual graphics processor respectively to obtain the plurality of second layers;
synthesizing the plurality of second image layers through the virtual graphic processor or the virtual display control unit to obtain the target image;
and sending the target image to the virtual display device for display through the virtual display interface.
5. The method of claim 4, wherein synthesizing the plurality of second layers by the virtual graphics processor or the virtual display control unit results in the target image, comprising:
Carrying out layer aliasing processing on the plurality of second layers to obtain a layer aliasing result;
determining the image layer aliasing result as the target image under the condition that the image layer aliasing result does not need small window display or split screen display;
and under the condition that the image layer aliasing result needs small window display or split screen display, carrying out small window processing or split screen processing on the image layer aliasing result to obtain the target image.
6. The method according to any one of claims 1 to 5, further comprising:
the virtual hardware resource also comprises a virtual memory;
storing the plurality of first layers, the plurality of second layers and the target image into the virtual memory;
the virtual memory is obtained by carrying out virtualization processing on memory devices in the Hardware library Hardware by the hypervisor Hypersivor.
7. An image display method of a virtual machine, applied to a main virtual machine, is characterized in that the method comprises the following steps:
receiving a use application requesting use of virtual hardware resources from a virtual machine;
and calling the virtual hardware resource, providing the virtual hardware resource for the slave virtual machine to instruct the slave virtual machine to respectively render a plurality of first layers generated by an application program according to the virtual hardware resource to obtain a plurality of second layers, synthesizing the plurality of second layers to obtain a target image, and displaying the target image.
8. The method of claim 7, wherein invoking the virtual hardware resource comprises:
The virtual hardware resources include: a virtual graphics processor, a virtual display control unit, a virtual display interface, and a virtual display device;
invoking the virtual graphics processor and providing the virtual graphics processor to the slave virtual machine, wherein the virtual graphics processor is used for indicating the slave virtual machine to render the plurality of first layers to obtain the plurality of second layers;
Invoking the virtual display control unit and providing the virtual display control unit for the slave virtual machine, wherein the virtual display control unit is used for indicating the slave virtual machine to synthesize the plurality of second image layers to obtain the target image;
And invoking the virtual display interface and the virtual display device, and providing the virtual display interface and the virtual display device to the slave virtual machine, wherein the virtual display interface is used for instructing the slave virtual machine to send the target image to the virtual display device, and the virtual display device is used for displaying the target image.
9. The method of claim 7, wherein receiving a use application requesting use of virtual hardware resources from a virtual machine comprises:
receiving an interrupt signal sent from a virtual machine through an event channel, wherein the interrupt signal is used for requesting to use the virtual hardware resource;
The virtual Hardware resource is obtained by carrying out virtualization processing on a Hardware resource in a Hardware library Hardware by using a hypervisor Hypersivor, and the Hardware resource comprises: the display device comprises a graphic processor, a display control unit, a display interface and a display device.
10. The method according to any one of claims 7 to 9, further comprising:
the virtual hardware resource also comprises a virtual memory;
And calling the virtual memory and providing the virtual memory to the slave virtual machine, wherein the virtual memory is used for the slave virtual machine to store the first image layers, the second image layers and the target image, and the virtual memory is obtained by carrying out virtualization processing on memory devices in a Hardware library Hardware by a hypervisor Hypersivor.
11. An image display apparatus of a virtual machine, applied to a slave virtual machine, characterized by comprising:
the first acquisition module is used for acquiring a plurality of first layers generated by the application program;
the second acquisition module is used for acquiring virtual hardware resources through the main virtual machine;
and the image processing module is used for respectively rendering the plurality of first image layers according to the virtual hardware resources to obtain a plurality of second image layers, synthesizing the plurality of second image layers to obtain a target image, and displaying the target image.
12. An image display device of a virtual machine, applied to a main virtual machine, characterized in that the device comprises:
the first receiving module is used for receiving a use application for acquiring virtual hardware resources from the virtual machine;
the calling module is used for calling the virtual hardware resources, feeding the virtual hardware resources back to the slave virtual machine, indicating the slave virtual machine to respectively render a plurality of first image layers generated by an application program according to the virtual hardware resources to obtain a plurality of second image layers, synthesizing the plurality of second image layers to obtain a target image, and displaying the target image.
13. An image display system of a virtual machine, the system comprising:
the host virtual machine is used for receiving a use application of the virtual hardware resources acquired by the slave virtual machine; invoking the virtual hardware resource and feeding back the virtual hardware resource to the slave virtual machine;
One or more slave virtual machines for obtaining a plurality of first layers generated by the application program; respectively rendering the first layers according to the virtual hardware resources to obtain a plurality of second layers, synthesizing the second layers to obtain a target image, and displaying the target image;
And the Hardware library Hardware is used for integrating Hardware resources required by image display, wherein the Hardware resources comprise: the display control unit is used for displaying the display interface and the display equipment;
The hypervisor Hypersivor is configured to virtualize the hardware resource to obtain the virtual hardware resource, where the virtual hardware resource includes: virtual graphics processor, virtual display control unit, virtual display interface and virtual display device.
14. A computer-readable storage medium, in which a computer program is stored, wherein the computer program, when run by a processor, performs the method of any one of claims 1 to 10.
15. An electronic device comprising a memory in which a computer program is stored and a processor arranged to run the computer program to perform the method of any of claims 1 to 10.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211366888.1A CN117992157A (en) | 2022-11-01 | 2022-11-01 | Image display method and device of virtual machine |
| PCT/CN2023/119000 WO2024093546A1 (en) | 2022-11-01 | 2023-09-15 | Image display method and device for virtual machine |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211366888.1A CN117992157A (en) | 2022-11-01 | 2022-11-01 | Image display method and device of virtual machine |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117992157A true CN117992157A (en) | 2024-05-07 |
Family
ID=90900228
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211366888.1A Pending CN117992157A (en) | 2022-11-01 | 2022-11-01 | Image display method and device of virtual machine |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN117992157A (en) |
| WO (1) | WO2024093546A1 (en) |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103403732B (en) * | 2012-10-15 | 2015-07-08 | 华为技术有限公司 | Processing method and device for input and output opeartion |
| KR101401523B1 (en) * | 2013-02-08 | 2014-06-03 | 한국과학기술정보연구원 | A coarse-grained sharing gpu scheduling method and apparatus among virtual machines |
| CN104660711A (en) * | 2015-03-13 | 2015-05-27 | 华存数据信息技术有限公司 | Remote visualized application method based on virtualization of graphic processor |
| KR101765723B1 (en) * | 2016-05-09 | 2017-08-08 | 한국과학기술정보연구원 | apparatus and method for interaction between a coarse-grained GPU resource scheduler and a GPU aware scheduler |
| US11055809B2 (en) * | 2019-06-28 | 2021-07-06 | Intel Corporation | Apparatus and method for provisioning virtualized multi-tile graphics processing hardware |
| US20220331702A1 (en) * | 2021-04-15 | 2022-10-20 | Nvidia Corporation | Cloud execution of audio/video compositing applications |
-
2022
- 2022-11-01 CN CN202211366888.1A patent/CN117992157A/en active Pending
-
2023
- 2023-09-15 WO PCT/CN2023/119000 patent/WO2024093546A1/en unknown
Also Published As
| Publication number | Publication date |
|---|---|
| WO2024093546A1 (en) | 2024-05-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8924959B2 (en) | Switching hardware devices in virtual machine systems using resource converting relationships | |
| CN107077377B (en) | Equipment virtualization method, device and system, electronic equipment and computer program product | |
| US9146762B2 (en) | Specialized virtual machine to virtualize hardware resource for guest virtual machines | |
| CN107003892B (en) | GPU virtualization method, device and system, electronic equipment and computer program product | |
| US11010859B2 (en) | Display resource scheduling method and device for embedded system | |
| CN100570562C (en) | Video card, the dummy machine system of using this video card and display processing method | |
| CN104714846A (en) | Resource processing method, operating system and equipment | |
| CN112148422A (en) | IO processing method and device | |
| CN109712060B (en) | Cloud desktop display card sharing method and system based on GPU container technology | |
| CN109656646B (en) | Remote desktop control method, device, equipment and virtualization chip | |
| CN113296884A (en) | Virtualization method, virtualization device, electronic equipment, virtualization medium and resource virtualization system | |
| CN113821308B (en) | System on chip, virtual machine task processing method and device and storage medium | |
| CN108737131B (en) | Method and device for realizing network equipment virtualization | |
| CN100568180C (en) | A kind of dummy machine system and graphics card access method thereof | |
| CN113419845A (en) | Calculation acceleration method and device, calculation system, electronic equipment and computer readable storage medium | |
| CN114691286A (en) | Server system, virtual machine creation method and device | |
| CN111857943B (en) | Data processing method, device and equipment | |
| US20240062450A1 (en) | Cloud Image Rendering for Concurrent Processes | |
| US20180052700A1 (en) | Facilitation of guest application display from host operating system | |
| CN111681157A (en) | Image rendering method and system | |
| CN117992157A (en) | Image display method and device of virtual machine | |
| CN110941408A (en) | KVM virtual machine graphical interface output method and device | |
| CN115904617A (en) | GPU virtualization implementation method based on SR-IOV technology | |
| CN111813355B (en) | Distributed system for displaying content | |
| CN113051032A (en) | Application picture processing method, device and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication |