CN118170338A - Display processing method, system, equipment and medium based on hardware virtualization - Google Patents

Display processing method, system, equipment and medium based on hardware virtualization Download PDF

Info

Publication number
CN118170338A
CN118170338A CN202410450174.1A CN202410450174A CN118170338A CN 118170338 A CN118170338 A CN 118170338A CN 202410450174 A CN202410450174 A CN 202410450174A CN 118170338 A CN118170338 A CN 118170338A
Authority
CN
China
Prior art keywords
virtual machine
video memory
target
display
display screen
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202410450174.1A
Other languages
Chinese (zh)
Inventor
霍安帮
刘亮
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ecarx Hubei Tech Co Ltd
Original Assignee
Ecarx Hubei Tech Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ecarx Hubei Tech Co Ltd filed Critical Ecarx Hubei Tech Co Ltd
Priority to CN202410450174.1A priority Critical patent/CN118170338A/en
Publication of CN118170338A publication Critical patent/CN118170338A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1423—Digital output to display device ; Cooperation and interconnection of the display device with other functional units controlling a plurality of local displays, e.g. CRT and flat panel display
    • G06F3/1431—Digital output to display device ; Cooperation and interconnection of the display device with other functional units controlling a plurality of local displays, e.g. CRT and flat panel display using a single graphics controller
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1454—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/147—Digital output to display device ; Cooperation and interconnection of the display device with other functional units using display panels
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/20—Processor architectures; Processor configuration, e.g. pipelining

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Human Computer Interaction (AREA)
  • Computer Graphics (AREA)
  • Controls And Circuits For Display Device (AREA)

Abstract

The embodiment of the application provides a display processing method, a system, equipment and a medium based on hardware virtualization, wherein the method specifically comprises the following steps: the method comprises the steps that a first virtual machine obtains the display resolution of a target display screen; the first virtual machine sends the display resolution to the second virtual machine; the second virtual machine utilizes an independent graphic processing unit to allocate a target video memory object for the data to be rendered according to the display resolution, and stores a rendering result corresponding to the data to be rendered to the target video memory object; determining the size of a video memory corresponding to the target video memory object according to the display resolution; the second virtual machine sends the video memory information of the target video memory object to the first virtual machine; and the first virtual machine displays rendering results corresponding to the video memory information by utilizing the integrated graphic processing unit and the target display screen. The embodiment of the application can meet the changing requirement of resolution.

Description

Display processing method, system, equipment and medium based on hardware virtualization
Technical Field
The present application relates to the field of hardware virtualization technologies, and in particular, to a display processing method, system, device, and medium based on hardware virtualization.
Background
Hardware virtualization technology allows multiple virtual machines to run simultaneously on the same set of hardware platforms, creating a secure, independent virtual machine running environment on the hardware platform using a virtual machine monitor (Hypervisor). The hardware virtualization technology can be applied to application scenes such as car machines, cloud computing, virtualized desktops and the like.
In the related art, a display processing method based on hardware virtualization generally includes: the client virtual machine performs rendering processing on the data to be rendered by using a DGPU (independent graphic processing unit, discreteProcessing Unit), and sends an address of a rendering result to the privileged virtual machine; the privileged virtual machine then displays the rendering results using IGPU (integrated graphics Processing Unit ).
In practical application, the resolution adopted in the rendering process of DGPU is specifically: the default resolution recorded in the configuration file of the graphics subsystem service cannot meet the resolution change requirement in the related art. For example, in the case of a display screen replacement, the resolution of the display screen is changed, and the default resolution in the related art cannot meet the resolution change requirement.
Disclosure of Invention
The embodiment of the application provides a display processing method based on hardware virtualization, which can meet the changing requirement of resolution.
Correspondingly, the embodiment of the application also provides a display processing system based on hardware virtualization, electronic equipment and a machine-readable medium, which are used for ensuring the realization and application of the method.
In order to solve the above problems, an embodiment of the present application discloses a display processing method based on hardware virtualization, which includes:
the method comprises the steps that a first virtual machine obtains the display resolution of a target display screen;
the first virtual machine sends the display resolution to the second virtual machine;
The second virtual machine utilizes an independent graphic processing unit to allocate a target video memory object to the data to be rendered according to the display resolution, performs rendering processing on the data to be rendered, and stores a rendering result corresponding to the data to be rendered to the target video memory object; determining the size of the video memory corresponding to the target video memory object according to the display resolution;
the second virtual machine sends the video memory information of the target video memory object to the first virtual machine;
and the first virtual machine displays rendering results corresponding to the video memory information by utilizing the integrated graphic processing unit and the target display screen.
The embodiment of the application also discloses a display processing system based on hardware virtualization, which comprises: a first virtual machine and a second virtual machine;
Wherein the first virtual machine includes: an equipment model module; the second virtual machine includes: a graphics subsystem driver module and a graphics processing unit driver module;
The device model module is used for acquiring the display resolution of the target display screen and sending the display resolution to the second virtual machine;
The graphics subsystem driving module is used for distributing target video memory objects for data to be rendered according to the display resolution by utilizing an independent graphics processing unit, rendering the data to be rendered, storing rendering results corresponding to the data to be rendered to the target video memory objects, and sending video memory information of the target video memory objects to a first virtual machine;
The equipment model module is also used for displaying the rendering result corresponding to the video memory information by utilizing the integrated graphic processing unit and the target display screen.
The embodiment of the application also discloses an electronic device, which comprises: a processor; and a memory having executable code stored thereon that, when executed, causes the processor to perform a method according to an embodiment of the present application.
Embodiments of the present application also disclose a machine readable medium having stored thereon executable code which, when executed, causes a processor to perform a method according to an embodiment of the present application.
The embodiment of the application has the following advantages:
In the technical scheme of the embodiment of the application, a first virtual machine acquires the display resolution of a target display screen and sends the display resolution to a second virtual machine; and the second virtual machine allocates a target video memory object for the data to be rendered according to the display resolution, performs rendering processing on the data to be rendered, stores a rendering result corresponding to the data to be rendered into the target video memory object, and sends the video memory information of the target video memory object to the first virtual machine so that the first virtual machine displays the rendering result corresponding to the video memory information.
The first virtual machine in the embodiment of the application sends the display resolution of the target display screen to the second virtual machine, so that the second virtual machine distributes the target video memory object according to the display resolution and acquires the rendering result. Thus, even if the target display screen is replaced, the second virtual machine can obtain the display resolution corresponding to the replaced display screen from the first virtual machine; or even if the resolution set by the target display screen is replaced, the second virtual machine can obtain the replaced display resolution from the first virtual machine; therefore, the embodiment of the application can meet the changing requirement of the resolution. In practical application, the embodiment of the application can be applied to different target display screens with different display resolutions, and can also be applied to scenes with different display resolutions set by the same target display screen.
Drawings
FIG. 1 is a schematic diagram of an application environment of a display processing method based on hardware virtualization according to one embodiment of the present application;
FIG. 2 is a flow chart illustrating steps of a display processing method based on hardware virtualization according to an embodiment of the present application;
FIG. 3 is a flow chart illustrating steps of a display processing method based on hardware virtualization according to an embodiment of the present application;
FIG. 4 is a schematic diagram of a display processing system based on hardware virtualization according to one embodiment of the present application;
fig. 5 is a schematic structural view of an apparatus according to an embodiment of the present application.
Detailed Description
In order that the above-recited objects, features and advantages of the present application will become more readily apparent, a more particular description of the application will be rendered by reference to the appended drawings and appended detailed description.
The method provided by the embodiment of the application can be used for application scenes such as vehicle-mounted devices, cloud computing, virtualized desktops and the like. Under the application scene, a plurality of virtual machines can be operated on a set of hardware platform, and the plurality of virtual machines can be independently operated in parallel, so that a plurality of functions under the application scene can be provided.
Taking a cabin application scene in a car machine as an example, the functions provided by the plurality of virtual machines specifically include at least one of the following functions:
vehicle and environmental information display function: display interaction of a central control, an instrument, head-up display, a streaming media rearview mirror and the like;
entertainment and browsing functions: local and online media play, station play, etc.;
Configuration, setting and control functions: vehicle/driving information setting;
in-vehicle monitoring: driver/occupant head, face, eye monitoring functions and health monitoring;
interaction function: voice interaction and gesture control;
Connection and navigation functions: bluetooth, wireless fidelity, universal serial bus, fourth generation mobile communication technology or fifth generation mobile communication technology, beidou, global positioning system and the like;
auxiliary driving: looking around, looking back, night vision, etc.
In practical applications, the plurality of virtual machines may include: one privileged virtual machine and at least one guest virtual machine. The application program operated by the privileged virtual machine is a control program and is used for controlling the client virtual machine. The controlling the guest virtual machine specifically includes: and distributing the shared memory for the client virtual machines so as to realize mutual communication among the virtual machines through the content of the shared memory, and the like.
Virtual machines typically require the provision of corresponding functionality by means of a display screen. And the display screen can display the display content corresponding to the function. Assuming that the function is a game function, the display content on the display screen may be game content of the game interface.
Taking a cabin application scene in a car machine as an example, the display screen may include: a first display screen and a second display screen. The first display screen and the second display screen may be disposed in different positions for use by users seated in different seats (e.g., front seats or rear seats). The first display screen may be a console display screen; the second display may be provided on the back of the rear seat of the cabin or in front of the passenger's seat, for example the second display may be a secondary display. The display content of the first display screen may be: the content is controlled. The display content of the second display screen may be game content.
The display processing method based on hardware virtualization is used for providing display content for a display screen. In the related art, a display processing method based on hardware virtualization generally includes: the client virtual machine performs rendering processing on the data to be rendered by using the DGPU, and sends an address of a rendering result to the privileged virtual machine; and the privileged virtual machine displays the rendering result by utilizing IGPU. Wherein the rendering result is available as display content.
In practical application, the resolution adopted in the rendering process of DGPU is specifically: the default resolution recorded in the configuration file of the graphics subsystem service cannot meet the resolution change requirement in the related art. For example, in the case of a display screen replacement, the resolution of the display screen is changed, and the default resolution in the related art cannot meet the resolution change requirement.
Aiming at the technical problem that the default resolution in the related art cannot meet the change requirement of the resolution, the embodiment of the application provides a display processing method based on hardware virtualization, which specifically comprises the following steps: the method comprises the steps that a first virtual machine obtains the display resolution of a target display screen; the first virtual machine sends the display resolution to the second virtual machine; the second virtual machine utilizes an independent graphic processing unit to allocate a target video memory object to the data to be rendered according to the display resolution, performs rendering processing on the data to be rendered, and stores a rendering result corresponding to the data to be rendered into the target video memory object; determining the size of the video memory corresponding to the target video memory object according to the display resolution; the second virtual machine sends the video memory information of the target video memory object to the first virtual machine; and the first virtual machine displays rendering results corresponding to the video memory information by utilizing the integrated graphic processing unit and the target display screen.
In the technical scheme of the embodiment of the application, the target display screen may be a display screen corresponding to the rendering result of the second virtual machine, and is used for displaying the rendering result of the second virtual machine. The target video memory object is used for storing the rendering result of the second virtual machine.
In the technical scheme of the embodiment of the application, a first virtual machine acquires the display resolution of a target display screen and sends the display resolution to a second virtual machine; and the second virtual machine allocates a target video memory object for the data to be rendered according to the display resolution, performs rendering processing on the data to be rendered, stores a rendering result corresponding to the data to be rendered into the target video memory object, and sends the video memory information of the target video memory object to the first virtual machine so that the first virtual machine displays the rendering result corresponding to the video memory information.
Because the first virtual machine in the embodiment of the application sends the display resolution of the target display screen to the second virtual machine, the second virtual machine distributes the target video memory object and acquires the rendering result according to the display resolution. Thus, even if the target display screen is replaced, the second virtual machine can obtain the display resolution corresponding to the replaced display screen from the first virtual machine; or even if the resolution set by the target display screen is replaced, the second virtual machine can obtain the replaced display resolution from the first virtual machine; therefore, the embodiment of the application can meet the changing requirement of the resolution. In practical application, the embodiment of the application can be applied to different target display screens with different display resolutions, and can also be applied to scenes with different display resolutions set by the same target display screen.
Referring to fig. 1, a schematic diagram of an application environment of a display processing method based on hardware virtualization according to an embodiment of the present application is shown, where a virtual machine monitor 102 may be run on a hardware platform 101, and a first virtual machine 103 and a second virtual machine 104 may be run on the virtual machine monitor 102.
In practical applications, the hardware platform 101 may be a control Chip such as a System On Chip (SOC) Chip. The hardware platform 101 may be connected to external devices such as a display screen and a handle.
Virtual machine monitor 102 is an intermediate software layer running between hardware and operating systems that allows multiple operating systems and applications to share the hardware. Virtual machine monitor 102 can create a secure independent virtual machine operating environment on a hardware platform, which can include: a first virtual machine 103 and a second virtual machine 104. The first operating system running on the first virtual machine 103 may be the same as or different from the second operating system running on the second virtual machine 104. The first operating system may be an android system or a Linux system, etc. It will be appreciated that embodiments of the present application are not limited to a particular first operating system or second operating system.
In practical applications, the first virtual machine 103 may be a privileged virtual machine, and the second virtual machine 104 may be a guest virtual machine. The user space of the first virtual machine 103 may be running DM (device model DeviceModel). The device model may be used to control guest virtual machines.
The device model may start the second virtual machine 104 and, after starting the second virtual machine 104, obtain the display resolution of the target display screen. The device model may also utilize the shared memory device between virtual machines to send the display resolution to the second virtual machine 104.
In the embodiment of the application, the implementation mode of memory sharing between virtual machines is specifically as follows: the virtual machine internal shared memory is mapped to a PCI (peripheral component interconnect standard, PERIPHERAL COMPONENT INTERCONNECT) device. The Inter-virtual machine shared memory device (ivshmem, inter-VM shared memory device) may represent a PCI device that shares memory inside the virtual machine.
To achieve memory sharing among virtual machines, the virtual machine monitor 102 may include: the first monitored state shared memory device and the second monitored state shared memory device, the kernel space of the first virtual machine 103 may include: the first user space input output driver and device node notification, the device model of the first virtual machine 101 may include: the first user state may share a memory device and a first input/output control (ioctl) interface, and the kernel space of the second virtual machine 104 may include: the second user space input output driver, the user space of the second virtual machine 104 may include: the second user mode shares the memory device and the second input-output control interface.
The first user space input/output driver or the second user space input/output driver may be used to expose a PCI device corresponding to a shared memory device between virtual machines as a user space input/output device, so that an application program in the user space performs control of the PCI device via an access interface (input/output control interface).
In the embodiment of the application, the control of the PCI equipment specifically comprises the following steps: the first virtual machine writes the display resolution into the PCI device, the second virtual machine reads the display resolution from the PCI device, the second virtual machine writes the video memory information into the PCI device, and the first virtual machine reads the video memory information from the PCI device.
The device node notification may represent a device node corresponding to the first user space input output driver for providing a notification service for data transfer between virtual machines. The notification service is specifically used for: the write direction reader informs of the arrival of new data. For example, after the first virtual machine writes the display resolution into the PCI device, the device node notification of the first virtual machine may send a first notification to the second virtual machine, so that the second virtual machine reads the display resolution according to the first notification. Similarly, for example, after the second virtual machine writes the video memory information into the PCI device, a device node notification (not shown in fig. 1) of the second virtual machine may send a second notification to the first virtual machine, so that the first virtual machine reads the video memory information according to the second notification.
The user space of the second virtual machine 104 may include: the graphics subsystem driver, the second user mode shared memory device and the second input output control interface may be part of the graphics subsystem driver. The graphics subsystem driver of the second virtual machine 104 may receive the display resolution using the shared memory device between the virtual machines.
The kernel space of the second virtual machine 104 may include: and the graphics processing unit driver can allocate a target video memory object to the data to be rendered according to the display resolution by using the independent graphics processing unit, render the data to be rendered, store a rendering result corresponding to the data to be rendered to the target video memory object, and send the video memory information of the target video memory object to the first virtual machine by using the shared memory device among the virtual machines.
And the equipment model of the first virtual machine can display rendering results corresponding to the video memory information by utilizing the integrated graphic processing unit and the target display screen.
In practical applications, the video memory information may include: and the integrated graphic processing unit of the first virtual machine can read the corresponding rendering result from the video memory address according to the display refresh rate so as to continuously display the rendering result by utilizing the target display screen. The display refresh rate may be determined by one skilled in the art according to the actual application requirements, and embodiments of the present application are not limited to a particular display refresh rate.
Method embodiment one
Referring to fig. 2, a schematic step flow diagram of a display processing method based on hardware virtualization according to an embodiment of the present application is shown, where the method may specifically include the following steps:
step 201, a first virtual machine obtains the display resolution of a target display screen;
step 202, a first virtual machine sends the display resolution to a second virtual machine;
Step 203, the second virtual machine allocates a target video memory object to the data to be rendered according to the display resolution by using an independent graphics processing unit, performs rendering processing on the data to be rendered, and stores a rendering result corresponding to the data to be rendered to the target video memory object; determining the size of the video memory corresponding to the target video memory object according to the display resolution;
204, the second virtual machine sends the video memory information of the target video memory object to the first virtual machine;
and 205, displaying the rendering result corresponding to the video memory information by the first virtual machine by utilizing the integrated graphic processing unit and the target display screen.
The embodiment of the application is used for meeting the changing requirement of the resolution. The resolution change scene may specifically include: under the condition that the display screen is replaced, the resolution of the display screen is changed; or in the case that the display screen is not replaced, the resolution of the display screen setting is changed.
In step 201, the target display screen may be a display screen corresponding to a rendering result of the second virtual machine, and is configured to display the rendering result of the second virtual machine.
In practical application, the number of the second virtual machines may be plural, and different second virtual machines may correspond to different target display screens. For example, according to the virtual machine identifier of the second virtual machine, a search may be performed in a mapping relationship between the virtual machine identifier and the identifier of the display screen, so as to obtain the identifier of the target display screen corresponding to the second virtual machine.
In a specific implementation, the first virtual machine may call a preset function of the graphics subsystem service according to the identifier of the target display screen, and obtain the display resolution of the target display screen. Examples of the preset function may include: XRRGetCrtcInfo (), which is used to obtain the display resolution currently set by the output display.
Examples of display resolutions may include: 2048×1080 pixels, 3200×1800 pixels, 4096×2160 pixels, and the like. It will be appreciated that embodiments of the application are not limited to a particular display resolution.
In step 202, when the first virtual machine is a privileged virtual machine, the device model included in the privileged virtual machine may send the display resolution to the second virtual machine using the shared memory device between virtual machines.
In a specific implementation, a first user space input/output driver can be operated on a first virtual machine, and a second user space input/output driver can be operated on a second virtual machine; the first user space input/output driver or the second user space input/output driver can be used for exposing PCI equipment corresponding to shared memory equipment among virtual machines as user space input/output equipment, so that an application program of the user space controls the PCI equipment through an access interface; the control of the PCI device comprises: the first virtual machine writes the display resolution into the PCI device, the second virtual machine reads the display resolution from the PCI device, the second virtual machine writes the video memory information into the PCI device, and the first virtual machine reads the video memory information from the PCI device. The access interface may be specifically an input/output control interface, and the device model in the embodiment of the present application may call the first input/output control interface and send the display resolution to the second virtual machine.
In step 203, the graphics subsystem driver of the second virtual machine may determine the display parameters according to the display resolution, and send the display parameters to the graphics processing unit driver. The display resolution may be one of the display parameters. In addition to display resolution, the above display parameters may include: pixel format, etc.
The second virtual machine can utilize the independent graphic processing unit to allocate a target video memory object to the data to be rendered according to the display resolution, render the data to be rendered, and store the rendering result corresponding to the data to be rendered to the target video memory object.
In the embodiment of the application, the video memory is a component for storing graphic information to be processed. The picture seen on the display screen is composed of individual pixels, each pixel usually uses 4 to 32 or even 64 bits of data to control the brightness and color of the pixel, the data is stored through a video memory, and then is allocated by a display chip and a processor, and finally, the operation result is converted into a graph to be output to the display screen. Therefore, the stored object of the video memory is: the graphics processing unit outputs information to each pixel on the display.
In particular, according to the embodiment of the present application, the target video memory object may represent a block area in the video memory, and the storage object may be: rendering results output by the independent graphics processing units.
The video memory information of the target video memory object specifically includes: memory address and memory size. And determining the size of the video memory corresponding to the target video memory object according to the display resolution.
According to one implementation a, the size of the data to be rendered may be determined according to the display resolution, and the size of the video memory corresponding to the target video memory object may be determined according to the size of the data to be rendered. For example, the size of the data to be rendered may be determined according to the above-described information of the display resolution, pixel format, and the like.
According to another implementation manner B, the embodiment of the present application may save a correspondence between resolution and a size of a video memory, and may search in the correspondence according to a display resolution, so as to obtain the size of the video memory corresponding to the target video memory object.
The video memory size may be equal to the size of the data to be rendered, or the video memory size may be greater than the size of the data to be rendered. Of course, the embodiment of the present application is not limited to a specific size of the video memory.
The graphics unit driver corresponding to the independent graphics processing unit can acquire the video memory block corresponding to the video memory size from the idle video memory blocks according to the video memory size, and the video memory block is used as a target video memory object. After the rendering process is completed, the graphic unit driver may further store the rendering result corresponding to the data to be rendered to the target video memory object.
In step 204, the second virtual machine may send the video memory information of the target video memory object to the first virtual machine by using the shared memory device between virtual machines, so that the first virtual machine obtains the video memory information.
In an optional implementation manner of the present application, after the first virtual machine uses the memory device shared between the virtual machines to send the display resolution to the second virtual machine, the first virtual machine may block the video memory information waiting for the second virtual machine to send, so as to save the resources of the first virtual machine.
In step 205, the first virtual machine displays a rendering result corresponding to the video memory information by using the integrated graphics processing unit and the target display screen.
In a specific implementation, the video memory information specifically includes: memory address and memory size; the memory address and the memory size can be used to determine the memory location corresponding to the target memory object.
The process of displaying the rendering result corresponding to the video memory information by the first virtual machine through the integrated graphics processing unit and the target display screen may specifically include: and the first virtual machine judges whether the size of the video memory is matched with the size of the video memory corresponding to the display resolution, and if so, the rendering result corresponding to the video memory address is displayed by utilizing the integrated graphic processing unit and the target display screen.
The embodiment of the application can utilize the implementation mode A or the implementation mode B to determine the size of the video memory corresponding to the display resolution, compare the size of the video memory corresponding to the display resolution with the size of the video memory corresponding to the target video memory object, and if the comparison result is matched, display the rendering result corresponding to the video memory address by utilizing the integrated graphic processing unit and the target display screen. It can be appreciated that if the comparison result is not matched, the rendering result corresponding to the video memory address may not be displayed.
In a specific implementation, the video memory information specifically includes: the process of displaying the rendering result corresponding to the video memory information by the first virtual machine through the integrated graphics processing unit and the target display screen may specifically include: and the first virtual machine reads the corresponding rendering result from the video memory address according to the display refresh rate by utilizing the integrated graphic processing unit, and continuously displays the rendering result by utilizing the target display screen. The display refresh rate may be determined by one skilled in the art according to the actual application requirements, and embodiments of the present application are not limited to a particular display refresh rate. The continuous display of rendering results may include: continuous display of the same rendering result and continuous display of different rendering results. Assuming that the rendering results correspond to game content of a game interface, different rendering results may correspond to different game content of the same game interface, or different rendering results may correspond to different game content of different game interfaces.
The embodiment of the application can process the video memory information by utilizing the double caches so as to relieve the visual cracking sense of the screen display content. Specifically, under the condition that the first virtual machine reads the video memory information of the ith frame from the first buffer memory, the second virtual machine stores the video memory information of the (i+1) th frame into the second buffer memory; under the condition that the first virtual machine reads the video memory information of the (i+1) th frame from the second buffer memory, the second virtual machine stores the video memory information of the (i+2) th frame into the first buffer memory; wherein i is a positive integer.
The operations of the first cache and the second cache may be in parallel. After performing a round of operations, pointers of the first cache and the second cache may be swapped. One round of operation is specifically: under the condition that the first virtual machine reads the video memory information of the ith frame from the first buffer memory, the second virtual machine stores the video memory information of the (i+1) th frame into the second buffer memory. After the pointers of the first cache and the second cache are exchanged, the first virtual machine may perform a read operation of the second cache, and the second virtual machine may perform a write operation of the first cache.
In summary, according to the display processing method based on hardware virtualization in the embodiment of the present application, since the first virtual machine sends the display resolution of the target display screen to the second virtual machine, the second virtual machine allocates the target video memory object and obtains the rendering result according to the display resolution. Thus, even if the target display screen is replaced, the second virtual machine can obtain the display resolution corresponding to the replaced display screen from the first virtual machine; or even if the resolution set by the target display screen is replaced, the second virtual machine can obtain the replaced display resolution from the first virtual machine; therefore, the embodiment of the application can meet the changing requirement of the resolution. In practical application, the embodiment of the application can be applied to different target display screens with different display resolutions, and can also be applied to scenes with different display resolutions set by the same target display screen.
Method embodiment II
Referring to fig. 3, a schematic step flow diagram of a display processing method based on hardware virtualization according to an embodiment of the present application is shown, where a first virtual machine is a privileged virtual machine and a second virtual machine is a guest virtual machine, and the method specifically may include the following steps:
step 301, loading a first user space input/output driver by a privileged virtual machine;
step 302, a device model of a privileged virtual machine starts a client virtual machine;
step 303, loading a second user space input/output driver, a graphics subsystem driver and a graphics processing unit driver by the client virtual machine;
step 304, the equipment model of the privileged virtual machine acquires the display resolution of the target display screen according to the display interface identification of the target display screen corresponding to the client virtual machine;
The display interface (DP, displayPort) is a digital video interface standard standardized by the video electronics standards association developed by the PC (personal computer ) and chip manufacturer alliance. The device model may call a preset function of the graphics subsystem service to obtain the display resolution of the target display screen. Wherein the display interface identifier can be used as a parameter of a preset function.
Step 305, the device model of the privileged virtual machine sends the display resolution to the client virtual machine by using the shared memory device among the virtual machines;
step 306, blocking the video memory information sent by the waiting client virtual machine by the equipment model of the privileged virtual machine;
Step 307, the graphics subsystem driver of the client virtual machine uses the shared memory device between the virtual machines to receive the display resolution and send the display resolution to the graphics processing unit driver;
308, the graphic processing unit of the client virtual machine drives to allocate a target video memory object for the data to be rendered according to the display resolution, performs rendering processing on the data to be rendered, and stores a rendering result corresponding to the data to be rendered to the target video memory object;
step 309, the graphic processing unit of the client virtual machine drives and utilizes the shared memory device among the virtual machines to send the video memory information of the target video memory object to the device model of the privileged virtual machine;
Step 310, the device model of the privileged virtual machine calls the integrated graphics processing unit, obtains the rendering result from the video memory address and displays the rendering result.
In summary, according to the display processing method based on hardware virtualization in the embodiment of the present application, since the privileged virtual machine sends the display resolution of the target display screen to the client virtual machine, the client virtual machine allocates the target video memory object and obtains the rendering result according to the display resolution. Thus, even if the target display screen is replaced, the client virtual machine can obtain the display resolution corresponding to the replaced display screen from the privileged virtual machine; or even if the resolution set by the target display screen is replaced, the client virtual machine can obtain the replaced display resolution from the privileged virtual machine; therefore, the embodiment of the application can meet the changing requirement of the resolution. In practical application, the embodiment of the application can be applied to different target display screens with different display resolutions, and can also be applied to scenes with different display resolutions set by the same target display screen.
It should be noted that, for simplicity of description, the method embodiments are shown as a series of acts, but it should be understood by those skilled in the art that the embodiments are not limited by the order of acts, as some steps may occur in other orders or concurrently in accordance with the embodiments. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred embodiments, and that the acts are not necessarily required by the embodiments of the application.
On the basis of the embodiment, the embodiment also provides a display processing system based on hardware virtualization.
Referring to FIG. 4, a schematic diagram of a display processing system based on hardware virtualization according to one embodiment of the present application is shown, which may specifically include: a first virtual machine 401 and a second virtual machine 402.
The first virtual machine 401 specifically includes: a device model module 411; the second virtual machine 402 specifically includes: a graphics subsystem driver module 421 and a graphics processing unit driver module 422;
The device model module 411 is configured to obtain a display resolution of a target display screen, and send the display resolution to a second virtual machine;
a graphics subsystem driver module 421 for receiving the display resolution;
the graphics processing unit driving module 422 is configured to allocate a target video memory object to the data to be rendered according to the display resolution by using an independent graphics processing unit, perform rendering processing on the data to be rendered, store a rendering result corresponding to the data to be rendered to the target video memory object, and send video memory information of the target video memory object to a first virtual machine;
the device model module 411 is further configured to display a rendering result corresponding to the video memory information by using the integrated graphics processing unit and the target display screen.
Optionally, the device model module 411 is specifically configured to call a preset function of a graphics subsystem service according to an identifier of a target display screen, and obtain a display resolution of the target display screen.
Optionally, the device model module 411 is specifically configured to obtain a display resolution of the target display screen after the second virtual machine is started.
Optionally, the video memory information includes: memory address and memory size; the device model module 411 is specifically configured to determine whether the size of the video memory matches the display resolution, and if so, display a rendering result corresponding to the video memory address by using the integrated graphics processing unit and the target display screen.
Optionally, the video memory information includes: the device model module 411 is specifically configured to read, by using the integrated graphics processing unit, a corresponding rendering result from the video memory address according to the display refresh rate, and continuously display the rendering result by using the target display screen.
Optionally, the device model module 411 is further configured to block waiting for the video memory information sent by the second virtual machine after sending the display resolution to the second virtual machine.
Optionally, the first virtual machine further includes: the first user space input output driver, the second virtual machine further comprises: a second user space input output driver;
The first user space input/output driver or the second user space input/output driver is used for exposing PCI equipment corresponding to shared memory equipment among virtual machines as user space input/output equipment, so that an application program of the user space controls the PCI equipment through an access interface; the control of the PCI device specifically comprises: the first virtual machine writes the display resolution into the PCI device, the second virtual machine reads the display resolution from the PCI device, the second virtual machine writes the video memory information into the PCI device, and the first virtual machine reads the video memory information from the PCI device.
The embodiment of the application can sample the double-buffer technology to process the video memory information. Specifically, under the condition that the first virtual machine reads the video memory information of the ith frame from the first buffer memory, the second virtual machine stores the video memory information of the (i+1) th frame into the second buffer memory; under the condition that the first virtual machine reads the video memory information of the (i+1) th frame from the second buffer memory, the second virtual machine stores the video memory information of the (i+2) th frame into the first buffer memory; wherein i is a positive integer.
In summary, in the display processing system based on hardware virtualization according to the embodiment of the present application, since the first virtual machine sends the display resolution of the target display screen to the second virtual machine, the second virtual machine allocates the target video memory object and obtains the rendering result according to the display resolution. Thus, even if the target display screen is replaced, the second virtual machine can obtain the display resolution corresponding to the replaced display screen from the first virtual machine; or even if the resolution set by the target display screen is replaced, the second virtual machine can obtain the replaced display resolution from the first virtual machine; therefore, the embodiment of the application can meet the changing requirement of the resolution. In practical application, the embodiment of the application can be applied to different target display screens with different display resolutions, and can also be applied to scenes with different display resolutions set by the same target display screen.
The embodiment of the application also provides a non-volatile readable storage medium, where one or more modules (programs) are stored, where the one or more modules are applied to a device, and the instructions (instructions) of each method step in the embodiment of the application may cause the device to execute.
Embodiments of the application provide one or more machine-readable media having instructions stored thereon that, when executed by one or more processors, cause an electronic device to perform a method as described in one or more of the above embodiments. In the embodiment of the application, the electronic equipment comprises various types of equipment such as terminal equipment, servers (clusters) and the like. The electronic device may be an in-vehicle device applied to a car scene such as a cabin controller.
Embodiments of the present disclosure may be implemented as an apparatus for performing a desired configuration using any suitable hardware, firmware, software, or any combination thereof, which may include: terminal equipment, servers (clusters), vehicle-mounted equipment and other electronic equipment. Fig. 5 schematically illustrates an exemplary apparatus 1100 that may be used to implement various embodiments described in the present disclosure.
For one embodiment, FIG. 5 illustrates an example apparatus 1100 having one or more processors 1102, a control module (chipset) 1104 coupled to at least one of the processor(s) 1102, a memory 1106 coupled to the control module 1104, a non-volatile memory 1108 coupled to the control module 1104, one or more input/output devices 1110 coupled to the control module 1104, and a network interface 1112 coupled to the control module 1104.
The processor 1102 may include one or more single-core or multi-core processors, and the processor 1102 may include any combination of general-purpose or special-purpose processors (e.g., graphics processors, application processors, baseband processors, etc.). In some embodiments, the apparatus 1100 can be used as a terminal device, a server (cluster), or the like in the embodiments of the present application.
In some embodiments, apparatus 1100 may include one or more computer-readable media (e.g., memory 1106 or NVM/storage 1108) having instructions 1114 and one or more processors 1102 combined with the one or more computer-readable media configured to execute instructions 1114 to implement modules to perform the actions described in this disclosure.
For one embodiment, the control module 1104 may include any suitable interface controller to provide any suitable interface to at least one of the processor(s) 1102 and/or any suitable device or component in communication with the control module 1104.
The control module 1104 may include a memory controller module to provide an interface to the memory 1106. The memory controller modules may be hardware modules, software modules, and/or firmware modules.
Memory 1106 may be used to load and store data and/or instructions 1114 for device 1100, for example. For one embodiment, memory 1106 may comprise any suitable volatile memory, such as, for example, a suitable DRAM. In some embodiments, memory 1106 may comprise double data rate type four synchronous dynamic random access memory (DDR 4 SDRAM).
For one embodiment, the control module 1104 may include one or more input/output controllers to provide interfaces to the NVM/storage 1108 and the input/output device(s) 1110.
For example, NVM/storage 1108 may be used to store data and/or instructions 1114. NVM/storage 1108 may include any suitable nonvolatile memory (e.g., flash memory) and/or may include any suitable nonvolatile storage device(s) (e.g., one or more Hard Disk Drives (HDDs), one or more Compact Disc (CD) drives, and/or one or more Digital Versatile Disc (DVD) drives).
NVM/storage 1108 may include storage resources that are physically part of the device on which apparatus 1100 is installed or may be accessible by the device without necessarily being part of the device. For example, NVM/storage 1108 may be accessed over a network via input/output device(s) 1110.
Input/output device(s) 1110 may provide an interface for apparatus 1100 to communicate with any other suitable device, input/output device 1110 may include communication components, audio components, sensor components, and the like. Network interface 1112 may provide an interface for device 1100 to communicate over one or more networks, and device 1100 may communicate wirelessly with one or more components of a wireless network in accordance with any of one or more wireless network standards and/or protocols, such as accessing a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G, 5G, etc., or a combination thereof.
For one embodiment, at least one of the processor(s) 1102 may be packaged together with logic of one or more controllers (e.g., memory controller modules) of the control module 1104. For one embodiment, at least one of the processor(s) 1102 may be packaged together with logic of one or more controllers of the control module 1104 to form a System In Package (SiP). For one embodiment, at least one of the processor(s) 1102 may be integrated on the same mold as logic of one or more controllers of the control module 1104. For one embodiment, at least one of the processor(s) 1102 may be integrated on the same die as logic of one or more controllers of the control module 1104 to form a system on chip (SoC).
In various embodiments, apparatus 1100 may be, but is not limited to being: a server, a desktop computing device, or a mobile computing device (e.g., a laptop computing device, a handheld computing device, a tablet, a netbook, etc.), among other terminal devices. In various embodiments, device 1100 may have more or fewer components and/or different architectures. For example, in some embodiments, the apparatus 1100 includes one or more cameras, keyboards, liquid Crystal Display (LCD) screens (including touch screen displays), non-volatile memory ports, multiple antennas, graphics chips, application Specific Integrated Circuits (ASICs), and speakers.
The detection device can adopt a main control chip as a processor or a control module, sensor data, position information and the like are stored in a memory or an NVM/storage device, a sensor group can be used as an input/output device, and a communication interface can comprise a network interface.
For the device embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and reference is made to the description of the method embodiments for relevant points.
In this specification, each embodiment is described in a progressive manner, and each embodiment is mainly described by differences from other embodiments, and identical and similar parts between the embodiments are all enough to be referred to each other.
Embodiments of the present application are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiment and all such alterations and modifications as fall within the scope of the embodiments of the application.
Finally, it is further noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or terminal device that comprises the element.
The foregoing has described in detail a display processing method and system based on hardware virtualization, an electronic device and a machine readable medium, and specific examples have been applied to illustrate the principles and embodiments of the present application, and the above examples are only used to help understand the method and core idea of the present application; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present application, the present description should not be construed as limiting the present application in view of the above.

Claims (10)

1. A display processing method based on hardware virtualization, the method comprising:
the method comprises the steps that a first virtual machine obtains the display resolution of a target display screen;
the first virtual machine sends the display resolution to the second virtual machine;
The second virtual machine utilizes an independent graphic processing unit to allocate a target video memory object to the data to be rendered according to the display resolution, performs rendering processing on the data to be rendered, and stores a rendering result corresponding to the data to be rendered to the target video memory object; determining the size of the video memory corresponding to the target video memory object according to the display resolution;
the second virtual machine sends the video memory information of the target video memory object to the first virtual machine;
and the first virtual machine displays rendering results corresponding to the video memory information by utilizing the integrated graphic processing unit and the target display screen.
2. The method of claim 1, wherein the first virtual machine obtaining a display resolution of a target display screen comprises:
after the first virtual machine starts the second virtual machine, calling a preset function of the graphic subsystem service according to the identification of the target display screen, and obtaining the display resolution of the target display screen.
3. The method of claim 1, wherein the video memory information comprises: memory address and memory size; the first virtual machine displays a rendering result corresponding to the video memory information by utilizing an integrated graphic processing unit and a target display screen, and the method comprises the following steps:
And the first virtual machine judges whether the size of the video memory is matched with the size of the video memory corresponding to the display resolution, and if so, the rendering result corresponding to the video memory address is displayed by utilizing the integrated graphic processing unit and the target display screen.
4. The method of claim 1, wherein the video memory information comprises: the video memory address, the first virtual machine displays the rendering result corresponding to the video memory information by using the integrated graphic processing unit and the target display screen, and the method comprises the following steps:
and the first virtual machine reads the corresponding rendering result from the video memory address according to the display refresh rate by utilizing the integrated graphic processing unit, and continuously displays the rendering result by utilizing the target display screen.
5. The method according to claim 1, wherein the method further comprises:
under the condition that the first virtual machine reads the video memory information of the ith frame from the first buffer memory, the second virtual machine stores the video memory information of the (i+1) th frame into the second buffer memory;
Under the condition that the first virtual machine reads the video memory information of the (i+1) th frame from the second buffer memory, the second virtual machine stores the video memory information of the (i+2) th FI frame into the first buffer memory; wherein i is a positive integer.
6. The method according to claim 1, wherein the method further comprises:
And after the first virtual machine sends the display resolution to the second virtual machine, blocking and waiting for the video memory information sent by the second virtual machine.
7. The method of claim 1, wherein a first user space input output driver is run on the first virtual machine and a second user space input output driver is run on the second virtual machine;
The first user space input/output driver or the second user space input/output driver is used for exposing PCI equipment corresponding to shared memory equipment among virtual machines as user space input/output equipment, so that an application program of the user space controls the PCI equipment through an access interface; the control of the PCI device comprises: the first virtual machine writes the display resolution into the PCI device, the second virtual machine reads the display resolution from the PCI device, the second virtual machine writes the video memory information into the PCI device, and the first virtual machine reads the video memory information from the PCI device.
8. A display processing system based on hardware virtualization, the system comprising: a first virtual machine and a second virtual machine;
Wherein the first virtual machine includes: an equipment model module; the second virtual machine includes: a graphics subsystem driver module and a graphics processing unit driver module;
The device model module is used for acquiring the display resolution of the target display screen and sending the display resolution to the second virtual machine;
the graphic subsystem driving module is used for receiving the display resolution;
The graphic processing unit driving module is used for distributing target video memory objects for data to be rendered according to the display resolution by utilizing an independent graphic processing unit, rendering the data to be rendered, storing rendering results corresponding to the data to be rendered to the target video memory objects, and sending video memory information of the target video memory objects to a first virtual machine by utilizing shared memory equipment among virtual machines;
The equipment model module is also used for displaying the rendering result corresponding to the video memory information by utilizing the integrated graphic processing unit and the target display screen.
9. An electronic device, comprising: a processor; and
A memory having executable code stored thereon that, when executed, causes the processor to perform the method of any of claims 1-7.
10. A machine readable medium having stored thereon executable code which when executed causes a processor to perform the method of any of claims 1-7.
CN202410450174.1A 2024-04-15 2024-04-15 Display processing method, system, equipment and medium based on hardware virtualization Pending CN118170338A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202410450174.1A CN118170338A (en) 2024-04-15 2024-04-15 Display processing method, system, equipment and medium based on hardware virtualization

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202410450174.1A CN118170338A (en) 2024-04-15 2024-04-15 Display processing method, system, equipment and medium based on hardware virtualization

Publications (1)

Publication Number Publication Date
CN118170338A true CN118170338A (en) 2024-06-11

Family

ID=91356457

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202410450174.1A Pending CN118170338A (en) 2024-04-15 2024-04-15 Display processing method, system, equipment and medium based on hardware virtualization

Country Status (1)

Country Link
CN (1) CN118170338A (en)

Similar Documents

Publication Publication Date Title
US11042341B2 (en) Integrated functionality of center display, driver display, and shared-experience display
CN107003892B (en) GPU virtualization method, device and system, electronic equipment and computer program product
US20160328254A1 (en) Vehicle with multiple user interface operating domains
CN112644276B (en) Screen display method, vehicle and computer storage medium
US11204790B2 (en) Display method for use in multi-operating systems and electronic device
JP2007528062A (en) Graphic device clustering using PCI Express
CN112199222B (en) Video display method and device, electronic equipment and storage medium
JP2013508869A (en) Application image display method and apparatus
CN107077376B (en) Frame buffer implementation method and device, electronic equipment and computer program product
US9984651B2 (en) Method and apparatus for displaying composition screen image by composing screen images of operating systems (OSs)
CN116774933A (en) Virtualization processing method of storage device, bridging device, system and medium
CN116821040B (en) Display acceleration method, device and medium based on GPU direct memory access
US12027087B2 (en) Smart compositor module
US10929079B2 (en) Video display method, electronic device and computer program product
US6791538B2 (en) Method and system for operating a combination unified memory and graphics controller
KR100941029B1 (en) Graphic accelerator and graphic accelerating method
US20240143360A1 (en) Signal processing device and display apparatus for vehicles including the same
CN118170338A (en) Display processing method, system, equipment and medium based on hardware virtualization
CN116527978A (en) Multi-screen interaction control method and device
US20230376307A1 (en) Streaming via hardware abstraction layer
US20200326898A1 (en) Distributed system for displaying a content
CN111785229A (en) Display method, device and system
CN116389702A (en) Data processing method and device
CN115904295B (en) Multi-screen display control method, device, medium, system, chip and panel
CN113924552A (en) Multimedia system with optimized performance

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination