CN114035851A - Multi-system graphic data processing method and device, electronic equipment and storage medium - Google Patents
Multi-system graphic data processing method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN114035851A CN114035851A CN202111315195.5A CN202111315195A CN114035851A CN 114035851 A CN114035851 A CN 114035851A CN 202111315195 A CN202111315195 A CN 202111315195A CN 114035851 A CN114035851 A CN 114035851A
- Authority
- CN
- China
- Prior art keywords
- operating system
- composer
- graphic data
- instance
- triggering
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/4401—Bootstrapping
- G06F9/4406—Loading of operating system
- G06F9/441—Multiboot arrangements, i.e. selecting an operating system to be loaded
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/451—Execution arrangements for user interfaces
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/544—Buffers; Shared memory; Pipes
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Security & Cryptography (AREA)
- Human Computer Interaction (AREA)
- User Interface Of Digital Computer (AREA)
- Processing Or Creating Images (AREA)
Abstract
The application relates to a multi-system graphic data processing method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring graphic data of at least one Surface corresponding to a current display window of a first operating system, and synthesizing the acquired graphic data into a buffer area through a composer; the graphic data in the buffer area is adapted to the back end of the corresponding composer; triggering the first operating system to use the backend of the composer as a Surface object of a Surface Finger instance of a second operating system through a dynamic library access protocol between the first operating system and the second operating system; and triggering the Surface finger instance of the second operating system to render and generate a graphical display interface including the composer back end and the Surface object of the second operating system, and outputting the graphical display interface. The graphical interface updating efficiency is higher, and the user experience is improved.
Description
Technical Field
The embodiment of the application relates to a graphical interface display technology, in particular to a multi-system graphical data processing method and device, electronic equipment and a readable non-transitory storage medium.
Background
At present, the Android system supports the installation of the Linux system, but when the Linux system such as Linux distribution, anLinux and the like is operated in the Android system, the Linux system is installed in a container of an Android system, Virtual Network Console (VNC) technology is adopted to realize the display on a Linux desktop, the principle of the method is that a screen is rendered and synthesized into an image at a Linux end, the image is compressed and then sent to an android VNC Client end through a VNC protocol for display, in the VNC-based graphical interface display mode, composite image transmission is required between an application in the Linux system and a VNC server, the Linux system and the Android system still need to transmit the composite image, and because the data size of the composite image is large and needs to be transmitted twice, therefore, the delay is relatively large, and the graphics processing task with low delay and high refresh cannot be performed, and the amount of transmitted data is increased when the image quality requirement is high, and good user experience cannot be provided based on the VNC protocol.
Content of application
In view of the above, embodiments of the present application provide a method and an apparatus for processing graphics data of multiple systems, an electronic device, and a storage medium, so as to at least solve the above technical problems in the prior art.
According to a first aspect of embodiments of the present application, a method for processing graphics data of multiple systems is provided, including:
acquiring graphic data of at least one layer transmission interface Surface corresponding to a current display window of a first operating system, and synthesizing the acquired graphic data into a buffer area through a synthesizer (synthesizer);
the graphic data in the buffer area is adapted to the back end of the corresponding composer;
triggering the first operating system to use the backend of the composer as a Surface object of a graphics rendering composition controller (Surface controller) instance of a second operating system through a dynamic library access protocol between the first operating system and the second operating system;
and triggering the Surface finger instance of the second operating system to render and generate a graphical display interface including the composer back end and the Surface object of the second operating system, and outputting the graphical display interface.
In an embodiment, the obtaining of the graphic data of at least one Surface corresponding to the current display window of the first operating system includes:
detecting an operating instruction aiming at the first operating system, determining a dirty area of a current display window of the first operating system, determining at least one Surface corresponding to the dirty area according to the dirty area, and acquiring graphic data corresponding to the at least one Surface.
In one embodiment, the compositing, by the composer, the obtained graphics data into a buffer includes:
the composer of the first operating system calls the graphic data of at least one application corresponding to the current display window through the Wayland or X protocol, and synthesizes the called graphic data into the buffer area.
In one embodiment, the triggering the first operating system to use the composer backend as a Surface object of a Surface field instance of a graphics rendering composition controller of the second operating system through a dynamic library access protocol between the first operating system and the second operating system includes:
and triggering the first operating system to send the composer back end to the Surface finger instance of the second operating system through the Libhybris, wherein the composer back end is used as a Surface object of the Surface finger instance.
In one embodiment, the triggering the surfefinger instance of the second operating system to render a Surface object including the composer backend and the second operating system to generate a graphical display interface includes:
and triggering the Surface Finger instance of the second operating system to acquire all Surface objects and rendering and generating the graphical display interface after detecting a vertical synchronization signal (Vsync) of the graphical interface.
According to a second aspect of embodiments of the present application, there is provided a multi-system graphics data processing apparatus, including:
the system comprises an acquisition unit, a display unit and a display unit, wherein the acquisition unit is used for acquiring the graphic data of at least one Surface corresponding to the current display window of a first operating system;
the composition unit is used for triggering the composer to compose the acquired graphic data into a buffer area;
the adaptation unit is used for adapting the graphic data in the buffer area to the rear end of the corresponding composer;
a transmission unit, configured to trigger the first operating system to use the backend of the composer as a Surface object of a Surface filler instance of a second operating system through a dynamic library access protocol between the first operating system and the second operating system;
and the output unit is used for triggering the Surface finger instance of the second operating system to render and generate a graphical display interface by the Surface object containing the composer rear end and the second operating system, and outputting the graphical display interface.
In one embodiment, the obtaining unit is further configured to:
detecting an operating instruction aiming at the first operating system, determining a dirty area of a current display window of the first operating system, determining at least one Surface corresponding to the dirty area according to the dirty area, and acquiring graphic data corresponding to the at least one Surface.
In one embodiment, the synthesis unit is further configured to:
and triggering the composer of the first operating system to call the graphic data of at least one application corresponding to the current display window through the Wayland or X protocol, and synthesizing the called graphic data into the buffer area.
In one embodiment, the output unit is further configured to:
and triggering the first operating system to send the composer back end to the Surface finger instance of the second operating system through the Libhybris, wherein the composer back end is used as a Surface object of the Surface finger instance.
In one embodiment, the output unit is further configured to:
and after detecting the Vsync of the graphical interface, triggering the SurfaceFinger instance of the second operating system to acquire all the Surface objects, and rendering to generate the graphical display interface.
According to a third aspect of the embodiments of the present application, there is provided an electronic device, including a processor, a memory, and an executable program stored on the memory and capable of being executed by the processor, where the processor executes the steps of the multi-system graphics data processing method when executing the executable program.
According to a fourth aspect of embodiments herein, there is provided a readable non-transitory storage medium having stored thereon an executable program, which when executed by a processor, performs the steps of the multi-system graphics data processing method.
In the embodiment of the application, when it is determined that the graphic display data corresponding to the current display window of the first operating system needs to be displayed through the second operating system, the graphic data of at least one Surface to be displayed on the interface of the first operating system is acquired, the graphic data is adapted to the rear end of the corresponding composer through the composer, and the rear end of the composer is used as a Surface object of the Surface field instance of the second operating system, so that the graphic data to be displayed of the first operating system and the Surface object of the second operating system are synthesized and rendered together and updated to the display interface of the second operating system for presentation. The embodiment of the application can realize the updating of the display image only by the transmission of the graphic data of the bottom layer, the transmission data volume is greatly reduced, the graphic display efficiency is higher, and the graphic data size to be displayed by the first operating system is hardly influenced. Due to the fact that the graphical interface updating efficiency is higher, user experience is improved.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below. It is obvious that the drawings in the following description are some embodiments of the invention, and that for a person skilled in the art, other drawings can be derived from them without inventive effort.
FIG. 1 is a flowchart illustrating a multi-system graphics data processing method according to an embodiment of the present disclosure;
FIG. 2 is a diagram illustrating an example of a method for processing graphics data for multiple systems according to an embodiment of the present disclosure;
FIG. 3 is a diagram illustrating a graphical data transfer of a first operating system according to an embodiment of the present application;
FIG. 4 is a schematic diagram of a multi-system graphics data processing apparatus according to an embodiment of the present application;
fig. 5 is a block diagram schematically illustrating the components of the electronic device according to the embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some, but not all, embodiments of the present application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The terms first, second and the like in the description and in the claims of the present application are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the application are capable of operation in sequences other than those illustrated or described herein. In addition, "and/or" in the specification and claims means at least one of connected objects, a character "/" generally means that a preceding and succeeding related objects are in an "or" relationship.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The method for modifying a root directory, the apparatus for modifying a root directory, the electronic device and the readable non-transitory storage medium provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings by specific embodiments and application scenarios thereof.
Fig. 1 is a schematic flowchart of a multisystem graphics data processing method according to an embodiment of the present application, and as shown in fig. 1, the multisystem graphics data processing method according to the embodiment of the present application includes the following processing steps:
As an example, the first operating system may include a Linux system such as Linux distribution, anLinux, etc., and the second operating system may include an Android system. In the embodiment of the present application, a Linux Operating System (OS) is installed on the Android System, and as an example, the Linux System is installed on the Android System as an application program.
The Surface in the embodiment of the application refers to the Surface required by the Rendering of the graphics, namely, the on-Screen Rendering Surface. When creating an On-Screen Rendering Surface, firstly a native Window of a local platform needs to be created, and then an eglSurface, namely a Surface object, is created.
In the embodiment of the application, the graphic data of the Surface corresponding to all the current display windows of the first operating system can be acquired; as an example, only the graphic data of the dirty areas corresponding to all current display windows may be obtained, that is, the area with the updated graphic may be updated on the graphic interface, and it is not necessary to update all the graphic data of the entire display window. Specifically, an operating instruction for the first operating system is detected, a dirty area of a current display window of the first operating system is determined, at least one Surface corresponding to the dirty area is determined according to the dirty area, and graphic data corresponding to the at least one Surface is acquired. The operation instruction herein includes opening or closing of a child window for an application, a click operation for a toolbar instruction on a window interface, a refresh instruction for display contents in a window, and the like. And responding to the operation instruction, determining dirty regions of all current display windows of the first operating system, rendering and synthesizing the Surface graphic data corresponding to the dirty regions, and updating the Surface graphic data to a graphic display interface of the second operating system.
And step 102, adapting the graphic data in the buffer area to the back end of the corresponding composer.
In the embodiment of the present application, the composer of the first operating system calls, through the Wayland or X protocol, the graphics data of at least one application corresponding to the current display window, and synthesizes the called graphics data into the buffer. Among them, Wayland is a set of communication protocol between display server (Wayland component) and client (Wayland client), such as application, and is mainly used for transmitting graphics data, and graphics data transmission is generally realized through Buffer.
Specifically, graphics synthesizers of the Linux operating system are various, such as kwin, x-server, and the like, a Surface of the Linux operating system is transmitted between an application (application) and a composer by using a wayland or x protocol in the Linux operating system, and finally the Surface is also synthesized into a buffer in the composer, and the content of the buffer is output to different virtual devices by adapting to different composer back ends (backs).
In this embodiment of the application, the dynamic library access protocol between the first operating system and the second operating system includes Libhybris protocol, where the first operating system may use the synthesized graphics data in the cache corresponding to the back end of the composer as the Surface object of the second operating system through Libhybris protocol, the Surface object of the second operating system is directly called by the Surface pointer instance of the second operating system, the graphics data corresponding to the Surface of the second operating system is rendered and synthesized, a graphics interface to be displayed is generated, and the graphics interface is refreshed on the current display screen for display after a vertical synchronization signal (Vsync) arrives. Libhybris is a set of adaptive libraries for solving the compatibility problem between GNU lib and Android lib, and aims to enable the Android lib to be called by an application program in standard Linux, such as a library (e.g., a closed source HAL library of an Android system) compiled with bionic, and the like, running on a GNU C library-based system.
Specifically, the first operating system is triggered to send the composer back end to the surfefinger instance of the second operating system through Libhybris, and the composer back end is used as a Surface object of the surfefinger instance.
And 104, triggering the Surface pointer instance of the second operating system to render and generate a graphical display interface by the Surface object containing the composer back end and the second operating system, and outputting the graphical display interface.
In the embodiment of the application, after the second operating system detects Vsync of the graphical interface, the Surface pointer instance of the second operating system is triggered to acquire all Surface objects, a graphical display interface is rendered, and the graphical display interface is refreshed on the current display screen for display.
Fig. 2 is a schematic diagram illustrating an example of a multisystem graphic data processing method according to an embodiment of the present application, and as shown in fig. 2, only an example of displaying a display window of Linux on a display screen of an Android system is shown.
As shown in fig. 2, in the embodiment of the application, a graphics composer for Android may include a Surface flag, all interfaces in an upper layer are finally composed by the Surface flag, and then are submitted to a display screen Buffer, when a Vsync signal arrives, the display screen displays graphics content in the display screen Buffer on the display screen, each display interface in the upper layer corresponds to one or more Surface objects in the Surface flag, the Surface is a carrier for further packaging graphics data Buffer, and the Buffer is a carrier for transferring graphics data.
The graphics compositors of the Linux operating system are various, and may include kwin, x-server, etc., for example, the Surface is transmitted between the application and the composer of the Linux operating system through the wayland or x protocol, the Surface of the Linux operating system is finally composited into a buffer in the composer, and the content of the buffer is output to different devices by adapting to different composer backings.
And finally, synthesizing a final display graphical interface by the SurfaceFlinger in the Android system and a display window of the Android system through libhybris, and outputting the final display graphical interface to a display screen for displaying.
Fig. 3 is a schematic diagram of transmitting graphics data of a first operating system according to an embodiment of the present application, and as shown in fig. 3, it mainly shows how a Linux operating system transmits its own graphics data to a second operating system through a transmission protocol, renders and synthesizes corresponding Surface objects into a to-be-displayed graphics interface through a Surface filler instance of the second operating system, and displays the to-be-displayed graphics interface on a display screen.
In the embodiment of the present application, the transmission of graphic data corresponding to a display window corresponding to application of a Linux operating system is taken as an example for description. As shown in fig. 3, the application serves as a wave client (client), the graphics data corresponding to the application is stored in the eglsswapbuffer buffer, the composer of Linux serves as a wave server (server), and the application and the composer perform transmission of the graphics data through a wave or X protocol, and specifically, the composer of Linux calls the graphics data in the eglsswapbuffer buffer based on the wave or X protocol through an eglccreatewindow surface instance and composes the graphics data in the eglsswapbuffer buffer.
And (4) putting a new graphic buffer in the previously drawn eglSufrace into a buffer queue in an eglSwapBuffer mode.
The method comprises the steps that the Surface of the Android system calls composite graphics data of a composite back through libhybris, the composite back of Linux is used as one Surface of a Surface maker, the Surface Finger instance of the Android system calls all Surface objects, graphics data corresponding to all Surface objects are rendered and synthesized into an interface image to be displayed, and when a Vsync signal is detected, the interface image to be displayed is updated to a display screen to be displayed.
In the embodiment of the application, the Surface corresponds to two buffers, one of the buffers is frontBuffer, the other Buffer is backing Buffer, and the graphic data is updated in the backing Buffer during updating.
Compared with VNC display interface image transmission, the technical scheme of the embodiment saves one-time cross-system image data transmission, window exchange between the Linux system and the Android system is realized through Surface, compression copying and data structure conversion of data are omitted, and therefore transmission efficiency and transmission real-time performance of graphic data between multiple systems are improved to a great extent, updated graphic data of the multiple systems can be presented on a display screen in time, and user experience is improved.
Fig. 4 is a schematic structural diagram of a multi-system graphics data processing apparatus according to an embodiment of the present application, and as shown in fig. 4, the multi-system graphics data processing apparatus according to the embodiment of the present application includes:
an obtaining unit 40, configured to obtain graphics data of at least one Surface corresponding to a current display window of a first operating system;
a composition unit 41 for triggering the composer to compose the acquired graphics data into a buffer;
an adapting unit 42, configured to adapt the graphics data in the buffer to a corresponding composer back end;
a transmission unit 43, configured to trigger the first operating system to use the backend of the composer as a Surface object of a Surface field instance of the second operating system through a dynamic library access protocol between the first operating system and the second operating system;
and the output unit 44 is configured to trigger the Surface pointer instance of the second operating system to render and generate a graphical display interface including the backend of the composer and the Surface object of the second operating system, and output the graphical display interface.
As an implementation manner, the obtaining unit 40 is further configured to:
detecting an operating instruction aiming at the first operating system, determining a dirty area of a current display window of the first operating system, determining at least one Surface corresponding to the dirty area according to the dirty area, and acquiring graphic data corresponding to the at least one Surface.
As an implementation manner, the synthesis unit 41 is further configured to:
and triggering the composer of the first operating system to call the graphic data of at least one application corresponding to the current display window through the Wayland or X protocol, and synthesizing the called graphic data into the buffer area.
As an implementation manner, the output unit 44 is further configured to:
and triggering the first operating system to send the composer back end to the Surface finger instance of the second operating system through the Libhybris, wherein the composer back end is used as a Surface object of the Surface finger instance.
As an implementation manner, the output unit 44 is further configured to:
and after detecting the Vsync of the graphical interface, triggering the SurfaceFinger instance of the second operating system to acquire all the Surface objects, and rendering to generate the graphical display interface.
In an exemplary embodiment, the obtaining Unit 40, the synthesizing Unit 41, the adapting Unit 42, the transmitting Unit 43, the output Unit 44, and the like may be implemented by one or more Central Processing Units (CPUs), Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Programmable Logic Devices (PLDs), Complex Programmable Logic Devices (CPLDs), Field Programmable Gate Arrays (FPGAs), general purpose processors, controllers, Micro Controllers (MCUs), microprocessors (processors), or other electronic elements for performing the steps of the multi-system graphic data Processing method of the foregoing embodiment.
In the embodiment of the present disclosure, the specific manner in which each unit in the multi-system graphics data processing apparatus shown in fig. 4 performs operations has been described in detail in the embodiment related to the method, and will not be described in detail here.
The operating system execution device in the embodiment of the present application may be a device, and may also be a component, an integrated circuit, or a chip in a terminal. The device can be mobile electronic equipment or non-mobile electronic equipment. By way of example, the mobile electronic device may be a mobile phone, a tablet Computer, a notebook Computer, a palm top Computer, an in-vehicle electronic device, a wearable device, an Ultra-mobile Personal Computer (UMPC), a netbook or a Personal Digital Assistant (PDA), and the like, and the non-mobile electronic device may be a server, a Network Attached Storage (NAS), a Personal Computer (Personal Computer, PC), a Television (TV), a teller machine, a self-service machine, and the like, and the embodiments of the present application are not particularly limited.
Next, an electronic apparatus 11 according to an embodiment of the present application is described with reference to fig. 5.
As shown in fig. 5, the electronic device 11 includes one or more processors 111 and memory 112.
The processor 111 may be a Central Processing Unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device 11 to perform desired functions.
Memory 112 may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, Random Access Memory (RAM), cache memory (cache), and/or the like. The non-volatile memory may include, for example, Read Only Memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium and executed by the processor 111 to implement the multi-system graphics data processing method of the various embodiments of the present application described above and/or other desired functions. Various contents such as an input signal, a signal component, a noise component, etc. may also be stored in the computer-readable storage medium.
In one example, the electronic device 11 may further include: an input device 113 and an output device 114, which are interconnected by a bus system and/or other form of connection mechanism (not shown).
The input device 113 may include, for example, a keyboard, a mouse, and the like.
The output device 114 may output various information including the determined distance information, direction information, and the like to the outside. The output devices 114 may include, for example, a display, speakers, a printer, and a communication network and its connected remote output devices, among others.
Of course, for the sake of simplicity, only some of the components of the electronic device 11 relevant to the present application are shown in fig. 5, and components such as buses, input/output interfaces, and the like are omitted. In addition, the electronic device 11 may include any other suitable components, depending on the particular application.
The embodiment of the application also describes a readable non-transitory storage medium, on which an executable program is stored, and the executable program is used by a processor to execute the steps of the multi-system graphic data processing method of the foregoing embodiment.
In addition to the above-described methods and apparatus, embodiments of the present application may also be a computer program product comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps in the methods according to the various embodiments of the present application described in the "exemplary methods" section of this specification, above.
The computer program product may be written with program code for performing the operations of embodiments of the present application in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server.
Furthermore, embodiments of the present application may also be a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, cause the processor to perform steps in a method according to various embodiments of the present application described in the "exemplary methods" section above of this specification.
The computer-readable storage medium may take any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The foregoing describes the general principles of the present application in conjunction with specific embodiments, however, it is noted that the advantages, effects, etc. mentioned in the present application are merely examples and are not limiting, and they should not be considered essential to the various embodiments of the present application. Furthermore, the foregoing disclosure of specific details is for the purpose of illustration and description and is not intended to be limiting, since the foregoing disclosure is not intended to be exhaustive or to limit the disclosure to the precise details disclosed.
The block diagrams of devices, apparatuses, systems referred to in this application are only given as illustrative examples and are not intended to require or imply that the connections, arrangements, configurations, etc. must be made in the manner shown in the block diagrams. These devices, apparatuses, devices, systems may be connected, arranged, configured in any manner, as will be appreciated by those skilled in the art. Words such as "including," "comprising," "having," and the like are open-ended words that mean "including, but not limited to," and are used interchangeably therewith. The words "or" and "as used herein mean, and are used interchangeably with, the word" and/or, "unless the context clearly dictates otherwise. The word "such as" is used herein to mean, and is used interchangeably with, the phrase "such as but not limited to".
It should also be noted that in the devices, apparatuses, and methods of the present application, the components or steps may be decomposed and/or recombined. These decompositions and/or recombinations are to be considered as equivalents of the present application.
The previous description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects without departing from the scope of the application. Thus, the present application is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
The foregoing description has been presented for purposes of illustration and description. Furthermore, the description is not intended to limit embodiments of the application to the form disclosed herein. While a number of example aspects and embodiments have been discussed above, those of skill in the art will recognize certain variations, modifications, alterations, additions and sub-combinations thereof.
Claims (12)
1. A multisystem graphic data processing method, characterized in that the method comprises:
acquiring graphic data of at least one layer transmission interface Surface corresponding to a current display window of a first operating system, and synthesizing the acquired graphic data to a buffer area through a synthesizer;
the graphic data in the buffer area is adapted to the back end of the corresponding composer;
triggering the first operating system to use the backend of the composer as a Surface object of a Surface Finger instance of a graphics rendering composition controller of a second operating system through a dynamic library access protocol between the first operating system and the second operating system;
and triggering the Surface finger instance of the second operating system to render and generate a graphical display interface including the composer back end and the Surface object of the second operating system, and outputting the graphical display interface.
2. The method of claim 1, wherein the obtaining of the graphics data of the at least one Surface corresponding to the current display window of the first operating system comprises:
detecting an operating instruction aiming at the first operating system, determining a dirty area of a current display window of the first operating system, determining at least one Surface corresponding to the dirty area according to the dirty area, and acquiring graphic data corresponding to the at least one Surface.
3. The method according to claim 1 or 2, wherein the composing the acquired graphics data into the buffer by the compositor comprises:
the composer of the first operating system calls the graphic data of at least one application corresponding to the current display window through the Wayland or X protocol, and synthesizes the called graphic data into the buffer area.
4. The method as claimed in claim 3, wherein said triggering the first operating system to use the composer back end as a Surface object of a graphics rendering composition controller Surface filler instance of the second operating system through a dynamic library access protocol between the first operating system and the second operating system comprises:
and triggering the first operating system to send the composer back end to the Surface finger instance of the second operating system through the Libhybris, wherein the composer back end is used as a Surface object of the Surface finger instance.
5. The method as claimed in claim 3, wherein said triggering the Surface finger instance of the second os to render a Surface object containing the composer backend and the second os into a gui comprises:
and after detecting a vertical synchronization signal Vsync of the graphical interface, triggering the SurfaceFinger instance of the second operating system to acquire all Surface objects, and rendering to generate the graphical display interface.
6. A multisystem graphics data processing apparatus, the apparatus comprising:
the acquisition unit is used for acquiring graphic data of at least one layer transmission interface Surface corresponding to a current display window of a first operating system;
the composition unit is used for triggering the composer to compose the acquired graphic data into a buffer area;
the adaptation unit is used for adapting the graphic data in the buffer area to the rear end of the corresponding composer;
a transmission unit, configured to trigger the first operating system to use the backend of the composer as a Surface object of a Surface field instance of a graphics rendering composition controller of a second operating system through a dynamic library access protocol between the first operating system and the second operating system;
and the output unit is used for triggering the Surface finger instance of the second operating system to render and generate a graphical display interface by the Surface object containing the composer rear end and the second operating system, and outputting the graphical display interface.
7. The apparatus of claim 6, wherein the obtaining unit is further configured to:
detecting an operating instruction aiming at the first operating system, determining a dirty area of a current display window of the first operating system, determining at least one Surface corresponding to the dirty area according to the dirty area, and acquiring graphic data corresponding to the at least one Surface.
8. The apparatus of claim 6 or 7, wherein the synthesis unit is further configured to:
and triggering the composer of the first operating system to call the graphic data of at least one application corresponding to the current display window through the Wayland or X protocol, and synthesizing the called graphic data into the buffer area.
9. The apparatus of claim 8, wherein the output unit is further configured to:
and triggering the first operating system to send the composer back end to the Surface finger instance of the second operating system through the Libhybris, wherein the composer back end is used as a Surface object of the Surface finger instance.
10. The apparatus of claim 8, wherein the output unit is further configured to:
and after detecting the Vsync of the graphical interface, triggering the SurfaceFinger instance of the second operating system to acquire all the Surface objects, and rendering to generate the graphical display interface.
11. An electronic device comprising a processor, a memory and a program or instructions stored on the memory and executable on the processor, the program or instructions when executed by the processor implementing the steps of the multi-system graphics data processing method of any one of claims 1 to 5.
12. A readable non-transitory storage medium, wherein the storage medium stores thereon a program or instructions which, when executed by a processor, implement the steps of the multi-system graphics data processing method of any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111315195.5A CN114035851B (en) | 2021-11-08 | 2021-11-08 | Multi-system graphic data processing method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111315195.5A CN114035851B (en) | 2021-11-08 | 2021-11-08 | Multi-system graphic data processing method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114035851A true CN114035851A (en) | 2022-02-11 |
| CN114035851B CN114035851B (en) | 2023-10-03 |
Family
ID=80136707
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111315195.5A Active CN114035851B (en) | 2021-11-08 | 2021-11-08 | Multi-system graphic data processing method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114035851B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115639977A (en) * | 2022-12-23 | 2023-01-24 | 北京海誉动想科技股份有限公司 | Android graph synthesis method and device, electronic equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104765594A (en) * | 2014-01-08 | 2015-07-08 | 联发科技（新加坡）私人有限公司 | Method and device for displaying graphical user interface |
| CN106327542A (en) * | 2016-08-29 | 2017-01-11 | 苏州蓝海彤翔系统科技有限公司 | Picture layer synthesis method and system, client and server |
| WO2017167127A1 (en) * | 2016-03-31 | 2017-10-05 | 阿里巴巴集团控股有限公司 | Image compositing method, window configuration method, and system |
| CN109240677A (en) * | 2017-07-07 | 2019-01-18 | 阿里巴巴集团控股有限公司 | A kind of figure layer process method and device |
| CN110020300A (en) * | 2017-11-07 | 2019-07-16 | 华为技术有限公司 | A kind of browser page synthetic method and terminal |
-
2021
- 2021-11-08 CN CN202111315195.5A patent/CN114035851B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104765594A (en) * | 2014-01-08 | 2015-07-08 | 联发科技（新加坡）私人有限公司 | Method and device for displaying graphical user interface |
| WO2017167127A1 (en) * | 2016-03-31 | 2017-10-05 | 阿里巴巴集团控股有限公司 | Image compositing method, window configuration method, and system |
| CN106327542A (en) * | 2016-08-29 | 2017-01-11 | 苏州蓝海彤翔系统科技有限公司 | Picture layer synthesis method and system, client and server |
| CN109240677A (en) * | 2017-07-07 | 2019-01-18 | 阿里巴巴集团控股有限公司 | A kind of figure layer process method and device |
| US20200193556A1 (en) * | 2017-07-07 | 2020-06-18 | Alibaba Group Holding Limited | Graphics layer processing in a multiple operating systems framework |
| CN110020300A (en) * | 2017-11-07 | 2019-07-16 | 华为技术有限公司 | A kind of browser page synthetic method and terminal |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115639977A (en) * | 2022-12-23 | 2023-01-24 | 北京海誉动想科技股份有限公司 | Android graph synthesis method and device, electronic equipment and storage medium |
| CN115639977B (en) * | 2022-12-23 | 2024-01-26 | 安徽海马云科技股份有限公司 | Android graph synthesis method and device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114035851B (en) | 2023-10-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107251004B (en) | Backward compatibility by using spoofed clocks and fine-grained frequency control | |
| EP2622463B1 (en) | Instant remote rendering | |
| CN112114916B (en) | Method and device for compatibly running Android application on Linux operating system | |
| US8957905B2 (en) | Cross-environment user interface mirroring | |
| KR102307163B1 (en) | Cross-platform rendering engine | |
| CN107292807B (en) | Graph synthesis method, window setting method and system | |
| US9984651B2 (en) | Method and apparatus for displaying composition screen image by composing screen images of operating systems (OSs) | |
| CN110073324B (en) | Method and apparatus for dynamically reducing application rendering time on screen in desktop environment | |
| CN116821040B (en) | Display acceleration method, device and medium based on GPU direct memory access | |
| US11024257B2 (en) | Android platform based display device and image display method thereof | |
| CN114035851A (en) | Multi-system graphic data processing method and device, electronic equipment and storage medium | |
| US20230052979A1 (en) | Image display method and apparatus, and medium | |
| CN108021366B (en) | Interface animation realization method and device, electronic equipment and storage medium | |
| WO2023221822A1 (en) | Data processing method, electronic device, and readable storage medium | |
| CN115373618B (en) | Multi-screen display method and device, vehicle machine and storage medium | |
| EP3198406B1 (en) | Facilitation of guest application display from host operating system | |
| EP3522530A1 (en) | System performance improvement method, system performance improvement device and display device | |
| CN115220803A (en) | Data processing method and equipment | |
| CN114020375A (en) | Display method and device | |
| CN113728622A (en) | Method and device for wirelessly transmitting image, storage medium and electronic equipment | |
| CN117234655B (en) | Method, device, equipment and medium for running Linux desktop program based on android system | |
| CN114327721B (en) | Display method of multiple operating systems and electronic equipment | |
| CN114727051B (en) | Media resource transmission device, system and method | |
| CN117492607A (en) | Multi-screen collaboration method, vehicle and computer storage medium | |
| CN117008860A (en) | Application screen projection method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20220330 Address after: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Applicant after: BEIJING BYTEDANCE NETWORK TECHNOLOGY Co.,Ltd. Address before: 2005, floor 2, No. 39, West Street, Haidian District, Beijing 100080 Applicant before: Beijing jingling Information System Technology Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |