CN112269603B - Graphic display method and device for compatibly running Android application on Linux - Google Patents
Graphic display method and device for compatibly running Android application on Linux Download PDFInfo
- Publication number
- CN112269603B CN112269603B CN202011257571.5A CN202011257571A CN112269603B CN 112269603 B CN112269603 B CN 112269603B CN 202011257571 A CN202011257571 A CN 202011257571A CN 112269603 B CN112269603 B CN 112269603B
- Authority
- CN
- China
- Prior art keywords
- graphics
- linux
- android
- gpu
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/445—Program loading or initiating
- G06F9/44505—Configuring for program initiating, e.g. using registry, configuration files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/445—Program loading or initiating
- G06F9/44536—Selecting among different versions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/451—Execution arrangements for user interfaces
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Image Generation (AREA)
Abstract
The invention discloses a graph display method and device for compatibly running Android application on Linux, and belongs to the field of computer basic software. The method comprises the following steps: judging whether the Android supports the current GPU hardware type, if so, directly importing the graphics drawn in the Android into the DRI of the Linux for synthesis and then sending the graphics to the display, otherwise, directly converting an Android graphics library OpenGL ES into an OpenGL library interface of the Linux, and then calling the GPU through the DRM of the Linux for graphics drawing, synthesis and sending the display; in addition, the situation that the old display card does not support the application graphic library is solved by a graphic soft acceleration method. The method solves the display problem caused by the fact that Android applications running on a Linux system are compatible with different types of GPU hardware, and different strategies are adopted to complete the whole graphic display process according to different GPU types in the current system environment so as to achieve the purpose of integrating graphic display.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a method and a device for displaying a fusion graph in the field of Linux compatible with Android operation.
Background
Generally, an Android graphics display mechanism is completed in an Android system, and in order to solve the problem of graphics display on an embedded device with a weak GPU capability, Android employs an embedded graphics GPU and an OpenGL ES graphics display interface library. Fig. 3 shows an Android graph display mechanism, specifically, the Android graph display is a process of drawing a graph by using a surfefinger in the Android, the surfefinger is a control unit, for a graph layer to be displayed by an upper application, the surfefinger selects a corresponding display device, and simultaneously calls an OpenGL ES graphics library interface through an EGL interface to complete corresponding drawing, and outputs the drawing to a framebuffernetivewindow to call a display device and a GPU device by using graceoc, and the GPU device completes a process of drawing and rendering the graph. And finally, sending the rendered graph to display equipment in a Linux kernel for displaying, thereby completing the whole process from drawing to displaying of the graph.
The method is greatly different from a Linux graphic display mechanism and a Linux graphic display framework, wherein Linux graphic display is an independent display card or an integrated display card with strong GPU facing capability, and an OpenGL graphic display library interface is adopted. Specifically, the Linux graphics display is based on the X protocol, which is a set of software specifications and protocols for displaying graphical interfaces with bitmap images. At present, XWindow System is realized by Xorg which adopts C/S architecture, and the server and the client can communicate based on network. Fig. 4 shows a graph display mechanism of Linux, where an OpenGL application calls an OpenGL graph library through a graph drawing interface, and OpenGL completes drawing work on various layers in the application through a base layer direct Rendering DRI (direct Rendering) graph Rendering framework calling a direct Rendering manager drm (direct Rendering manager), where the base layer direct Rendering DRI is responsible for completing calling a hardware GPU when a specific graph is drawn, the GPU completes corresponding graph drawing and Rendering and stores the graph drawing and Rendering into a FrameBuffer, and data in the FrameBuffer is transmitted to a display controller LCDC through a KMS (Kernel Mode-Setting) to complete the graph display work. So far, the whole process from drawing all the graphs to displaying is completed.
Obviously, the Android and Linux display mechanisms and oriented hardware have great differences, and when the Android application is compatibly run on a Linux system, different types of GPU physical devices are encountered, and the different types of GPU devices themselves support different operating systems, for example: the GPU which can be directly operated on the original Linux does not support the Android, and the GPU supported on the original Android system does not support the graphic display of the Linux. Therefore, when the Android is required to run on the Linux operating system in a compatible layer manner, the graphics display mechanisms of the Android and the Linux operating system must be opened.
In addition, if the Android application is to be directly run on Linux, various different types of graphics card problems must also be faced. At present, when Android is compatible to run, a part of old display cards cannot support the high version of the Open GL graphics library due to the display card, and a part of Android applications depend on the high version graphics library, so that the situation that the applications cannot run is caused.
Disclosure of Invention
The invention aims to solve the main technical problems that when an Android application is compatibly operated in a Linux system in a container mode, due to the fact that the Android graphic display and the Linux graphic display mechanism have large difference, the graphic display function cannot be normally completed, and due to the fact that incompatibility is caused by the fact that different types of GPU physical equipment face, a graphic display method and a graphic display device which are compatibly operated on the Linux are provided, the Android and Linux graphic display mechanisms can be unified, different strategies are respectively adopted according to the GPU types of different display cards in the current system environment to complete the whole process of graphic display, the purpose of fusing graphic display is achieved, and compared with a single strategy, consumption of system resources can be reduced, and graphic display efficiency is improved. In addition, the situation that the old display card does not support the graphic library required by the application is solved by a graphic soft acceleration method.
According to one aspect of the invention, a graphics display method for compatibly running an Android application on Linux is provided, wherein the Android application is compatibly run in a Linux system in a container mode, and when a GPU hardware type of the Linux system can support a graphics library required by the Android application, the method comprises the following steps: s1: judging whether the Android side of the container supports the GPU hardware type or not by a graph drawing control process, if so, starting S2, and if not, starting S3 if the Linux side of the container supports the GPU hardware type; s2: calling OpenGL ES by the Android side of the container, calling a GPU by the OpenGL ES according to a graph drawing request to generate first graph data, transmitting the first graph data to a base layer direct rendering DRI of the Linux side of the container by the graph drawing control process, and acquiring the first graph data and completing the synthetic drawing of graphs by the base layer direct rendering DRI to generate second graph data; the base layer direct rendering DRI sends the second graphics data to the current Linux system display hardware; s3: calling an OpenGL ES by the Android side of the container, wherein the OpenGL ES transmits the graph drawing request to the graph drawing control process; the graphics drawing control process converts the graphics drawing request from an OpenGL ES interface format to an OpenGL interface format, and then OpenGL calls a GPU to generate third graphics data; the graphics drawing control process transmits the first graphics data to a base layer direct rendering DRI on the Linux side of the container, the base layer direct rendering DRI acquires the third graphics data and completes the synthesis drawing of graphics to generate fourth graphics data, and the base layer direct rendering DRI transmits the fourth graphics data to the current Linux system display hardware through a Linux kernel; when the hardware type of the GPU of the Linux system cannot support the graphics library required by the Android application, executing the following steps: s4: judging whether the Android side of the container supports the GPU hardware type or not by a graph drawing control process, if so, executing a step S5: calling a graphics soft acceleration process by the Android side of the container, calling a CPU (central processing unit) by the graphics soft acceleration process according to the graphics drawing request to generate fifth graphics data, and storing the fifth graphics data into a virtual display screen of the Android side of the container through an Android hardware abstraction layer; the graphics rendering control process transmits the fifth graphics data to the base layer direct rendering DRI on the Linux side of the container; the base layer direct rendering DRI sends the fifth graphic data to the current Linux system display hardware;
as a further improvement of the present invention, the OpenGL ES invoking a GPU to generate the first graphics data includes: and the OpenGL ES calls a FrameBufferNativeWindow interface, the FrameBufferNativeWindow interface delivers the graphics drawing request to the GPU, and the GPU generates the first graphics data according to the graphics drawing request and writes the first graphics data into a frame cache of the GPU.
As a further improvement of the present invention, the invoking, by the OpenGL, the GPU to generate the first graphics data includes: the Open GL invokes a direct rendering manager DRM, which passes the graphics drawing request to the GPU, which generates the first graphics data.
As a further improvement of the present invention, the graphics soft acceleration process emulates a graphics interface that implements OPEN GL and OPEN GL ES.
According to another aspect of the present invention, there is provided a graphics display apparatus compatible with running an Android application on Linux, wherein the Android application is compatible with running in a Linux system in a container manner, and when a GPU hardware type of the Linux system can support a graphics library required by the Android application, the apparatus includes: the graphics drawing control module is used for acquiring the GPU hardware type, judging whether the Android supports the GPU hardware type, and starting a first drawing module if the Android supports the GPU hardware type; if not, starting a second drawing module if the container Linux side supports the GPU hardware type; a first rendering module: calling OpenGL ES from the Android side of the container in the first drawing module, calling a GPU to generate first graphic data according to an obtained graphic drawing request by the OpenGL ES, transmitting the first graphic data to a base layer direct rendering DRI of the Linux side by the graphic drawing control module, and acquiring the first graphic data and completing the synthetic drawing of a graphic by the base layer direct rendering DRI to generate second graphic data; the base layer direct rendering DRI sends the second graphic data to the current Linux system display hardware for display through a Linux kernel; a second rendering module: calling OpenGL ES from the Android side of the container in the second drawing module, transmitting a graphics drawing request to the graphics drawing control module by the OpenGL ES, converting the graphics drawing request from an OpenGL ES interface format to an OpenGL interface format by the graphics drawing control module, calling a GPU by the OpenGL to generate third graphics data, transmitting the third graphics data to the base layer direct rendering DRI of the Linux side of the container by the graphics drawing control module, acquiring the third graphics data by the base layer direct rendering DRI and completing the synthetic drawing of graphics to generate fourth graphics data, and transmitting the fourth graphics data to the current Linux system display hardware by the base layer direct rendering DRI through a Linux kernel for display; when the Linux system GPU hardware type cannot support the graphics library required by the Android application, the graphics drawing control module judges whether the container Android side supports the GPU hardware type, if so, the container Android side calls: a graphics soft acceleration module: the graphics soft acceleration module calls a CPU to generate fifth graphics data according to the graphics drawing request, and the fifth graphics data are stored in a virtual display screen on the Android side of the container through an Android hardware abstraction layer; the graphics drawing control module acquires the fifth graphics data from the virtual display screen and transmits the fifth graphics data to a base layer direct rendering DRI on the Linux side of the container; the base layer direct rendering DRI sends the fifth graphic data to the current Linux system display hardware for display;
as a further improvement of the present invention, the graphics rendering system further includes a translation module, and the graphics rendering control module calls the translation module to convert the graphics rendering request from an OpenGL ES interface format to an OpenGL interface format.
As a further improvement of the present invention, the OpenGL ES invoking a GPU to generate the first graphics data includes: the graphics soft acceleration module simulates a graphics interface for realizing OPEN GL and OPEN GL ES.
The invention has the beneficial effects that:
(1) unifying graph display mechanisms of Android and Linux;
the Android graph display mechanism and the Linux graph display mechanism and hardware oriented to the Android graph display mechanism and the Linux graph display mechanism are different greatly, so that the problem that the graph display function cannot be completed normally when the Android runs on a Linux operating system in a compatible layer mode occurs.
(2) Different strategies are adopted to support different types of display cards respectively;
the method provided by the invention can enable the Android graph drawing to respectively adopt different strategies according to the GPU types of different display cards in the current system environment so as to complete the whole process of graph display, so that the purpose of fusing graph display is achieved, and compared with a single strategy, the method can reduce the consumption of system resources and improve the graph display efficiency.
(3) Under the condition that an old display card of the Linux system cannot support the high version of the Open GL graphic library required by the Android application, the soft acceleration of the graphics is realized.
The foregoing description is only an overview of the technical solutions of the present invention, and the embodiments of the present invention are described below in order to make the technical means of the present invention more clearly understood and to make the above and other objects, features, and advantages of the present invention more clearly understandable.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
fig. 1 shows an implementation schematic diagram of a graph display method for compatibly running an Android application on Linux according to an embodiment of the present invention;
fig. 2 shows a flowchart of a graphics display method for compatibly running an Android application on Linux according to an embodiment of the present invention;
FIG. 3 illustrates an Android graphics display mechanism schematic;
FIG. 4 shows a Linux graphical display mechanism schematic;
fig. 5 shows a diagram of implementing soft acceleration of a graph display method for compatibly running an Android application on Linux according to an embodiment of the present invention;
fig. 6 shows a flowchart of implementing graphics soft acceleration of a graphics display method for compatibly running an Android application on Linux according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
It will be understood that the description and claims of the present invention and the method and apparatus of the drawings are referred to one another with respect to the features thereof. Furthermore, the terms "first," "second," and the like in the description and in the claims, and in the drawings, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein.
First, partial terms or terms appearing in the description of the embodiments of the present invention are applied to the following explanations:
HAL: HAL is an abbreviation of Hardware Abstraction Layer Hardware Abstraction Layer, is a set of dynamic libraries built on Linux driver and has a standard calling interface. The set of dynamic libraries does not belong to the Linux kernel, but belongs to a system operation library layer above the Linux kernel layer. As the HAL has a standard hardware calling interface, the complicated and non-uniform interface of the Linux drive can be shielded.
The container technology comprises the following steps: the container effectively partitions resources managed by a single operating system into isolated groups to better balance conflicting resource usage requirements among the isolated groups.
JStack: the Android operating environment comprises a container.
EGL: interface between OpenGL ES and underlying platform windowing systems.
Dri (direct Rendering architecture) base layer direct Rendering.
DRM: the Direct Rendering Manager is a kernel module giving the DRI client Direct access to hardware.
KMS: kemel model-Setting kernel Mode Setting.
LCDC: and displaying the controller.
For compatibly running the Android running environment on the Linux operating system, the Android running environment is run in a container. Starting an Android running environment through a container technology on the existing Linux operating system, marking the Android running environment as JStack, and running the Android application frame and the Android running environment as a system process of Linux in the JStack.
The core technical problem to be solved by the invention is that when Android runs on a Linux operating system in a compatible layer mode, the problem that the graphic display function cannot be normally completed due to the large difference between Android graphic display and Linux graphic display mechanisms and the problem of incompatibility caused by the fact that different types of GPU physical equipment are faced occur.
Aiming at the technical problem, the invention provides a graph display method and a graph display device for compatibly operating an Android operating system on Linux, wherein firstly, a GPU (graphics processing Unit) supported by the Android is modified, the graph which is stored in a FrameBuffer and drawn in the Android is directly imported into DRI of the Linux, the Linux graph and the Android graph are synthesized, and then the DRI of the Linux can directly complete the work of sending the graph data to display equipment; secondly, for a GPU which is supported by Linux and not supported by Android, directly converting an Android graphics library OpenGL ES into an OpenGL library interface of Linux through a graphics library conversion interface, calling the GPU through the DRM of the Linux to perform graphics rendering and graphics synthesis, and calling DRI to complete the work of sending graphics data to display equipment; and thirdly, for the condition that the old display card cannot support the high version of the Open GL graphic library required by the Android application, calling graphic soft acceleration software simulating a graphic interface for realizing the OPEN GL/ES, calling a CPU (central processing unit) by the graphic soft acceleration software to perform specific graphic drawing, and calling DRI to finish the work of sending graphic data to display equipment.
The method and the device can enable the Android graph drawing to respectively adopt different strategies according to different GPU types in the current system environment so as to complete the whole process of graph display, so that the purpose of fusing graph display is achieved, and compared with a single strategy, the method and the device can reduce the consumption of system resources and improve the efficiency of graph display.
Example 1
The following specifically describes the implementation steps of the present invention in the case that the Linux system GPU hardware type can support the graphics library required by the Android application through two application scenarios.
The application scene one:
fig. 2 is a flowchart of a method for displaying a graph compatibly running an Android application on Linux according to an embodiment of the present invention, and it can be seen from the flowchart that the method according to the embodiment includes the following steps:
s1: judging whether the Android side of the container supports the GPU hardware type or not by a graph drawing control process, and specifically, the method comprises the following steps:
1.1: after the Linux is started, automatically starting a Graphic Controller process of a Graphic drawing control module in JStack to serve as a core Controller for the compatible display of the whole Graphic; the Graphic Controller acquires the hardware information of the GPU in the current system, compares the hardware information with the configuration file in the Graphic Controller and determines the Graphic drawing mode. Fig. 1 shows an implementation schematic diagram of a graphics display method for compatibly running an Android application on Linux according to an embodiment of the present invention, as shown in a left branch of fig. 1, when Android supports the current Linux system GPU hardware type, it is determined to execute step S2.
1.2: the user clicks to open the Android application, the application sends a graph drawing request to the surfaceFlinger, the surfaceFlinger sends graph layer drawing information to the EGL, and the EGL calls an OpenGL ES graph library interface to perform specific graph drawing work.
S2: as shown in the left branch of fig. 1, when the Android supports the current Linux system GPU hardware type, OpenGL ES invokes a GPU to generate first graphics data according to the obtained graphics drawing request, transmits the first graphics data to a base layer direct rendering DRI, and the base layer direct rendering DRI obtains the first graphics data and completes the synthetic rendering of graphics to generate second graphics data; the DRI directly rendered by the base layer sends the second graphics data to the current GPU hardware of the Linux system through a Linux kernel, and the DRI is displayed by combining the flow chart of the figure 2, and specifically comprises the following steps:
2.1: OpenGL ES calls a FrameBufferNativeWindow interface;
2.2: the FrameBufferNativeWindow delivers the detailed information of the graph layer graph to a GPU for specific graph drawing, and writes first graph data generated by drawing into a GPU frame buffer;
2.3: sending a graph drawing transfer request to a graph drawing control process graph Controller, and sending a video memory initial address in a GPU frame cache to the graph Controller;
2.4: the Graphic Controller controls to transmit the first Graphic data in the GPU frame cache to the base layer direct rendering DRI;
2.5: and directly rendering the DRI on the base layer to perform graphic synthesis to finish graphic drawing work, generating second graphic data and transmitting the second graphic data to the KMS, finishing the process of transmitting the graphic data to display equipment, and displaying the graphics in a window at the Linux desktop end.
Application scenario two:
s1: judging whether the Android side of the container supports the GPU hardware type or not by a graph drawing control process, and specifically, the method comprises the following steps:
1.1: after the Linux is started, automatically starting a Graphic Controller process of a Graphic drawing control module in JStack to serve as a core Controller for the compatible display of the whole Graphic; the Graphic Controller acquires the hardware information of the GPU in the current system, compares the hardware information with the configuration file in the Graphic Controller and determines the Graphic drawing mode. Fig. 1 shows an implementation schematic diagram of a graphics display method for compatibly running an Android operating system on Linux according to an embodiment of the present invention, as shown in a right branch of fig. 1, when Android does not support the GPU hardware type of the current Linux system, step S3 is determined to be executed.
1.2: the user clicks to open the Android application, the application sends a graph drawing request to the surfaceFlinger, the surfaceFlinger sends graph layer drawing information to the EGL, and the EGL calls an OpenGL ES graph library interface to perform specific graph drawing work.
S3: as shown in the right branch of fig. 1, when the Android does not support the GPU hardware type of the current Linux system, after the obtained graphics drawing request is converted from the OpenGL ES interface format to the OpenGL interface format, OpenGL calls GPU to generate the third graphics data, and transfers the third graphics data to the base layer direct rendering DRI, the base layer direct rendering DRI obtains the third graphics data and completes the synthesis drawing of the graphics, and generates fourth graphics data, and the base layer direct rendering DRI sends the fourth graphics data to the GPU hardware of the current Linux system through the Linux kernel, which is displayed by combining the flowchart in fig. 2, specifically, the following steps are performed:
3.1: the OpenGL ES does not carry out Graphic drawing work, but transfers the Graphic work needing to be drawn to a Graphic drawing control module Graphic Controller;
3.2: a graph drawing control process graph Controller calls a translation module OpenGL Translator to perform graph interface conversion work, graph conversion from OpenGL ES to OpenGL is completed, and at the moment, drawing requests of all graphs are converted into an OpenGL interface format;
3.3: the Open GL calls DRM, the DRM transfers the graph drawing to the GPU, the specific graph drawing work is completed, third graph data are generated and stored in a frame cache of the GPU;
3.4: and directly rendering the DRI on the base layer to perform graphic synthesis to finish graphic drawing work, generating fourth graphic data and transmitting the fourth graphic data to the KMS, finishing the process of transmitting the graphic data to display equipment, and displaying the graphics in a window at the Linux desktop end.
The following specifically describes the implementation steps of the method in the case that the Linux system GPU hardware type cannot support the graphics library required by the Android application through an application scenario.
Application scenario three:
fig. 5 is a diagram illustrating a soft graphics acceleration implementation principle of a graphics display method for compatibly running an Android application on Linux according to an embodiment of the present invention.
As shown in fig. 5, after the Linux system is started, the graphics card type is discriminated by a graphics rendering control process graphics Controller, and the graphics rendering layer of the Android is notified through an interface that a graphics soft acceleration function needs to be enabled for some types of GPUs.
When the Graphic Controller identifies that the current display card is a display card supported by Android, Native Graphic drawing is carried out through a SurfaceFlinger of the Android, a Graphic soft acceleration process is called through a window drawing interface EGL in the Graphic drawing process, the Graphic soft acceleration process simulates to realize a Graphic interface of OPEN GL/ES, a CPU is called to carry out specific Graphic drawing, the drawn Graphic is output to a virtual display screen in the Android HAL, a Graphic starting address pointer of the display screen is sent to the Graphic Controller through the interface, the Graphic Controller sends the Graphic to a corresponding interface of Linux DRI, and the DRI is told to directly send the Graphic data to display equipment, so that the Graphic display work of the Android application is completed.
The above-mentioned graphics soft acceleration process can be implemented by using the graphics soft acceleration software with a part of open sources in the current Linux world, such as swift shader of google, which can implement the capability of OpenGL graphics library.
Fig. 6 is a flowchart of implementing soft acceleration of a graph display method for compatibly running an Android application on Linux according to an embodiment of the present invention, and it can be seen from the diagram that the method according to the embodiment includes the following steps:
s4: judging whether the Android side of the container supports the GPU hardware type or not by a graph drawing control process, and specifically, the method comprises the following steps:
1.1: after the Linux is started, automatically starting a Graphic Controller process of a Graphic drawing control module in JStack to serve as a core Controller for the compatible display of the whole Graphic; the Graphic Controller acquires the hardware information of the GPU of the current system, compares the hardware information with a configuration file in the Graphic Controller and determines a Graphic drawing mode;
s5: calling a graphics soft acceleration process by the Android side of the container, calling a CPU (central processing unit) by the graphics soft acceleration process according to a graphics drawing request to generate fifth graphics data, and storing the fifth graphics data into a virtual display screen on the Android side of the container through an Android hardware abstraction layer; the graph drawing control process transmits the fifth graph data to a base layer of a container Linux side to directly render the DRI; the base layer directly renders the DRI and sends the fifth graphic data to the display hardware of the current Linux system;
as can be seen in fig. 6, the following steps are specifically plotted:
1.2: the user clicks to open the Android application, the application sends a graph drawing request to the surfaceFlinger, and the surfaceFlinger informs the EGL to start a graph soft acceleration process according to the graph drawing mode obtained in 1.1;
1.3: the EGL starts the graphic soft acceleration module and sends the graphic information to be drawn to the graphic soft acceleration module through an interface;
1.4: the graphics soft acceleration module simulates a corresponding API of an Open GL/ES graphics library, sends a specific drawing requirement to the CPU, and the CPU performs graphics drawing according to the requirement, generates fifth graphics data and returns the fifth graphics data to the graphics software acceleration module;
1.5: the graphic soft acceleration module informs the Android HAL to send graphic data to the virtual display screen;
1.6: the Android HAL writes the fifth graphic data into the virtual display screen;
1.7: and the Android HAL informs the Graphic Controller that the graph is drawn completely and sends fifth graph data to the virtual display screen.
As can be seen in fig. 6, in particular the sending of the graphics data to the display device has the following steps:
2.1: the Graphic Controller informs the DRM that the Graphic sending work needs to be carried out and transmits the initial address pointer in the Graphic memory which is drawn;
2.2: the DRM acquires fifth graphic data from the virtual display screen according to the initial address pointer in the memory;
2.3: the DRM calls the DRI GEM to prepare for starting to send the fifth graphic data;
2.4: the DRI GEM passes the fifth graphics data to the KMS;
2.5: and refreshing the image in the appointed area of the display screen of the KMS control box, and finishing the work of sending the fifth graphic data to the display equipment.
Example 2
Further, as an implementation of the method shown in the foregoing embodiment, another embodiment of the present invention further provides a graphics display apparatus compatible with the Android operating system running on Linux. The embodiment of the apparatus corresponds to the embodiment of the method, and for convenience of reading, details in the embodiment of the apparatus are not repeated one by one, but it should be clear that the apparatus in the embodiment can correspondingly implement all the contents in the embodiment of the method. In the apparatus of this embodiment, there are the following modules:
firstly, the method comprises the following steps: a Graphic drawing control module Graphic Controller (corresponding to the Graphic drawing control process in the implementation 1), acquiring the current Linux system GPU hardware type, judging whether the Android supports the current Linux system GPU hardware type, if so, starting a first drawing module, and if not, starting a second drawing module; specifically, the method comprises the following steps:
1.1: after the Linux is started, automatically starting a Graphic Controller process of a Graphic drawing control module in JStack to serve as a core Controller for the compatible display of the whole Graphic; the Graphic Controller acquires the hardware information of the GPU in the current system, compares the hardware information with the configuration file in the Graphic Controller and determines the Graphic drawing mode. As shown in the left branch of fig. 1, specifically, when the Android supports the current Linux system GPU hardware type, it is determined to invoke the first rendering module to perform rendering work; as shown in the right branch of fig. 1, when the Android does not support the current Linux system GPU hardware type, it is determined to invoke a second rendering module to perform rendering work;
1.2: the user clicks to open the Android application, the application sends a graph drawing request to the surfaceFlinger, the surfaceFlinger sends graph layer drawing information to the EGL, and the EGL calls an OpenGL ES graph library interface to perform specific graph drawing work.
II, secondly: a first drawing module (corresponding to S2 in implementation 1), as shown in the right branch of fig. 1, OpenGLES invokes a GPU according to an acquired graphics drawing request to generate first graphics data, a graphics drawing control module transfers the first graphics data to a base layer direct rendering DRI, and the base layer direct rendering DRI acquires the first graphics data and completes the synthetic drawing of graphics to generate second graphics data; and the base layer direct rendering DRI sends the second graphic data to the GPU hardware of the current Linux system for display through a Linux kernel. As shown in the flowchart of fig. 2, the following steps are specifically performed:
2.1: OpenGL ES calls a FrameBufferNativeWindow interface;
2.2: the FrameBufferNativeWindow delivers the detailed information of the graph layer graph to the GPU for specific graph drawing, and writes the drawn first graph data into the GPU FrameBuffer;
2.3: sending a graph drawing transmission request to a graph drawing control module graph Controller, and sending a video memory starting address in the FrameBuffer to the graph drawing control module graph Controller;
2.4: the Graphic drawing control module Graphic Controller controls the GPU FrameBuffer data to be transmitted to the base layer direct rendering DRI;
2.5: and directly rendering the DRI on the base layer to perform graphic synthesis to finish graphic drawing work, generating second graphic data, transmitting the second graphic data to the KMS, finishing the process of sending the graphic data to display equipment, and displaying the graphics in a window at the Linux desktop end.
Thirdly, the method comprises the following steps: as shown in the right branch of fig. 1, the second rendering module (corresponding to S3 in embodiment 1) sends a graphics rendering request to the graphics Controller by OpenGL ES, converts the graphics rendering request from the OpenGL ES interface format to the OpenGL interface format, calls GPU by OpenGL to generate third graphics data, transmits the third graphics data to the base layer direct rendering DRI by the graphics Controller, obtains the third graphics data and completes the synthetic rendering of the graphics by the base layer direct rendering DRI, generates fourth graphics data, and transmits the fourth graphics data to the current Linux system GPU hardware by the base layer direct rendering DRI through the Linux kernel. As shown in the flowchart of fig. 2, the following steps are specifically performed:
3.1: OpenGL ES does not carry out Graphic drawing work, but transfers the Graphic work needing drawing to the Graphic Controller;
3.2: the Graphic Controller calls a translation module OpenGL Translator to perform Graphic interface conversion work, graphics conversion from OpenGL ES to OpenGL is completed, and at the moment, drawing requests of all graphics are converted into an OpenGL interface format;
3.3: the Open GL calls DRM, the DRM transfers the graph drawing to the GPU, the specific graph drawing work is completed, third graph data are generated and stored in FramBuffer of the GPU;
3.4: and directly rendering the DRI on the base layer to perform graphic synthesis to finish graphic drawing work, generating fourth graphic data and transmitting the fourth graphic data to the KMS, finishing the process of transmitting the graphic data to display equipment, and displaying the graphics in a window at the Linux desktop end.
Fourthly, the method comprises the following steps: a graphics soft acceleration module (corresponding to the graphics soft acceleration process in the implementation 1) calls a CPU (central processing unit) to generate fifth graphics data according to the graphics drawing request, and the fifth graphics data is stored in a virtual display screen on the Android side of the container through an Android hardware abstraction layer; the graphics drawing control module transmits the fifth graphics data to a base layer of a container Linux side to directly render the DRI; and the base layer direct rendering DRI sends the fifth graphic data to the display hardware of the current Linux system for display.
The flowchart in connection with fig. 6 shows, in particular, the following steps:
1.2: the user clicks to open the Android application, the application sends a graph drawing request to the surfaceFlinger, and the surfaceFlinger informs the EGL to start a graph soft acceleration process according to the graph drawing mode obtained in 1.1;
1.3: the EGL starts the graphic soft acceleration module and sends the graphic information to be drawn to the graphic soft acceleration module through an interface;
1.4: the graphics soft acceleration module simulates an Open GL/ES graphics library corresponding API, sends a specific drawing requirement to the CPU, the CPU draws graphics according to the requirement and returns the drawing completed fifth graphics data to the graphics software acceleration module;
1.5: the graphic soft acceleration module informs the Android HAL to send fifth graphic data to the virtual display screen;
1.6: the Android HAL writes the fifth graphic data into the virtual display screen;
1.7: and the Android HAL informs the Graphic Controller that the graph is drawn and sends the graph to the virtual display screen.
As shown in the flowchart of fig. 6, the step of sending the fifth graphic data to the display device specifically includes:
2.1: the graphical Controller informs the DRM that the fifth Graphic data needs to be sent to the display equipment, and transmits the initial address pointer in the drawn Graphic memory;
2.2: the DRM acquires fifth graphic data from the virtual display screen according to the initial address pointer in the memory;
2.3: the DRM calls the DRI GEM to prepare for starting the work of sending the fifth graphic data to the display equipment;
2.4: the DRI GEM passes the fifth graphics data to the KMS;
2.5: and refreshing the image in the appointed area of the display screen of the KMS control box to finish the image display work.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments of the apparatus.
It will be appreciated that the relevant features of the above methods and systems may be referred to one another. In addition, "first", "second", and the like in the above embodiments are for distinguishing the embodiments, and do not represent merits of the embodiments.
The algorithms and displays presented herein are not inherently related to any particular computer, virtual machine, or other apparatus. Various general purpose systems may also be used with the teachings herein. The required structure for constructing such a system will be apparent from the description above. Moreover, the present invention is not directed to any particular programming language. It is appreciated that a variety of programming languages may be used to implement the teachings of the present invention as described herein, and any descriptions of specific languages are provided above to disclose the best mode of the invention.
In the description provided herein, numerous specific details are set forth. It is understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the invention and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be interpreted as reflecting an intention that: that the invention as claimed requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.
Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features included in other embodiments, rather than other features, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments may be used in any combination.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names.
Claims (7)
1. A graphic display method for compatible operation of Android applications on Linux, wherein the Android applications are compatible to operate in a Linux system in a container mode,
when the hardware type of the GPU in the Linux system can support the graphics library required by the Android application, the method comprises the following steps:
s1: judging whether the Android side of the container supports the GPU hardware type or not by a graph drawing control process, and starting S2 if the Android side of the container supports the GPU hardware type; if not, and the container Linux side supports the GPU hardware type, starting an S3 step;
s2: calling OpenGL ES by the Android side of the container, calling a GPU by the OpenGL ES according to a graph drawing request to generate first graph data, transmitting the first graph data to a base layer direct rendering DRI of the Linux side of the container by the graph drawing control process, and acquiring the first graph data and completing the synthetic drawing of graphs by the base layer direct rendering DRI to generate second graph data; the base layer direct rendering DRI sends the second graphics data to the current Linux system display hardware for display;
s3: calling an OpenGL ES by the Android side of the container, wherein the OpenGL ES transmits the graph drawing request to the graph drawing control process; the graphics drawing control process converts the graphics drawing request from an OpenGL ES interface format to an OpenGL interface format, and then OpenGL calls a GPU to generate third graphics data; the graphics drawing control process transmits the third graphics data to a base layer direct rendering DRI on the Linux side of the container, the base layer direct rendering DRI acquires the third graphics data and completes the synthesis drawing of graphics to generate fourth graphics data, and the base layer direct rendering DRI transmits the fourth graphics data to the display hardware of the current Linux system through a Linux kernel for displaying;
when the hardware type of the GPU of the Linux system cannot support the graphics library required by the Android application, the method comprises the following steps:
s4: judging whether the Android side of the container supports the GPU hardware type or not by a graph drawing control process, if so, executing a step S5;
s5: calling a graphics soft acceleration process by the Android side of the container, calling a CPU (central processing unit) by the graphics soft acceleration process according to the graphics drawing request to generate fifth graphics data, and storing the fifth graphics data into a virtual display screen of the Android side of the container through an Android hardware abstraction layer; the graphics rendering control process transmits the fifth graphics data to the base layer direct rendering DRI on the Linux side of the container; and the base layer direct rendering DRI sends the fifth graphic data to the display hardware of the current Linux system for display.
2. The method for displaying the graph compatibly running the Android application on Linux according to claim 1, comprising: the OpenGL ES invoking a GPU to generate the first graphics data comprises: and the OpenGL ES calls a FrameBufferNativeWindow interface, the FrameBufferNativeWindow interface delivers the graphics drawing request to the GPU, and the GPU generates the first graphics data according to the graphics drawing request and writes the first graphics data into a frame cache of the GPU.
3. The method for displaying the graph compatibly running the Android application on Linux according to claim 1, comprising: the invoking, by the OpenGL, the GPU to generate the first graphics data includes: the Open GL invokes a direct rendering manager DRM, which passes the graphics drawing request to the GPU, which generates the first graphics data.
4. The method for displaying the graph compatibly running the Android application on Linux according to claim 1, comprising: the graphics soft acceleration process simulates a graphics interface that implements OPEN GL and OPEN GL ES.
5. A graphics display device compatible with running Android applications on Linux, wherein the Android applications are compatible with running in a Linux system in a container mode,
when the hardware type of the GPU in the Linux system can support the graphics library required by the Android application, the method comprises the following steps:
the graphics drawing control module is used for acquiring the GPU hardware type, judging whether the Android supports the GPU hardware type, and starting a first drawing module if the Android supports the GPU hardware type; if not, starting a second drawing module if the container Linux side supports the GPU hardware type;
the first drawing module calls OpenGL ES from the Android side of the container, the OpenGL ES calls a GPU according to an obtained graph drawing request to generate first graph data, the graph drawing control module transmits the first graph data to a base layer direct rendering DRI of the Linux side of the container, and the base layer direct rendering DRI acquires the first graph data and completes the synthetic drawing of graphs to generate second graph data; the base layer direct rendering DRI sends the second graphic data to the current Linux system display hardware for display through a Linux kernel;
a second rendering module, in which OpenGL ES is called by the container Android side, the OpenGL ES transfers the graphics rendering request to the graphics rendering control module, the graphics rendering control module converts the graphics rendering request from an OpenGL ES interface format to an OpenGL interface format, OpenGL calls a GPU to generate the third graphics data, the graphics rendering control module transfers the third graphics data to the base layer direct rendering DRI of the container Linux side, the base layer direct rendering DRI acquires the third graphics data and completes the synthesis rendering of graphics to generate fourth graphics data, and the base layer direct rendering DRI sends the fourth graphics data to the current Linux system display hardware through a Linux kernel for display;
when the Linux system GPU hardware type cannot support the graphics library required by the Android application, the graphics drawing control module judges whether the container Android side supports the GPU hardware type, if so, the container Android side calls:
the graphics soft acceleration module calls a CPU (central processing unit) to generate fifth graphics data according to the graphics drawing request, and the fifth graphics data are stored in a virtual display screen on the Android side of the container from an Android hardware abstraction layer;
the graphics drawing control module acquires the fifth graphics data from the virtual display screen and transmits the fifth graphics data to a base layer direct rendering DRI on the Linux side of the container; and the base layer direct rendering DRI sends the fifth graphic data to the display hardware of the current Linux system for display.
6. A Linux-compatible Android application-running graphics display apparatus according to claim 5, wherein: the graphics rendering control module calls the translation module to convert the graphics rendering request from an OpenGL ES interface format to an OpenGL interface format.
7. A Linux-compatible Android application-running graphics display apparatus according to claim 5, wherein: the graphics soft acceleration module simulates a graphics interface for realizing OPEN GL and OPEN GL ES.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2020111147096 | 2020-10-16 | ||
| CN202011114709 | 2020-10-16 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112269603A CN112269603A (en) | 2021-01-26 |
| CN112269603B true CN112269603B (en) | 2021-05-28 |
Family
ID=74339935
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011257571.5A Active CN112269603B (en) | 2020-10-16 | 2020-11-11 | Graphic display method and device for compatibly running Android application on Linux |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112269603B (en) |
Families Citing this family (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113032025B (en) * | 2021-03-15 | 2022-10-28 | 厦门亿联网络技术股份有限公司 | Method, device, terminal and storage medium for rendering graph |
| CN112988357B (en) * | 2021-04-14 | 2021-07-27 | 湖北亿咖通科技有限公司 | Panoramic image display method, computer storage medium and electronic device |
| CN113064728B (en) * | 2021-04-16 | 2022-11-04 | 上海众链科技有限公司 | High-load application image display method, terminal and readable storage medium |
| CN113687888B (en) * | 2021-05-18 | 2023-08-08 | 麒麟软件有限公司 | File sharing method compatible with Android application running on Linux system |
| CN113515396B (en) * | 2021-07-09 | 2024-01-30 | 北京字节跳动网络技术有限公司 | Graphics rendering method, graphics rendering device, electronic equipment and storage medium |
| CN113867820B (en) * | 2021-09-29 | 2024-05-28 | 深圳市智微智能软件开发有限公司 | Method, device, equipment and storage medium for dynamically modifying frame buffer |
| CN114095781A (en) * | 2021-11-02 | 2022-02-25 | 北京鲸鲮信息系统技术有限公司 | Multimedia data processing method and device, electronic equipment and storage medium |
| CN114092312A (en) * | 2022-01-19 | 2022-02-25 | 北京鲸鲮信息系统技术有限公司 | Image generation method, image generation device, electronic equipment and storage medium |
| CN114125558B (en) * | 2022-01-26 | 2022-04-22 | 麒麟软件有限公司 | Linux desktop video card-based android video decoding hardware acceleration method |
| CN115660940B (en) * | 2022-11-11 | 2023-04-28 | 北京麟卓信息科技有限公司 | Graphic application frame rate synchronization method based on vertical blanking simulation |
| CN117234655B (en) * | 2023-11-13 | 2024-01-26 | 成都阿加犀智能科技有限公司 | Method, device, equipment and medium for running Linux desktop program based on android system |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106844066A (en) * | 2017-01-22 | 2017-06-13 | 腾讯科技（深圳）有限公司 | One kind application operation method, apparatus and system |
| CN107547474A (en) * | 2016-06-27 | 2018-01-05 | 中兴通讯股份有限公司 | A kind of graphic display method and equipment of heterogeneous dual system |
| CN108762815A (en) * | 2018-05-16 | 2018-11-06 | 北京麟卓信息科技有限公司 | A kind of Android running environment implementation methods based on non-virtualized architectural framework |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9009651B2 (en) * | 2011-03-30 | 2015-04-14 | Infosys Limited | System and method for management of a program throughout its lifecycle in an organization |
-
2020
- 2020-11-11 CN CN202011257571.5A patent/CN112269603B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107547474A (en) * | 2016-06-27 | 2018-01-05 | 中兴通讯股份有限公司 | A kind of graphic display method and equipment of heterogeneous dual system |
| CN106844066A (en) * | 2017-01-22 | 2017-06-13 | 腾讯科技（深圳）有限公司 | One kind application operation method, apparatus and system |
| CN108762815A (en) * | 2018-05-16 | 2018-11-06 | 北京麟卓信息科技有限公司 | A kind of Android running environment implementation methods based on non-virtualized architectural framework |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112269603A (en) | 2021-01-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112269603B (en) | Graphic display method and device for compatibly running Android application on Linux | |
| US8773443B2 (en) | Compression for co-processing techniques on heterogeneous graphics processing units | |
| US8042094B2 (en) | Architecture for rendering graphics on output devices | |
| US8537166B1 (en) | System and method for rendering and displaying high-resolution images | |
| WO2017167127A1 (en) | Image compositing method, window configuration method, and system | |
| CN102239483B (en) | Command remoting | |
| US5982399A (en) | Immediate mode drawing interface for the construction of graphics software | |
| US7995003B1 (en) | System and method for rendering and displaying high-resolution images | |
| CN113515396B (en) | Graphics rendering method, graphics rendering device, electronic equipment and storage medium | |
| TW200935309A (en) | Method and system for remote visualization of a graphics application | |
| CN114741044B (en) | Cross-operation environment display output sharing method based on heterogeneous rendering | |
| US20110169844A1 (en) | Content Protection Techniques on Heterogeneous Graphics Processing Units | |
| JPH0727571B2 (en) | Raster scan display device and graphic data transfer method | |
| CN114972607B (en) | Data transmission method, device and medium for accelerating image display | |
| CN112114916A (en) | Method and device for compatibly running Android application on Linux operating system | |
| CN113051047B (en) | Method and device for identifying android system drawing threads, mobile terminal and storage medium | |
| US8327387B2 (en) | Method for acquisition of GDI and directX data | |
| CN116821040B (en) | Display acceleration method, device and medium based on GPU direct memory access | |
| US11200019B2 (en) | Bypassing desktop composition | |
| CN114741081A (en) | Cross-operation environment display output sharing method based on heterogeneous cache access | |
| CN114968152A (en) | Method for reducing additional performance loss of VIRTIO-GPU | |
| US10733689B2 (en) | Data processing | |
| CN117170883B (en) | Method, device, equipment and storage medium for rendering display | |
| WO2023221822A1 (en) | Data processing method, electronic device, and readable storage medium | |
| CN115373618B (en) | Multi-screen display method and device, vehicle machine and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |