CN114092312A - Image generation method, image generation device, electronic equipment and storage medium - Google Patents
Image generation method, image generation device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN114092312A CN114092312A CN202210057895.7A CN202210057895A CN114092312A CN 114092312 A CN114092312 A CN 114092312A CN 202210057895 A CN202210057895 A CN 202210057895A CN 114092312 A CN114092312 A CN 114092312A
- Authority
- CN
- China
- Prior art keywords
- layer
- hardware
- renderer
- type
- rendering
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/20—Processor architectures; Processor configuration, e.g. pipelining
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/005—General purpose rendering architectures
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Graphics (AREA)
- Advance Control (AREA)
Abstract
The invention provides an image generation method, an image generation device, electronic equipment and a storage medium, wherein the method comprises the following steps: dividing the image layers into a first type image layer and a second type image layer according to the rendering requirements of each image layer in the target image to be generated; the first type of layer is a layer supporting the rendering of the processor unit, and the second type of layer is a layer supporting the rendering of the HWC; calling the processor unit through a hook function preset in a GLES thread to realize the rendering of the first type of layer; calling a hardware renderer through a hardware hybrid renderer client in a Linux operating system and a hardware hybrid renderer server in the hardware abstraction layer to realize the rendering of the second type layer; and synthesizing the rendered first type layer and the rendered second type layer through the HWC server to generate a target image. Thereby reducing problems such as image refresh rate drop due to insufficient performance of the processor unit.
Description
Technical Field
The present invention relates to the field of terminal technologies, and in particular, to an image generation method and apparatus, an electronic device, and a storage medium.
Background
When a user views a video through a video Application (APP) installed in a terminal device, a screen displayed by the terminal device is generally composed of a plurality of layers. The fluency of video playing is related to the rendering and compositing speed of the multiple layers.
In the prior art, when a Linux desktop system runs in an X86 environment, a Graphics Processing Unit (GPU) or a Central Processing Unit (CPU) is usually used to complete the composition of a layer, however, if the Linux operating system runs on some intelligent terminal devices that employ processors with high cost performance and low energy consumption, for example, the Linux operating system runs on a smart phone or a tablet computer that employs an arm (advanced RISC machines) processor, if the rendering and composition of the layer are completed only by using a processor unit, the problem of frame dropping due to insufficient performance of the processor unit is likely to occur.
Therefore, how to improve the performance of layer rendering and composition when the Linux operating system runs on some intelligent terminal devices adopting processors with high cost performance and low energy consumption, and avoid the phenomenon of frame dropping of image refresh rate, so that the video playing has better fluency is a problem to be solved urgently at present.
Disclosure of Invention
The invention provides an image generation method, an image generation device, electronic equipment and a storage medium, which are used for solving the defect that in the prior art, if a Linux operating system runs on some intelligent terminal equipment adopting a processor with high cost performance and low energy consumption, if only a processor unit is used for finishing the rendering and synthesis of a layer, the performance of the processor unit is insufficient, so that the frame dropping of an image refresh rate is caused, the problems that when the Linux operating system runs on an intelligent mobile phone or a tablet personal computer adopting an ARM (advanced RISC machine) framework, the frame dropping of the image refresh rate is caused due to the performance insufficiency of the processor unit and the like are reduced, and the video playing can also have better fluency.
The invention provides an image generation method, which is applied to a Linux operating system with an Android subsystem, wherein the Android subsystem comprises a hardware abstraction layer; the method comprises the following steps:
dividing the layers into a first layer type and a second layer type according to the rendering requirements of each layer in the target image to be generated; the first type of layer is a layer supporting rendering of a processor unit, and the second type of layer is a layer supporting rendering of a hardware mixing renderer;
calling the processor unit through a hook function preset in a GLES thread to realize the rendering of the first type of layer;
calling a hardware renderer through a hardware hybrid renderer client in a Linux operating system and a hardware hybrid renderer server in the hardware abstraction layer to realize the rendering of the second type layer;
and synthesizing the rendered first type layer and the rendered second type layer through the hardware mixing renderer server to generate the target image.
The invention provides an image generation method, which realizes the rendering of a first type of layer by calling a processor unit through a hook function preset in a GLES thread, and comprises the following steps:
transmitting the first type layer to a GLES thread in a Linux operating system;
calling a first interface in the hardware abstraction layer through a preset hook function in the GLES thread; and calling a processor unit through the first interface to realize the rendering of the first type of layer.
The invention provides an image generation method, where the first interface is a vendor GLES interface in the hardware abstraction layer, and correspondingly, the calling the first interface in the hardware abstraction layer through a preset hook function in a GLES thread includes:
and calling a vendor GLES interface in the hardware abstraction layer by a preset hook function in the GLES thread through the hybris running in the Linux operating system and the hybris component running in the hardware abstraction layer.
The invention provides an image generation method, which is used for realizing the rendering of a second type of layer by calling a hardware renderer through a hardware hybrid renderer client in a Linux operating system and a hardware hybrid renderer server in a hardware abstraction layer, and comprises the following steps:
transmitting the second type of graph layer to a hardware hybrid renderer client in a Linux operating system;
calling a hardware hybrid renderer server in the hardware abstraction layer by the hardware hybrid renderer client;
and calling the hardware renderer through the hardware mixing renderer server to realize the rendering of the second type layer.
According to the present invention, an image generation method is provided, where the invoking of a hardware hybrid renderer server in the hardware abstraction layer by the hardware hybrid renderer client includes:
the hardware hybrid renderer client calls a hardware hybrid renderer server in the hardware abstraction layer through the hybris running in the Linux operating system and the hybris component running in the hardware abstraction layer.
The invention provides an image generation method, wherein the step of generating a target image by synthesizing a rendered first type layer and a rendered second type layer through a hardware mixing renderer server comprises the following steps:
transmitting the rendered first type layer to a Linux operating system;
transmitting the rendered first type layer to a hardware hybrid renderer client in a Linux operating system, and transmitting the rendered first type layer to a hardware hybrid renderer server in a hardware abstraction layer by the hardware hybrid renderer client through a hybrid running in the Linux operating system and a hybrid running in the hardware abstraction layer;
and synthesizing the rendered first type layer and the rendered second type layer through the hardware mixing renderer server to generate the target image.
Before dividing layers into a first type of layer and a second type of layer according to rendering requirements of each layer in a target image to be generated, the method further includes:
initializing and starting an Android subsystem deployed in a Linux operating system;
in the process of starting the Android subsystem, starting a hardware hybrid renderer server in a hardware abstraction layer;
in the Linux operating system, a hardware hybrid renderer client is started.
The invention also provides an image generation device, which is applied to a Linux operating system with an Android subsystem, wherein the Android subsystem comprises a hardware abstraction layer; the device comprises:
the distinguishing module is used for dividing the layers into a first layer and a second layer according to the rendering requirements of each layer in the target image to be generated; the first type of layer is a layer supporting rendering of a processor unit, and the second type of layer is a layer supporting rendering of a hardware mixing renderer;
the first processing module is used for calling the processor unit through a hook function preset in a GLES thread to realize the rendering of the first type of layer;
the second processing module is used for calling a hardware renderer through a hardware hybrid renderer client in a Linux operating system and a hardware hybrid renderer server in the hardware abstraction layer to realize the rendering of the second type layer;
and the generating module is used for synthesizing the rendered first type layer and the rendered second type layer through the hardware mixing renderer server to generate the target image.
The present invention also provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the image generation method as described in any of the above when executing the program.
The invention also provides a non-transitory computer-readable storage medium having stored thereon a computer program which, when executed by a processor, carries out the steps of the image generation method as described in any of the above.
According to the image generation method, the image generation device, the electronic equipment and the storage medium, the target image is divided into the layer supporting rendering of the processor unit and the layer supporting rendering of the hardware mixing renderer, and corresponding hardware is called by corresponding methods to respectively complete rendering of the two layers aiming at the two layers, so that the hardware mixing renderer in the Android subsystem can assist the processor unit to complete rendering and synthesis of the layers, the problems of frame dropping of an image refresh rate and the like caused by insufficient performance of the processor unit are solved, and a Linux operating system can have better fluency during video playing when running on an ARM framework smart phone or a tablet computer.
Drawings
In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
FIG. 1 is one of the flow diagrams of an image generation method provided by the present invention;
FIG. 2 is a second schematic flowchart of an image generating method according to the present invention;
FIG. 3 is a schematic structural diagram of an image generation apparatus provided by the present invention;
fig. 4 is a schematic structural diagram of an electronic device provided in the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
For ease of understanding, the terms to which the present invention relates will be explained first.
(1) Android Hardware Abstraction Layer (HAL)
The hardware abstraction layer of the Android can be abbreviated as the Android HAL, and predefined interfaces are issued in the Android HAL in a manufacturer dynamic library form and are used for calling some device drivers to complete corresponding functions.
It can be understood that the device driver can be called through an interface in the Android HAL to enable the corresponding device to complete the corresponding function.
Illustratively, the interfaces predefined by the Android HAL include interfaces related to hardware hybrid renderer (HWC) servers, and calling the related interfaces of the hardware hybrid renderer servers may further call hardware hybrid renderer drivers, so that the hardware hybrid renderer devices complete corresponding functions.
The device driver is usually located in a kernel layer of the Android system, and is used for implementing various functions and call interfaces of the hardware, and how to use the functions of the hardware is determined by the call of the application program.
It will be appreciated that the interface through which the application calls the device driver is substantially a uniform standard interface, commonly referred to as a system call interface. In general, the Android HAL may be regarded as encapsulating functions implemented by calling device drivers into individual classes for java to call. In the invention, the encapsulated classes in the Android HAL can provide interfaces for related services of a Linux operating system.
(2) Container with a lid
A container is a process or set of processes that is isolated from the rest of the system. All files needed to run these processes are provided by another image.
And deploying the Android subsystem in the container, so that the Linux operating system accesses the HWC of the Android system through the container, and the Android subsystem and the GPU finish drawing of the layer together.
（3）libhybris
libhybris mainly functions to solve the libc library compatibility problem, and aims to run libraries compiled with bionic (mainly closed source HAL library under Android) on GNU C library-based systems.
It can be understood that libhybris is a set of software libraries for implementing the Android system dynamic library under the Linux operating system, and is mainly divided into two parts, namely, hybris running on the Linux end and hybris compot running on the Android end.
Illustratively, the libhybris includes hwc _ component.so library, and the hwc _ component.so library includes some functions for calling the interfaces in the Android HAL. The composer in the Linux operating system can access the relevant interfaces of the Android-side hardware hybrid renderer server side through the library.
The composition can be understood as a module corresponding to the Linux system and used for synthesizing the layer, and is a hardware hybrid renderer client in the invention, which corresponds to a hardware hybrid renderer server of the Android HAL layer.
（4）Binder
The Binder framework mainly relates to 4 roles which are respectively Client, Server, Service Manager and Binder driver, wherein the Client, Server and Service Manager run in a user space, and the Binder driver runs in a kernel space. The Client represents a Client process; the Server represents various services provided by the client process, such as audio and video; the Service Manager is used for managing various system services; the Binder driver provides the ability to communicate between processes.
(6) Vertical synchronization (VSync) signals
The refreshing process of the screen is that each row is from left to right (row refreshing, horizontal refreshing) and from top to bottom (screen refreshing, vertical refreshing). When the whole screen is refreshed, namely one vertical refreshing period is completed, a short blank period exists, and a VSync signal is sent out at the moment.
The Android system sends a VSync signal every 16ms to trigger rendering of a User Interface (UI), which can be simply regarded as a timer interrupt. And a VSync mechanism is introduced into the Android system and used for synchronous rendering, so that the AppUI and the SurfaceFlinger can work according to VSync rhythm generated by hardware.
It can be understood that the image generation method provided by the invention can be applied to a scene that the Linux desktop system runs on a smart phone or a tablet computer adopting an ARM architecture. As described above, the Linux desktop system operates on a smart phone or a tablet computer that uses an ARM processor, and if only a GPU or a CPU is used to complete the composition of layers, the problem of insufficient performance of a processor unit is likely to occur, which leads to a problem of frame dropping of an image refresh rate. The smart phone or the tablet computer generally uses an Android system, and the Android system can complete the composition of the layer by using a hardware mixing renderer and a processor unit together, so that the composition of the layer can be completed by using a GPU and the hardware mixing renderer in the Android system together. However, the Linux operating system cannot directly access the hardware hybrid renderer of the Android subsystem, and the Android system is an open source mobile device operating system based on a Linux kernel, so that the Android subsystem is deployed in the Linux container, the Linux operating system can conveniently access the hardware hybrid renderer of the Android subsystem through the Linux container, and the hardware hybrid renderer and the GPU can jointly complete rendering and synthesis of the layer.
The image generation method, apparatus, electronic device, and storage medium of the present invention are described below with reference to fig. 1 to 4.
Fig. 1 is a flowchart of an image generation method provided by the present invention. It can be understood that the image generation method is applied to a Linux operating system with an Android subsystem deployed, and the Android subsystem comprises a hardware abstraction layer. As shown in fig. 1, the image generation method provided by the present invention includes:
The first type of layer is a layer supporting rendering of a processor unit, and the second type of layer is a layer supporting rendering of a hardware mixing renderer.
In one embodiment, the processor unit may be a CPU or a GPU.
It can be understood that the target image to be generated includes a plurality of image layers, and each image layer corresponds to its own rendering requirement. For example, layers that support hardware hybrid renderer rendering may be set for some layers that do not require high frequency rendering, such as layers of a mouse, wallpaper, and the like.
It should be understood that, because the composition of the layers by using the hardware hybrid renderer has the characteristic of saving GPU resources compared with the composition of the layers by using the GPU, the layers which support the rendering by using the hardware hybrid renderer are synthesized by using the hardware hybrid renderer, and the drawing of the target image is completed by using the hardware hybrid renderer and the GPU together.
For ease of understanding, the example of cursor is used herein to illustrate that compositing layers with a hardware hybrid renderer saves GPU resources compared to compositing layers with a GPU. For example, using the vendor GLES rendering first requires the entire cursor layer to be redrawn and rendered, and then using the GPU to fit the cursor on the screen, which consumes a lot of GPU resources. The hardware hybrid renderer only needs to tell the position of the cursor coordinate of the hardware, the hardware hybrid renderer can automatically complete the updating and fitting process of the cursor coordinate of the cursor layer before the image is displayed on a screen, and the process is asynchronous, namely the GPU waiting time is hardly occupied, GPU resources are not consumed, and therefore the problems that the image refreshing rate is dropped due to the fact that the performance of a processor unit is insufficient and the like can be solved.
And step 120, calling the processor unit through a hook function preset in a GLES thread to realize the rendering of the first type of layer.
The hook function can be used to interrupt the executing function and execute the function of the preset target function. In this embodiment, the hook function may be configured to call a corresponding function to call a processor unit to implement rendering on the first type layer.
It can be understood that the GLES thread in the Linux operating system cannot invoke hardware to perform rendering of the layer by itself. Therefore, a hook function is set in the thread, and the hook function calls the processor unit to realize the rendering of the first type of layer.
Step 130, calling a hardware renderer through a hardware hybrid renderer client in the Linux operating system and a hardware hybrid renderer server in the hardware abstraction layer to realize rendering of the second type layer.
The hardware hybrid renderer client can receive a request for rendering and synthesizing the second type of layers from the Linux operating system, and can call the hardware hybrid renderer server according to the request. The hardware hybrid renderer server can be understood as a service defined in a hardware abstraction layer in the Android subsystem and used for controlling the hardware hybrid renderer, an interface related to the hardware hybrid renderer is provided, and a hardware hybrid renderer driver can be called, so that the hardware hybrid renderer can complete rendering and synthesis of the second type of layers.
It can be understood that the hardware hybrid renderer is a piece of hardware corresponding to the Android subsystem, and can be called by a hardware hybrid renderer server in a hardware abstraction layer in the Android subsystem. Therefore, the hardware mixing renderer server can be called through the hardware mixing renderer client, and the hardware mixing renderer server calls the hardware renderer to complete the rendering of the second type of layers.
Step 140, synthesizing the rendered first type layer and the rendered second type layer by the hardware mixing renderer server to generate the target image.
According to the image generation method provided by the invention, the target image is divided into the layer supporting the rendering of the processor unit and the layer supporting the rendering of the hardware mixing renderer, and the rendering of the two layers is respectively completed by calling corresponding hardware by adopting corresponding methods aiming at the two layers, so that the hardware mixing renderer in the Android subsystem can assist the processor unit to complete the rendering and synthesis of the layers, the problems of frame dropping of an image refresh rate and the like caused by insufficient performance of the processor unit are solved, and when a Linux operating system runs on an intelligent mobile phone or a tablet computer adopting an ARM (advanced RISC machine) framework, the video playing can also have better fluency.
Based on any one of the above embodiments, in this embodiment, the invoking a processor unit through a hook function preset in a GLES thread to implement rendering of the first type layer includes:
transmitting the first type layer to a GLES thread in a Linux operating system;
calling a first interface in the hardware abstraction layer through a preset hook function in the GLES thread;
and calling a processor unit through the first interface to realize the rendering of the first type of layer.
As mentioned above, since the GLES thread in the Linux operating system cannot call hardware to render layers. Therefore, a hook function is set in the thread, and then the hook function calls the processor unit to render the first type of layer through calling the first interface. The hook function is used for calling a first interface in the hardware abstraction layer. The first interface is a rendering interface for implementing an image layer supporting rendering by the processor unit, and may call the processor unit, for example, may be a vendor GLES interface.
According to the image generation method provided by the invention, the hook function is preset in the GLES thread, the rendering interface of the layer which supports the rendering of the processor unit in the hardware abstraction layer is realized through the hook function, and the rendering of the layer which supports the rendering of the processor unit is completed, so that the layers are rendered by using the processor unit in the Linux operating system and the Android subsystem.
Based on any one of the foregoing embodiments, in this embodiment, the first interface is a vendor GLES interface in the hardware abstraction layer, and correspondingly, the invoking the first interface in the hardware abstraction layer through a preset hook function in the GLES thread includes:
and calling a vendor GLES interface in the hardware abstraction layer by a preset hook function in the GLES thread through the hybris running in the Linux operating system and the hybris component running in the hardware abstraction layer.
The hybris and the hybris complex are two components of libhybris respectively and are used for calling some interfaces in the Android hardware abstraction layer under the Linux operating system. In this embodiment, the vendor GLES interface in the hardware abstraction layer may be called through the hybris and hybris complex.
It can be understood that since some interfaces in the Android hardware abstraction layer are functions compiled based on the bionic, library compatibility needs to be realized through libhybris to call the interfaces in the Android hardware abstraction layer on the Linux system, so as to call the interfaces in the Android hardware abstraction layer on the Linux system.
According to the image generation method provided by the invention, the hook function is preset in the GLES thread, and the vendor GLES interface in the hardware abstraction layer in the Android subsystem is called through the hybris and the hybris component, so that the rendering of the layer which supports the rendering of the processor unit can be completed by the Linux operating system and the Android subsystem together, and the problem of compatibility of the Linux operating system and the Android subsystem in the rendering of the layer which uses the processor unit is solved.
Based on any one of the above embodiments, in this embodiment, the invoking a hardware renderer by a hardware hybrid renderer client in the Linux operating system and a hardware hybrid renderer server in the hardware abstraction layer to realize rendering of the second type layer includes:
transmitting the second type of graph layer to a hardware hybrid renderer client in a Linux operating system;
calling a hardware hybrid renderer server in the hardware abstraction layer by the hardware hybrid renderer client;
and calling the hardware renderer through the hardware mixing renderer server to realize the rendering of the second type layer.
As mentioned above, the hardware hybrid renderer client may receive a request for rendering and composition of the second type layer from the Linux operating system, and the hardware hybrid renderer server may understand that one service defined in the hardware abstraction layer in the Android subsystem is used for controlling the hardware hybrid renderer, provides an interface related to the hardware hybrid renderer, and may call a hardware hybrid renderer driver, so that the hardware hybrid renderer completes rendering and composition of the second type layer. Therefore, the second type of layer can be transmitted to a hardware hybrid renderer client in the Linux operating system, the hardware hybrid renderer server is called through the hardware hybrid renderer client, and the hardware hybrid renderer is called by the hardware hybrid renderer server to achieve rendering of the second type of layer.
According to the image generation method provided by the invention, the hardware mixing renderer server in the hardware abstraction layer in the Android subsystem is called through the hardware mixing renderer client, so that the hardware renderer is called, and the rendering of the second type of layer is realized, therefore, the hardware renderer in the Android subsystem can be used by a Linux operating system to complete the rendering of the layer supporting the rendering of the hardware renderer, the hardware mixing renderer in the Android subsystem can assist the processor unit to complete the rendering and synthesis of the layer, the problems of frame dropping of an image refresh rate and the like caused by the insufficient performance of the processor unit are solved, and when the Linux operating system runs on an ARM (advanced RISC machines) framework smart phone or a tablet personal computer, the video playing can have better fluency.
Based on any one of the foregoing embodiments, in this embodiment, the invoking, by the hardware hybrid renderer client, the hardware hybrid renderer server in the hardware abstraction layer includes:
the hardware hybrid renderer client calls a hardware hybrid renderer server in the hardware abstraction layer through the hybris running in the Linux operating system and the hybris component running in the hardware abstraction layer.
It can be understood that, since the hardware hybrid renderer server may be understood as a service interface defined in a hardware abstraction layer in the Android subsystem for controlling the hardware hybrid renderer, and the service interface is compiled based on bionic, library compatibility may be achieved through libhybris when called in the Linux operating system, that is, hybris and hybris component enable the hardware hybrid renderer server in the hardware abstraction layer in the Android system to be called under the Linux system, so as to implement rendering of the second type layer.
According to the image generation method provided by the invention, the hardware mixing renderer server in the hardware abstraction layer in the Android subsystem is called through the hybris and the hybris complex, so that the hardware renderer is called, and the rendering of the second type of layer is realized, therefore, the hardware renderer in the Android subsystem can be used by a Linux operating system to complete the rendering of the layer supporting the rendering of the hardware renderer, the hardware mixing renderer in the Android subsystem can assist a processor unit to complete the rendering and synthesis of the layer, the problems of frame dropping of an image refresh rate and the like caused by insufficient performance of the processor unit are solved, and when the Linux operating system runs on an ARM (advanced RISC machines) framework smart phone or a tablet personal computer, the video playing can have better fluency.
Based on any one of the foregoing embodiments, in this embodiment, the synthesizing, by the hardware hybrid renderer server, the rendered first type layer and the rendered second type layer to generate the target image includes:
transmitting the rendered first type layer to a Linux operating system;
transmitting the rendered first type layer to a hardware hybrid renderer client in a Linux operating system, and transmitting the rendered first type layer to a hardware hybrid renderer server in a hardware abstraction layer by the hardware hybrid renderer client through a hybrid running in the Linux operating system and a hybrid running in the hardware abstraction layer;
and synthesizing the rendered first type layer and the rendered second type layer through the hardware mixing renderer server to generate the target image.
It can be understood that the rendered first type of layer is obtained by rendering one or more layers through a vendor GLES interface respectively. Therefore, the first type of image layer rendered through the vector GLES interface needs to be sent to the hardware mixing renderer server, so that the hardware mixing renderer server generates a target image after combining the first type of image layer with the second type of image layer.
Specifically, one or more rendered first type layers may be placed in a cache block, and marked as a rendered layer supporting the rendering of the processor unit, a hardware hybrid renderer client sets the cache block to a hardware hybrid renderer server by calling a relevant interface in libhybris, and the hardware hybrid renderer server calls a hardware hybrid renderer based on a call instruction of the hardware hybrid renderer client, thereby completing the rendering and composition of the layer supporting the rendering of the hardware hybrid renderer and the layer supporting the rendering of the processor unit.
According to the image generation method provided by the invention, the first type of layer and the second type of layer are synthesized by the hardware mixing renderer server, so that the hardware mixing renderer and the processor unit in the Android subsystem complete rendering and synthesis of the layers together, the problems of frame dropping of an image refresh rate and the like caused by insufficient performance of the processor unit are solved, and when a Linux operating system runs on an ARM (advanced RISC machines) architecture smart phone or tablet personal computer, a video can have better fluency during playing.
Based on any of the foregoing embodiments, in this embodiment, before dividing, according to the rendering requirement of each layer in the target image to be generated, the layer into a first type layer and a second type layer, the method further includes:
initializing and starting an Android subsystem deployed in a Linux operating system;
in the process of starting the Android subsystem, starting a hardware hybrid renderer server in a hardware abstraction layer;
in the Linux operating system, a hardware hybrid renderer client is started.
It can be understood that the Android subsystem can be deployed in the Linux operating system in a container mode, and the Android subsystem is initialized and started in the container. And in the process of starting the Android subsystem, the hardware mixing renderer driver is automatically loaded, so that the hardware mixing renderer is conveniently accessed through the hardware mixing renderer driver subsequently. And because the corresponding hardware hybrid renderer client in the Android subsystem is a surfaceflag, but the rendering and synthesizing module of the corresponding control layer in the Linux operating system is a composition, the hardware hybrid renderer client is modified into the composition in the Linux operating system.
According to the image generation method provided by the invention, the Android subsystem, the hardware mixing renderer server and the hardware mixing renderer client are started, and the hardware mixing renderer synthesizes the first type of layer and the second type of layer through the Android subsystem, so that the hardware mixing renderer and the processor unit in the Android subsystem jointly complete rendering and synthesis of the layers, and therefore the problems of frame dropping of an image refresh rate and the like caused by insufficient performance of the processor unit are solved, and a Linux operating system can have better fluency during video playing when running on an ARM (advanced RISC machine) framework smart phone or a tablet personal computer.
Fig. 2 is a second schematic flow chart of the image generating method provided by the present invention. Fig. 2 describes the image generation method provided by the present invention in an interactive process between the Linux synthesizer module composition and the Android HAL. Wherein, the composer can be understood as a module for controlling rendering and composition of layers in a Linux system; a window manager (window manager) is responsible for generating and classifying layers in the window; the hardware hybrid renderer client (hwc client) corresponds to the composer; the hybris and the hybris complex are two components of libhybris respectively, and form an interaction channel formed by a composition and an Android HAL, so that the composition and the Android HAL can complete layer synthesis together through interaction; the hardware mixing renderer client is used for calling the data in each layer to a relevant interface in a hardware mixing renderer server (hwc server) through libhybris to complete the rendering and synthesis of each layer; a hook function is preset in the GLES thread and used for calling a vendor GLES in the Android HAL to realize rendering of an image layer supporting rendering (GLES layer) of the processor unit, and a vendor GLES interface is used for calling the processor unit to complete rendering of the image layer synthesized by the processor unit and storing the rendered image layer in a target cache region. And the hardware hybrid renderer server calls the hardware hybrid renderer to complete corresponding functions based on a calling instruction of the hardware hybrid renderer client. The image generation method is described below with reference to fig. 2.
First, a window manager in a composition receives content included in a target image, which may include: an interface corresponding to an application (application), a cursor (cursor) corresponding to a mouse, a status bar (status bar), and a background (background).
And secondly, the window manager distinguishes the layers corresponding to the contents included in the target image respectively to obtain the layers of different types.
Wherein, the different types of layers may include, for example: the layers (gles layer) supporting the rendering of the processor unit, the layers (hwc client layer) supporting the rendering of the processor unit which have already been rendered, and the layers (hwc device layer, hwc color layer and hwc current layer) supporting the rendering of the hardware blending renderer.
Before the process, the composer may call, according to the content included in the target image, a relevant interface of the hardware hybrid renderer server through libhybris to create at least one layer and set information of the at least one layer, and cause, through libhybris, the hardware hybrid renderer to determine, according to the information of the layer, a layer that supports rendering by the processor unit and a layer that supports rendering by the hardware hybrid renderer in the at least one layer, and return the information of the layer to the window manager.
Specifically, a process of creating at least one layer and setting information of the at least one layer will be described in detail below.
The method comprises the steps that a relevant interface of a hardware hybrid renderer server side is called through an interface of libhybris, firstly, a channel between a Linux system and the hardware hybrid renderer needs to be established through the libhybris, and the Linux system can access the hardware hybrid renderer of the Android system.
It is understood that hwc _ component.so, hwc _ component.so, contained in the libhybris framework, are compiled using bionic, run in an Android environment, and contain some functions for calling an Android interface inside, so that the linux system can access the hardware hybrid renderer of the Android system.
Specifically, when the correlation interface of the hardware hybrid renderer server is used, the correlation interface is exported to a Linux header file, the header file is referred to in a compositor of Linux, and when a method of the correlation interface is called, an hwc _ composition.so library can be loaded in an android _ dlopen manner, so that the correlation interface between the Linux (glibc environment) and the hardware hybrid renderer server is called. Calling a second interface in related interfaces of the hardware hybrid renderer server through a first interface of libhybris to create at least one layer according to content contained in a target image; and calling one or more third interfaces of the libhybris to set information of the at least one layer, wherein the information comprises the rendering type of the layer.
The first interface and the third interface of the libhybris may be, for example, one of the aforementioned hwc _ child. The first interface of the libhybris is used for calling a related interface of a hardware hybrid renderer service end, and the second interface of the hardware hybrid renderer service is used for creating at least one layer with a preset rendering type. And the third interface of the libhybris is used for setting the information of the at least one image layer. The layer of the preset rendering type may be: layers that support hardware mix renderer rendering and layers that support processor unit rendering.
It can be understood that, the relevant interface of the hardware hybrid renderer server is called through the interface of libhybris, and the aforementioned establishing of the channel between the Linux system and the hardware hybrid renderer through libhybris is performed first, which is similar to the aforementioned method for calling the relevant interface of the android (bionic environment) through implementing Linux (glibc environment), and is not described here any more.
Specifically, the first interface of the libhybris may be, for example, one of the interfaces hwc _ composition.so, for example, an interface hwc2_ composition _ display _ create _ layer, and the second interface of the hardware hybrid renderer service may be, for example, hwc2_ display _ create _ layer.
The information of the layer includes a rendering type of the layer, which corresponds to the different types of layers described above, and may be: the hardware mix renderer rendering support layer, the processor unit rendering support layer that has already been rendered, and the processor unit rendering support layer.
Specifically, if the rendering type of the at least one layer is set by calling the third interface of the libhybris, the third interface of the libhybris may be, for example, an hwc2_ composition _ layer _ set _ composition _ type interface.
In addition, the information of the layer may further include a display area parameter of the layer.
Specifically, if a third interface of the libhybris is called to set a display area parameter corresponding to each of the at least one layer, the third interface of the libhybris may be, for example, one of the interfaces hwc _ component, such as hwc2_ component _ layer _ set _ source _ crop interface, hwc2_ component _ layer _ set \ u
display _ frame interface or hwc2_ composition _ layer _ set _ visible _ region interface.
It can be understood that, before layer rendering and compositing, a control signal (VSync signal) for layer compositing of the hardware hybrid renderer needs to be accessed to a compositor module of Linux according to a preset communication mode for controlling rendering and compositing of the layers.
The preset communication method may be Binder communication, for example. Binder communication is used for interprocess communication. The compositor module of Linux corresponds to a client process. It can be understood that since the Android subsystem runs in the container, the synthesizer module of the Linux system and the hardware hybrid renderer service of the Android subsystem are in different processes, and therefore, the two need to communicate through the Binder.
As mentioned above, the control signal for layer composition is sent by the hardware mixing renderer to control the rendering and composition of the layers. Therefore, when the control signal for layer composition comes, the rendering and composition process of the layers is executed. The rendering and compositing processes of the layers are described in detail below.
As described above, the layer is set with a corresponding rendering type when being created, and thus the rendering type of the layer needs to be determined in the rendering process, and the layer can be allocated to different synthesis hardware for synthesis according to the set rendering type.
Specifically, determining that the layer supporting the rendering by the processor unit and the layer supporting the rendering by the hardware hybrid renderer in the at least one layer may be: and calling a fourth interface of the libhybris to enable the hardware hybrid renderer to determine a layer which supports rendering of the processor unit and a layer which supports rendering of the hardware hybrid renderer in the at least one layer.
It should be noted that, the fourth interface of libhybris may be one interface in hwc _ component. The fourth interface may be, for example, hwc2_ composition _ display _ valid.
Then, the composer calls a relevant interface in the service end of the hardware mixing renderer through hwc _ composition.so that the hardware mixing renderer determines a layer supporting processor unit rendering and a layer supporting hardware mixing renderer rendering in the at least one layer according to the rendering type of the layer, different layers are respectively sent to a GELS thread and a hardware mixing renderer client according to the rendering type of the layer, and the GELS thread and the hardware mixing renderer client are respectively forwarded to a vendor GELS and the hardware mixing renderer service end.
Specifically, the process corresponding to fig. 2 is: and respectively sending the layers of different types to a hardware mixing renderer client and a GLES thread, and respectively calling a hardware mixing renderer server and a vendor GLES by the hardware mixing renderer client and the GLES thread through a hybris and a hybris component. And finishing the rendering of the layer which supports the rendering of the processor unit by the realization of the vendor GLES, storing the layer which supports the rendering of the processor unit in a cache block, returning the layer to the window manager, and using the layer which supports the rendering of the processor unit as the rendered layer which supports the rendering of the processor unit by the window manager. And calling the hardware mixing renderer by the hardware mixing renderer server based on a calling instruction of the hardware mixing renderer client to complete the rendering and synthesis of the layer supporting the hardware mixing renderer to the rendered layer supporting the processor unit. The specific process can refer to the related description below.
Specifically, the cache block may be set to the hardware mix renderer by calling a fifth interface of libhybris; and calling a sixth interface of libhybris to enable the hardware mixing renderer to complete the composition of the rendered layer supporting the rendering of the processor unit and the rendered layer supporting the hardware mixing renderer, and obtaining the target image.
Wherein, the fifth interface and the sixth interface of libhybris are respectively one interface of hwc _ compot. The fifth interface may be, for example, hwc2_ composition _ display _ set _ client _ target, and the sixth interface may be, for example, hwc2_ composition _ display _ present.
And finally, sending the target image to a display screen for display.
The image generating apparatus provided by the present invention is described below, and the image generating apparatus based on the following description and the image generating method described above may be referred to in correspondence with each other.
Fig. 3 is a schematic diagram of an image generating apparatus provided by the present invention, and as shown in fig. 3, the image generating apparatus provided by the embodiment of the present invention is applied to a Linux operating system deployed with an Android subsystem, where the Android subsystem includes a hardware abstraction layer; the device comprises:
the distinguishing module 310 is configured to divide layers into a first type layer and a second type layer according to rendering requirements of each layer in a target image to be generated; the first type of layer is a layer supporting rendering of a processor unit, and the second type of layer is a layer supporting rendering of a hardware mixing renderer;
a first processing module 320, configured to call the processor unit through a hook function preset in a GLES thread, so as to implement rendering on the first type layer;
the second processing module 330 is configured to invoke a hardware renderer through a hardware hybrid renderer client in the Linux operating system and a hardware hybrid renderer server in the hardware abstraction layer, so as to implement rendering on the second type layer;
the generating module 340 is configured to generate the target image by synthesizing the rendered first type layer and the rendered second type layer through the hardware hybrid renderer server.
According to the image generation device provided by the invention, the target image is divided into the layer supporting the rendering of the processor unit and the layer supporting the rendering of the hardware mixing renderer, and the rendering of the two layers is respectively completed by calling corresponding hardware by adopting a corresponding method aiming at the two layers, so that the hardware mixing renderer in the Android subsystem can assist the processor unit to complete the rendering and synthesis of the layers, the problems of frame dropping of an image refresh rate and the like caused by insufficient performance of the processor unit are solved, and when a Linux operating system runs on an intelligent mobile phone or a tablet computer adopting an ARM (advanced RISC machines) framework, the video playing can also have better fluency.
Based on any one of the above embodiments, in this embodiment, the first processing module 320 includes:
the first transmission unit is used for transmitting the first type layer to a GLES thread in a Linux operating system;
the first calling unit is used for calling a first interface in the hardware abstraction layer through a preset hook function in the GLES thread;
and the first rendering unit is used for calling the processor unit through the first interface to realize the rendering of the first type of layer.
According to the image generation device provided by the invention, the hook function is preset in the GLES thread, the rendering interface of the layer which supports the rendering of the processor unit in the hardware abstraction layer is realized through the hook function, and the rendering of the layer which supports the rendering of the processor unit is completed, so that the layers are rendered by using the processor unit in the Linux operating system and the Android subsystem.
Based on any one of the foregoing embodiments, in this embodiment, the first invoking unit includes:
and the second calling unit is used for calling the vendor GLES interface in the hardware abstraction layer by the preset hook function in the GLES thread through the hybris running in the Linux operating system and the hybris component running in the hardware abstraction layer.
According to the image generation device provided by the invention, the hook function is preset in the GLES thread, and the vendor GLES interface in the hardware abstraction layer in the Android subsystem is called through the hybris and the hybris complex, so that the rendering of the layer which supports the rendering of the processor unit can be completed by the Linux operating system and the Android subsystem together, and the problem of compatibility of the Linux operating system and the Android subsystem in the rendering of the layer which uses the processor unit is solved.
Based on any one of the above embodiments, in this embodiment, the second processing module 330 includes:
the second transmission unit is used for transmitting the second type of graph layer to a hardware hybrid renderer client in a Linux operating system;
a third calling unit, configured to call, by the hardware hybrid renderer client, a hardware hybrid renderer server in the hardware abstraction layer;
and the second rendering unit is used for calling a hardware renderer through the hardware hybrid renderer server to realize the rendering of the second type layer.
According to the image generation device provided by the invention, the hardware mixing renderer server in the hardware abstraction layer in the Android subsystem is called through the hardware mixing renderer client, so that the hardware renderer is called, and the rendering of the second type of layer is realized, so that the hardware renderer in the Android subsystem can be used by a Linux operating system to complete the rendering of the layer supporting the rendering of the hardware renderer, the hardware mixing renderer in the Android subsystem can assist the processor unit to complete the rendering and synthesis of the layer, the problems of frame dropping of an image refresh rate and the like caused by the insufficient performance of the processor unit are solved, and when the Linux operating system runs on an ARM (advanced RISC machines) framework smart phone or a tablet personal computer, the video playing can have better fluency.
Based on any one of the foregoing embodiments, in this embodiment, the third invoking unit includes:
and the fourth calling unit is used for calling the hardware hybrid renderer server in the hardware abstraction layer by the hardware hybrid renderer client running in the Linux operating system and the hardware component running in the hardware abstraction layer.
According to the image generation device provided by the invention, the hardware mixing renderer server side in the hardware abstraction layer in the Android subsystem is called through the hybris and the hybris complex, so that the hardware renderer is called, and the rendering of the second type of layer is realized, therefore, the hardware renderer in the Android subsystem can be used for finishing the rendering of the layer supporting the rendering of the hardware renderer in the Linux operating system, the hardware mixing renderer in the Android subsystem can assist the processor unit to finish the rendering and synthesis of the layer, the problems of frame dropping of an image refresh rate and the like caused by insufficient performance of the processor unit are solved, and when the Linux operating system runs on an ARM (advanced RISC machines) framework smart phone or a tablet personal computer, the video playing can have better fluency.
Based on any one of the above embodiments, in this embodiment, the generating module 340 includes:
the third transmission unit is used for transmitting the rendered first layer to a Linux operating system;
a fourth transmission unit, configured to transmit the rendered first type layer to a hardware hybrid renderer client in a Linux operating system, and transmit the rendered first type layer to a hardware hybrid renderer server in a hardware abstraction layer by the hardware hybrid renderer client via a hybrid running in the Linux operating system and a hybrid running in the hardware abstraction layer;
and the synthesizing unit is used for synthesizing the rendered first type layer and the rendered second type layer through the hardware mixing renderer server to generate the target image.
According to the image generation device provided by the invention, the first type of layer and the second type of layer are synthesized by the hardware mixing renderer server, so that the hardware mixing renderer and the processor unit in the Android subsystem complete rendering and synthesis of the layers together, the problems of frame dropping of an image refresh rate and the like caused by insufficient performance of the processor unit are solved, and when a Linux operating system runs on an ARM (advanced RISC machines) architecture smart phone or tablet personal computer, a video can be played with better fluency.
Based on any one of the above embodiments, in this embodiment, the apparatus further includes:
the Android subsystem starting module is used for initializing and starting an Android subsystem deployed in a Linux operating system;
the first starting module is used for starting a hardware hybrid renderer server side in a hardware abstraction layer in the process of starting the Android subsystem;
and the second starting module is used for starting the hardware hybrid renderer client in the Linux operating system.
According to the image generation device provided by the invention, the Android subsystem, the hardware mixing renderer server and the hardware mixing renderer client are started, and the hardware mixing renderer synthesizes the first type of layer and the second type of layer through the Android subsystem, so that the hardware mixing renderer and the processor unit in the Android subsystem jointly complete rendering and synthesis of the layers, and therefore the problems of frame dropping of an image refresh rate and the like caused by insufficient performance of the processor unit are solved, and a Linux operating system can have better fluency during video playing when running on an ARM (advanced RISC machine) framework smart phone or a tablet personal computer.
Fig. 4 illustrates a physical structure diagram of an electronic device, which may include, as shown in fig. 4: a processor (processor)410, a communication Interface 420, a memory (memory)430 and a communication bus 440, wherein the processor 410, the communication Interface 420 and the memory 430 are communicated with each other via the communication bus 440. Processor 410 may invoke logic instructions in memory 430 to perform an image generation method comprising: dividing the layers into a first layer type and a second layer type according to the rendering requirements of each layer in the target image to be generated; the first type of layer is a layer supporting rendering of a processor unit, and the second type of layer is a layer supporting rendering of a hardware mixing renderer; calling the processor unit through a hook function preset in a GLES thread to realize the rendering of the first type of layer; calling a hardware renderer through a hardware hybrid renderer client in a Linux operating system and a hardware hybrid renderer server in the hardware abstraction layer to realize the rendering of the second type layer; and synthesizing the rendered first type layer and the rendered second type layer through the hardware mixing renderer server to generate the target image.
In addition, the logic instructions in the memory 430 may be implemented in the form of software functional units and stored in a computer readable storage medium when the software functional units are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions which, when executed by a computer, enable the computer to perform the image generation method provided by the present invention, the method comprising: dividing the layers into a first layer type and a second layer type according to the rendering requirements of each layer in the target image to be generated; the first type of layer is a layer supporting rendering of a processor unit, and the second type of layer is a layer supporting rendering of a hardware mixing renderer; calling the processor unit through a hook function preset in a GLES thread to realize the rendering of the first type of layer; calling a hardware renderer through a hardware hybrid renderer client in a Linux operating system and a hardware hybrid renderer server in the hardware abstraction layer to realize the rendering of the second type layer; and synthesizing the rendered first type layer and the rendered second type layer through the hardware mixing renderer server to generate the target image.
In yet another aspect, the present invention also provides a non-transitory computer-readable storage medium having stored thereon a computer program which, when executed by a processor, is implemented to perform the image generation method provided by the present invention, the method comprising: dividing the layers into a first layer type and a second layer type according to the rendering requirements of each layer in the target image to be generated; the first type of layer is a layer supporting rendering of a processor unit, and the second type of layer is a layer supporting rendering of a hardware mixing renderer; calling the processor unit through a hook function preset in a GLES thread to realize the rendering of the first type of layer; calling a hardware renderer through a hardware hybrid renderer client in a Linux operating system and a hardware hybrid renderer server in the hardware abstraction layer to realize the rendering of the second type layer; and synthesizing the rendered first type layer and the rendered second type layer through the hardware mixing renderer server to generate the target image.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. The image generation method is characterized by being applied to a Linux operating system with an Android subsystem deployed, wherein the Android subsystem comprises a hardware abstraction layer; the method comprises the following steps:
dividing the layers into a first layer type and a second layer type according to the rendering requirements of each layer in the target image to be generated; the first type of layer is a layer supporting rendering of a processor unit, and the second type of layer is a layer supporting rendering of a hardware mixing renderer;
calling the processor unit through a hook function preset in a GLES thread to realize the rendering of the first type of layer;
calling a hardware renderer through a hardware hybrid renderer client in a Linux operating system and a hardware hybrid renderer server in the hardware abstraction layer to realize the rendering of the second type layer;
and synthesizing the rendered first type layer and the rendered second type layer through the hardware mixing renderer server to generate the target image.
2. The image generation method according to claim 1, wherein the rendering of the first type of layer is implemented by calling a processor unit through a hook function preset in a GLES thread, and includes:
transmitting the first type layer to a GLES thread in a Linux operating system;
calling a first interface in the hardware abstraction layer through a preset hook function in the GLES thread;
and calling a processor unit through the first interface to realize the rendering of the first type of layer.
3. The image generation method of claim 2, wherein the first interface is a vendor GLES interface in the hardware abstraction layer, and accordingly, the calling the first interface in the hardware abstraction layer through a preset hook function in the GLES thread includes:
and calling a vendor GLES interface in the hardware abstraction layer by a preset hook function in the GLES thread through the hybris running in the Linux operating system and the hybris component running in the hardware abstraction layer.
4. The image generation method according to claim 1, wherein the rendering of the second type layer is implemented by calling a hardware renderer through a hardware hybrid renderer client in a Linux operating system and a hardware hybrid renderer server in the hardware abstraction layer, and includes:
transmitting the second type of graph layer to a hardware hybrid renderer client in a Linux operating system;
calling a hardware hybrid renderer server in the hardware abstraction layer by the hardware hybrid renderer client;
and calling the hardware renderer through the hardware mixing renderer server to realize the rendering of the second type layer.
5. The image generation method of claim 4, wherein the invoking, by the hardware hybrid renderer client, a hardware hybrid renderer server in the hardware abstraction layer comprises:
the hardware hybrid renderer client calls a hardware hybrid renderer server in the hardware abstraction layer through the hybris running in the Linux operating system and the hybris component running in the hardware abstraction layer.
6. The image generation method according to claim 1, wherein the synthesizing, by the hardware hybrid renderer server, the rendered first type layer and the rendered second type layer to generate the target image includes:
transmitting the rendered first type layer to a Linux operating system;
transmitting the rendered first type layer to a hardware hybrid renderer client in a Linux operating system, and transmitting the rendered first type layer to a hardware hybrid renderer server in a hardware abstraction layer by the hardware hybrid renderer client through a hybrid running in the Linux operating system and a hybrid running in the hardware abstraction layer;
and synthesizing the rendered first type layer and the rendered second type layer through the hardware mixing renderer server to generate the target image.
7. The image generation method according to any one of claims 1 to 6, wherein before the dividing the layers into the first type of layer and the second type of layer according to the rendering requirement of each layer in the target image to be generated, the method further includes:
initializing and starting an Android subsystem deployed in a Linux operating system;
in the process of starting the Android subsystem, starting a hardware hybrid renderer server in a hardware abstraction layer;
in the Linux operating system, a hardware hybrid renderer client is started.
8. An image generation method device is characterized by being applied to a Linux operating system with an Android subsystem, wherein the Android subsystem comprises a hardware abstraction layer; the device comprises:
the distinguishing module is used for dividing the layers into a first layer and a second layer according to the rendering requirements of each layer in the target image to be generated; the first type of layer is a layer supporting rendering of a processor unit, and the second type of layer is a layer supporting rendering of a hardware mixing renderer;
the first processing module calls a processor unit through a hook function preset in a GLES thread to realize the rendering of the first type of layer;
the second processing module is used for calling a hardware renderer through a hardware hybrid renderer client in a Linux operating system and a hardware hybrid renderer server in the hardware abstraction layer to realize the rendering of the second type layer;
and the generating module is used for synthesizing the rendered first type layer and the rendered second type layer through the hardware mixing renderer server to generate the target image.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the steps of the image generation method according to any of claims 1 to 7 are implemented when the processor executes the program.
10. A non-transitory computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the image generation method according to any one of claims 1 to 7.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210057895.7A CN114092312A (en) | 2022-01-19 | 2022-01-19 | Image generation method, image generation device, electronic equipment and storage medium |
| PCT/CN2022/108439 WO2023138020A1 (en) | 2022-01-19 | 2022-07-28 | Image generation method and apparatus, electronic device, and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210057895.7A CN114092312A (en) | 2022-01-19 | 2022-01-19 | Image generation method, image generation device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114092312A true CN114092312A (en) | 2022-02-25 |
Family
ID=80308576
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210057895.7A Pending CN114092312A (en) | 2022-01-19 | 2022-01-19 | Image generation method, image generation device, electronic equipment and storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN114092312A (en) |
| WO (1) | WO2023138020A1 (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023138020A1 (en) * | 2022-01-19 | 2023-07-27 | 北京字节跳动网络技术有限公司 | Image generation method and apparatus, electronic device, and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140366021A1 (en) * | 2013-06-07 | 2014-12-11 | American Megatrends, Inc. | Methods, Devices and Computer Readable Storage Devices for Emulating an Accelerometer in a Guest Operating System from a Host Operating System |
| CN112200712A (en) * | 2020-09-08 | 2021-01-08 | 成都安易迅科技有限公司 | GLES image rendering method and device, storage medium and computer equipment |
| CN112269603A (en) * | 2020-10-16 | 2021-01-26 | 北京技德系统技术有限公司 | Graphic display method and device for compatibly running Android application on Linux |
| CN113110910A (en) * | 2021-04-20 | 2021-07-13 | 上海卓易科技股份有限公司 | Method, system and equipment for implementing android container |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107292807B (en) * | 2016-03-31 | 2020-12-04 | 阿里巴巴集团控股有限公司 | Graph synthesis method, window setting method and system |
| CN111768330B (en) * | 2019-03-30 | 2024-09-06 | 华为技术有限公司 | Image processing method and computer system |
| CN112099923B (en) * | 2020-11-17 | 2021-02-05 | 江苏北弓智能科技有限公司 | Android virtualization system running on Linux operating system |
| CN114092312A (en) * | 2022-01-19 | 2022-02-25 | 北京鲸鲮信息系统技术有限公司 | Image generation method, image generation device, electronic equipment and storage medium |
-
2022
- 2022-01-19 CN CN202210057895.7A patent/CN114092312A/en active Pending
- 2022-07-28 WO PCT/CN2022/108439 patent/WO2023138020A1/en unknown
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140366021A1 (en) * | 2013-06-07 | 2014-12-11 | American Megatrends, Inc. | Methods, Devices and Computer Readable Storage Devices for Emulating an Accelerometer in a Guest Operating System from a Host Operating System |
| CN112200712A (en) * | 2020-09-08 | 2021-01-08 | 成都安易迅科技有限公司 | GLES image rendering method and device, storage medium and computer equipment |
| CN112269603A (en) * | 2020-10-16 | 2021-01-26 | 北京技德系统技术有限公司 | Graphic display method and device for compatibly running Android application on Linux |
| CN113110910A (en) * | 2021-04-20 | 2021-07-13 | 上海卓易科技股份有限公司 | Method, system and equipment for implementing android container |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023138020A1 (en) * | 2022-01-19 | 2023-07-27 | 北京字节跳动网络技术有限公司 | Image generation method and apparatus, electronic device, and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2023138020A1 (en) | 2023-07-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10127628B2 (en) | Method and system to virtualize graphic processing services | |
| EP4122568A1 (en) | Data processing method and device and storage medium | |
| CN107292807B (en) | Graph synthesis method, window setting method and system | |
| US20150261726A1 (en) | Screenshot method and device | |
| US11204790B2 (en) | Display method for use in multi-operating systems and electronic device | |
| CN112269603A (en) | Graphic display method and device for compatibly running Android application on Linux | |
| KR101279680B1 (en) | Motion desktop | |
| WO2011050683A1 (en) | Method and device for displaying application image | |
| KR102426012B1 (en) | A method device for dynamically reducing the time from rendering an application to on-screen in a desktop environment | |
| CN111488196A (en) | Rendering method and device, storage medium and processor | |
| CN111522670A (en) | GPU virtualization method, system and medium for Android system | |
| US10949226B2 (en) | Display method of multi-application based on Android system, and terminal device | |
| CN114741044A (en) | Cross-operating environment display output sharing method based on heterogeneous rendering | |
| CN116821040B (en) | Display acceleration method, device and medium based on GPU direct memory access | |
| CN114092312A (en) | Image generation method, image generation device, electronic equipment and storage medium | |
| CN110545415A (en) | Data transmission method and device and server | |
| US10929079B2 (en) | Video display method, electronic device and computer program product | |
| CN114116227A (en) | Display method, device and equipment based on Wayland protocol without GPU support | |
| US20240161670A1 (en) | Dynamic refresh rate switching | |
| CN114924837A (en) | Data processing method, electronic device and readable storage medium | |
| CN114968229A (en) | Picture rendering method, medium and electronic device | |
| CN115826898B (en) | Cross-screen display method, system, device, equipment and storage medium | |
| CN117372594A (en) | Display method, electronic device and storage medium | |
| CN117724797A (en) | Free scaling optimization method based on heterogeneous mixed rendering | |
| CN113379589A (en) | Dual-system graphic processing method and device and terminal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20220425 Address after: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Applicant after: BEIJING BYTEDANCE NETWORK TECHNOLOGY Co.,Ltd. Address before: 2005, floor 2, No. 39, West Street, Haidian District, Beijing 100080 Applicant before: Beijing jingling Information System Technology Co.,Ltd. |
|
| TA01 | Transfer of patent application right |