WO2023138020A1 - 图像生成方法、装置、电子设备及存储介质 - Google Patents

图像生成方法、装置、电子设备及存储介质 Download PDF

Info

Publication number
WO2023138020A1
WO2023138020A1 PCT/CN2022/108439 CN2022108439W WO2023138020A1 WO 2023138020 A1 WO2023138020 A1 WO 2023138020A1 CN 2022108439 W CN2022108439 W CN 2022108439W WO 2023138020 A1 WO2023138020 A1 WO 2023138020A1
Authority
WO
WIPO (PCT)
Prior art keywords
layer
renderer
hardware
type
rendering
Prior art date
Application number
PCT/CN2022/108439
Other languages
English (en)
French (fr)
Inventor
邓宝安
Original Assignee
北京字节跳动网络技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京字节跳动网络技术有限公司 filed Critical 北京字节跳动网络技术有限公司
Publication of WO2023138020A1 publication Critical patent/WO2023138020A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/20—Processor architectures; Processor configuration, e.g. pipelining
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—3D [Three Dimensional] image rendering
    • G06T15/005—General purpose rendering architectures

Definitions

  • the present application relates to the technical field of terminals, for example, to an image generation method, device, electronic equipment, and storage medium.
  • the picture displayed on the terminal device is usually composed of multiple layers.
  • the smoothness of video playback is related to the speed of multiple layer rendering and compositing.
  • the present application provides an image generation method, device, electronic device and storage medium, which are used to solve the problem of image refresh rate frame drop caused by insufficient performance of the processor unit if the Linux operating system runs on some intelligent terminal devices using processors with high cost performance and low energy consumption in the related art.
  • the application provides a method for generating an image, which is applied to a Linux operating system deployed with an Android subsystem, and the Android subsystem includes a hardware abstraction layer; including:
  • the multiple layers are divided into a first type layer and a second type layer; wherein, the first type layer is a layer that supports processor unit rendering, and the second type layer is a layer that supports hardware hybrid renderer rendering;
  • the rendered first-type layer and the rendered second-type layer are synthesized by the hardware hybrid renderer server to generate the target image.
  • the present application also provides an image generating device, which is applied to a Linux operating system deployed with an Android subsystem, and the Android subsystem includes a hardware abstraction layer; the device includes:
  • the distinguishing module is configured to divide the multiple layers into a first type layer and a second type layer according to the rendering requirements of the multiple layers in the target image to be generated; wherein, the first type layer is a layer that supports processor unit rendering, and the second type layer is a layer that supports hardware hybrid renderer rendering;
  • the first processing module is configured to call the processor unit through a preset hook function in the GLES thread, so as to realize the rendering of the first type of layer;
  • the second processing module is configured to call the hardware hybrid renderer through the hardware hybrid renderer client in the Linux operating system and the hardware hybrid renderer server in the hardware abstraction layer, so as to realize the rendering of the second type of layer;
  • the generation module is configured to synthesize the rendered first-type layer and the rendered second-type layer through the hardware hybrid renderer server to generate the target image.
  • the present application also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and operable on the processor, and the processor implements the above-mentioned image generation method when executing the program.
  • the present application also provides a non-transitory computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned image generation method is implemented.
  • Fig. 1 is one of the schematic flow charts of the image generation method provided by the present application.
  • Fig. 2 is the second schematic flow diagram of the image generation method provided by the present application.
  • FIG. 3 is a schematic structural diagram of an image generation device provided by the present application.
  • FIG. 4 is a schematic structural diagram of an electronic device provided by the present application.
  • Android's hardware abstraction layer can be referred to as Android HAL for short.
  • Android HAL publishes some predefined interfaces in the form of manufacturer dynamic libraries. These interfaces are used to call some device drivers to complete corresponding functions.
  • the device driver can be called through the interface in the Android HAL, so that the corresponding device can complete the corresponding function.
  • the Android HAL predefined interface includes an interface related to a hardware hybrid renderer (Hard Ware Composer, HWC) server, calling the relevant interface of the hardware hybrid renderer server can further call the hardware hybrid renderer driver, so that the hardware hybrid renderer device completes the corresponding function.
  • HWC Hardware hybrid renderer
  • the device driver is usually located in the kernel layer of the Android system, and is used to implement various functions and call interfaces of the hardware. How to use the functions of the hardware is determined by the call of the application program.
  • the interface for the application program to call the device driver is basically a unified standard interface, which is usually called a system call interface.
  • the Android HAL can be regarded as encapsulating the functions implemented by calling the device driver into individual classes for java to call.
  • the classes encapsulated in the Android HAL can provide interfaces for related services of the Linux operating system.
  • a container is a process or group of processes isolated from the rest of the system. All the files needed to run these processes are provided by another mirror.
  • libhybris The main function of libhybris is to solve the compatibility problem of the libc library, and the purpose is to run those libraries compiled with bionic (mainly the closed-source HAL library under Android) on the GNU Library-based system.
  • libhybris is a set of software libraries for calling Android system dynamic libraries under the Linux operating system. It is mainly divided into two parts, namely hybris running on the Linux side and hybris compat running on the Android side.
  • libhybris includes the hwc_compat.so library, and the hwc_compat.so library contains some functions for calling interfaces in the Android HAL.
  • the compositor in the Linux operating system can use this library to access the relevant interfaces of the hardware hybrid renderer server on the Android side.
  • compositor can be understood as a module corresponding to the Linux system for compositing layers, which is the hardware hybrid renderer client in this application, corresponding to the hardware hybrid renderer server of the Android HAL layer.
  • the Binder framework mainly involves four roles, namely Client, Server, Service Manager and Binder driver.
  • Client, Server and Service Manager run in user space, and Binder driver runs in kernel space.
  • Client represents the client process
  • Server represents various services provided by the client process, such as audio and video
  • Service Manager is used to manage various system services
  • Binder driver provides inter-process communication capabilities.
  • the refresh process of the screen is from left to right (row refresh, horizontal refresh) and from top to bottom (screen refresh, vertical refresh) for each row.
  • a vertical refresh cycle is completed, there will be a short blank period, and the VSync signal is sent at this time.
  • the Android system sends a VSync signal every 16ms to trigger the rendering of the user interface (UI), which can be simply regarded as a timing interrupt.
  • UI user interface
  • the VSync mechanism is introduced in the Android system to synchronize rendering, so that AppUI and SurfaceFlinger can work according to the VSync rhythm generated by the hardware.
  • the image generation method provided in the present application can be applied to a scenario where a Linux desktop system runs on a smart phone or a tablet computer using an ARM architecture.
  • the Linux desktop system runs on a smartphone or tablet computer with an ARM processor.
  • the GPU or CPU is used to render and synthesize layers, problems such as insufficient performance of the processor unit may cause image refresh rate to drop frames and other problems.
  • the smart phone or tablet computer usually uses the Android system, and the Android system can use the hardware hybrid renderer and the processor unit to jointly complete the rendering and synthesis of the layer. Therefore, this application considers that the GPU and the hardware hybrid renderer in the Android system can be used to jointly complete the layer rendering and synthesis.
  • this application considers deploying the Android subsystem in a linux container, so that the Linux operating system can access the hardware hybrid renderer of the Android subsystem through the Linux container, so that it and the GPU can jointly complete the rendering and synthesis of layers.
  • FIG. 1 is a flow chart of the image generation method provided by the present application. It can be understood that the image generation method is applied to a Linux operating system deployed with an Android subsystem, and the Android subsystem includes a hardware abstraction layer. As shown in Figure 1, the image generation method provided by this application includes:
  • Step 110 according to the rendering requirements of the multiple layers in the target image to be generated, divide the multiple layers into a first type of layer and a second type of layer.
  • the first type of layer is a layer that supports processor unit rendering
  • the second type of layer is a layer that supports hardware hybrid renderer rendering.
  • the processor unit may be a CPU or a GPU.
  • the target image to be generated includes multiple layers, and each layer has its own rendering requirements.
  • some layers that do not require high-frequency rendering, such as mouse and wallpaper can be set as layers that support hardware hybrid renderer rendering.
  • layer rendering using a hardware hybrid renderer has the feature of saving GPU resources compared with using a GPU to render a layer
  • this application will use the hardware hybrid renderer to render the layers rendered by the hardware hybrid renderer, and use the hardware hybrid renderer and the GPU to jointly complete the drawing of the target image.
  • the cursor is used as an example here to illustrate that using a hardware hybrid renderer to render a layer saves GPU resources compared to using a GPU to render a layer.
  • using vendor GLES rendering first requires the entire cursor layer to be redrawn for rendering, and then uses the GPU to fit the cursor on the screen, which consumes a lot of GPU resources.
  • you only need to tell the hardware the cursor coordinate position and the hardware hybrid renderer will automatically complete the cursor coordinate position updating and pasting process of the cursor layer before the graphics are displayed on the screen. This process is asynchronous, which means that it hardly takes up GPU waiting time and does not consume GPU resources. Therefore, problems such as image refresh rate drop due to insufficient performance of the processor unit can be reduced.
  • Step 120 calling the processor unit through a preset hook function in the GLES thread, so as to render the layer of the first type.
  • the hook function can be used to interrupt the function being executed and execute the function of the preset target function.
  • the hook function may be used to call a corresponding function to call a processor unit to render the layer of the first type.
  • Step 130 calling the hardware hybrid renderer through the hardware hybrid renderer client in the Linux operating system and the hardware hybrid renderer server in the hardware abstraction layer, so as to render the second type of layer.
  • the hardware hybrid renderer client can receive a request from the Linux operating system about the rendering and synthesis of the second type of layer, and can call the hardware hybrid renderer server according to the request.
  • the hardware hybrid renderer server can be understood as a service defined in the hardware abstraction layer in the Android subsystem for controlling the hardware hybrid renderer, which provides an interface related to the hardware hybrid renderer, and can call the hardware hybrid renderer driver, so that the hardware hybrid renderer completes the rendering and synthesis of the second type of layer.
  • the hardware hybrid renderer is a piece of hardware corresponding to the Android subsystem, and can be called by the hardware hybrid renderer server in the hardware abstraction layer of the Android subsystem. Therefore, the hardware hybrid renderer server can be called by the hardware hybrid renderer client, and the hardware hybrid renderer server can call the hardware hybrid renderer to complete the rendering of the second type of layer.
  • Step 140 Synthesize the rendered first-type layer and the rendered second-type layer through the hardware hybrid renderer server to generate the target image.
  • the image generation method provided by this application divides the target image into a layer that supports processor unit rendering and a layer that supports hardware hybrid renderer rendering, and respectively for these two types of layers, adopts a corresponding method to call corresponding hardware to complete the rendering of these two types of layers, so that the hardware hybrid renderer in the Android subsystem can assist the processor unit in completing the rendering and synthesis of the layers, thereby reducing problems such as image refresh rate drop due to insufficient performance of the processor unit, so that when the Linux operating system runs on a smart phone or tablet computer that adopts the ARM architecture, video playback can also have better fluency.
  • the calling of the processor unit through the preset hook function in the GLES thread to realize the rendering of the first type of layer includes:
  • the first type of layer is transferred to the GLES thread in the Linux operating system
  • the processor unit is invoked through the first interface to render the layer of the first type.
  • the GLES thread in the Linux operating system itself cannot call the hardware to render the layer. Therefore, a hook function is set in the thread, and then the hook function invokes the first interface, and the processor unit is called by the first interface to render the first type of layer.
  • the hook function is used to call the first interface in the hardware abstraction layer.
  • the first interface is a rendering interface for implementing a layer that supports processor unit rendering, and may call the processor unit, for example, may be a vendor GLES interface.
  • the image generation method provided by this application implements the rendering interface of the layer that supports processor unit rendering in the hardware abstraction layer through the hook function preset in the GLES thread, and completes the rendering of the layer that supports processor unit rendering, thereby realizing the rendering of the layer by the Linux operating system and the Android subsystem using the processor unit.
  • the first interface is the vendor GLES interface in the hardware abstraction layer, and correspondingly, calling the first interface in the hardware abstraction layer through the preset hook function in the GLES thread includes:
  • the preset hook function in the GLES thread calls the vendor GLES interface in the hardware abstraction layer through hybris running in the Linux operating system and hybris compat running in the hardware abstraction layer.
  • hybris and hybris compat are two components of libhybris respectively, which are used to implement some interfaces in the Android hardware abstraction layer called under the Linux operating system.
  • the vendor GLES interface in the hardware abstraction layer can be called through hybris and hybris compat.
  • the image generation method provided by this application presets the hook function in the GLES thread, and calls the vendor GLES interface in the hardware abstraction layer in the Android subsystem through hybris and hybris compat, so that the Linux operating system and the Android subsystem can jointly complete the rendering of the layer that supports processor unit rendering, thereby solving the compatibility problem between the Linux operating system and the Android subsystem for layer rendering using the processor unit.
  • the hardware hybrid renderer is invoked by the hardware hybrid renderer client in the Linux operating system and the hardware hybrid renderer server in the hardware abstraction layer to implement the rendering of the second type of layer, including:
  • the second type of layer is transmitted to the hardware hybrid renderer client in the Linux operating system
  • the hardware hybrid renderer is invoked by the hardware hybrid renderer server to render the second type of layer.
  • the hardware hybrid renderer client can receive requests from the Linux operating system for rendering and compositing of the second type of layer
  • the hardware hybrid renderer server can be understood as a service for controlling the hardware hybrid renderer defined in the hardware abstraction layer in the Android subsystem, provides an interface related to the hardware hybrid renderer, and can call the hardware hybrid renderer driver, so that the hardware hybrid renderer completes the rendering and composition of the second type of layer. Therefore, the layer of the second type can be transmitted to the hardware hybrid renderer client in the Linux operating system, the hardware hybrid renderer client calls the hardware hybrid renderer server, and the hardware hybrid renderer server calls the hardware hybrid renderer to realize the rendering of the second type of layer.
  • the hardware hybrid renderer client calls the hardware hybrid renderer server in the hardware abstraction layer in the Android subsystem to make it call the hardware hybrid renderer to realize the rendering of the second type of layer, so that the hardware hybrid renderer in the Android subsystem can be used in the Linux operating system to complete the rendering of the layer that supports the hardware hybrid renderer rendering, so that the hardware hybrid renderer in the Android subsystem can assist the processor unit to complete the rendering and synthesis of the layers, thereby reducing problems such as image refresh rate drops due to insufficient performance of the processor unit, so that the Linux operating system runs on the system using the hardware hybrid renderer.
  • the video playback can also have better fluency.
  • the calling the hardware hybrid renderer server in the hardware abstraction layer by the hardware hybrid renderer client includes:
  • the hardware hybrid renderer client invokes the hardware hybrid renderer server in the hardware abstraction layer through hybris running in the Linux operating system and hybris compat running in the hardware abstraction layer.
  • the hardware hybrid renderer server can be understood as the service interface for controlling the hardware hybrid renderer defined in the hardware abstraction layer in the Android subsystem, the service interface is compiled based on bionic, therefore, library compatibility can be realized by libhybris when called in the Linux operating system, that is, hybris and hybris compat realize calling the hardware hybrid renderer server in the hardware abstraction layer of the Android system under the linux system, so that it calls the hardware hybrid renderer to realize the rendering of the second type of layer.
  • the image generation method provided by the present application calls the hardware hybrid renderer server in the hardware abstraction layer in the Android subsystem through hybris and hybris compat, so that it calls the hardware hybrid renderer to realize the rendering of the second type of layer, so that the hardware hybrid renderer in the Android subsystem can be used in the Linux operating system to complete the rendering of the layer that supports the hardware hybrid renderer rendering, so that the hardware hybrid renderer in the Android subsystem can assist the processor unit to complete the rendering and synthesis of the layers, thereby reducing problems such as image refresh rate dropped due to insufficient performance of the processor unit, and the Linux operating system.
  • the video playback can also have a good fluency.
  • the hardware hybrid renderer server synthesizes the rendered first type layer and the rendered second type layer to generate the target image, including:
  • the rendered first-type layer is transmitted to a hardware hybrid renderer client, and the hardware hybrid renderer client transmits the rendered first-type layer to the hardware hybrid renderer server in the hardware abstraction layer via hybris running in the Linux operating system and hybris compat running in the hardware abstraction layer;
  • the rendered first-type layer and the rendered second-type layer are synthesized by the hardware hybrid renderer server to generate the target image.
  • the first type of layer after rendering is obtained by rendering at least one layer through the vendor GLES interface. Therefore, it is also necessary to send the first type of layer rendered through the vendor GLES interface to the hardware hybrid renderer server, so that the hardware hybrid renderer server combines it with the second type of layer to generate the target image.
  • the rendered at least one layer of the first type can be put into the cache block, and marked as a rendered layer that supports processor unit rendering, and the hardware hybrid renderer client sets the cache block to the hardware hybrid renderer server by calling the relevant interface in libhybris, and the hardware hybrid renderer server invokes the hardware hybrid renderer based on the calling instruction of the hardware hybrid renderer client, and completes the rendering and synthesis of the layer that supports hardware hybrid renderer rendering and the already rendered layer that supports processor unit rendering.
  • the image generation method provided by this application by making the hardware hybrid renderer server synthesize the first layer and the second layer, the hardware hybrid renderer in the Android subsystem and the processor unit jointly complete the rendering and synthesis of the layers, thereby reducing problems such as image refresh rate drop due to insufficient performance of the processor unit, and enabling the Linux operating system to run on a smart phone or a tablet computer using an ARM architecture, and the video playback can also have better fluency.
  • the method before dividing the layers into the first type layer and the second type layer according to the rendering requirements of the multiple layers in the target image to be generated, the method further includes:
  • start the hardware hybrid renderer client In the Linux operating system, start the hardware hybrid renderer client.
  • the Android subsystem can be deployed in the Linux operating system through a container, and the Android subsystem can be initialized and started in the container.
  • the hardware hybrid renderer driver is automatically loaded to facilitate subsequent access to the hardware hybrid renderer through the hardware hybrid renderer driver.
  • the corresponding hardware hybrid renderer client in the Android subsystem is surfaceflinger, but the corresponding module for controlling layer rendering and compositing in the Linux operating system is compositor, therefore, in the Linux operating system, modify the hardware hybrid renderer client to compositor.
  • the image generation method provided by the application starts the Android subsystem, the hardware hybrid renderer server and the hardware hybrid renderer client, through which the hardware hybrid renderer synthesizes the first layer and the second layer, so that the hardware hybrid renderer and the processor unit in the Android subsystem jointly complete the rendering and synthesis of the layers, thereby reducing problems such as the image refresh rate dropping frames due to insufficient performance of the processor unit, and enabling the Linux operating system to run on smart phones or tablet computers using the ARM architecture.
  • FIG. 2 is the second schematic flow diagram of the image generation method provided by the present application.
  • Fig. 2 describes the image generation method provided by the application with the interactive process between the Linux synthesizer module compositor and the Android HAL.
  • compositor can be understood as a module used to control the rendering and composition of layers in the Linux system;
  • the window manager (window manager) is responsible for the generation and classification of layers in the window;
  • the hardware hybrid renderer client (hwc client) corresponds to the compositor;
  • hybris and hybris compat are two components of libhybris, which constitute the interactive channel between compositor and Android HAL, so that the two can complete layer composition through interaction;
  • hybris calls the relevant interfaces in the hardware hybrid renderer server (hwc server) to complete layer rendering and compositing;
  • the GLES thread is preset with a hook function, and the hook function is used to call the vendor GLES in the Android HAL to
  • the vendor GLES interface is set to call the processor unit to complete the rendering of the layer that supports processor unit rendering, and save the rendered layer to the target buffer area.
  • the hardware hybrid renderer server invokes the hardware hybrid renderer to complete corresponding functions based on the calling instruction of the hardware hybrid renderer client.
  • the image generation method is introduced below in combination with FIG. 2 .
  • the window manager in the compositor receives the contents of the target image, which may include: an interface corresponding to an application, a cursor corresponding to a mouse, a status bar and a background.
  • the window manager distinguishes the layers corresponding to the contents included in the target image to obtain different types of layers.
  • layers may include, for example: a layer supporting processor unit rendering (gles layer), a rendered layer supporting processor unit rendering (hwc client layer), and a layer supporting hardware hybrid renderer rendering (hwc device layer, hwc color layer and hwc cursor layer).
  • a layer supporting processor unit rendering gles layer
  • a rendered layer supporting processor unit rendering hwc client layer
  • a layer supporting hardware hybrid renderer rendering hwc device layer, hwc color layer and hwc cursor layer.
  • the compositor can call the relevant interface of the hardware hybrid renderer server through libhybris to create at least one layer and set the information of the at least one layer according to the content included in the target image, and let the hardware hybrid renderer determine the layers that support processor unit rendering and the layers that support hardware hybrid renderer rendering in the at least one layer according to the information of the layers through libhybris, and return the layer information to the window manager.
  • the libhybris framework contains hwc_compat.so, hwc_compat.so is compiled with bionic, runs in the Android environment, and contains some functions for calling the Android interface, so that the Linux system can access the hardware hybrid renderer of the Android system.
  • At least one layer is created by calling the second interface in the relevant interface of the hardware hybrid renderer server through the first interface of libhybris; calling at least one third interface of libhybris to set the information of the at least one layer, and the information includes the rendering type of the layer.
  • the first interface and the third interface of libhybris may be, for example, one of the interfaces in the aforementioned hwc_compat.so.
  • the first interface of libhybris is set to call the relevant interface of the hardware hybrid renderer server, and the second interface of the hardware hybrid renderer service is set to create at least one layer of preset rendering type.
  • the third interface of libhybris is configured to set the information of the at least one layer.
  • the layer of the preset rendering type may be: a layer supporting hardware hybrid renderer rendering and a layer supporting processor unit rendering.
  • the first interface of libhybris may be, for example, an interface in the aforementioned hwc_compat.so, such as the hwc2_compat_display_create_layer interface, and the second interface of the hardware hybrid renderer server may be, for example, hwc2_display_create_layer.
  • the layer information includes the rendering type of the layer, which corresponds to the different types of layers mentioned above, and may be: a layer supporting hardware hybrid renderer rendering, a rendered layer supporting processor unit rendering, and a layer supporting processor unit rendering.
  • the third interface of libhybris may be, for example, the hwc2_compat_layer_set_composition_type interface.
  • the information of the layer may also include the display area parameter of the layer.
  • the third interface of libhybris may be, for example, an interface in the aforementioned hwc_compat.so, such as the interface hwc2_compat_layer_set_source_crop, the interface hwc2_compat_layer_set_display_frame or the interface hwc2_compat_layer_set_visible_region.
  • VSync signal layer compositing control signal
  • the preset communication mode may be Binder communication, for example.
  • Binder communication is used for inter-process communication.
  • the compositor module of Linux corresponds to a client process. It can be understood that since the Android subsystem runs in a container, the compositor module of the Linux system and the hardware hybrid renderer service of the Android subsystem are in different processes. Therefore, the two need to communicate through the Binder.
  • control signal of layer compositing is sent by the hardware composite renderer, and the control signal of layer compositing is used to control the rendering and compositing of layers. Therefore, when the control signal for layer composition arrives, the layer rendering and composition process will be executed. The following describes the layer rendering and compositing process in detail.
  • the corresponding rendering type will be set when the layer is created, so the rendering type of the layer needs to be determined during the rendering process, and then the layer can be assigned to different compositing hardware for compositing according to the already set rendering type.
  • determining the layer that supports processor unit rendering and the layer that supports hardware hybrid renderer rendering in the at least one layer may be: calling the fourth interface of libhybris to let the hardware hybrid renderer determine the layer that supports processor unit rendering and the layer that supports hardware hybrid renderer rendering in the at least one layer.
  • the fourth interface of libhybris may be an interface in hwc_compat.so, which is set to determine the rendering type of the at least one layer.
  • the fourth interface may be hwc2_compat_display_validate, for example.
  • the compositor calls the relevant interface in the hardware hybrid renderer server through hwc_compat.so in libhybris to let the hardware hybrid renderer determine the layer that supports processor unit rendering and the layer that supports hardware hybrid renderer rendering in the at least one layer according to the rendering type of the layer, and sends different layers to the GELS thread and the hardware hybrid renderer client respectively according to the rendering type of the layer, and is forwarded to the vendor GELS and the hardware hybrid renderer server by the GELS thread and the hardware hybrid renderer client respectively.
  • the process corresponding to Figure 2 is: sending different types of layers to the hardware hybrid renderer client and the GLES thread respectively, and the hardware hybrid renderer client and the GLES thread respectively call the hardware hybrid renderer server and vendor GLES through hybris and hybris compat.
  • the rendering of the layer that supports processor unit rendering is completed by vendor GLES, and the rendered layer of the layer that supports processor unit rendering is saved in the buffer block, and returned to the window manager, and the window manager uses the layer that is rendered by the layer that supports processor unit rendering as the already rendered layer that supports processor unit rendering.
  • the hardware hybrid renderer server calls the hardware hybrid renderer based on the call instruction of the hardware hybrid renderer client, and completes the rendering and compositing of the layer that supports hardware hybrid renderer rendering and the already rendered layer that supports processor unit rendering. For the process, please refer to the relevant description below.
  • the cache block can be set to the hardware hybrid renderer by calling the fifth interface of libhybris; calling the sixth interface of libhybris allows the hardware hybrid renderer to complete the composite of the already rendered layer supporting processor unit rendering and the already rendered layer supporting hardware hybrid renderer rendering to obtain the target image.
  • the fifth interface and the sixth interface of libhybris are respectively an interface in hwc_compat.so.
  • the fifth interface may be, for example, hwc2_compat_display_set_client_target, and the sixth interface may be, for example, hwc2_compat_display_present.
  • the target image is sent to the display for display.
  • the image generation device provided by the present application is described below, and the image generation device described below and the image generation method described above can be referred to in correspondence.
  • Fig. 3 is a schematic diagram of the image generating device provided by the present application. As shown in Fig. 3, the image generating device provided by the embodiment of the present application is applied to a Linux operating system deployed with an Android subsystem, and the Android subsystem includes a hardware abstraction layer; the device includes:
  • the distinguishing module 310 is configured to divide the plurality of layers into a first type of layer and a second type of layer according to the rendering requirements of the plurality of layers in the target image to be generated; wherein, the first type of layer is a layer that supports processor unit rendering, and the second type of layer is a layer that supports hardware hybrid renderer rendering;
  • the first processing module 320 is configured to call the processor unit through a preset hook function in the GLES thread, so as to realize the rendering of the first type of layer;
  • the second processing module 330 is configured to call the hardware hybrid renderer through the hardware hybrid renderer client in the Linux operating system and the hardware hybrid renderer server in the hardware abstraction layer to implement the rendering of the second type of layer;
  • the generating module 340 is configured to synthesize the rendered first-type layer and the rendered second-type layer through the hardware hybrid renderer server to generate the target image.
  • the image generation device divides the target image into a layer that supports processor unit rendering and a layer that supports hardware hybrid renderer rendering, and uses corresponding methods to call corresponding hardware to complete the rendering of these two types of layers respectively for these two types of layers, so that the hardware hybrid renderer in the Android subsystem can assist the processor unit to complete the rendering and synthesis of layers, thereby reducing problems such as image refresh rate dropped due to insufficient performance of the processor unit, and when the Linux operating system runs on a smart phone or tablet computer that adopts an ARM architecture, video playback can also have better fluency.
  • the first processing module 320 includes:
  • the first transmission unit is configured to transmit the first type of layer to the GLES thread in the Linux operating system
  • the first calling unit is configured to call the first interface in the hardware abstraction layer through the preset hook function in the GLES thread;
  • the first rendering unit is configured to call the processor unit through the first interface to implement rendering of the first type of layer.
  • the image generation device realizeds the rendering interface of the layer supporting processor unit rendering in the hardware abstraction layer through the hook function preset in the GLES thread, and completes the rendering of the layer supporting processor unit rendering, thereby realizing the rendering of the layer by the processor unit used by the Linux operating system and the Android subsystem.
  • the first calling unit includes:
  • the second calling unit is set as the preset hook function in the GLES thread calls the vendor GLES interface in the hardware abstraction layer through hybris running in the Linux operating system and hybris compat running in the hardware abstraction layer.
  • the image generation device presets a hook function in the GLES thread, and calls the vendor GLES interface in the hardware abstraction layer in the Android subsystem through hybris and hybris compat, so that the Linux operating system and the Android subsystem can jointly complete the rendering of the layer that supports processor unit rendering, thereby solving the compatibility problem between the Linux operating system and the Android subsystem for layer rendering using the processor unit.
  • the second processing module 330 includes:
  • the second transmission unit is configured to transmit the second type of layer to the hardware hybrid renderer client in the Linux operating system
  • the third calling unit is configured to call the hardware hybrid renderer server in the hardware abstraction layer by the hardware hybrid renderer client;
  • the second rendering unit is configured to call the hardware hybrid renderer through the hardware hybrid renderer server to render the second type of layer.
  • the image generating device calls the hardware hybrid renderer server in the hardware abstraction layer in the Android subsystem through the hardware hybrid renderer client, so that it calls the hardware hybrid renderer to realize the rendering of the second type of layer, so that the hardware hybrid renderer in the Android subsystem can be used in the Linux operating system to complete the rendering of the layers that support the hardware hybrid renderer rendering, so that the hardware hybrid renderer in the Android subsystem can assist the processor unit in completing the rendering and synthesis of layers, thereby reducing problems such as image refresh rate drop due to insufficient performance of the processor unit.
  • the video playback can also have better fluency.
  • the third calling unit includes:
  • the fourth calling unit is set to the hardware hybrid renderer client calling the hardware hybrid renderer server in the hardware abstraction layer through hybris running in the Linux operating system and hybris compat running in the hardware abstraction layer.
  • the image generating device calls the hardware hybrid renderer server in the hardware abstraction layer in the Android subsystem through hybris and hybris compat, so that it calls the hardware hybrid renderer to realize the rendering of the second type of layer, so that the hardware hybrid renderer in the Android subsystem can be used in the Linux operating system to complete the rendering of the layer that supports the hardware hybrid renderer rendering, so that the hardware hybrid renderer in the Android subsystem can assist the processor unit to complete the rendering and synthesis of the layers, thereby reducing problems such as image refresh rate drop due to insufficient performance of the processor unit, and the Linux operating system.
  • the video playback can also have a good fluency.
  • the generating module 340 includes:
  • the third transmission unit is configured to transmit the rendered first type layer to the Linux operating system
  • the fourth transmission unit is configured to transmit the rendered first-type layer to the hardware hybrid renderer client in the Linux operating system, and the hardware hybrid renderer client transmits the rendered first-type layer to the hardware hybrid renderer server in the hardware abstraction layer via hybris running in the Linux operating system and hybris compat running in the hardware abstraction layer;
  • the synthesis unit is configured to synthesize the rendered first-type layer and the rendered second-type layer through the hardware hybrid renderer server to generate the target image.
  • the image generating device makes the hardware hybrid renderer server synthesize the first layer and the second layer, so that the hardware hybrid renderer in the Android subsystem and the processor unit jointly complete the rendering and synthesis of the layers, thereby reducing problems such as image refresh rate drop due to insufficient performance of the processor unit, and enabling the Linux operating system to run on a smart phone or a tablet computer using an ARM architecture, and the video playback can also have better fluency.
  • the device further includes:
  • the Android subsystem startup module is configured to initialize and start the Android subsystem deployed in the Linux operating system
  • the first startup module is configured to start the hardware hybrid renderer server in the hardware abstraction layer in the process of starting the Android subsystem;
  • the second startup module is configured to start the hardware hybrid renderer client in the Linux operating system.
  • the image generation device starts the Android subsystem, the hardware hybrid renderer server and the hardware hybrid renderer client, through which the hardware hybrid renderer synthesizes the first layer and the second layer, so that the hardware hybrid renderer in the Android subsystem and the processor unit jointly complete the rendering and synthesis of the layers, thereby reducing problems such as image refresh rate drop due to insufficient performance of the processor unit, and enabling the Linux operating system to run on smart phones or tablet computers using ARM architecture, and the video playback can also have better fluency.
  • FIG. 4 illustrates a schematic diagram of the physical structure of an electronic device.
  • the electronic device may include: a processor (processor) 410, a communication interface (Communications Interface) 420, a memory (memory) 430, and a communication bus 440, wherein the processor 410, the communication interface 420, and the memory 430 complete mutual communication through the communication bus 440.
  • processor processor
  • Communication interface Communication interface
  • memory memory
  • the processor 410 can call the logical instructions in the memory 430 to execute the image generation method in this application, the method includes: according to the rendering requirements of the multiple layers in the target image to be generated, divide the multiple layers into a first type layer and a second type layer; wherein, the first type layer is a layer that supports processor unit rendering, and the second type layer is a layer that supports hardware hybrid renderer rendering; the processor unit is called by a hook function preset in the GLES thread to realize the rendering of the first type layer; through the hardware hybrid renderer client in the Linux operating system and the described layer
  • the hardware hybrid renderer server in the hardware abstraction layer calls the hardware hybrid renderer to render the second-type layer; the hardware hybrid renderer server synthesizes the rendered first-type layer and the rendered second-type layer to generate the target image.
  • the above logic instructions in the memory 430 may be implemented in the form of software function units and be stored in a computer-readable storage medium when sold or used as an independent product.
  • the technical solution of the present application is essentially or the part that contributes to the related technology or the part of the technical solution can be embodied in the form of a software product.
  • the computer software product is stored in a storage medium and includes several instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in each embodiment of the application.
  • the aforementioned storage medium includes: various media that can store program codes such as U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk.
  • the present application also provides a computer program product.
  • the computer program product includes a computer program stored on a non-transitory computer-readable storage medium.
  • the computer program includes program instructions. When the program instructions are executed by the computer, the computer can execute the image generation method provided in the present application.
  • the method includes: according to the rendering requirements of the multiple layers in the target image to be generated, dividing the multiple layers into a first type of layer and a second type of layer; wherein, the first type of layer is a layer that supports processor unit rendering, and the second type of layer is a layer that supports rendering by a hardware hybrid renderer;
  • the preset hook function in the thread calls the processor unit to realize the rendering of the first type of layer;
  • the hardware hybrid renderer client in the Linux operating system and the hardware hybrid renderer server in the hardware abstraction layer call the hardware hybrid renderer to realize the rendering of the second type of layer;
  • the rendered first type layer and the rendered second type layer are synthesized by the hardware hybrid renderer server to generate the target image.
  • the present application also provides a non-transitory computer-readable storage medium, on which a computer program is stored.
  • the image generation method provided by the present application is implemented. The method includes: according to the rendering requirements of the multiple layers in the target image to be generated, dividing the multiple layers into a first type of layer and a second type of layer; wherein, the first type of layer is a layer that supports rendering by a processor unit, and the second type of layer is a layer that supports rendering by a hardware hybrid renderer; the processor unit is implemented by calling the preset hook function in the GLES thread.
  • Rendering of the first type of layer calling the hardware hybrid renderer through the hardware hybrid renderer client in the Linux operating system and the hardware hybrid renderer server in the hardware abstraction layer to realize the rendering of the second type of layer; synthesizing the rendered first type layer and the rendered second type layer through the hardware hybrid renderer server to generate the target image.
  • the device embodiments described above are only illustrative, and the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may also be distributed to multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. It can be understood and implemented by those skilled in the art without any creative effort.
  • each implementation can be implemented by means of software plus a necessary general-purpose hardware platform, and of course also by hardware.
  • the computer software product can be stored in a computer-readable storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., and includes several instructions to make a computer device (which can be a personal computer, server, or network device, etc.) execute the methods described in various embodiments or some parts of the embodiments.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Graphics (AREA)
  • Advance Control (AREA)

Abstract

本申请提供一种图像生成方法、装置、电子设备及存储介质，方法包括：根据待生成的目标图像中的多个图层的渲染需求，将多个图层分为第一类图层与第二类图层；第一类图层为支持处理器单元渲染的图层，第二类图层为支持HWC渲染的图层；通过在GLES线程中预设的钩子函数调用所述处理器单元，实现对所述第一类图层的渲染；通过Linux操作系统中的硬件混合渲染器客户端和所述硬件抽象层中的硬件混合渲染器服务端调用硬件混合渲染器，实现对所述第二类图层的渲染；通过HWC服务端对渲染后的第一类图层与渲染后的第二类图层进行合成，生成目标图像。

Description

图像生成方法、装置、电子设备及存储介质
本申请要求在2022年1月19日提交中国专利局、申请号为202210057895.7的中国专利申请的优先权，该申请的全部内容通过引用结合在本申请中。
技术领域
本申请涉及终端技术领域，例如涉及一种图像生成方法、装置、电子设备及存储介质。
背景技术
用户通过终端设备安装的视频应用程序(Application，APP)观看视频时，终端设备显示的画面通常是由多个图层合成的。视频播放的流畅度和多个图层渲染及合成的速度有关。
相关技术中，Linux桌面系统运行在X86环境下时，通常用图形处理器(graphicsprocessing unit，GPU)或者中央处理器(central processing unit，CPU)完成对图层的合成，然而若Linux操作系统运行在一些采用高性价比、耗能低的处理器的智能终端设备上，例如Linux操作系统运行在采用ARM(Advanced RISC Machines)处理器的智能手机或平板电脑上，如果仅使用处理器单元完成对图层的渲染及合成，就容易出现处理器单元的性能不足导致图像刷新率掉帧等问题。
因此，如何提高Linux操作系统运行在一些采用高性价比、耗能低的处理器的智能终端设备上时图层渲染及合成的性能，避免出现图像刷新率掉帧的现象，使视频播放时具有较好的流畅度，是目前亟待解决的一个问题。
发明内容
本申请提供一种图像生成方法、装置、电子设备及存储介质，用以解决相关技术中若Linux操作系统运行在一些采用高性价比、耗能低的处理器的智能终端设备上，如果仅使用处理器单元完成对图层的渲染及合成，出现处理器单元 的性能不足导致图像刷新率掉帧的缺陷，实现Linux操作系统运行在采用ARM架构的智能手机或平板电脑上时，减少因为处理器单元的性能不足导致图像刷新率掉帧等问题，视频播放时也能具有较好的流畅度。
本申请提供一种图像生成方法，应用于部署有Android子系统的Linux操作系统中，所述Android子系统包括硬件抽象层；包括：
根据待生成的目标图像中的多个图层的渲染需求，将所述多个图层分为第一类图层与第二类图层；其中，所述第一类图层为支持处理器单元渲染的图层，所述第二类图层为支持硬件混合渲染器渲染的图层；
通过在GLES线程中预设的钩子函数调用所述处理器单元，实现对所述第一类图层的渲染；
通过Linux操作系统中的硬件混合渲染器客户端和所述硬件抽象层中的硬件混合渲染器服务端调用硬件混合渲染器，实现对所述第二类图层的渲染；
通过所述硬件混合渲染器服务端对渲染后的第一类图层与渲染后的第二类图层进行合成，生成所述目标图像。
本申请还提供一种图像生成装置，应用于部署有Android子系统的Linux操作系统中，所述Android子系统包括硬件抽象层；所述装置包括：
区分模块，设置为根据待生成的目标图像中的多个图层的渲染需求，将所述多个图层分为第一类图层与第二类图层；其中，所述第一类图层为支持处理器单元渲染的图层，所述第二类图层为支持硬件混合渲染器渲染的图层；
第一处理模块，设置为通过在GLES线程中预设的钩子函数调用所述处理器单元，实现对所述第一类图层的渲染；
第二处理模块，设置为通过Linux操作系统中的硬件混合渲染器客户端和所述硬件抽象层中的硬件混合渲染器服务端调用硬件混合渲染器，实现对所述第二类图层的渲染；
生成模块，设置为通过所述硬件混合渲染器服务端对渲染后的第一类图层与渲染后的第二类图层进行合成，生成所述目标图像。
本申请还提供一种电子设备，包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，所述处理器执行所述程序时实现如上述图像生成方法。
本申请还提供一种非暂态计算机可读存储介质，其上存储有计算机程序，该计算机程序被处理器执行时实现如上述图像生成方法。
附图说明
图1是本申请提供的图像生成方法的流程示意图之一；
图2是本申请提供的图像生成方法的流程示意图之二；
图3是本申请提供的图像生成装置的结构示意图；
图4是本申请提供的电子设备的结构示意图。
具体实施方式
下面将结合本申请中的附图，对本申请中的技术方案进行清楚、完整地描述。
为了便于理解，首先对本申请涉及的术语进行解释。
(1)Android的硬件抽象层(hardware abstraction layer，HAL)
Android的硬件抽象层可简称为Android HAL，Android HAL中通过厂商动态库的形式发布了一些预定义的接口，这些接口用于调用一些设备驱动完成相应的功能。
可以理解为，通过Android HAL中的接口可以调用设备驱动程序，使相应的设备完成相应的功能。
示例性地，所述Android HAL预定义的接口中包括与硬件混合渲染器(Hard Ware Composer，HWC)服务端相关的接口，调用硬件混合渲染器服务端的相关接口即可以进一步调用硬件混合渲染器驱动程序，使硬件混合渲染器设备完成相应的功能。
设备驱动通常位于Android系统的内核层，用于实现硬件的各种功能及调用接口，如何使用硬件的功能由应用程序的调用来决定的。
可以理解，应用程序调用设备驱动的接口基本是统一的标准接口，通常被称为系统调用接口。通常情况下，Android HAL可以看作是把调用设备驱动实现的功能封装成一个个的类，供java来调用。在本申请中，Android HAL中封装好的类可以给Linux操作系统的相关服务提供接口。
(2)容器
容器是与系统其他部分隔离开的一个或一组进程。运行这些进程所需的所有文件都由另一个镜像提供。
将Android子系统部署在容器中，以便Linux操作系统通过容器访问Android系统的HWC，使其与GPU共同完成对图层的绘制。
(3)libhybris
libhybris主要作用是为了解决libc库的兼容问题，目的是为了在基于GNU Clibrary的系统上运行那些用bionic编译的库(主要是Android下的闭源HAL库)。
可以理解为，libhybris是一套用于实现在Linux操作系统下调用Android系统动态库的软件库，主要分为两部分，分别为运行在Linux端的hybris和运行在Android端的hybris compat。
示例性地，libhybris中包含hwc_compat.so库，hwc_compat.so库内部包含一些用于调用Android HAL中接口的函数。Linux操作系统中的compositor可以通过这个库去访问Android端的硬件混合渲染器服务端的相关接口。
其中，compositor可以理解为Linux系统对应的用于合成图层的一个模块，是本申请中的硬件混合渲染器客户端，与Android HAL层的硬件混合渲染器服务端相对应。
(4)Binder
Binder框架中主要涉及到4个角色，分别为Client、Server、Service Manager 及Binder驱动，其中Client、Server、Service Manager运行在用户空间，Binder驱动运行在内核空间。Client代表客户端进程；Server代表客户端进程提供的各种服务，如音视频等；Service Manager用来管理各种系统服务；Binder驱动提供进程间通信的能力。
(6)垂直同步(vertical synchronization，VSync)信号
屏幕的刷新过程是每一行从左到右(行刷新，水平刷新)，从上到下(屏幕刷新，垂直刷新)。当整个屏幕刷新完毕，即一个垂直刷新周期完成，会有短暂的空白期，此时发出VSync信号。
Android系统每隔16ms发出VSync信号，触发对用户界面(user interface,UI)进行渲染，可以简单的把它认为是一种定时中断。而在Android系统中引入VSync机制，用来同步渲染，让AppUI和SurfaceFlinger可以按硬件产生的VSync节奏进行工作。
可以理解，本申请提供的图像生成方法可以应用于Linux桌面系统运行在采用ARM架构的智能手机或平板电脑上的场景。如前所述，Linux桌面系统运行在采用ARM处理器的智能手机或平板电脑上，如果仅采用GPU或者CPU完成对图层的渲染与合成，容易出现处理器单元的性能不足的问题导致图像刷新率掉帧等问题。而所述智能手机或平板电脑通常使用Android系统，而Android系统可以利用硬件混合渲染器与处理器单元共同完成对图层的渲染及合成，因此，本申请考虑可以采用GPU和Android系统中硬件混合渲染器共同完成图层的渲染及合成。但是由于Linux操作系统无法直接访问Android子系统的硬件混合渲染器，且由于Android系统是一个以Linux内核为基础的开源移动设备操作系统，因此本申请考虑将Android子系统部署在linux容器中，以便Linux操作系统通过Linux容器访问Android子系统的硬件混合渲染器，使其与GPU共同完成对图层的渲染及合成。
下面结合图1-图4描述本申请的图像生成方法、装置、电子设备与存储介质。
图1为本申请提供的图像生成方法的流程图。可理解，所述图像生成方法应用于部署有Android子系统的Linux操作系统中，所述Android子系统包括硬件抽象层。如图1所示，本申请提供的图像生成方法包括：
步骤110、根据待生成的目标图像中的多个图层的渲染需求，将所述多个图层分为第一类图层与第二类图层。
其中，所述第一类图层为支持处理器单元渲染的图层，所述第二类图层为支持硬件混合渲染器渲染的图层。
在一个实施例中，所述处理器单元可以为CPU或GPU。
可以理解，所述待生成的目标图像包括多个图层，每个图层对应有自己的渲染需求。例如，可以针对一些不需要高频渲染的图层如鼠标，壁纸等图层，设置为支持硬件混合渲染器渲染的图层。
应理解，由于采用硬件混合渲染器进行图层的渲染与采用GPU进行图层的渲染相比具有节省GPU资源的特点，因此，本申请将支持硬件混合渲染器渲染的图层使用硬件混合渲染器渲染，使用硬件混合渲染器和GPU共同完成对目标图像的绘制。
为了便于理解，此处以光标为例说明采用硬件混合渲染器进行图层的渲染与采用GPU进行图层的渲染相比具有节省GPU资源。示例性地，使用vendor GLES渲染首先需要整个光标图层重新绘制渲染，然后使用GPU将光标贴合在屏幕上，此过程需要消耗大量GPU资源。而使用硬件混合渲染器只需要告诉硬件光标坐标位置，硬件混合渲染器会在图形上屏前自动完成光标图层的光标坐标位置更新和贴合过程，这个过程是异步的，也就是说几乎不占GPU等待时间，不用消耗GPU资源，因此可以减少因为处理器单元的性能不足导致图像刷新率掉帧等问题。
步骤120、通过在GLES线程中预设的钩子函数调用所述处理器单元，实现对所述第一类图层的渲染。
其中，钩子函数可以用于中断正在执行的函数，执行预设的目标函数的功 能。在本实施例中，钩子函数可以用于调用相应的函数以调用处理器单元实现对所述第一类图层的渲染。
可以理解，由于Linux操作系统中的GLES线程本身不能调用硬件进行图层的渲染。因此在该线程中设置一个钩子函数，由该钩子函数调用处理器单元，实现对所述第一类图层的渲染。
步骤130、通过Linux操作系统中的硬件混合渲染器客户端和所述硬件抽象层中的硬件混合渲染器服务端调用硬件混合渲染器，实现对所述第二类图层的渲染。
其中，硬件混合渲染器客户端可以接收来自Linux操作系统中的关于第二类图层的渲染和合成的请求，并能够根据所述请求调用硬件混合渲染器服务端。硬件混合渲染器服务端可以理解为Android子系统中的硬件抽象层中定义的一个用于控制硬件混合渲染器的服务，提供了与硬件混合渲染器相关的接口，可以调用硬件混合渲染器驱动器，从而使硬件混合渲染器完成第二类图层的渲染和合成。
可以理解，所述硬件混合渲染器是Android子系统对应的一个硬件，可以被Android子系统中的硬件抽象层中的硬件混合渲染器服务端调用。因此，可以通过硬件混合渲染器客户端调用硬件混合渲染器服务端，由硬件混合渲染器服务端调用硬件混合渲染器完成对所述第二类图层的渲染。
步骤140、通过所述硬件混合渲染器服务端对渲染后的第一类图层与渲染后的第二类图层进行合成，生成所述目标图像。
本申请提供的图像生成方法，通过将目标图像分为支持处理器单元渲染的图层和支持硬件混合渲染器渲染的图层，并且分别针对这两类图层，采用相应的方法调用相应的硬件分别完成这两类图层的渲染，使Android子系统中的硬件混合渲染器能够协助处理器单元完成对图层的渲染和合成，从而减少因为处理器单元的性能不足导致图像刷新率掉帧等问题，使Linux操作系统运行在采用ARM架构的智能手机或平板电脑上时，视频播放时也能具有较好的流畅度。
基于上述任一实施例，在本实施例中，所述通过在GLES线程中预设的钩子函数调用处理器单元，实现对所述第一类图层的渲染，包括：
将所述第一类图层传输至Linux操作系统中的GLES线程；
通过所述GLES线程中的预设钩子函数调用所述硬件抽象层中的第一接口；
通过所述第一接口调用处理器单元，实现对所述第一类图层的渲染。
如前所述，由于Linux操作系统中的GLES线程本身不能调用硬件进行图层的渲染。因此在该线程中设置一个钩子函数，再由该钩子函数通过调用第一接口，由第一接口调用处理器单元进行第一类图层的渲染。钩子函数用于调用所述硬件抽象层中的第一接口。其中，第一接口是用于实现支持处理器单元渲染的图层的渲染接口，可以调用处理器单元，例如可以为vendor GLES接口。
本申请提供的图像生成方法，通过在GLES线程中预设钩子函数，通过钩子函数实现对硬件抽象层中支持处理器单元渲染的图层的渲染接口，完成支持处理器单元渲染的图层的渲染，从而实现了Linux操作系统和Android子系统使用处理器单元进行图层的渲染。
基于上述任一实施例，在本实施例中，所述第一接口为所述硬件抽象层中的vendor GLES接口，相应的，所述通过所述GLES线程中的预设钩子函数调用所述硬件抽象层中的第一接口，包括：
所述GLES线程中的预设钩子函数通过运行在Linux操作系统中的hybris以及运行在所述硬件抽象层中的hybris compat调用所述硬件抽象层中的vendor GLES接口。
其中，hybris和hybris compat分别为libhybris的两个组成部分，用于实现在Linux操作系统下调用Android硬件抽象层中的一些接口。在本实施例中，通过hybris和hybris compat可以调用硬件抽象层中的vendor GLES接口。
可以理解，由于Android硬件抽象层中的一些接口是基于bionic编译的函数，因此要想在Linux系统上调用Android硬件抽象层中的接口，需要通过libhybris实现库兼容，以实现在Linux系统上调用Android硬件抽象层中的接口。
本申请提供的图像生成方法，通过在GLES线程中预设钩子函数，并且通过hybris和hybris compat调用Android子系统中的硬件抽象层中的vendor GLES接口，从而使Linux操作系统与Android子系统能够共同完成支持处理器单元渲染的图层的渲染，从而解决了Linux操作系统和Android子系统对使用处理器单元进行图层的渲染的兼容问题。
基于上述任一实施例，在本实施例中，所述通过Linux操作系统中的硬件混合渲染器客户端和所述硬件抽象层中的硬件混合渲染器服务端调用硬件混合渲染器，实现对所述第二类图层的渲染，包括：
将所述第二类图层传输至Linux操作系统中的硬件混合渲染器客户端；
由所述硬件混合渲染器客户端调用所述硬件抽象层中的硬件混合渲染器服务端；
通过所述硬件混合渲染器服务端调用所述硬件混合渲染器，实现对所述第二类图层的渲染。
如前所述，硬件混合渲染器客户端可以接收来自Linux操作系统中的关于第二类图层的渲染和合成的请求，且硬件混合渲染器服务端可以理解为Android子系统中的硬件抽象层中定义的一个用于控制硬件混合渲染器的服务，提供了与硬件混合渲染器相关的接口，可以调用硬件混合渲染器驱动器，从而使硬件混合渲染器完成第二类图层的渲染和合成。因此，可以将所述第二类图层传输至Linux操作系统中的硬件混合渲染器客户端，通过硬件混合渲染器客户端调用硬件混合渲染器服务端，由硬件混合渲染器服务端调用硬件混合渲染器实现对所述第二类图层的渲染。
本申请提供的图像生成方法，通过硬件混合渲染器客户端调用Android子系统中的硬件抽象层中的硬件混合渲染器服务端，使其调用硬件混合渲染器，实现对所述第二类图层的渲染，从而在Linux操作系统能够使用Android子系统中的硬件混合渲染器完成支持硬件混合渲染器渲染的图层的渲染，使Android子系统中的硬件混合渲染器能够协助处理器单元完成对图层的渲染和合成，从而减 少因为处理器单元的性能不足导致图像刷新率掉帧等问题，使Linux操作系统运行在采用ARM架构的智能手机或平板电脑上时，视频播放时也能具有较好的流畅度。
基于上述任一实施例，在本实施例中，所述由所述硬件混合渲染器客户端调用所述硬件抽象层中的硬件混合渲染器服务端，包括：
所述硬件混合渲染器客户端通过运行在Linux操作系统中的hybris以及运行在硬件抽象层中的hybris compat调用所述硬件抽象层中的硬件混合渲染器服务端。
可以理解，由于硬件混合渲染器服务端可理解为Android子系统中的硬件抽象层中定义的用于控制硬件混合渲染器的服务接口，服务接口是基于bionic编译的，因此，在Linux操作系统中调用时可以通过libhybris实现库兼容，也即hybris和hybris compat实现在linux系统下调用Android系统中硬件抽象层中的硬件混合渲染器服务端，使其调用硬件混合渲染器，实现对所述第二类图层的渲染。
本申请提供的图像生成方法，通过hybris和hybris compat调用Android子系统中的硬件抽象层中的硬件混合渲染器服务端，使其调用硬件混合渲染器，实现对所述第二类图层的渲染，从而在Linux操作系统能够使用Android子系统中的硬件混合渲染器完成支持硬件混合渲染器渲染的图层的渲染，使Android子系统中的硬件混合渲染器能够协助处理器单元完成对图层的渲染和合成，从而减少因为处理器单元的性能不足导致图像刷新率掉帧等问题，使Linux操作系统运行在采用ARM架构的智能手机或平板电脑上时，视频播放时也能具有较好的流畅度。
基于上述任一实施例，在本实施例中，所述通过所述硬件混合渲染器服务端对渲染后的第一类图层与渲染后的第二类图层进行合成，生成所述目标图像，包括：
将渲染后的第一类图层传输至Linux操作系统；
在Linux操作系统内，将所述渲染后的第一类图层传输至硬件混合渲染器客户端，由所述硬件混合渲染器客户端经由运行在Linux操作系统中的hybris以及运行在硬件抽象层中的hybris compat，将所述渲染后的第一类图层传输至所述硬件抽象层中的硬件混合渲染器服务端；
通过所述硬件混合渲染器服务端对渲染后的第一类图层与渲染后的第二类图层进行合成，生成所述目标图像。
可以理解，渲染后的第一类图层是至少一个图层分别通过vendor GLES接口进行渲染得到。因此，还需要将通过vendor GLES接口渲染之后的第一类图层发送给硬件混合渲染器服务端，使硬件混合渲染器服务端将其与第二类图层合成之后生成目标图像。
示例性地，可以将渲染后的至少一个第一类图层放入到缓存块中，并将其标记为已经渲染过的支持处理器单元渲染的图层，由硬件混合渲染器客户端通过调用libhybris中的相关接口，将所述缓存块设置给硬件混合渲染器服务端，由硬件混合渲染器服务端基于硬件混合渲染器客户端的调用指令调用硬件混合渲染器，完成支持硬件混合渲染器渲染的图层与已经渲染过的支持处理器单元渲染的图层的渲染及合成。
本申请提供的图像生成方法，通过使硬件混合渲染器服务端将第一类图层和第二图层合成，使Android子系统中的硬件混合渲染器与处理器单元共同完成对图层的渲染和合成，从而减少因为处理器单元的性能不足导致图像刷新率掉帧等问题，使Linux操作系统运行在采用ARM架构的智能手机或平板电脑上时，视频播放时也能具有较好的流畅度。
基于上述任一实施例，在本实施例中，在所述根据待生成的目标图像中的多个图层的渲染需求，将所述图层分为第一类图层与第二类图层之前，方法还包括：
初始化并启动部署在Linux操作系统中的Android子系统；
在启动Android子系统的过程中，启动硬件抽象层中的硬件混合渲染器服务 端；
在Linux操作系统中，启动硬件混合渲染器客户端。
可以理解，Android子系统可以通过容器的方式部署在Linux操作系统中，在容器中初始化并启动Android子系统。在启动Android子系统的过程中，自动加载硬件混合渲染器驱动，便于后续通过硬件混合渲染器驱动访问硬件混合渲染器。并且由于在Android子系统中对应的硬件混合渲染器客户端为surfaceflinger，但在Linux操作系统中对应的控制图层的渲染与合成的模块为compositor，因此，在Linux操作系统中，将硬件混合渲染器客户端修改为compositor。
本申请提供的图像生成方法，启动Android子系统及硬件混合渲染器服务端和硬件混合渲染器客户端，通过它们使硬件混合渲染器将第一类图层和第二图层合成，使Android子系统中的硬件混合渲染器与处理器单元共同完成对图层的渲染和合成，从而减少因为处理器单元的性能不足导致图像刷新率掉帧等问题，使Linux操作系统运行在采用ARM架构的智能手机或平板电脑上时，视频播放时也能具有较好的流畅度。
图2为本申请提供的图像生成方法的流程示意图之二。图2以Linux合成器模块compositor和Android HAL之间的交互过程描述本申请提供的图像生成方法。其中，compositor可以理解为Linux系统中用于控制图层的渲染与合成的模块；窗口管理器(window manager)负责窗口中图层的生成与分类；硬件混合渲染器客户端(hwc client)对应compositor；hybris和hybris compat分别为libhybris的两个组成部分，构成compositor和Android HAL的交互通道，使两者可以通过交互共同完成图层合成；硬件混合渲染器客户端设置为将图层中的数据通过libhybris调用硬件混合渲染器服务端(hwc server)中的相关接口完成图层的渲染及合成；GLES线程中预设有钩子函数，钩子函数用于调用Android HAL中的vendor GLES，实现支持处理器单元渲染的图层(gles layer)的渲染，vendor GLES接口设置为调用处理器单元完成支持处理器单元渲染的图层的渲染，并将渲染 后的图层保存到目标缓存区。硬件混合渲染器服务端基于硬件混合渲染器客户端的调用指令调用硬件混合渲染器完成相应的功能。下面结合图2介绍图像生成方法。
首先，compositor中的窗口管理器接收到目标图像包括的内容，所述目标图像可以包括：应用程序(application)对应的界面、鼠标对应的光标(cursor)、状态栏(statusbar)以及背景(background)。
其次，窗口管理器将目标图像包括的内容分别对应的图层进行区分，得到不同类型的图层。
其中，不同类型的图层，例如可以包括：支持处理器单元渲染的图层(gles layer)、已经渲染过的支持处理器单元渲染的图层(hwc client layer)、支持硬件混合渲染器渲染的图层(hwc device layer、hwc color layer和hwc cursor layer)。
可以理解，在此之前，compositor可以根据目标图像包括的内容通过libhybris调用硬件混合渲染器服务端的相关接口创建至少一个图层并设置所述至少一个图层的信息，并通过libhybris让硬件混合渲染器根据所述图层的信息确定所述至少一个图层中支持处理器单元渲染的图层和支持硬件混合渲染器渲染的图层，并将图层的信息返回给窗口管理器。
创建至少一个图层并设置所述至少一个图层的信息的过程将在下文详述。
通过libhybris的接口调用硬件混合渲染器服务端的相关接口，首先需要通过libhybris建立Linux系统与硬件混合渲染器之间的通道，使linux系统可以访问Android系统的硬件混合渲染器。
可以理解，libhybris框架中包含hwc_compat.so，hwc_compat.so使用bionic编译，运行在Android环境下，其内部包含一些用于调用Android接口的函数，使linux系统可以访问Android系统的硬件混合渲染器。
示例性地，在使用硬件混合渲染器服务端的相关接口时，将相关接口导出到Linux头文件中，在Linux的合成器模块compositor中引用头文件，调用其方法时可以通过android_dlopen的方式加载hwc_compat.so库，以此来实现在Linux (glibc环境)调用Android(bionic环境)中的硬件混合渲染器服务端的相关接口。根据目标图像中包含的内容，通过libhybris的第一接口调用所述硬件混合渲染器服务端的相关接口中的第二接口创建至少一个图层；调用libhybris的至少一个第三接口设置所述至少一个图层的信息，所述信息包括图层的渲染类型。
其中，所述libhybris的第一接口和第三接口例如均可以为前述hwc_compat.so中的一个接口。libhybris的第一接口设置为调用硬件混合渲染器服务端的相关接口，所述硬件混合渲染器服务的第二接口设置为创建至少一个预设渲染类型的图层。libhybris的第三接口设置为设置所述至少一个图层的信息。所述预设渲染类型的图层可以为：支持硬件混合渲染器渲染的图层和支持处理器单元渲染的图层。
可以理解，通过libhybris的接口调用所述硬件混合渲染器服务端的相关接口，首先执行前面所述的通过libhybris建立Linux系统与硬件混合渲染器之间的通道，与前述的通过实现Linux(glibc环境)调用Android(bionic环境)的相关接口的方法类似，此处不再一一赘述。
示例性地，libhybris的第一接口例如可以为前述hwc_compat.so中的一个接口，例如可以为hwc2_compat_display_create_layer接口，硬件混合渲染器服务端的第二接口例如可以为hwc2_display_create_layer。
其中，图层的信息中包括图层的渲染类型，与前面所述的不同类型的图层对应，可以为：支持硬件混合渲染器渲染的图层、已经渲染过的支持处理器单元渲染的图层和支持处理器单元渲染的图层。
示例性地，若调用libhybris的第三接口设置所述至少一个图层的渲染类型，则libhybris的第三接口例如可以为hwc2_compat_layer_set_composition_type接口。
此外，图层的信息还可以包括图层的显示区域参数。
示例性地，若调用libhybris的第三接口设置所述至少一个图层分别对应的显示区域参数，则libhybris的第三接口例如可以为前述hwc_compat.so中的一个 接口，例如可以为hwc2_compat_layer_set_source_crop接口、hwc2_compat_layer_set_display_frame接口或hwc2_compat_layer_set_visible_region接口。
可以理解，在进行图层渲染和合成之前，需要根据预设的通信方式将硬件混合渲染器的图层合成的控制信号(VSync信号)接入到Linux的合成器模块，用于控制图层的渲染与合成。
其中，预设的通信方式例如可以为Binder通信。Binder通信用于进程间通信。Linux的合成器模块compositor，对应一个客户端进程。可以理解，由于Android子系统运行在容器中，因此Linux系统的合成器模块和Android子系统的硬件混合渲染器服务处于不同的进程，因此，两者需要通过Binder进行通信。
如前所述，图层合成的控制信号由硬件混合渲染器发出，图层合成的控制信号用于控制图层的渲染和合成。因此，在图层合成的控制信号到来时，将执行图层的渲染及合成过程。下面详细介绍图层的渲染与合成过程。
如前所述，图层在创建时会被设置对应的渲染类型，因此在渲染的过程中需要先确定图层的渲染类型，才能根据已经设置好的渲染类型将图层分配给不同的合成硬件进行合成。
可选地，确定所述至少一个图层中支持处理器单元渲染的图层和支持硬件混合渲染器渲染的图层可以为：调用libhybris的第四接口让硬件混合渲染器确定所述至少一个图层中支持处理器单元渲染的图层和支持硬件混合渲染器渲染的图层。
需要说明的是，libhybris的第四接口可以为hwc_compat.so中的一个接口，设置为确定所述至少一个图层的渲染类型。第四接口例如可以为hwc2_compat_display_validate。
然后，compositor通过libhybris中的hwc_compat.so调用所述硬件混合渲染器服务端中的相关接口让硬件混合渲染器根据所述图层的渲染类型确定所述至少一个图层中支持处理器单元渲染的图层和支持硬件混合渲染器渲染的图层， 并根据图层的渲染类型将不同的图层分别发送给GELS线程和硬件混合渲染器客户端，由GELS线程和硬件混合渲染器客户端分别转发至vendor GELS和硬件混合渲染器服务端。
示例性地，对应图2的过程为：将不同类型的图层分别发送给硬件混合渲染器客户端和GLES线程，由硬件混合渲染器客户端和GLES线程通过hybris和hybris compat分别调用硬件混合渲染器服务端和vendor GLES。由vendor GLES完成支持处理器单元渲染的图层的渲染，并将支持处理器单元渲染的图层渲染后的图层保存在缓存块中，返回给窗口管理器，窗口管理器将所述支持处理器单元渲染的图层渲染后的图层作为已经渲染过的支持处理器单元渲染的图层。由硬件混合渲染器服务端基于硬件混合渲染器客户端的调用指令调用硬件混合渲染器，完成支持硬件混合渲染器渲染的图层与已经渲染过的支持处理器单元渲染的图层的渲染及合成。过程可参考下文的相关描述。
示例性地，可以通过调用libhybris的第五接口把缓存块设置给硬件混合渲染器；调用libhybris的第六接口让硬件混合渲染器完成对已经渲染过的支持处理器单元渲染的图层和已经渲染过的支持硬件混合渲染器渲染的图层合成，得到目标图像。
其中，libhybris的第五接口和第六接口分别为hwc_compat.so中的一个接口。第五接口例如可以为hwc2_compat_display_set_client_targe，第六接口例如可以为hwc2_compat_display_present。
最后，将目标图像送到显示屏上显示。
下面对本申请提供的图像生成装置进行描述，下文描述的基于图像生成装置与上文描述的图像生成方法可相互对应参照。
图3是本申请提供的图像生成装置的示意图，如图3所示，本申请实施例提供的图像生成装置，应用于部署有Android子系统的Linux操作系统中，所述Android子系统包括硬件抽象层；所述装置包括：
区分模块310，设置为根据待生成的目标图像中的多个图层的渲染需求，将 所述多个图层分为第一类图层与第二类图层；其中，所述第一类图层为支持处理器单元渲染的图层，所述第二类图层为支持硬件混合渲染器渲染的图层；
第一处理模块320，设置为通过在GLES线程中预设的钩子函数调用所述处理器单元，实现对所述第一类图层的渲染；
第二处理模块330，设置为通过Linux操作系统中的硬件混合渲染器客户端和所述硬件抽象层中的硬件混合渲染器服务端调用硬件混合渲染器，实现对所述第二类图层的渲染；
生成模块340，设置为通过所述硬件混合渲染器服务端对渲染后的第一类图层与渲染后的第二类图层进行合成，生成所述目标图像。
本申请提供的图像生成装置，通过将目标图像分为支持处理器单元渲染的图层和支持硬件混合渲染器渲染的图层，并且分别针对这两类图层，采用相应的方法调用相应的硬件分别完成这两类图层的渲染，使Android子系统中的硬件混合渲染器能够协助处理器单元完成对图层的渲染和合成，从而减少因为处理器单元的性能不足导致图像刷新率掉帧等问题，使Linux操作系统运行在采用ARM架构的智能手机或平板电脑上时，视频播放时也能具有较好的流畅度。
基于上述任一实施例，在本实施例中，所述第一处理模块320包括：
第一传输单元，设置为将所述第一类图层传输至Linux操作系统中的GLES线程；
第一调用单元，设置为通过所述GLES线程中的预设钩子函数调用所述硬件抽象层中的第一接口；
第一渲染单元，设置为通过所述第一接口调用处理器单元，实现对所述第一类图层的渲染。
本申请提供的图像生成装置，通过在GLES线程中预设钩子函数，通过钩子函数实现对硬件抽象层中支持处理器单元渲染的图层的渲染接口，完成支持处理器单元渲染的图层的渲染，从而实现了Linux操作系统和Android子系统使用处理器单元进行图层的渲染。
基于上述任一实施例，在本实施例中，所述第一调用单元包括：
第二调用单元，设置为所述GLES线程中的预设钩子函数通过运行在Linux操作系统中的hybris以及运行在所述硬件抽象层中的hybris compat调用所述硬件抽象层中的vendor GLES接口。
本申请提供的图像生成装置，通过在GLES线程中预设钩子函数，并且通过hybris和hybris compat调用Android子系统中的硬件抽象层中的vendor GLES接口，从而使Linux操作系统与Android子系统能够共同完成支持处理器单元渲染的图层的渲染，从而解决了Linux操作系统和Android子系统对使用处理器单元进行图层的渲染的兼容问题。
基于上述任一实施例，在本实施例中，所述第二处理模块330包括：
第二传输单元，设置为将所述第二类图层传输至Linux操作系统中的硬件混合渲染器客户端；
第三调用单元，设置为由所述硬件混合渲染器客户端调用所述硬件抽象层中的硬件混合渲染器服务端；
第二渲染单元，设置为通过所述硬件混合渲染器服务端调用硬件混合渲染器，实现对所述第二类图层的渲染。
本申请提供的图像生成装置，通过硬件混合渲染器客户端调用Android子系统中的硬件抽象层中的硬件混合渲染器服务端，使其调用硬件混合渲染器，实现对所述第二类图层的渲染，从而在Linux操作系统能够使用Android子系统中的硬件混合渲染器完成支持硬件混合渲染器渲染的图层的渲染，使Android子系统中的硬件混合渲染器能够协助处理器单元完成对图层的渲染和合成，从而减少因为处理器单元的性能不足导致图像刷新率掉帧等问题，使Linux操作系统运行在采用ARM架构的智能手机或平板电脑上时，视频播放时也能具有较好的流畅度。
基于上述任一实施例，在本实施例中，所述第三调用单元包括：
第四调用单元，设置为所述硬件混合渲染器客户端通过运行在Linux操作系 统中的hybris以及运行在硬件抽象层中的hybris compat调用所述硬件抽象层中的硬件混合渲染器服务端。
本申请提供的图像生成装置，通过hybris和hybris compat调用Android子系统中的硬件抽象层中的硬件混合渲染器服务端，使其调用硬件混合渲染器，实现对所述第二类图层的渲染，从而在Linux操作系统能够使用Android子系统中的硬件混合渲染器完成支持硬件混合渲染器渲染的图层的渲染，使Android子系统中的硬件混合渲染器能够协助处理器单元完成对图层的渲染和合成，从而减少因为处理器单元的性能不足导致图像刷新率掉帧等问题，使Linux操作系统运行在采用ARM架构的智能手机或平板电脑上时，视频播放时也能具有较好的流畅度。
基于上述任一实施例，在本实施例中，所述生成模块340包括：
第三传输单元，设置为将渲染后的第一类图层传输至Linux操作系统；
第四传输单元，设置为在Linux操作系统内，将所述渲染后的第一类图层传输至硬件混合渲染器客户端，由所述硬件混合渲染器客户端经由运行在Linux操作系统中的hybris以及运行在硬件抽象层中的hybris compat，将所述渲染后的第一类图层传输至所述硬件抽象层中的硬件混合渲染器服务端；
合成单元，设置为通过所述硬件混合渲染器服务端对渲染后的第一类图层与渲染后的第二类图层进行合成，生成所述目标图像。
本申请提供的图像生成装置，通过使硬件混合渲染器服务端将第一类图层和第二图层合成，使Android子系统中的硬件混合渲染器与处理器单元共同完成对图层的渲染和合成，从而减少因为处理器单元的性能不足导致图像刷新率掉帧等问题，使Linux操作系统运行在采用ARM架构的智能手机或平板电脑上时，视频播放时也能具有较好的流畅度。
基于上述任一实施例，在本实施例中，所述装置还包括：
Android子系统启动模块，设置为初始化并启动部署在Linux操作系统中的Android子系统；
第一启动模块，设置为在启动Android子系统的过程中，启动硬件抽象层中的硬件混合渲染器服务端；
第二启动模块，设置为在Linux操作系统中，启动硬件混合渲染器客户端。
本申请提供的图像生成装置，通过启动Android子系统及硬件混合渲染器服务端和硬件混合渲染器客户端，通过它们使硬件混合渲染器将第一类图层和第二图层合成，使Android子系统中的硬件混合渲染器与处理器单元共同完成对图层的渲染和合成，从而减少因为处理器单元的性能不足导致图像刷新率掉帧等问题，使Linux操作系统运行在采用ARM架构的智能手机或平板电脑上时，视频播放时也能具有较好的流畅度。
图4示例了一种电子设备的实体结构示意图，如图4所示，该电子设备可以包括：处理器(processor)410、通信接口(Communications Interface)420、存储器(memory)430和通信总线440，其中，处理器410，通信接口420，存储器430通过通信总线440完成相互间的通信。处理器410可以调用存储器430中的逻辑指令，以执行本申请中的图像生成方法，该方法包括：根据待生成的目标图像中的多个图层的渲染需求，将所述多个图层分为第一类图层与第二类图层；其中，所述第一类图层为支持处理器单元渲染的图层，所述第二类图层为支持硬件混合渲染器渲染的图层；通过在GLES线程中预设的钩子函数调用所述处理器单元，实现对所述第一类图层的渲染；通过Linux操作系统中的硬件混合渲染器客户端和所述硬件抽象层中的硬件混合渲染器服务端调用硬件混合渲染器，实现对所述第二类图层的渲染；通过所述硬件混合渲染器服务端合成渲染后的第一类图层与渲染后的第二类图层，生成所述目标图像。
此外，上述的存储器430中的逻辑指令可以通过软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对相关技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计 算机，服务器，或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。
另一方面，本申请还提供一种计算机程序产品，所述计算机程序产品包括存储在非暂态计算机可读存储介质上的计算机程序，所述计算机程序包括程序指令，当所述程序指令被计算机执行时，计算机能够执行本申请所提供的图像生成方法，该方法包括：根据待生成的目标图像中的多个图层的渲染需求，将所述多个图层分为第一类图层与第二类图层；其中，所述第一类图层为支持处理器单元渲染的图层，所述第二类图层为支持硬件混合渲染器渲染的图层；通过在GLES线程中预设的钩子函数调用所述处理器单元，实现对所述第一类图层的渲染；通过Linux操作系统中的硬件混合渲染器客户端和所述硬件抽象层中的硬件混合渲染器服务端调用硬件混合渲染器，实现对所述第二类图层的渲染；通过所述硬件混合渲染器服务端合成渲染后的第一类图层与渲染后的第二类图层，生成所述目标图像。
又一方面，本申请还提供一种非暂态计算机可读存储介质，其上存储有计算机程序，该计算机程序被处理器执行时实现以执行本申请所提供的图像生成方法，该方法包括：根据待生成的目标图像中的多个图层的渲染需求，将所述多个图层分为第一类图层与第二类图层；其中，所述第一类图层为支持处理器单元渲染的图层，所述第二类图层为支持硬件混合渲染器渲染的图层；通过在GLES线程中预设的钩子函数调用所述处理器单元，实现对所述第一类图层的渲染；通过Linux操作系统中的硬件混合渲染器客户端和所述硬件抽象层中的硬件混合渲染器服务端调用硬件混合渲染器，实现对所述第二类图层的渲染；通过所述硬件混合渲染器服务端合成渲染后的第一类图层与渲染后的第二类图层，生成所述目标图像。
以上所描述的装置实施例仅仅是示意性的，其中所述作为分离部件说明的 单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。本领域普通技术人员在不付出创造性的劳动的情况下，即可以理解并实施。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到各实施方式可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件。基于这样的理解，上述技术方案本质上或者说对相关技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品可以存储在计算机可读存储介质中，如ROM/RAM、磁碟、光盘等，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行各个实施例或者实施例的某些部分所述的方法。

Claims (10)

  1. 一种图像生成方法，应用于部署有Android子系统的Linux操作系统中，所述Android子系统包括硬件抽象层；方法包括：
    根据待生成的目标图像中的多个图层的渲染需求，将所述多个图层分为第一类图层与第二类图层；其中，所述第一类图层为支持处理器单元渲染的图层，所述第二类图层为支持硬件混合渲染器渲染的图层；
    通过在GLES线程中预设的钩子函数调用所述处理器单元，实现对所述第一类图层的渲染；
    通过Linux操作系统中的硬件混合渲染器客户端和所述硬件抽象层中的硬件混合渲染器服务端调用硬件混合渲染器，实现对所述第二类图层的渲染；
    通过所述硬件混合渲染器服务端对渲染后的第一类图层与渲染后的第二类图层进行合成，生成所述目标图像。
  2. 根据权利要求1所述的图像生成方法，其中，所述通过在GLES线程中预设的钩子函数调用处理器单元，实现对所述第一类图层的渲染，包括：
    将所述第一类图层传输至Linux操作系统中的GLES线程；
    通过所述GLES线程中的预设钩子函数调用所述硬件抽象层中的第一接口；
    通过所述第一接口调用处理器单元，实现对所述第一类图层的渲染。
  3. 根据权利要求2所述的图像生成方法，其中，所述第一接口为所述硬件抽象层中的vendor GLES接口，相应的，所述通过所述GLES线程中的预设钩子函数调用所述硬件抽象层中的第一接口，包括：
    所述GLES线程中的预设钩子函数通过运行在Linux操作系统中的hybris以及运行在所述硬件抽象层中的hybris compat调用所述硬件抽象层中的vendor GLES接口。
  4. 根据权利要求1所述的图像生成方法，其中，所述通过Linux操作系统中的硬件混合渲染器客户端和所述硬件抽象层中的硬件混合渲染器服务端调用硬件混合渲染器，实现对所述第二类图层的渲染，包括：
    将所述第二类图层传输至Linux操作系统中的硬件混合渲染器客户端；
    由所述硬件混合渲染器客户端调用所述硬件抽象层中的硬件混合渲染器服务端；
    通过所述硬件混合渲染器服务端调用所述硬件混合渲染器，实现对所述第二类图层的渲染。
  5. 根据权利要求4所述的图像生成方法，其中，所述由所述硬件混合渲染器客户端调用所述硬件抽象层中的硬件混合渲染器服务端，包括：
    所述硬件混合渲染器客户端通过运行在Linux操作系统中的hybris以及运行在硬件抽象层中的hybris compat调用所述硬件抽象层中的硬件混合渲染器服务端。
  6. 根据权利要求1所述的图像生成方法，其中，所述通过所述硬件混合渲染器服务端对渲染后的第一类图层与渲染后的第二类图层进行合成，生成所述目标图像，包括：
    将渲染后的第一类图层传输至Linux操作系统；
    在Linux操作系统内，将所述渲染后的第一类图层传输至硬件混合渲染器客户端，由所述硬件混合渲染器客户端经由运行在Linux操作系统中的hybris以及运行在硬件抽象层中的hybris compat，将所述渲染后的第一类图层传输至所述硬件抽象层中的硬件混合渲染器服务端；
    通过所述硬件混合渲染器服务端对渲染后的第一类图层与渲染后的第二类图层进行合成，生成所述目标图像。
  7. 根据权利要求1至6任一项所述的图像生成方法，在所述根据待生成的目标图像中的多个图层的渲染需求，将所述多个图层分为第一类图层与第二类图层之前，还包括：
    初始化并启动部署在Linux操作系统中的Android子系统；
    在启动Android子系统的过程中，启动硬件抽象层中的硬件混合渲染器服务端；
    在Linux操作系统中，启动硬件混合渲染器客户端。
  8. 一种图像生成方法装置，应用于部署有Android子系统的Linux操作系统中，所述Android子系统包括硬件抽象层；所述装置包括：
    区分模块，设置为根据待生成的目标图像中的多个图层的渲染需求，将所述多个图层分为第一类图层与第二类图层；其中，所述第一类图层为支持处理器单元渲染的图层，所述第二类图层为支持硬件混合渲染器渲染的图层；
    第一处理模块，设置为通过在GLES线程中预设的钩子函数调用处理器单元，实现对所述第一类图层的渲染；
    第二处理模块，设置为通过Linux操作系统中的硬件混合渲染器客户端和所述硬件抽象层中的硬件混合渲染器服务端调用硬件混合渲染器，实现对所述第二类图层的渲染；
    生成模块，设置为通过所述硬件混合渲染器服务端对渲染后的第一类图层与渲染后的第二类图层进行合成，生成所述目标图像。
  9. 一种电子设备，包括存储器、处理器及存储在所述存储器上并可在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现如权利要求1至7任一项所述的图像生成方法。
  10. 一种非暂态计算机可读存储介质，所述非暂态计算机可读存储介质上存储有计算机程序，所述计算机程序被处理器执行时实现如权利要求1至7任一项所述的图像生成方法。
PCT/CN2022/108439 2022-01-19 2022-07-28 图像生成方法、装置、电子设备及存储介质 WO2023138020A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210057895.7A CN114092312A (zh) 2022-01-19 2022-01-19 图像生成方法、装置、电子设备及存储介质
CN202210057895.7 2022-01-19

Publications (1)

Publication Number Publication Date
WO2023138020A1 true WO2023138020A1 (zh) 2023-07-27

Family

ID=80308576

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/108439 WO2023138020A1 (zh) 2022-01-19 2022-07-28 图像生成方法、装置、电子设备及存储介质

Country Status (2)

Country Link
CN (1) CN114092312A (zh)
WO (1) WO2023138020A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114092312A (zh) * 2022-01-19 2022-02-25 北京鲸鲮信息系统技术有限公司 图像生成方法、装置、电子设备及存储介质

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140366021A1 (en) * 2013-06-07 2014-12-11 American Megatrends, Inc. Methods, Devices and Computer Readable Storage Devices for Emulating an Accelerometer in a Guest Operating System from a Host Operating System
WO2017167127A1 (zh) * 2016-03-31 2017-10-05 阿里巴巴集团控股有限公司 一种图形合成方法、窗口设置方法及系统
CN111768330A (zh) * 2019-03-30 2020-10-13 华为技术有限公司 图像处理方法及计算机系统
CN112099923A (zh) * 2020-11-17 2020-12-18 江苏北弓智能科技有限公司 一种运行于Linux操作系统上的Android虚拟化系统
CN112200712A (zh) * 2020-09-08 2021-01-08 成都安易迅科技有限公司 Gles图像渲染方法及装置、存储介质、计算机设备
CN113110910A (zh) * 2021-04-20 2021-07-13 上海卓易科技股份有限公司 一种安卓容器实现的方法、系统及设备
CN114092312A (zh) * 2022-01-19 2022-02-25 北京鲸鲮信息系统技术有限公司 图像生成方法、装置、电子设备及存储介质

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112269603B (zh) * 2020-10-16 2021-05-28 北京技德系统技术有限公司 一种在Linux上兼容运行Android应用的图形显示方法与装置

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140366021A1 (en) * 2013-06-07 2014-12-11 American Megatrends, Inc. Methods, Devices and Computer Readable Storage Devices for Emulating an Accelerometer in a Guest Operating System from a Host Operating System
WO2017167127A1 (zh) * 2016-03-31 2017-10-05 阿里巴巴集团控股有限公司 一种图形合成方法、窗口设置方法及系统
CN111768330A (zh) * 2019-03-30 2020-10-13 华为技术有限公司 图像处理方法及计算机系统
CN112200712A (zh) * 2020-09-08 2021-01-08 成都安易迅科技有限公司 Gles图像渲染方法及装置、存储介质、计算机设备
CN112099923A (zh) * 2020-11-17 2020-12-18 江苏北弓智能科技有限公司 一种运行于Linux操作系统上的Android虚拟化系统
CN113110910A (zh) * 2021-04-20 2021-07-13 上海卓易科技股份有限公司 一种安卓容器实现的方法、系统及设备
CN114092312A (zh) * 2022-01-19 2022-02-25 北京鲸鲮信息系统技术有限公司 图像生成方法、装置、电子设备及存储介质

Also Published As

Publication number Publication date
CN114092312A (zh) 2022-02-25

Similar Documents

Publication Publication Date Title
US8957905B2 (en) Cross-environment user interface mirroring
US9047102B2 (en) Instant remote rendering
US8819705B2 (en) User interaction support across cross-environment applications
CN112269603B (zh) 一种在Linux上兼容运行Android应用的图形显示方法与装置
KR101394094B1 (ko) 응용 프로그램 화상의 표시 방법 및 장치
JP5149411B2 (ja) グラフィックス処理システムにおける統一された合成エンジンのためのシステムおよび方法
US20120086716A1 (en) User interaction across cross-environment applications through an extended graphics context
CN111240626A (zh) 一种基于Hypervisor智能座舱操作系统双屏互动的方法和系统
WO2021013019A1 (zh) 一种图片处理方法及装置
WO2017167127A1 (zh) 一种图形合成方法、窗口设置方法及系统
WO2023279993A1 (zh) 图形渲染方法、装置、电子设备与存储介质
CN114741044B (zh) 一种基于异构渲染的跨运行环境显示输出共享方法
CN113051047B (zh) 识别安卓系统绘制线程的方法、装置、移动终端及存储介质
WO2023138020A1 (zh) 图像生成方法、装置、电子设备及存储介质
WO2023093776A1 (zh) 界面生成方法及电子设备
US20240161670A1 (en) Dynamic refresh rate switching
WO2023221822A1 (zh) 数据处理方法、电子设备和可读存储介质
WO2023035619A1 (zh) 一种场景渲染方法、装置、设备及系统
CN113051032A (zh) 应用画面处理方法、装置和系统
WO2019079940A1 (zh) 图形处理方法及相关装置和设备
CN117724797B (zh) 一种基于异构混合渲染的自由缩放优化方法
WO2024016798A9 (zh) 图像显示方法和相关装置
WO2024022257A1 (zh) 一种内容显示方法、设备及系统
CN114968229A (zh) 画面的渲染方法，介质和电子设备
CN116700860A (zh) 基于Web操作系统上的原生窗口实现窗口绘制的方法及系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22921449

Country of ref document: EP

Kind code of ref document: A1