CN113838180A

CN113838180A - Rendering instruction processing method and related equipment thereof

Info

Publication number: CN113838180A
Application number: CN202010587225.7A
Authority: CN
Inventors: 尚钦; 张志博; 彭新超
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2020-06-24
Filing date: 2020-06-24
Publication date: 2021-12-24

Abstract

The embodiment of the application discloses a rendering instruction processing method, which is applied to terminal equipment, wherein a GPU (graphics processing Unit) in the terminal equipment supports target expansion in openGL ES; in the application, a CPU can obtain a first rendering instruction which is not realized based on the target extension, the first rendering instruction is used for realizing a rendering task, a second rendering instruction is realized based on the target extension, the second rendering instruction is used for realizing the rendering task, and the GPU is triggered to execute the rendering task according to the second rendering instruction. According to the method and the device, the rendering instruction based on GPU expansion is used for replacing the rendering instruction which is not based on GPU expansion originally, and the performance loss of the terminal device is reduced.

Description

Rendering instruction processing method and related equipment thereof

Technical Field

The present application relates to the field of computers, and in particular, to a rendering instruction processing method and related apparatus.

Background

OpenGL (open graphics library) refers to a professional graphical program interface defining a cross-programming language and cross-platform programming interface specification, and is applied to the fields of industries including content creation, energy, entertainment, game development, manufacturing industry, pharmaceutical industry, virtual reality and the like, and OpenGL can help programmers to realize the development of high-performance and high-visual-expressive graphics processing software on hardware devices such as Personal Computers (PCs), workstations, supercomputers and the like. OpenGL (OpenGL for embedded systems) is a subset of an OpenGL three-dimensional graphics Application Program Interface (API), and is designed for embedded devices such as mobile phones and game hosts.

The problems of frame dropping and power consumption of games are always the pain problems which bother mobile phone manufacturers. Graphics rendering technology is also continuously advancing, OpenGL ES is evolving from 1.0 to 3.2, supporting more features, and also continuously improving rendering efficiency. GPU manufacturers are also actively spreading various technologies for reducing loads, improving image quality, and the like, and pushing out OpenGL ES extensions based on their own GPU characteristics. New extensions are derived to promote rendering efficiency, promote GPU self-owned features, and so on.

However, there is a long evolution of the extension before entering the standard, and the game engine will not generally adapt to the latest extension until it becomes the standard or is approved by the standard organization, which results in additional terminal device performance loss and poor rendering.

Disclosure of Invention

In a first aspect, the present application provides a rendering instruction processing method, where the method is applied to a terminal device, where the terminal device includes a GPU, and the GPU supports a target extension in an open graphics library openGL ES of an embedded system, and the method includes:

obtaining a first rendering instruction which is not realized based on the target extension, wherein the first rendering instruction is used for realizing a rendering task; acquiring a second rendering instruction according to the first rendering instruction; wherein the second rendering instruction is a rendering instruction implemented based on the target extension, and the second rendering instruction is used for implementing the rendering task;

taking an application program needing to perform graphic rendering as an example of a game, when the game is started, a rendering instruction for a rendering task in the game can be acquired, and then the rendering task corresponding to the rendering instruction is executed; wherein the first rendering instruction is one of the fetched rendering instructions; in the embodiment of the application, the terminal device may pre-configure some rules, where the rules define preset conditions that the instruction should meet, and if the first rendering instruction meets the preset conditions, it may indicate that the current rendering scene needs to be optimized; in this embodiment, the first rendering instruction is configured not to be implemented based on the target extension, and the first rendering instruction and the second rendering instruction are used to implement the same rendering task;

that is, the current GPU supports the target extension on which the second rendering instruction depends, but in the application, the second rendering instruction is not used but the first rendering instruction that is realized without depending on the target extension is used for the rendering task, and at this time, the second rendering instruction can be obtained and used as a subsequent rendering instruction for executing the rendering task;

in an alternative implementation, the device power consumption for triggering the GPU to execute the rendering task according to the first rendering instruction is greater than the device power consumption for triggering the GPU to execute the rendering task according to the second rendering instruction; and/or the memory data copy amount corresponding to the condition that the GPU is triggered to execute the rendering task according to the first rendering instruction is larger than the memory data copy amount corresponding to the condition that the GPU is triggered to execute the rendering task according to the second rendering instruction; and/or the corresponding GPU load when the GPU is triggered to execute the rendering task according to the first rendering instruction is larger than the corresponding GPU load when the GPU is triggered to execute the rendering task according to the second rendering instruction; and/or the CPU load corresponding to the GPU when the first rendering instruction triggers the GPU to execute the rendering task is larger than the CPU load corresponding to the GPU when the second rendering instruction triggers the GPU to execute the rendering task.

That is, the rendering efficiency of the second rendering instruction is better than that of the first rendering task under the condition of executing the same rendering task;

triggering the GPU to execute the rendering task according to the second rendering instruction;

in an alternative implementation, after acquiring the second rendering instruction, the CPU transmits the second rendering instruction to the GPU driver. For example, the GPU driver may compile second rendering instructions into GPU executable objects or machine code, after which the second rendering instructions can be executed by the GPU, and the GPU may execute the corresponding rendering task after receiving the compiled second rendering instructions.

In the embodiment of the application, the game can reduce the terminal performance loss caused by frequent memory mapping and the like or reduce the load of a CPU/GPU by executing the instruction realized based on GPU expansion without modifying codes. Meanwhile, the embodiment of the application is a general expansion optimization framework based on GPU expansion, the performance of a mobile phone supporting the expansion can be improved, optimization cannot be performed on a mobile phone not supporting the expansion, and the compatibility is better.

In an alternative implementation, the fetching second rendering instructions according to the first rendering instructions includes:

and acquiring the second rendering instruction corresponding to the first rendering instruction based on a mapping relation, wherein the mapping relation comprises a preset corresponding relation between the first rendering instruction and the second rendering instruction.

The CPU may obtain the second rendering instruction corresponding to the first rendering instruction based on a mapping relationship, where the mapping relationship includes a preset corresponding relationship between the first rendering instruction and the second rendering instruction, where the mapping relationship includes a corresponding relationship between rendering instructions, and in the mapping relationship, the first rendering instruction corresponds to the second rendering instruction.

In an alternative implementation, the fetching of the second rendering instruction according to the first rendering instruction comprises: when the first rendering instruction meets a preset condition, acquiring the second rendering instruction according to the first rendering instruction; the preset conditions include: the first rendering instruction belongs to a preset instruction set.

The terminal device may specify in advance instructions that are not implemented based on the target extension, which are less efficient in rendering, and the current GPU supports an extension based on which there may be another instruction that can implement the same rendering task. At this time, the terminal device may specify some instructions in advance as candidate optimization instructions (for example, the first rendering instruction in this embodiment), and correspondingly, may also specify optimization instructions in advance as candidate optimization instructions (for example, the second rendering instruction in this embodiment).

It should be noted that the first rendering instruction may be an instruction set including a plurality of instructions, and at this time, the preset instruction set may include a plurality of instructions and a time sequence order between the instructions, and it may be understood that the first rendering instruction belongs to the preset instruction set, and the first rendering instruction includes names of the plurality of instructions and the execution order thereof belongs to the preset instruction set.

In an alternative implementation, the first rendering instruction is for operating a target cache, and the preset condition includes at least one of:

the cache type of the target cache meets a preset condition;

the historical information of the target cache operated by the first rendering instruction meets a preset condition; or the like, or, alternatively,

and the context information of the target cache meets a preset condition.

It should be noted that, the type of the target cache meets a preset condition, which may be whether the type of data stored in the target cache meets the condition, or whether some attributes of the target cache meet the condition, for example, when the first rendering instruction is a glBufferData instruction, it may be determined whether the target cache is used for storing vertex or index data, and whether its own purpose is dynamically updated, and if so, it is determined that the cache type of the target cache meets the preset condition.

It should be noted that, the number of historical operations performed on the target cache by the first rendering instruction is greater than a preset value; whether the historical operation times of the first rendering instruction are larger than a preset value within a certain time of the target cache can be obtained. For example, when the first rendering instruction is a glBlitFramebuffer instruction, it may be determined whether the number of times (frequency) that the bound Read Framebuffer is called within a certain time of the history is greater than a preset value.

It should be noted that the context information may indicate a rendering state, for example, when the first rendering instruction is a glBlitFramebuffer instruction, it may be determined whether the color and depth template attachment of the currently bound Read Framebuffer is greater than 1, and whether the sampling value is greater than 1.

In an optional implementation, the first rendering instruction is configured to operate on a texture object, and the preset condition includes that context information of the texture object satisfies a preset condition.

In an alternative implementation, the second rendering instructions are configured to invoke an interface of the target extension.

Illustratively, the first rendering instruction may be a glbufferData instruction and the second rendering instruction may be a glBufferStorageEXT instruction implemented based on an EXT _ buffer _ storage extension. It should be noted that GL _ MAP _ PERSISTENT _ BIT _ EXT, GL _ MAP _ coheren _ BIT _ EXT, and GL _ DYNAMIC _ STORAGE _ BIT _ EXT attributes may be additionally specified in the glBufferStorageEXT instruction to create a buffer supporting persistent address mapping.

In an optional implementation, the first rendering instruction is located in an objective function table, where the first rendering instruction includes a plurality of first sub-instructions, and correspondingly, if the first rendering instruction satisfies a preset condition, the method includes:

if at least one of the plurality of first sub-instructions included in the first rendering instruction meets a preset condition.

It should be noted that the preset condition satisfied by the at least one first sub-instruction may refer to the preset condition satisfied by the first rendering instruction.

In an alternative implementation, the second rendering instruction comprises a plurality of second sub-instructions, each corresponding to one of the first sub-instructions; the executing the second rendering instructions in response to the first rendering instructions comprises:

and responding to each first sub-instruction, and executing a second sub-instruction corresponding to each first sub-instruction.

In an alternative implementation, at least one of the plurality of second sub-instructions is configured to invoke an interface of the target extension.

In an alternative implementation, at least one of the plurality of second sub-instructions is configured to call an interface of the target extension and is an instruction related to a function that implements the target extension.

In an alternative implementation, the first rendering instructions include first target rendering instructions and second target rendering instructions; the method includes the steps that a first target rendering instruction is a rendering instruction corresponding to a first frame image, a second target rendering instruction is a rendering instruction corresponding to a second frame image, the first frame image is an image frame before the second frame image, and if the first rendering instruction meets a preset condition, the method includes:

and if the first target rendering instruction meets a preset condition, the second target rendering instruction meets the preset condition.

Specifically, before each frame (first frame image) is finished, the scene is identified based on the analysis of the first target rendering instruction, and whether to enter optimization is determined. For the intercepted first target rendering instruction, analyzing specific parameters, the number of calls, current OpenGL ES state information, recorded history information, current context characteristics, and the like, to match a preset scene in the extended optimization algorithm library (specifically, refer to a part in the above embodiment where it is determined whether the first rendering instruction satisfies a preset condition), then, a second target rendering instruction intercepted in the second frame image may be analyzed to analyze a specific parameter, the number of calls, current OpenGL ES state information, recorded history information, current context characteristics, and the like, to match a preset scene in the extended optimization algorithm library (specifically, refer to a part in the above embodiment, if it is determined that the first rendering instruction satisfies a preset condition, or not), and if the second target rendering instruction satisfies the preset condition, the second rendering instruction is obtained.

In a second aspect, the present application provides a rendering instruction processing apparatus, where the apparatus is applied to a terminal device, where the terminal device includes a GPU, and the GPU supports a target extension in an open graphics library openGL ES of an embedded system, and the apparatus includes:

an instruction obtaining module, configured to obtain a first rendering instruction that is not implemented based on the target extension, where the first rendering instruction is used to implement a rendering task;

acquiring a second rendering instruction according to the first rendering instruction; wherein the second rendering instruction is a rendering instruction implemented based on the target extension, and the second rendering instruction is used for implementing the rendering task; and the number of the first and second groups,

and the instruction execution module is used for triggering the GPU to execute the rendering task according to the second rendering instruction.

In an alternative implementation, the instruction obtaining module is configured to:

In an alternative implementation, the device power consumption for triggering the GPU to execute the rendering task according to the first rendering instruction is greater than the device power consumption for triggering the GPU to execute the rendering task according to the second rendering instruction; and/or the presence of a gas in the gas,

triggering the GPU to execute the rendering task according to the first rendering instruction, wherein the corresponding memory data copy amount is larger than that when triggering the GPU to execute the rendering task according to the second rendering instruction; and/or the presence of a gas in the gas,

triggering the GPU to execute the rendering task according to the first rendering instruction, wherein the corresponding GPU load is larger than that when triggering the GPU to execute the rendering task according to the second rendering instruction; and/or the presence of a gas in the gas,

and triggering the GPU to execute the rendering task according to the first rendering instruction, wherein the CPU load of a corresponding central processing unit is larger than that of the corresponding CPU load when triggering the GPU to execute the rendering task according to the second rendering instruction.

In an alternative implementation, the preset conditions include: the first rendering instruction belongs to a preset instruction set.

the cache type of the target cache meets a preset condition;

and the context information of the target cache meets a preset condition.

In an optional implementation, the first rendering instruction includes a plurality of first sub-instructions, and correspondingly, the instruction obtaining module is configured to obtain the second rendering instruction if at least one first sub-instruction of the plurality of first sub-instructions included in the first rendering instruction satisfies a preset condition.

In an alternative implementation, the second rendering instruction comprises a plurality of second sub-instructions, each corresponding to one of the first sub-instructions; the instruction execution module is configured to:

In a third aspect, the present application provides a terminal device, where the terminal device includes a processor and a memory, and the processor acquires a code stored in the memory to execute any one of the first aspect and optional implementations thereof.

In a fourth aspect, the present application provides a non-transitory computer-readable storage medium containing computer instructions for executing the rendering instruction processing method of any one of the first aspect and optional implementations thereof,

in a fifth aspect, the present application further provides a computer program product, which includes computer instructions executed by a processor of a host device for performing operations performed by the processor in any one of the possible implementations of the embodiment.

The embodiment of the application provides a rendering instruction processing method, which is applied to terminal equipment, wherein the terminal equipment comprises a GPU (graphics processing unit), the GPU supports the target expansion in an open graphics library openGL ES of an embedded system, and the method comprises the following steps: obtaining a first rendering instruction which is not realized based on the target extension, wherein the first rendering instruction is used for realizing a rendering task; acquiring a second rendering instruction according to the first rendering instruction; wherein the second rendering instruction is a rendering instruction implemented based on the target extension, and the second rendering instruction is used for implementing the rendering task; and triggering the GPU to execute the rendering task according to the second rendering instruction. By the method, for the specific scene, the rendering instruction based on the GPU expansion is used for replacing the rendering instruction which is not based on the GPU expansion originally, so that the performance loss of the terminal equipment can be reduced.

Drawings

FIG. 1 is an architectural illustration of a system;

FIG. 2 is a block diagram illustrating a computing device that may implement the techniques described in this disclosure;

fig. 3 is a flowchart illustrating a rendering instruction processing method according to an embodiment of the present application;

FIG. 4 is an alternative illustration of a rendering instruction in this embodiment;

FIG. 5 is a schematic diagram of a rendering instruction processing method according to an embodiment of the present application;

FIG. 6a is a schematic diagram of a rendering instruction process according to an embodiment of the present application;

FIG. 6b is a schematic diagram of a rendering instruction process according to an embodiment of the present application;

FIG. 7 is a schematic diagram of a rendering instruction processing method according to an embodiment of the present application;

FIG. 8 is a schematic diagram of a rendering instruction processing method according to an embodiment of the present application;

FIG. 9 is a schematic structural diagram of a rendering instruction processing apparatus according to the present application;

fig. 10 is a structural schematic diagram of a terminal device provided in the present application.

Detailed Description

Embodiments of the present application will now be described with reference to the accompanying drawings, and it is to be understood that the described embodiments are merely illustrative of some, but not all, embodiments of the present application. As can be known to those skilled in the art, with the development of technology and the emergence of new scenarios, the technical solution provided in the embodiments of the present application is also applicable to similar technical problems.

The terms "first," "second," and the like in the description and in the claims of the present application and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that the embodiments described herein may be practiced otherwise than as specifically illustrated or described herein. Moreover, the terms "comprises," "comprising," and any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or modules is not necessarily limited to those steps or modules explicitly listed, but may include other steps or modules not expressly listed or inherent to such process, method, article, or apparatus. The naming or numbering of the steps appearing in the present application does not mean that the steps in the method flow have to be executed in the chronological/logical order indicated by the naming or numbering, and the named or numbered process steps may be executed in a modified order depending on the technical purpose to be achieved, as long as the same or similar technical effects are achieved.

OpenGL (open graphics library) refers to a professional graphical program interface defining a cross-programming language and cross-platform programming interface specification, and is applied to the fields of industries including content creation, energy, entertainment, game development, manufacturing industry, pharmaceutical industry, virtual reality and the like, and OpenGL can help programmers to realize the development of high-performance and high-visual-expressive graphics processing software on hardware devices such as Personal Computers (PCs), workstations, supercomputers and the like.

OpenGL (OpenGL for embedded systems) is a subset of an OpenGL three-dimensional graphics Application Program Interface (API), and is designed for embedded devices such as mobile phones and game hosts. OpenGL ES is based on OpenGL, and removes many non-essential characteristics such as complex primitives like glBegin/glEnd, quadrilateral (GL _ QUADS), polygon (GL _ polygon), etc. Over the years, there are now two major versions, OpenGL ES 1.x for fixed pipeline hardware and OpenGL ES 2.x for programmable pipeline hardware. OpenGL ES 1.0 is based on the OpenGL 1.3 specification, and OpenGL ES 1.1 is based on the OpenGL 1.5 specification. OpenGL ES 2.0 is defined with reference to the OpenGL 2.0 specification.

Enabling scenarios for OpenGL ES include, but are not limited to: picture processing such as picture tone conversion, beauty, and the like; camera preview effect processing such as beauty camera, etc., video processing, 3D games, etc.

In one implementation, OpenGL is implemented as a client-server system, where an application corresponds to a client and OpenGL corresponds to a server. As shown in fig. 1, a client program needs to call an OpenGL interface to implement 3D rendering, so OpenGL commands and data are cached in a Random Access Memory (RAM), under a certain condition, the commands and data are sent to a Video Random Access Memory (VRAM) through a CPU clock, rendering of graphics is completed by using the data and commands in the VRAM under control of a GPU, and a result is stored in a frame buffer, and a frame in the frame buffer is finally sent to a display to display the result. In modern graphics hardware systems, it is also supported to send data directly from RAM to VRAM or directly from frame buffer to RAM without a CPU clock (e.g. VBO, PBO in OpenGL).

In some OpenGL implementations, such as those associated with X windows systems, the client and server execute on different machines, both connected via a network. In this case, the client sends OpenGL commands, which are translated into windowing system related protocols, and sent to the server over the shared network.

In the field of games, the problems of game frame dropping and power consumption are always pain problems which bother terminal manufacturers. Accordingly, graphics rendering technology is also continuously advancing, OpenGL ES is developing from 1.0 to 3.2, supporting more features, and also continuously improving rendering efficiency. GPU manufacturers are also actively promoting various technologies for reducing loads, improving image quality, and the like, and promoting more efficient OpenGL ES extensions, and a new OpenGL ES extension can improve rendering efficiency, however, there is a long evolution road before the extension enters the standard. For example, a GPU manufacturer proposes a new extension, which can improve rendering efficiency of a specific scene, promote the extension to other GPU manufacturers, become a general extension, promote Khronos organization approval, and finally promote incorporation standards.

However, until becoming standard or being approved by a standard organization, a game engine generally does not adapt to use the latest extension so soon, that is, even if a GPU of a terminal device currently running a game supports a certain non-standard OpenGL ES extension, since the extension does not become standard, rendering instructions in the game are not based on the extension but implemented based on other instructions for the same rendering task, and the final rendering efficiency is reduced (for example, the rendered display effect is poor or the power consumption caused to the terminal device is high, etc.). In order to solve the above problem, the present application provides a rendering instruction processing method.

A system architecture schematic of an embodiment of the present application is described next.

FIG. 2 is a block diagram illustrating a computing device 30 that may implement the techniques described in this disclosure. Computing device 30 may be a rendering instruction processing device in embodiments of the present application, examples of computing device 30 include, but are not limited to, wireless devices, mobile or cellular telephones (including so-called smartphones), Personal Digital Assistants (PDAs), video game consoles including video displays, mobile video game devices, mobile video conferencing units, laptop computers, desktop computers, television set-top boxes, tablet computing devices, e-book readers, fixed or mobile media players, and so forth.

In the example of fig. 2, computing device 30 includes a Central Processing Unit (CPU) 32 having a CPU memory 34, a Graphics Processing Unit (GPU) 36 having a GPU memory 38 and one or more shading units 40, a display unit 42, a display buffer unit 44, a user interface unit 46, and a storage unit 48. Further, storage unit 48 may store a GPU driver 50 with a compiler 54, a GPU program 52, and a natively compiled GPU program 56.

Examples of CPU 32 include, but are not limited to, a Digital Signal Processor (DSP), a general purpose microprocessor, an Application Specific Integrated Circuit (ASIC), a field programmable logic array (FPGA), or other equivalent integrated or discrete logic circuitry. Although CPU 32 and GPU36 are illustrated as separate units in the example of fig. 2, in some examples CPU 32 and GPU36 may be integrated into a single unit. CPU 32 may execute one or more applications. Examples of applications may include web browsers, email applications, spreadsheets, video games, audio and/or video capture, playback or editing applications, or other applications that initiate the generation of image data to be presented via display unit 42.

In the example shown in FIG. 2, CPU 32 includes CPU memory 34. The CPU memory 34 may represent on-chip storage or memory used in executing machine or object code. The CPU memories 34 may each include a hardware memory register capable of storing a fixed number of digital bits. CPU 32 may be able to read values from local CPU memory 34 or write values to local CPU memory 34 more quickly than reading values from storage unit 48 (which may be accessed, for example, via a system bus) or writing values to storage unit 48.

GPU36 represents one or more special-purpose processors for performing graphics operations. That is, for example, GPU36 may be a dedicated hardware unit having fixed functionality and programmable components for rendering graphics and executing GPU applications. GPU36 may also include a DSP, general purpose microprocessor, ASIC, FPGA, or other equivalent integrated or discrete logic circuitry.

GPU36 also includes GPU memory 38, which may represent on-chip storage or memory used in executing machine or object code. GPU memories 38 may each include hardware memory registers capable of storing a fixed number of digital bits. GPU36 may be able to read values from local GPU memory 38 or write values to local GPU memory 38 more quickly than reading values from storage unit 48 (which may be accessed, for example, via a system bus) or writing values to storage unit 48.

GPU36 also includes a shading unit 40. As described in more detail below, shading unit 40 may be configured as a programmable pipeline of processing components. In some examples, shading units 40 may be referred to as "shader processors" or "unified shaders," and may perform geometry, vertex, pixel, or other shading operations to render graphics. Shading unit 40 may include one or more components not specifically shown in fig. 2 for clarity, such as components for fetching and decoding instructions, one or more Arithmetic and Logic Units (ALUs) for performing arithmetic computations, and one or more memories, caches, or registers.

Display unit 42 represents a unit capable of displaying video data, images, text, or any other type of data. The display unit 42 may include a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display, an Organic LED (OLED), an active-matrix OLED (AMOLED) display, and the like.

Display buffer unit 44 represents memory or storage dedicated to storing data for display unit 42 for presentation of images (e.g., photographs or video frames). Display buffer unit 44 may represent a two-dimensional buffer containing a plurality of storage locations. The number of storage locations within display buffer unit 44 may be substantially similar to the number of pixels to be displayed on display unit 42. For example, if display unit 42 is configured to include 640x480 pixels, display buffer unit 44 may include 640x480 storage locations. Display buffer unit 44 may store final pixel values for each of the pixels processed by GPU 36. Display unit 42 may retrieve the final pixel values from display buffer unit 44 and display the final image based on the pixel values stored in display buffer unit 44.

User interface unit 46 represents a unit that a user may use to interact with or otherwise interface with other units of computing device 30 (e.g., CPU 32) to communicate with other units of computing device 30. Examples of user interface units 46 include, but are not limited to, a trackball, a mouse, a keyboard, and other types of input devices. The user interface unit 46 may also be a touch screen and may be incorporated as part of the display unit 42.

Storage unit 48 may include one or more computer-readable storage media. Examples of storage unit 48 include, but are not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer or processor.

In some example implementations, memory unit 48 may contain instructions that cause CPU 32 and/or GPU36 to perform the functions of the present invention for implementing CPU 32 and GPU 36. In some examples, storage unit 48 may be considered a non-transitory storage medium. The term "non-transitory" may indicate that the storage medium is not embodied in a carrier wave or propagated signal. However, the term "non-transitory" should not be construed to mean that the storage unit 48 is immobile. As one example, storage unit 48 may be removed from computing device 30 and moved to another device. As another example, a storage unit substantially similar to storage unit 48 may be inserted into computing device 30. In some instances, a non-transitory storage medium may store data that may change over time (e.g., in RAM).

Storage unit 48 stores GPU driver 50 and compiler 54, GPU program 52, and natively compiled GPU program 56. GPU driver 50 represents a computer program or executable code that provides an interface to access GPU 36. CPU 32 executes GPU driver 50, or portions thereof, to connect with GPU36, and for this reason GPU driver 50 is shown in the example of fig. 2 as GPU driver 50 labeled with a dashed box within CPU 32. GPU driver 50 may access programs or other executable files executed by CPU 32, including GPU program 52.

GPU program 52 may contain code written in a high-level (HL) programming language (e.g., using an Application Programming Interface (API)). An example of an API includes the open graphics library (OpenGL). In general, an API contains a predetermined standardized set of commands that are executed by associated hardware. API commands allow a user to instruct the hardware components of the GPU to execute the commands without the user being aware of the specifics of the hardware components.

GPU program 52 may invoke or otherwise include one or more functions provided by GPU driver 50. CPU 32 generally executes a program in which GPU program 52 is embedded, and upon encountering GPU program 52, passes GPU program 52 to GPU driver 50 (e.g., in the form of a command stream). CPU 32 executes GPU driver 50 in this context to process GPU program 52. For example, GPU driver 50 may process GPU program 52 by compiling the GPU program into an object or machine code that GPU36 may execute. This object code is shown in the example of fig. 2 as a natively compiled GPU program 56.

In some examples, compiler 54 may operate in real-time or near real-time to compile GPU program 52 during execution of the program in which GPU program 52 is embedded. For example, compiler 54 generally represents a module that reduces HL instructions defined according to an HL programming language to low-level (LL) instructions of an LL programming language. After compilation, these LL instructions can be executed by a particular type of processor or other type of hardware (e.g., FPGA, ASIC, etc., (including, for example, CPU 32 and GPU 36).

In the example of fig. 2, compiler 54 may receive GPU program 52 from CPU 32 while executing HL code that includes GPU program 52. Compiler 54 may compile GPU program 52 into a natively compiled GPU program 56 that conforms to the LL programming language. Compiler 54 then outputs a natively compiled GPU program 56 that includes the LL instructions.

GPU36 generally receives natively compiled GPU program 56 (as shown by the "natively compiled GPU program 56" labeled by the dashed box within GPU 36), after which, in some examples, GPU36 renders the image and outputs the rendered portion of the image to display buffer unit 44. For example, GPU36 may generate a plurality of primitives to be displayed at display unit 42. Primitives may include one or more lines (including curves, splines, etc.), points, circles, ellipses, polygons (where a polygon is typically defined as a collection of one or more triangles), or any other two-dimensional (2D) primitive. The term "primitive" may also refer to three-dimensional (3D) primitives such as cubes, cylinders, spheres, cones, pyramids, circles, and the like. In general, the term "primitive" refers to any geometric shape or element that is rendered by GPU36 for display as an image (or frame in the context of video data) via display unit 42.

GPU36 may transform the primitives or other state data for the primitives (e.g., which defines texture, brightness, camera configuration, or other aspects of the primitives) into a so-called "world space" by applying one or more model transforms (which may also be specified in the state data). Once transformed, GPU36 may apply a view transform of the active camera (which may also be specified in the state data defining the camera as well) to transform the coordinates of the primitives and light into camera or eye space. GPU36 may also perform vertex shading to render the appearance of primitives in any view of available light. GPU36 may perform vertex shading in one or more of the model, world, or view spaces described above (although vertex shading is typically performed in world space).

Once the primitives are shaded, GPU36 may perform projections to project the image into, as one example, unit cubes having poles at (-1, -1, -1) and (1, 1, 1). This unit cube is commonly referred to as a typical view volume. After transforming the model from eye space to a typical view volume, GPU36 may perform clipping to remove any primitives that do not reside at least partially in the view volume. In other words, GPU36 may remove any primitives that are not within the camera frame. GPU36 may then map the coordinates of the primitives from the view volume to screen space, effectively reducing the 3D primitives of the primitives to the 2D coordinates of the screen.

Given the transformed and projected vertices that define the primitives with their associated shading data, GPU36 may then rasterize the primitives. For example, GPU36 may calculate and set the colors of the pixels of the screen covered by the primitives. During rasterization, GPU36 may apply any textures associated with the primitives (where the textures may include state data). GPU36 may also perform a Z-buffer algorithm (also referred to as a depth test) during rasterization to determine whether any primitives and/or objects are obscured by any other objects. The Z-buffer algorithm orders the primitives according to their depth so that GPU36 knows the order in which each primitive is drawn onto the screen. GPU36 outputs the rendered pixels to display buffer unit 44.

Display buffer unit 44 may temporarily store the rendered pixels of the rendered image until the entire image is rendered. In this context, the display buffer unit 44 may be considered as an image frame buffer. Display buffer unit 44 may then transmit the rendered image to be displayed on display unit 42. In some alternative examples, rather than temporarily storing the image in display buffer unit 44, GPU36 may output the rendered portion of the image directly to display unit 42 for display. The display unit 42 may then display the images stored in the display buffer unit 78.

In this embodiment of the application, when a program that needs to perform graphics rendering runs, the CPU 32 may obtain a rendering instruction for each frame of image from the central processing unit memory, and execute the rendering instruction to implement graphics rendering. Specifically, the CPU 32 may obtain a function table (hook table) for each frame image from the central processing unit memory, where the function table may include a plurality of rendering instructions, and the CPU may implement graphics rendering by executing each rendering instruction in the function table (the specific execution flow may refer to the description in the embodiment corresponding to fig. 2, and is not described here again).

In particular, a GPU may support multiple extensions in OpenGL ES, however, when an application developer writes a program, the application developer may not write the program with a more efficient extension due to the extension not being generalized to a standard or the like, aiming at graphics rendering.

In order to solve the above problem, a rendering instruction processing method provided by an embodiment of the present application is described in detail below.

Referring to fig. 3, fig. 3 is a flowchart illustrating a rendering instruction processing method according to an embodiment of the present application, and as shown in fig. 3, the rendering instruction processing method according to the embodiment of the present application includes:

301. obtaining a first rendering instruction that is not implemented based on the target extension, the first rendering instruction being used to implement a rendering task.

Taking an application program needing to perform graphic rendering as an example of a game, when the game is started, a rendering instruction for a rendering task in the game can be acquired, and then the rendering task corresponding to the rendering instruction is executed; wherein the first rendering instruction is one of the fetched rendering instructions.

In an embodiment of the application, the first rendering instruction is not implemented based on the target extension.

302. Acquiring a second rendering instruction according to the first rendering instruction; wherein the second rendering instruction is a rendering instruction implemented based on the target extension, and the second rendering instruction is used to implement the rendering task.

In the embodiment of the application, if the first rendering instruction meets a preset condition, a second rendering instruction is obtained according to the first rendering instruction; wherein the second rendering instruction is a rendering instruction implemented based on the target extension.

In the embodiment of the application, in the process of replacing the instruction which does not adopt the target extension in the application program code with the instruction realized based on the extension, it is first required to identify which instructions need to be converted.

In the embodiment of the application, after the first rendering instruction is acquired, the first rendering instruction is analyzed to determine whether the current scene meets a preset requirement, and if the first rendering instruction meets a preset condition, it is determined that instruction conversion is required when the first rendering instruction is met.

In this embodiment, the terminal device may pre-configure some rules, where the rules define preset conditions that the instruction should satisfy, where the first rendering instruction is used to operate the target cache, and the preset conditions at least include one of the following conditions: the first rendering instruction belongs to a preset instruction set; the cache type of the target cache meets a preset condition; the historical operation times of the target cache by the first rendering instruction are greater than a preset value; or the context information of the target cache meets a preset condition.

The first rendering instruction comprises a plurality of first sub-instructions, and correspondingly, whether at least one first sub-instruction in the plurality of first sub-instructions included in the first rendering instruction meets a preset condition needs to be judged.

Specifically, for a preset condition that the first rendering instruction belongs to a preset instruction set, the terminal device may specify some instructions in advance, where the instructions are not implemented based on the target extension, rendering efficiency is poor, and the current GPU supports a certain extension, and there may be another instruction that can implement the same rendering task based on the extension. At this time, the terminal device may specify some instructions in advance as candidate optimization instructions (for example, the first rendering instruction in this embodiment), and correspondingly, may also specify optimization instructions in advance as candidate optimization instructions (for example, the second rendering instruction in this embodiment).

In one implementation, the first rendering instruction may be a separate instruction.

For example, the glmapbufferrrange instruction and the glBufferData instruction in openGL ES have a large number of map operations when they are executed in a scenario where vertex and index data are frequently updated, which may cause high GPU load and CPU load. For another example, the glBlitFramebuffer instruction, when executed, may copy a relatively large amount of data from the DDR memory. The preset instruction set in the embodiment of the present application may be an instruction set including the above instructions.

It should be noted that, whether the instruction belongs to the preset instruction set can be directly determined by identifying the instruction name.

In one implementation, the first rendering instruction may be an instruction set including a plurality of instructions, and in this case, the preset instruction set may include a plurality of instructions and a time sequence order between the instructions, and it may be understood that the first rendering instruction belongs to the preset instruction set, and the first rendering instruction includes names of the plurality of instructions and a sequential execution order thereof belongs to the preset instruction set. For example, the predetermined instruction set includes a subset of instructions { glBindBuffer; glBufferData; glMapBufferRange; memcpy; glumapBuffer }.

It should be noted that the first rendering instruction belongs to a preset instruction set, and may only satisfy one condition of the preset condition for the first rendering instruction.

In one implementation, the first rendering instruction is to operate a target cache, wherein the target cache may be one of a frame cache frame buffer, a render cache renderbuffer.

Specifically, the type of the cache of the target cache meets a preset condition, which may be whether the type of data stored in the target cache meets the condition, or whether some attributes of the target cache meet the condition, for example, when the first rendering instruction is a glBufferData instruction, it may be determined whether the target cache is used for storing vertex or index data, and whether its own purpose is dynamically updated, and if so, it is determined that the cache type of the target cache meets the preset condition.

Specifically, the number of times of the historical operation of the target cache by the first rendering instruction is greater than a preset value; whether the historical operation times of the first rendering instruction are larger than a preset value within a certain time of the target cache can be obtained. For example, when the first rendering instruction is a glBlitFramebuffer instruction, it may be determined whether the number of times (frequency) that the bound Read Framebuffer is called within a certain time of the history is greater than a preset value.

Specifically, the context information for the target cache meets a preset condition; context information of the target cache may be obtained, where the context information may indicate a rendering state, for example, when the first rendering instruction is a glBlitFramebuffer instruction, it may be determined whether a color and depth template attachment of the currently bound Read Framebuffer is present, and whether a sampling value of the color and depth template attachment is greater than 1, or the like.

In the embodiment of the application, whether the first rendering instruction belongs to a preset instruction set or not can be determined; whether the cache type of the target cache meets a preset condition or not; whether the historical operation times of the target cache by the first rendering instruction are larger than a preset value or not; whether the context information of the target cache meets at least one preset condition in preset conditions is judged to judge whether the current scene meets the conditions, and whether the first rendering instruction meets the preset conditions is further determined.

In the embodiment of the application, if the first rendering instruction meets a preset condition, a second rendering instruction can be obtained; wherein the second rendering instructions are configured to be implemented based on the target extension.

Specifically, a mapping relationship between a first rendering instruction and a second rendering instruction may be configured in advance, where the mapping relationship includes a preset corresponding relationship between the first rendering instruction and the second rendering instruction; after it is determined that the first rendering instruction meets the preset condition, a second rendering instruction corresponding to the first rendering instruction may be obtained in the mapping relationship.

In one implementation, the second rendering instructions are configured to invoke an interface of the target extension.

In one implementation, the second rendering instruction includes a plurality of second sub-instructions, each corresponding to one of the first sub-instructions.

In one implementation, the first rendering instruction is used to operate a texture object, and the preset condition includes that context information of the texture object satisfies a preset condition.

For example, referring to fig. 4, fig. 4 is an alternative illustration of a rendering instruction in the present embodiment, as shown in fig. 4, at the time of the first glmapbuffer range, the first rendering instruction may be an instruction set { glBindBuffer; glBufferData; glMapBufferRange; memcpy; glumapbuffer }, and correspondingly, the second rendering instruction may be an instruction set { glBindBuffer; glBufferStorageEXT; glMapBufferRange; memcpy; return }; wherein glBindBuffer corresponds to glBindBuffer, glBufferData corresponds to glBufferStoragEXT, glMapBufferRange corresponds to glMapBufferRange, memcpy corresponds to memcpy, and glUnmapBuffer corresponds to return. It can be seen that at least one second sub-instruction of the plurality of second sub-instructions is configured to invoke an interface of the target extension. At least one second sub-instruction of the plurality of second sub-instructions is configured to call an interface of the target extension and is an instruction related to a function of implementing the target extension.

When not the first glMapBufferRange, the first rendering instruction may be the set of instructions { glBindBuffer; glBufferData; glMapBufferRange; memcpy; glumapbuffer }, and correspondingly, the second rendering instruction may be an instruction set { glBindBuffer; return; returning the address; memcpy; return }; wherein glBindBuffer corresponds to glBindBuffer, glBufferData corresponds to return, glMapBufferRange corresponds to return address, memcpy corresponds to memcpy, and glUnmapBuffer corresponds to return.

As another example, the first rendering instruction may be a glBlitFramebuffer instruction and the second rendering instruction may be a glRenderBurage multisampleEXT instruction implemented based on an EXT _ multisampled _ render _ to _ texture extension.

In the embodiment of the application, the device power consumption for triggering the GPU to execute the rendering task according to the first rendering instruction is greater than the device power consumption for triggering the GPU to execute the rendering task according to the second rendering instruction; and/or the memory data copy amount corresponding to the condition that the GPU is triggered to execute the rendering task according to the first rendering instruction is larger than the memory data copy amount corresponding to the condition that the GPU is triggered to execute the rendering task according to the second rendering instruction; and/or the corresponding GPU load when the GPU is triggered to execute the rendering task according to the first rendering instruction is larger than the corresponding GPU load when the GPU is triggered to execute the rendering task according to the second rendering instruction; and/or the CPU load corresponding to the GPU when the first rendering instruction triggers the GPU to execute the rendering task is larger than the CPU load corresponding to the GPU when the second rendering instruction triggers the GPU to execute the rendering task.

In an embodiment of the application, the first rendering instruction includes a first target rendering instruction and a second target rendering instruction; and if the first target rendering instruction meets a preset condition and the second target rendering instruction meets the preset condition, the second target rendering instruction is a rendering instruction corresponding to a second frame image, and the first frame image is an image frame before the second frame image.

303. And triggering the GPU to execute the rendering task according to the second rendering instruction.

In the embodiment of the application, after the CPU obtains the second rendering instruction, the CPU transmits the second rendering instruction to the GPU for driving. For example, the GPU driver may compile second rendering instructions into GPU executable objects or machine code, after which the second rendering instructions can be executed by the GPU, and the GPU may execute the corresponding rendering task after receiving the compiled second rendering instructions.

In an alternative implementation, the code corresponding to the second rendering instruction may be executed with the first rendering instruction as a pointer.

In this embodiment of the application, the second rendering instruction may include a plurality of second sub-instructions, each corresponding to one of the first sub-instructions; then, in response to each first sub-instruction, a second sub-instruction corresponding to said each first sub-instruction may be executed. That is, after the first rendering instruction from the application program is acquired, the code corresponding to the first rendering instruction is not executed, but the code corresponding to the second rendering instruction is executed with the first rendering instruction as a pointer. Executing the second rendering instruction may be understood as sending the second rendering instruction to the corresponding driver, and executing a rendering task corresponding to the second rendering instruction based on the GPU.

Taking a game rendering scene as an example, in the embodiment of the application, the game can reduce terminal performance loss caused by frequent memory mapping and the like or reduce loads of a CPU/GPU by executing instructions realized based on GPU expansion without modifying codes. Meanwhile, the embodiment of the application is a general expansion optimization framework based on GPU expansion, the performance of a mobile phone supporting the expansion can be improved, optimization cannot be performed on a mobile phone not supporting the expansion, and the compatibility is better.

The embodiment of the application provides a rendering instruction processing method, which is applied to terminal equipment, wherein the terminal equipment comprises a GPU (graphics processing unit), the GPU supports the target expansion in an open graphics library openGL ES of an embedded system, and the method comprises the following steps: acquiring a first rendering instruction; if the first rendering instruction meets a preset condition, acquiring a second rendering instruction; wherein the second rendering instructions are configured to be implemented based on the target extension; and, in response to the first rendering instruction, executing the second rendering instruction. By the method, for the specific scene, the rendering instruction based on the GPU expansion is used for replacing the rendering instruction which is not based on the GPU expansion originally, so that the performance loss of the terminal equipment can be reduced.

Next, taking a scene rendered by a game as an example, an embodiment is given that includes more details than the corresponding embodiment of fig. 3.

Referring to fig. 5, fig. 5 is an illustration of a rendering instruction processing method 500 provided by an embodiment of the present application, as shown in fig. 5, when a game starts 501, an initialization process may be performed 502, and the initialization process may include: traversing a directory where the extended optimization algorithm is located, and loading all installed dynamic library files, wherein the extended optimization algorithm in the embodiment of the present application must include the following functions: CheckExtEnabled: the method is used for verifying whether the extended optimization algorithm is supported; OnEnterHookMode: calling the expanded optimization algorithm after enabling the expanded optimization algorithm, and replacing a hook table of OpenGL ES; OnExitHookMode: the method comprises the steps that after an extended optimization algorithm is closed, a hook table of OpenGL ES is restored; OnInterceptEndCommand: and calling each frame before finishing to finish scene recognition and determine whether the subsequent frame enters optimization.

Based on GetProcAddress 503, function pointers of external interfaces of all the extended optimization algorithms are obtained, and an algorithm enabling detection function CheckExtEnabled 504 of each extended optimization algorithm is called to determine whether the algorithm is enabled or not. Specifically, all openGL ES extensions supported by the GPU of the current device may be obtained, whether an extension dependent on the current extension optimization algorithm is supported is checked, and if so, the oneterhookmode 505 is enabled.

Before the end of each frame, an OnInterceptEndCommand function 506 is called to identify the scene and decide whether to enter optimization. Intercepting all OpenGL ES and EGL function calls, and jumping to an extended optimization framework instruction analysis module. For the intercepted instruction, analyzing specific parameters, the number of times of calling, the current OpenGL ES state information, the recorded historical information and the current context characteristics to match with a preset scene 507 in the extended optimization algorithm library.

Each algorithm presets a supported scenario, which is a set of a series of rules, and may include, for example:

1, instruction name: each scene is a combination of a series of instructions, and the rough scene can be deduced through the instructions;

2, instruction parameters: further narrowing the scene range based on the specific parameter information of the instruction;

3, instruction calling times: analyzing instruction calling in a period of time, only optimizing a general scene, and not optimizing an occasional scene;

4, current context state information: obtaining current OpenGL ES context state information, and only optimizing scenes meeting the expected range;

5, history information: a command call needs to be recorded for a certain period of time as a reference for matching information, etc.

And for the intercepted instruction, matching a preset scene in the extended optimization algorithm library. If the scenes match, instruction conversion optimization begins to be performed. And calling an algorithm enabling function OnEnterHookMode of each extended optimization algorithm to replace the GL function table of the current rendering thread.

In the next frame, if optimization can be entered, instruction conversion 508 is performed and the next frame 509 is rendered.

Then the optimization is turned off, restoring the function table 510

An embodiment comprising more details than the corresponding embodiment of fig. 3 is given next.

Referring to fig. 6a, fig. 6a is a schematic diagram of a rendering instruction processing provided in an embodiment of the present application, and as shown in fig. 6a, conversion of a rendering instruction may be implemented by an extended optimization framework in the embodiment, unlike the embodiment of fig. 3, fig. 6a shows a flow that may perform multiple optimization judgments and optimization execution.

As shown in fig. 6a, the first rendering instruction (glFunction) that the extended optimization framework may acquire implements one optimization based on scene matching, transformation enabling (which may be specifically described with reference to the embodiment corresponding to fig. 5) and instruction transformation, and then implements another optimization based on scene matching, transformation enabling (which may be specifically described with reference to the embodiment corresponding to fig. 5) and instruction transformation, where the two optimizations are different in that the instruction is replaced with a rendering instruction supporting a different extension based on different scene identifications.

In more detail, referring to fig. 6b, fig. 6b is an illustration of a rendering instruction processing provided in the embodiment of the present application, and as shown in fig. 6b, with respect to glFunction1, based on scene matching, conversion enabling (which may be described with reference to the embodiment corresponding to fig. 5 in particular), and instruction conversion, only optimization corresponding to algorithm 2 is implemented (algorithm 1 may be due to failure of scene matching). Aiming at the glFunction2, based on scene matching, transformation enabling and instruction transformation, the corresponding optimization of algorithm 1 and algorithm 2 is realized. For glFunction3, which is based on scene matching, transformation enabling, and instruction transformation, no optimization is achieved (algorithm 1 and algorithm 2 may be due to scene matching failure). For glFunction4, which is based on scene matching, transformation enabling, and instruction transformation, only the corresponding optimization of algorithm 1 is achieved (algorithm 2 may be due to scene matching failure).

An embodiment comprising more details than the corresponding embodiment of fig. 3 is given next. It is directed to scenes where vertex and index data are frequently updated in game rendering.

In this embodiment, the glmapbufferrrange instruction may be identified, and it is checked whether the buffer currently bound is vertex or index data, and whether the buffer is dynamically writable. If so, if the map instruction calls up to a certain proportion, the scenes are matched, and then the expansion conversion is enabled.

In this embodiment, a buffer dictionary may be maintained, and all buffer information, including target, size, usage, and other information, may be recorded.

Referring to FIG. 7, when the glBufferData instruction 701 is fetched, it can be checked whether the target cache it operates on is vertex or index data, whether the purpose is dynamic update, and whether the size meets the limit 702. If the condition is satisfied, the glBufferStorage EXT interface 703 is called to additionally specify the GL _ MAP _ PERSISTENT _ BIT _ EXT, GL _ MAP _ COHERENT _ BIT _ EXT, and GL _ DYNAMIC _ STORAGE _ BIT _ EXT attributes, creating a buffer that supports persistent address mapping. If the interface has been previously converted and there is no change in size or attributes, then return is straightforward. In the first time of glmapbuffer range704, GL _ MAP _ PERSISTENT _ BIT _ EXT and GL _ MAP _ COHERENT _ BIT _ EXT attributes need to be added to the access parameter to obtain the persistent memory address 705. The next time the glMapBufferRange704 is in the same buffer, the address 706 obtained before is directly returned. In glUnmapBuffer, return is performed directly.

By the mode, in a scene that a large amount of map operations exist in the vertex and index data, the calling proportion of the glBufferData instruction, the glMapBufferRange instruction and the glUnmapBuffer instruction is reduced, and the loads of a CPU and a GPU can be effectively reduced.

An embodiment comprising more details than the corresponding embodiment of fig. 3 is given next. It is directed to anti-aliasing scenes in a rendering process. As in fig. 8, the glBlitFramebuffer instruction 801 may be recognized to check if the color and depth template attachment of the currently bound Read frame is greater than 1. If so, if the Blit instruction calls a certain percentage, then the scene matches 802, and then the expansion transition is enabled.

A new framebuffer803 may be created first, a new single sample color texture created 804 and space allocated 805, a new depth template attachment created 806, and the color and depth template attachments of the single samples bound using the extension provision interface. RenderBuffer allocates space glRenderBufferStorage multisampleEXT 807, binds textures to glframebufferTexture2 DMultiampleEXT 808 on Framebuffer, binds RBOs to glFramebufferRenderBuffer 809, and renders directly onto a new Framebuffer in the next frame 810.

When the multisampling frame buffer is bound, the multisampling frame buffer is replaced by the newly created frame buffer ID, so that the multisampling result is output to the single sampling accessory. When the framebuffer where the Blit is located is bound, the operations of binding the framebuffer and the Blit are skipped. When binding to the following framebuffer, the bound texture is replaced with the texture corresponding to the newly created framebuffer, since the single-sample texture of the previous Blit output is used.

In the embodiment, for the anti-aliasing scene, the game can be expanded through the GPU without modifying codes, thereby reducing 60% of DDR data copy and reducing the power consumption of the mobile phone.

Referring to fig. 9, fig. 9 is a schematic structural diagram of a rendering instruction processing apparatus 900, which is applied to a terminal device including a GPU supporting a target extension in an open graphics library openGL ES of an embedded system, and the apparatus includes:

an instruction obtaining module 901, configured to obtain a first rendering instruction that is not implemented based on the target extension, where the first rendering instruction is used to implement a rendering task; acquiring a second rendering instruction according to the first rendering instruction; wherein the second rendering instruction is a rendering instruction implemented based on the target extension, and the second rendering instruction is used for implementing the rendering task;

an instruction execution module 902, configured to trigger the GPU to execute the rendering task according to the second rendering instruction.

the cache type of the target cache meets a preset condition;

and the context information of the target cache meets a preset condition.

Referring to fig. 10, fig. 10 is a schematic structural diagram of a terminal device 1000 provided in the present application, and as shown in fig. 10, the terminal device includes a processor 1001 and a memory 1003, the processor includes a central processing unit CPU and a graphics processing unit GPU, the GPU supports a target extension in an embedded system open graphics library openGL ES, and the CPU is configured to obtain a code of the memory to execute:

the cache type of the target cache meets a preset condition;

and the context information of the target cache meets a preset condition.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, a division of a unit is merely a logical division, and an actual implementation may have another division, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or other network devices) to execute all or part of the steps of the method described in the embodiment of fig. 2 of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

The above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A rendering instruction processing method is applied to a terminal device, the terminal device comprises a GPU (graphics processing Unit), the GPU supports a target extension in an open graphics library OpenGL ES of an embedded system, and the method comprises the following steps:

obtaining a first rendering instruction which is not realized based on the target extension, wherein the first rendering instruction is used for realizing a rendering task;

and triggering the GPU to execute the rendering task according to the second rendering instruction.

2. The method of claim 1, wherein fetching second rendering instructions according to the first rendering instructions comprises:

3. The method according to claim 1 or 2, wherein the device power consumption for triggering the GPU to execute the rendering task according to the first rendering instruction is greater than the device power consumption for triggering the GPU to execute the rendering task according to the second rendering instruction; and/or the presence of a gas in the gas,

4. The method of any of claims 1 to 3, wherein said fetching second rendering instructions according to the first rendering instructions comprises: when the first rendering instruction meets a preset condition, acquiring the second rendering instruction according to the first rendering instruction; the preset condition is one or more of the following conditions:

the first rendering instruction belongs to a preset instruction set;

the first rendering instruction is used for operating a target cache, and the cache type of the target cache meets a preset condition;

the historical information of the target cache operated by the first rendering instruction meets a preset condition;

the context information of the target cache meets a preset condition; or the like, or, alternatively,

the first rendering instruction is used for operating a texture object, and the preset condition comprises that the context information of the texture object meets a preset condition.

5. The method of any of claims 1 to 4, wherein the second rendering instruction is an interface to invoke the target extension.

6. The method according to any of claims 1 to 5, wherein the first rendering instruction comprises a plurality of first sub-instructions, and wherein, if the first rendering instruction satisfies a predetermined condition, the method comprises:

7. The method of claim 6, wherein the second rendering instruction comprises a plurality of second sub-instructions, each second sub-instruction corresponding to a first sub-instruction; the executing the second rendering instructions in response to the first rendering instructions comprises:

8. The method of claim 6 or 7, wherein at least one of the plurality of second sub-instructions is configured to invoke an interface of the target extension.

9. The method of any of claims 6 to 8, wherein at least one of the second sub-instructions is configured as an instruction that does not invoke an interface of the target extension and is related to a function that implements the target extension.

10. The method of any of claims 1 to 5, wherein the first rendering instructions comprise first target rendering instructions and second target rendering instructions; the method includes the steps that a first target rendering instruction is a rendering instruction corresponding to a first frame image, a second target rendering instruction is a rendering instruction corresponding to a second frame image, the first frame image is an image frame before the second frame image, and if the first rendering instruction meets a preset condition, the method includes:

11. A rendering instruction processing apparatus, the apparatus being applied to a terminal device including a graphics processor GPU that supports a target extension in an embedded system open graphics library openGL ES, the apparatus comprising:

12. The apparatus of claim 11, wherein the instruction fetch module is configured to:

13. The apparatus according to claim 11 or 12, wherein the device power consumption for triggering the GPU to execute the rendering task according to the first rendering instruction is greater than the device power consumption for triggering the GPU to execute the rendering task according to the second rendering instruction; and/or the presence of a gas in the gas,

14. The apparatus according to any one of claims 11 to 13, wherein the instruction obtaining module is configured to:

when the first rendering instruction meets a preset condition, acquiring the second rendering instruction according to the first rendering instruction; the preset condition is one or more of the following conditions:

the first rendering instruction belongs to a preset instruction set;

15. The apparatus of any of claims 11 to 14, wherein the second rendering instruction is configured to invoke an interface of the target extension.

16. The apparatus according to any of claims 11 to 15, wherein the first rendering instruction comprises a plurality of first sub-instructions, and correspondingly, the instruction fetching module is configured to fetch the second rendering instruction if at least one of the plurality of first sub-instructions included in the first rendering instruction satisfies a predetermined condition.

17. The apparatus of claim 16, wherein the second rendering instruction comprises a plurality of second sub-instructions, each second sub-instruction corresponding to a first sub-instruction; the instruction execution module is configured to:

18. The apparatus according to claim 16 or 17, wherein at least one of the plurality of second sub-instructions is configured to invoke an interface of the target extension.

19. The apparatus according to any of claims 16 to 18, wherein at least one of the plurality of second sub-instructions is configured to invoke an interface of the target extension and is an instruction related to a function that implements the target extension.

20. The apparatus of any of claims 11 to 15, wherein the first rendering instructions comprise first target rendering instructions and second target rendering instructions; the method includes the steps that a first target rendering instruction is a rendering instruction corresponding to a first frame image, a second target rendering instruction is a rendering instruction corresponding to a second frame image, the first frame image is an image frame before the second frame image, and if the first rendering instruction meets a preset condition, the method includes:

21. A non-transitory computer-readable storage medium containing computer instructions for executing the rendering instruction processing method according to any one of claims 1 to 10.

22. An arithmetic device comprising a memory in which code is stored and a processor for retrieving the code to execute the rendering instruction processing method according to any one of claims 1 to 10.