WO2022089512A1

WO2022089512A1 - Load control method and apparatus, and device

Info

Publication number: WO2022089512A1
Application number: PCT/CN2021/126924
Authority: WO
Inventors: 陈秋林; 吴江铮; 杨程云; 徐超
Original assignee: 华为技术有限公司
Priority date: 2020-10-29
Filing date: 2021-10-28
Publication date: 2022-05-05
Also published as: CN114511438A

Abstract

Disclosed in embodiments of the present application are a load control method and apparatus, and a device, used for controlling the load during an image rendering process. The method has good applicability and a wide application range. The method in the embodiments of the present application comprises: determining load data according to an instruction for performing a rendering operation; dynamically selecting, from among multiple load control schemes according to the load data, a load control scheme matching a current load condition; and applying the scheme to the subsequent image processing process, thereby achieving load control.

Description

A method, device and device for controlling load

This application claims the priority of the Chinese patent application filed on October 29, 2020 with the Chinese application number 202011186852.6 and the invention titled "A method, device and device for controlling load", the entire contents of which are incorporated herein by reference Applying.

technical field

The embodiments of the present application relate to the field of computer technologies, and in particular, to a method, an apparatus, and a device for controlling a load.

Background technique

With the rapid development of communication technology and chip technology, mobile terminals have become the most important deployment platform for applications. Taking game applications as an example, more and more large-scale game programs with high image quality and high frame rate are deployed on mobile terminals, which undoubtedly puts forward higher requirements on the image rendering performance of mobile terminals. However, the image rendering performance of the mobile terminal is limited, so when the image rendering performance cannot meet the load requirements of the game program, the user's game experience will be affected.

In order to avoid the above situation, currently the load of game programs is mainly reduced by the following methods: load analysis is performed for a specific game program in an offline state, and then a load reduction scheme is determined, and the load reduction scheme is solidified into the mobile terminal.

Since the solution fixed in the mobile terminal cannot be upgraded with the upgrade of the game program, this method of reducing the load has poor applicability and limited application scope.

SUMMARY OF THE INVENTION

Embodiments of the present application provide a method, apparatus, and device for controlling load, which are used to control the load in an image rendering process. The method has good applicability and wide application range.

A first aspect of the embodiments of the present application provides a method for controlling a load, including:

Acquiring at least one first instruction, where the at least one first instruction is used to call an application program interface API, so that the graphics processor GPU performs a first rendering operation; the first rendering operation can be used for one or more objects in a frame of image Rendering can also be used to render objects in multi-frame images; the load data of the process in which the GPU performs the first rendering operation is determined based on at least one first instruction, wherein the embodiment of the present application does not limit the type and specific load data. Determine the target scheme for controlling the load based on the load data, and the embodiment of the present application does not limit the specific method of determining the target scheme; obtain at least one second instruction, and at least one second instruction is used to call the API, so that the GPU When executing the second rendering operation, the second instruction is essentially the same as the first instruction, and both belong to the instructions for calling the API; the load in the process of executing the second rendering operation on the GPU is controlled based on the target scheme.

The process of controlling the load based on the target scheme is related to the specific content of the target scheme. If the target scheme instructs to process the second instruction, after obtaining the at least one second instruction, the obtained at least one second instruction is first processed according to the target scheme, and then the processed at least one second instruction is passed on. If the target solution does not instruct to process the second instruction, after obtaining the at least one second instruction, the at least first second instruction is not processed, but is passed before the second instruction is passed according to the specific content of the target solution. During the process of the second instruction or after the second instruction is transmitted, other forms of control means are adopted for the load.

Since the load data is determined based on the first instruction actually obtained in the image rendering process, the method provided by the embodiment of the present application can be applied to any game application, not limited to one or several game applications. The latter game application program and the newly added game application program are also applicable, so the applicability is strong and the application range is wide; in addition, the method provided by the embodiment of the present application does not need to be executed offline, and can realize dynamic control of the load.

As an implementation manner, determining the load data of the process in which the GPU performs the first rendering operation based on the at least one first instruction includes: acquiring the load data of the GPU performing the first rendering operation based on the at least one first instruction and at least one preset load characteristic factor. At least one kind of load data of the process, and each load characteristic factor corresponds to one kind of load data. Among them, the load characteristic factor can be defined according to actual needs. The load characteristic factor can be understood as the characteristic parameter of the load data. Each load characteristic factor indicates a load, and the value of each load characteristic factor can be regarded as a specific load. data. Exemplarily, the load characteristic factor may indicate a kind of information of computer hardware resources, and the value of the load characteristic factor may be used as a kind of load data; for example, the load characteristic factor indicates the frequency of the CPU, and correspondingly, the actual The frequency point can be used as a kind of load data. Exemplarily, the load characteristic factor may also indicate the quantity of the first instructions, and correspondingly, the specific quantity of the first instructions may be used as a kind of load data.

Since at least one load characteristic factor can be preset as required, and one kind of load data can be obtained corresponding to each load characteristic factor, it can be ensured that the load data is comprehensive and representative.

As an implementation manner, the at least one load characteristic factor includes at least one of the following three types of load characteristic factors: a first type of load characteristic factor, a second type of load characteristic factor, and a third type of load characteristic factor; the first type of load characteristic factor The factor is used to represent the consumption of computer hardware resources in the process of the GPU performing the first rendering operation. Information of the rendered mesh Mesh model, the information of the mesh Mesh model may include the number of vertices of the mesh Mesh model and the number of mesh Mesh models and other information; the third type of load characteristic factor is used to represent at least one first. The information of the instruction, the information of the first instruction may include various information such as the quantity of the first instruction and the relationship between the instructions.

At least one load characteristic factor includes at least one of the above three types of load characteristic factors; when at least one load characteristic factor includes the above two types or the above three types of load characteristic factors, it is guaranteed to obtain a variety of load data from multiple dimensions, thereby It is convenient to determine a more suitable target scheme for reducing the load.

As an implementation manner, the load data includes at least one of the first type of load data, the second type of load data, and the third type of load data; the first type of load data indicates that the process of the GPU performing the first rendering operation affects the computer hardware resources The second type of load data represents the information of the mesh model rendered by the GPU performing the first rendering operation; the third type of load data represents the information of at least one first instruction.

In this implementation manner, the load data may include at least one of the above three types of load data, which ensures the diversification of the load data.

As an implementation manner, the first type of load characteristic factor includes at least one of the following: calculation time in the process of the GPU performing the first rendering operation, operating parameters of the central processing unit CPU, operating parameters of the GPU, and operation of the internal memory parameter; the second type of load characteristic factor includes at least one of the following: the number of vertices of each Mesh model rendered in the process of the GPU performing the first rendering operation, the number of vertices in each frame of the image rendered in the process of the GPU performing the first rendering operation. The number of vertices, the number of Mesh models in each frame of images rendered by the GPU in the process of performing the first rendering operation, the number of repetitions of Mesh models in each frame of images rendered by the GPU in the process of performing the first rendering operation, and the first rendering by the GPU The distance of each Mesh model rendered by the operation process relative to the screen; the third type of load characteristic factor includes at least one of the following: the number of drawing commands DC formed by at least one first instruction, the rendering corresponding to at least one first instruction The number of channel Render Passes, the frame buffer Frame Buffer size, the reference relationship between rendering channels and frame buffers, and the reference relationship between DC and frame buffers.

This implementation provides multiple choices for each type of load characteristic factor, thereby ensuring the diversification of load data.

As an implementation manner, the first type of load data may include at least one of the following: calculation time, CPU load data, GPU load data, and DDR memory load data; the second type of load data may include the following At least one of: the number of vertices of each Mesh model rendered by the process of the GPU performing the first rendering operation, the number of vertices in each frame of the image rendered by the process of the GPU performing the first rendering operation, the number of vertices of the GPU performing the first rendering operation. The number of Mesh models in each frame of image rendered by the process, the number of repetitions of Mesh models in each frame of image rendered by the GPU in the process of performing the first rendering operation, and the number of Mesh models rendered in the process of the GPU performing the first rendering operation The distance relative to the screen; the third type of load data can include at least one of the following: the number of drawing commands (Draw Call, DC) formed by at least one first instruction, the number of rendering channels Render Pass corresponding to at least one first instruction , Frame Buffer Frame Buffer size, the reference relationship between the rendering channel and the frame buffer, and the reference relationship between the DC and the frame buffer.

There can also be multiple choices for each type of load data, thereby ensuring the diversification of load data.

As an implementation manner, the at least one load characteristic factor includes a second type of load characteristic factor; and acquiring at least one load data of a process in which the GPU performs the first rendering operation based on the at least one first instruction and the at least one load characteristic factor includes: based on at least one of the first instructions and the at least one load characteristic factor. The parameters carried by a first instruction respectively determine the data of the mesh Mesh model rendered by the GPU executing the first rendering operation; based on the data of the mesh Mesh model rendered by the GPU executing the first rendering operation, obtain the data of the mesh model that the GPU executes the first rendering operation. The process corresponds to the load data of the second type of load characteristic factor.

In this implementation manner, the parameter carried by the first instruction indicates the data of the mesh model to be rendered, and according to the data of the mesh model to be rendered, load data corresponding to the second type of load characteristic factor can be determined.

As an implementation manner, determining the target solution for controlling the load based on the load data includes: determining a load control label based on the load data, where the load control label indicates the content of the load control, wherein, for the same load data, one or more loads may be determined A control label; based on the load control label, a load control scheme is selected from at least one preset load control scheme as a target scheme for controlling the load. For example, if the load data is the frequency of the CPU, the first load tag can indicate the frequency of the CPU to be controlled; if the load data is the number of the first instructions, the first load tag can indicate the frequency of the first instruction. quantity is controlled.

In this implementation manner, the setting of the load control tag enables the developer to flexibly expand the load control scheme, that is, preset more load control schemes associated with the load control tag for selection.

As an implementation manner, the load data includes the first type of load data, and the first type of load data represents the consumption of computer hardware resources in the process of the GPU performing the first rendering operation; determining the load control label based on the load data includes: based on the first type of load data The load data indicates that the consumption of computer hardware resources in the process of performing the first rendering operation on the GPU satisfies the first condition, and a first load control tag is determined, and the first load control tag indicates the operation of the computer hardware in the process of performing the second rendering operation on the GPU parameters are adjusted.

In this implementation, when the load data includes the first type of load data, the first load control label instructs to adjust the operating parameters of the computer hardware, so that the purpose of reducing the load can be achieved by adjusting the operating parameters of the computer hardware.

As an implementation manner, the first load control tag instructs to adjust the frequency point of the CPU, the GPU or the internal memory in the process of performing the second rendering operation on the GPU.

This implementation provides a specific content of the first load control label, that is, instructs to adjust the frequency point of the CPU, GPU or internal memory.

As an implementation manner, the load data includes a second type of load data, and the second type of load data represents information of a mesh model rendered by the GPU performing the first rendering operation; determining the load control label based on the load data includes: based on the first type of load data The load data indicates that the information of the mesh model rendered by the GPU performing the first rendering operation satisfies the second condition, and a second load control label is determined, and the second load control label indicates the Mesh rendered in the process of performing the second rendering operation on the GPU The model is simplified.

In this implementation manner, when the load data includes the second type of load data, the second load control tag instructs to simplify the rendered Mesh model, so as to reduce the load by adjusting the operating parameters of the computer hardware.

As an implementation manner, the second load control tag indicates the characteristics of the Mesh model to be simplified during the GPU performing the second rendering operation, the simplified algorithm applicable to the Mesh model to be simplified, and the parameters of the simplified algorithm.

This implementation provides a specific content of the second load control label, ie, indicates the characteristics of the Mesh model to be simplified, the simplified algorithm applicable to the Mesh model to be simplified, and the parameters of the simplified algorithm.

As an implementation manner, the second type of load data includes the number of vertices and the number of Mesh models in each frame of image rendered by the first rendering operation; the second condition includes the number of vertices in each frame of image rendered by the first rendering operation is greater than the first threshold, or the number of Mesh models in each frame of image rendered by the first rendering operation is greater than the second threshold.

In this implementation, the second condition is defined from two dimensions, the number of vertices in each frame of image and the number of Mesh models.

As an implementation manner, the second load control tag indicates that among the Mesh models rendered by the first rendering operation, the number of vertices is greater than the third threshold, the number of repetitions is greater than the fourth threshold, or the distance from the screen is greater than The Mesh model of the fifth threshold is simplified.

In this implementation, the Mesh model to be simplified is defined from the number of vertices, the number of repetitions, and the distance from the screen.

As an implementation manner, the load data includes a third type of load data, and the third type of load data represents information of at least one first instruction; determining the load control tag based on the load data includes: indicating at least one first instruction based on the third type of load data The instruction information of the satisfies the third condition, and the third load control tag is determined, and the third load control tag instructs to process at least one second instruction.

In this implementation manner, when the load data includes the third type of load data, the third load control tag instructs to process the at least one second instruction, so that the purpose of reducing the load can be achieved by processing the at least one second instruction.

As an implementation manner, the number of second instructions is multiple; the third load control label indicates at least one of the following content: combining multiple DCs formed by multiple second instructions, and combining multiple DCs corresponding to multiple second instructions Combine multiple render passes.

This implementation provides a specific content of the third load control tag, that is, indicating that multiple DCs are merged or multiple rendering channels are merged.

A second aspect of the embodiments of the present application provides a device for controlling a load, including:

an instruction acquisition module, configured to acquire at least one first instruction, where the at least one first instruction is used to call an application programming interface API, so that the graphics processor GPU performs the first rendering operation;

a load data determination module, configured to determine load data of a process in which the GPU performs the first rendering operation based on the at least one first instruction;

a target scheme determination module for determining a target scheme for controlling the load based on the load data;

The instruction acquisition module is further configured to acquire at least one second instruction, and the at least one second instruction is used to call the API, so that the GPU performs the second rendering operation;

The control module is configured to control the load of the GPU in the process of performing the second rendering operation based on the target scheme.

As an implementation manner, a load data determination module, configured to acquire at least one kind of load data of a process in which the GPU performs the first rendering operation based on at least one first instruction and at least one load characteristic factor, where each load characteristic factor corresponds to one kind of load data.

As an implementation manner, the at least one load characteristic factor includes at least one of the following three types of load characteristic factors: a first type of load characteristic factor, a second type of load characteristic factor, and a third type of load characteristic factor; the first type of load characteristic factor The factor is used to represent the consumption of computer hardware resources in the process of the GPU performing the first rendering operation; the second type of load characteristic factor is used to represent the information of the mesh model rendered by the GPU in the process of performing the first rendering operation; the third type The load characteristic factor is used to represent information of at least one first instruction.

As an implementation manner, the at least one load characteristic factor includes the second type of load characteristic factor; the load data determination module is configured to determine, based on the parameters carried by the at least one first instruction, the mesh model rendered by the GPU by performing the first rendering operation Based on the data of the mesh Mesh model rendered by the GPU performing the first rendering operation, obtain the load data of the second type of load characteristic factor corresponding to the process of the GPU performing the first rendering operation.

As an implementation manner, the target solution determination module is configured to determine a load control label based on the load data, where the load control label indicates the content of the load control; and determine the target solution for controlling the load based on the load control label.

As an implementation manner, the load data includes the first type of load data, and the first type of load data represents the consumption of computer hardware resources in the process of the GPU performing the first rendering operation; the target solution determination module is configured to be based on the first type of load data. The consumption of computer hardware resources in the process of instructing the GPU to perform the first rendering operation satisfies the first condition, and a first load control label is determined. adjust.

As an implementation manner, the load data includes the second type of load data, and the second type of load data represents the information of the mesh model rendered by the GPU performing the first rendering operation; the target solution determination module is configured to be based on the first type of load data. Instructing the GPU to perform the information of the mesh model rendered by the first rendering operation satisfies the second condition, and determining a second load control label, the second load control label instructs the GPU to perform the second rendering operation on the rendered Mesh model. simplify.

As an implementation manner, the load data includes a third type of load data, and the third type of load data represents information of at least one first instruction; the target solution determination module is configured to indicate at least one instruction of the first instruction based on the third type of load data When the information satisfies the third condition, a third load control tag is determined, and the third load control tag instructs to process at least one second instruction.

For the specific implementation, related descriptions and technical effects of the above units, please refer to the description of the first aspect of the embodiment of the present application.

A third aspect of the embodiments of the present application provides a terminal device, including: one or more processors and a memory; wherein, the memory stores computer-readable instructions; the one or more processors read the Computer readable instructions to cause the computer device to implement the method according to any implementation of the first aspect.

A fourth aspect of the embodiments of the present application provides a computer-readable storage medium, including computer-readable instructions, when the computer-readable instructions are executed on a computer, the computer is made to execute any implementation manner of the first aspect. method described.

A fifth aspect of the embodiments of the present application provides a chip, including one or more processors. Part or all of the processor is used to read and execute the computer program stored in the memory, so as to execute the method in any possible implementation manner of the first aspect.

Optionally, the chip includes a memory, and the memory and the processor are connected to the memory through a circuit or a wire. Further optionally, the chip further includes a communication interface, and the processor is connected to the communication interface. The communication interface is used to receive data and/or information to be processed, and the processor obtains the data and/or information from the communication interface, processes the data and/or information, and outputs the processing result through the communication interface. The communication interface may be an input-output interface.

In some implementations, some of the one or more processors may also implement some steps in the above method by means of dedicated hardware, for example, the processing involving the neural network model may be performed by a dedicated neural network processor or graphics processor.

The methods provided in the embodiments of the present application may be implemented by one chip, or may be implemented collaboratively by multiple chips.

A sixth aspect of the embodiments of the present application provides a computer program product, where the computer program product includes computer software instructions, and the computer software instructions can be loaded by a processor to implement the method described in any one of the implementation manners of the first aspect above .

As can be seen from the above technical solutions, the embodiments of the present application have the following advantages:

Acquire at least one first instruction, then determine the load data of the process in which the GPU performs the first rendering operation based on the first instruction, and then determine the target scheme according to the load data. After acquiring at least one second instruction, the GPU can be executed according to the target scheme The load in the process of executing the second rendering operation is controlled, thereby realizing the online dynamic control of the load in the image rendering process; since the load data is determined based on the first instruction actually obtained in the image rendering process, the embodiment of the present application provides The method can be applied to any game application, not limited to one or several game applications. It is also applicable to upgraded game applications and newly added game applications, so it has strong applicability and wide application range. .

Description of drawings

1 is a schematic diagram of a system architecture in an embodiment of the present application;

FIG. 2 is a schematic diagram of an embodiment of a method for controlling a load in an embodiment of the present application;

FIG. 3 is a schematic flowchart of an embodiment of the control load in the implementation of the present application;

4 is a schematic diagram of an embodiment of a first load control label in an embodiment of the present application;

Fig. 5 is the characteristic schematic diagram of the Mesh model to be simplified in the embodiment of the application;

6 is a schematic structural diagram of an umbrella body in an embodiment of the present application;

7 is a schematic structural diagram of a closure body in an embodiment of the present application;

8 is a schematic structural diagram of an extension body in an embodiment of the present application;

FIG. 9 is a schematic process diagram of step 103 in an embodiment of the present application;

10 is a schematic diagram of a process of determining a third load control label in an embodiment of the present application;

FIG. 11 is a schematic diagram of an embodiment of a third load control label in an embodiment of the application;

FIG. 12 is a schematic flowchart of another embodiment of load control in the implementation of the present application;

13 is a schematic diagram of a device for controlling a load in an embodiment of the present application;

FIG. 14 is a schematic structural diagram of a terminal device in an embodiment of the present application.

Detailed ways

The embodiments of the present application will be described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Those of ordinary skill in the art know that with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

The terms "first", "second" and the like in the description and claims of the present application and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It is to be understood that data so used may be interchanged under appropriate circumstances so that the embodiments described herein can be practiced in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" and "having", and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device comprising a series of steps or modules is not necessarily limited to those expressly listed Rather, those steps or modules may include other steps or modules not expressly listed or inherent to the process, method, product or apparatus. The naming or numbering of the steps in this application does not mean that the steps in the method flow must be executed according to the time or logical sequence indicated by the naming or numbering, and the named or numbered process steps can be implemented according to the The technical purpose is to change the execution order, as long as the same or similar technical effects can be achieved.

The embodiments of the present application may be applied to the system architecture shown in FIG. 1 . The system architecture includes an application program layer, a graphics application program interface (API) layer, a kernel layer and a hardware layer.

It should be noted that the system framework can be applied to any terminal device that needs to reduce the load during image rendering, especially some terminal devices with limited image rendering performance. For example, the system framework can be applied to mobile terminals with limited image rendering performance such as mobile phones, tablet computers, and notebook computers, and the system framework can also be applied to terminal devices with limited image rendering performance due to special scenarios such as car computers.

As shown in Figure 1, the application layer includes three game applications. It should be noted that, in this embodiment, the application program in the application program layer mainly refers to the game application program, that is, the embodiment of the present application is mainly applied to the scene of game rendering. However, in other embodiments of the present application, the applications in the application layer may also be other applicable applications, which are not specifically limited in this application; in addition, the number of game applications is not limited to three, less or less. More is possible.

The graphics API layer includes but is not limited to three graphics APIs, Open Graphics Library (OpenGL), OpenGL ES (OpenGL for Embedded Systems) and Vulkan. Among them, the graphics API layer can be used to render 2D or 3D scenes to the screen of the terminal device; for example, OpenGL is used to render 2D scenes, and OpenGL ES and Vulkan are used to render 3D scenes.

The kernel layer includes the system kernel and drivers.

The hardware layer includes but is not limited to the central processing unit CPU, graphics processing unit GPU, and internal memory, among which the internal memory can be double-rate SDRAM (Double Data Rate Synchronous Dynamic Random-access Memory, DDR SDRAM), or other types of internal memory.

Based on the system framework shown in Figure 1, when the game application is running, the rendering process Pipe Line includes:

In the first step, the game engine in the game application traverses the game scene and cuts out invisible objects.

In the second step, corresponding to each visible object, the game engine prepares data resources and sends an instruction to call the graphics API, and the instruction to call the graphics API will reach the kernel layer.

In the third step, after receiving the instruction to call the graphics API, the driver of the kernel layer will transfer the data resource to the internal memory of the GPU, and send a rendering instruction to the GPU to instruct the GPU to render.

In the fourth step, after receiving the rendering instruction, the GPU will read the data resources from the internal memory, and execute operations such as vertex shader Vertex shader, pixel shader Fragment shader or Compute shader.

A vertex shader is a program fragment that processes vertex transformations on the GPU. The vertex shader is executed once for each vertex of the mesh Mesh model. Therefore, the more vertices in the scene to be rendered, the heavier the computational load of the vertex shader.

A pixel shader is a program fragment that handles pixel shading on the GPU. After the model has undergone vertex transformation, the pixel shader is executed once for each pixel on the screen that is covered by the Mesh model. Therefore, the higher the screen resolution, the more computationally loaded the pixel shader is.

Compute shader refers to a program fragment executed on the GPU, which can be used for calculations other than graphics rendering. Specifically, the Compute shader can be transferred to the GPU through OpenGL ES or Vulkan API to be executed by the GPU.

Game applications based on high image quality and high frame rate bring a high load to image rendering, so in order to reduce the load during image rendering, as shown in FIG. 1 , a load control layer is added in this embodiment of the present application. The control layer can intercept the instruction for calling the graphics API from the graphics API layer, and then obtain the required information from the intercepted instruction for calling the graphics API, or process the instruction for calling the graphics API. Finally, the instruction for calling the graphics API is returned to the graphics API layer, so that the instruction for calling the graphics API continues to be passed to the kernel layer.

Specifically, after obtaining the instruction to call the graphics API, the load control layer will dynamically analyze the load in the image rendering process based on the obtained instruction to call the graphics API, and then select the corresponding load reducing method based on the analysis result. Program. Ultimately, the load in subsequent image rendering processes is reduced using the scheme for load reduction. The method for controlling load provided by the embodiment of the present application will be specifically introduced below with reference to FIG. 2 . As shown in FIG. 2 , an embodiment of the present application provides an embodiment of a method for controlling a load, which can be applied to a CPU, including:

Step 101: Acquire at least one first instruction, where the at least one first instruction is used to call an application program interface API, so that the graphics processor GPU performs a first rendering operation.

This embodiment does not specifically limit the number of the first instructions. For example, the number of the first instruction may be one or multiple. When the number of the first instructions is multiple, the multiple first instructions may constitute one or more instruction streams, and each instruction stream is used to call an API to make the GPU render a frame of image.

Correspondingly, the first rendering operation may be used to render one or more objects (person, background, etc.) in one frame of image, and may also be used to render objects in multiple frames of images. In the embodiment of the present application, the API refers to the graphics API; correspondingly, in the system architecture shown in FIG. 1 , the first instruction is an instruction issued by the game application to call the graphics API.

It should be noted that acquiring at least one first instruction may be understood as sensing and recording at least one first instruction, and will not prevent the downward transmission of the first instruction.

Based on the relevant description of the system framework shown in FIG. 1 , after the first instruction for calling the API is passed to the driver, the driver will send a rendering instruction to the GPU, so that the GPU performs the first rendering operation.

The graphics processing process mainly includes multiple links such as modeling, physical simulation, rendering, and post-processing, and the first rendering operation in the embodiment of the present application mainly refers to the rendering link in the above process.

Step 102: Determine load data of a process in which the GPU performs a first rendering operation based on at least one first instruction.

In the embodiment of the present application, the determination of the load data is based on the acquired GPU of the at least one first instruction, and does not depend on the actual process of the GPU performing the first rendering operation, and can also be understood as the process of determining the load data and the GPU. The actual process of performing the first rendering operation may be parallel.

It should be noted that there are various methods for determining load data, which are not specifically limited in this embodiment. For example, the load data can be determined according to the quantity, specific type, function, etc. of the at least one first instruction; it is also possible to acquire computer hardware that may be used by the GPU in the process of performing the first rendering operation after acquiring the at least one first instruction Some information of the resource is used to determine load data; the object of the first rendering operation may also be determined according to at least one first instruction, and then the load data may be determined according to the information of the object of the first rendering operation.

Based on the above description, it can be known that there may be various types of load data. To this end, in this embodiment, a load characteristic factor is pre-defined, and the load characteristic factor can be understood as a characteristic parameter of the load data, each load characteristic factor indicates a type of load, and the value of each load characteristic factor can be used as a type of load. specific load data. In this way, the load data can be determined from the load characteristic factor.

Exemplarily, the load characteristic factor may indicate a kind of information of computer hardware resources, and the value of the load characteristic factor may be used as a kind of load data; for example, the characteristic factor indicates the frequency point of the CPU, and correspondingly, the actual frequency of the CPU. Points can be used as a kind of payload data.

Exemplarily, the characteristic factor may also indicate the number of the first instructions, and correspondingly, the specific number of the first instructions may be used as a kind of load data.

Exemplarily, determining load data of a process in which the GPU performs the first rendering operation based on the at least one first instruction includes:

At least one type of load data of a process in which the GPU performs the first rendering operation is acquired based on at least one first instruction and at least one load characteristic factor, and each load characteristic factor corresponds to one type of load data.

In this embodiment of the present application, corresponding to each load characteristic factor, one type of load data may be acquired.

It should be noted that the load characteristic factor can be defined according to actual needs, and there can be many dimensions for defining the load characteristic factor. The load characteristic factor will be introduced from multiple dimensions below.

In this embodiment of the present application, since at least one load characteristic factor can be preset as required, and one type of load data can be obtained corresponding to each load characteristic factor, it can be ensured that the load data is comprehensive and representative.

Step 103 , determining a target solution for controlling the load based on the load data.

After the load data is acquired, the load data can be analyzed, and then a target scheme for controlling the load can be determined. It can be understood that this target scheme is usually used to reduce the load during image rendering.

It should be noted that, there are various methods for determining the target solution based on the load data, which is not specifically limited in this embodiment of the present application. For example, a load control scheme library can be established in advance, and the load control scheme library contains a variety of load control schemes. According to the analysis result of the load data, a load control scheme can be selected from a variety of load control schemes as the target scheme for controlling load.

A method for determining a target scheme based on load data is described below. Exemplarily, determining the target solution for controlling the load based on the load data includes:

First, a load control tag is determined based on the load data, the load control tag indicates the content of the load control; then, a load control scheme is selected from at least one load control scheme based on the load control tag as a target scheme for controlling the load.

It can be understood that the load control label indicates the content of the load control and is used to associate with a specific load control scheme. Wherein, for the same load data, one or more load control tags can be determined.

The embodiments of the present application do not specifically limit the form and content of the load control label; for different load data, the form and content of the obtained load control label may be different. The possible forms and contents of the control label will be described in detail below with reference to FIG. 4 , FIG. 11 , and Tables 1 to 3.

Wherein, at least one load control scheme is preset.

In the embodiment of the present application, the setting of the load control label enables the developer to flexibly expand the load control scheme, that is, preset more load control schemes associated with the load control label for selection.

Step 104: Acquire at least one second instruction, where the at least one second instruction is used to call an API, so that the GPU performs a second rendering operation.

It should be noted that the second instruction is essentially the same as the first instruction, and both belong to the instruction to call the API; in step 104, for any second instruction in the at least one second instruction, it may be the first instruction The instruction for calling the API issued later may also be the first instruction, or when the second instruction and the first instruction respectively contain multiple instructions, the two have the same instruction.

Therefore, step 104 can be understood by referring to the relevant description of step 101 for details.

Step 105 , controlling the load in the process of performing the second rendering operation on the GPU based on the target scheme.

Wherein, similar to the first rendering operation, the second rendering operation in this embodiment of the present application mainly refers to the rendering link in the above process.

It should be noted that the process of controlling the load based on the target solution is related to the specific content of the target solution, which is not specifically limited in this embodiment of the present application.

For example, if the target scheme instructs to process the second instruction, after step 104, the at least one second instruction obtained in step 104 is processed according to the target scheme, and then the processed at least one second instruction is sent to the graphic API pass.

If the target solution does not instruct to process the second command, after step 104, at least one second command can be transferred to the graphics API, and the process of transferring the second command before transferring the second command according to the specific content of the target solution During or after passing the second command, the load is controlled.

Based on steps 101 to 105 , the load control method provided by the embodiment of the present application can be summarized as the solution shown in FIG. 3 . As shown in Figure 3, the first instruction is issued by the application, and is transmitted to the Driver Development Kit (DDK) through the graphics API, and is acquired by the instruction acquisition module in the process of transmitting to the driver, and then based on the acquisition The received first instruction collects data related to the first instruction. After that, load data is identified based on the load characteristic factor in the characteristic database, and scheme matching is performed according to the load data, that is, the target scheme is determined from all load control schemes in the load control scheme library. When the final target scheme takes effect, the target scheme can be used to control the load in the process of the GPU performing the second rendering operation.

In this embodiment of the present application, the load data of the process in which the GPU performs the first rendering operation is determined based on the first instruction, then the target scheme is determined according to the load data, and finally the load in the process of the GPU performing the second rendering operation is controlled according to the target scheme. , so as to realize the online analysis and control of the load in the image rendering process; since the load data is determined based on the first instruction actually obtained in the image rendering process, the method provided by the embodiment of the present application can be applied to any game application, without Limited to one or several game applications, it is also applicable to upgraded game applications and newly added game applications, so it has strong applicability and wide application range; in addition, the methods provided in the embodiments of the present application also No offline execution is required, and dynamic control of the load can be achieved.

Based on the foregoing description, it can be known that the load characteristic factor can be defined from multiple dimensions. The following describes the load characteristic factors from three dimensions, each of which corresponds to a type of load characteristic factor.

Exemplarily, the load characteristic factor includes at least one of the following three types of load characteristic factors: a first type of load characteristic factor, a second type of load characteristic factor, and a third type of load characteristic factor.

Specifically, the first type of load characteristic factor is used to represent the consumption of computer hardware resources in the process of performing the first rendering operation by the GPU, wherein the hardware resources may be CPU, GPU, internal memory, etc., and the internal memory may be DDR SDRAM.

It should be noted that, there may be various specific types of the first type of load characteristic factors, which are not specifically limited in this embodiment of the present application.

Exemplarily, the first type of load characteristic factor includes at least one of the following: calculation time in the process of the GPU performing the first rendering operation, operating parameters of the central processing unit CPU, operating parameters of the GPU, and operating parameters of the internal memory.

The calculation time may specifically include logic operation time and GPU call time; CPU operating parameters may specifically include CPU frequency and CPU usage; GPU operating parameters may specifically include GPU frequency and GPU usage.

The second type of load characteristic factor is used to represent the information of the mesh model rendered by the GPU in the process of performing the first rendering operation.

In the rendering scene, the 3D model is generally represented by a triangular mesh, so the 3D model can also be called a Mesh model. Among them, each triangle is composed of three vertices, and the vertex data includes position, normal, UV coordinates, etc.; a vertex can be shared by multiple adjacent triangles. The more complex the geometry of the mesh model, the larger the number of triangle vertices.

Therefore, the information of the Mesh model can be used as the second type of load characteristic factor to indicate how much the load is.

In this embodiment of the present application, the information of the mesh model may include various information such as the number of vertices of the mesh model and the number of mesh models.

It should be noted that, there may be multiple specific types of the second type of load characteristic factors, which are not specifically limited in this embodiment of the present application.

Exemplarily, the second type of load characteristic factor includes at least one of the following: the number of vertices of each Mesh model rendered by the GPU in the process of performing the first rendering operation, the number of vertices in each frame rendered by the GPU in the process of performing the first rendering operation The number of vertices in the image, the number of Mesh models in each frame of images rendered by the GPU in the process of performing the first rendering operation, the number of repetitions of the Mesh models in each frame of images rendered by the GPU in the process of performing the first rendering operation, and the number of Mesh models in each frame of images rendered by the GPU in the process of performing the first rendering operation. The distance of each Mesh model rendered relative to the screen during a rendering operation.

Among them, the number of vertices of the Mesh model indicates the complexity of the geometric structure of the Mesh model. The larger the number of vertices of the Mesh model, the more complex the geometric structure of the Mesh model, and accordingly, the greater the load brought by rendering the Mesh model. . Therefore, the number of vertices of the Mesh model can be used as the second type of feature factor.

The number of vertices in each frame of image is the total number of vertices that need to be traversed in the process of rendering each frame of image. The larger the total number of vertices to be traversed, the greater the load of rendering the frame image. Therefore, the number of vertices in each frame of image can be used as the second kind of feature factor.

The number of Mesh models in each frame of image means how many Mesh models need to be rendered in the process of rendering each frame of image. In the process of rendering a frame of image, the more mesh models that need to be rendered, the greater the load. Therefore, the number of mesh models in each frame of image can be used as the second type of feature factor.

In a frame of image, when the number of repetitions of a Mesh model is large, it means that the total number of vertices of the Mesh model is large, so the number of repetitions of the Mesh model in each frame of image can be used as the second type of feature factor. For example, the tree mesh model in one frame of image contains a small number of vertices, but the number of repetitions of the tree mesh model in the frame image is large, which means the load brought by rendering the tree mesh model in the frame image. larger. The number of repetitions of the Mesh model in each frame of image can also be referred to as the number of instances of the Mesh model.

When a Mesh model is relatively close to the screen, theoretically, the Mesh model needs to be rendered more clearly to ensure the quality of the picture; when a Mesh model is relatively far from the screen, the rendering of the Mesh model can be reduced. Require. That is, the distance of the Mesh model relative to the screen is different and can represent different loads. Therefore, the distance of each Mesh model relative to the screen can be used as the second type of feature factor.

The third type of load characteristic factor is used to represent the information of at least one first instruction, where the information of the first instruction may include various information such as the number of the first instructions and the relationship between the instructions.

It should be noted that, there may be various specific types of the third type of load characteristic factors, which are not specifically limited in the embodiment of the present application.

Exemplarily, the third type of load characteristic factor includes at least one of the following: the number of drawing commands (Draw Call, DC) formed by at least one first instruction, the number of rendering channels Render Pass corresponding to at least one first instruction, the number of frames Buffer Frame Buffer size, reference relationship between rendering channel and frame buffer, reference relationship between DC and frame buffer.

For ease of understanding, the following describes the first instruction, the DC, the rendering channel, the reference relationship between the rendering channel and the frame buffer, and the reference relationship between the DC and the frame buffer.

Taking rendering a frame of image as an example, it may include multiple rendering channels, each rendering channel may correspond to multiple DCs, and each DC may include multiple API calling instructions. Therefore, the number of drawing commands DC formed by at least one first instruction and the number of rendering channels Render Pass corresponding to at least one first instruction can be used as the third type of characteristic factor.

In the process of image rendering, the rendering result of each rendering channel will be output to a corresponding frame buffer, so the corresponding relationship between the rendering channel and the frame buffer can also be called the reference relationship between the rendering channel and the frame buffer.

Similarly, the rendering result corresponding to each DC will be output to a corresponding frame buffer, so the corresponding relationship between DC and frame buffer can also be called the reference relationship between DC and frame buffer.

It should be noted that, since the first type of load characteristic factor is used to represent the consumption of computer hardware resources in the process of GPU performing the first rendering operation, when the load characteristic factor includes the first type of load characteristic factor, it can be calculated according to the computer hardware. The running parameters of the GPU obtain the load data corresponding to the first type of load characteristic factor in the process of the GPU performing the first rendering operation. The third type of load characteristic factor is used to represent the information of at least one first instruction. Therefore, when the load characteristic factor includes the third type of load characteristic factor, the corresponding process of the GPU performing the first rendering operation can be obtained according to at least one first instruction. Load data for the third type of load characteristic factor.

The second type of load characteristic factor is used to represent the information of the mesh model rendered by the GPU in the process of performing the first rendering operation. Therefore, when the load characteristic factor includes the second type of load characteristic factor, it is necessary to first determine that the GPU performs the first rendering. Only the information of the mesh model rendered during the operation process can determine the load data.

Specifically, as an implementation manner, the at least one load characteristic factor includes the second type of load characteristic factor.

Acquiring at least one kind of load data of a process in which the GPU performs a first rendering operation based on at least one first instruction and at least one load characteristic factor includes:

Determine the data of the mesh Mesh model rendered by the GPU performing the first rendering operation based on the parameters carried by the at least one first instruction;

Based on the data of the mesh model rendered by the GPU performing the first rendering operation, the load data of the second type of load characteristic factor corresponding to the process of the GPU performing the first rendering operation is acquired.

It can be understood that the first instruction carries corresponding parameters, and the parameters carried in the first instruction can determine the mesh Mesh model rendered by the GPU performing the first rendering operation, and then obtain the rendering of the GPU performing the first rendering operation. The mesh data for the mesh model.

Corresponding to the above three types of load characteristic factors, three types of load data can be obtained. Therefore, the acquired load data may also include at least one of the first type of load data, the second type of load data, and the third type of load data.

Among them, the first type of load data represents the consumption of computer hardware resources in the process of the GPU performing the first rendering operation; the second type of load data represents the information of the mesh model rendered by the GPU performing the first rendering operation; the third type of load The data represents information of at least one first instruction.

Specifically, the first type of load data may include at least one of the following: calculation time consumption, load data of central processing unit CPU, load data of GPU, and load data of DDR memory.

The second type of load data may include at least one of the following: the number of vertices of each Mesh model rendered by the GPU in the process of performing the first rendering operation, the number of vertices in each frame of the image rendered by the GPU in the process of performing the first rendering operation , the number of Mesh models in each frame of images rendered by the process of performing the first rendering operation by the GPU, the number of repetitions of the Mesh models in each frame of images rendered by the process of performing the first rendering operation by the GPU, and the number of repetitions of the Mesh models in each frame of images rendered by the GPU in the process of performing the first rendering operation. The distance of each mesh model rendered by the pass relative to the screen.

The third type of load data may include at least one of the following: the number of drawing commands (Draw Call, DC) formed by at least one first instruction, the number of rendering passes corresponding to at least one first instruction, and the size of the frame buffer. , the reference relationship between the rendering channel and the frame buffer, and the reference relationship between the DC and the frame buffer.

Based on the foregoing description, it can be known that the process of determining the load control label is different based on different load control labels. The above three kinds of load data are taken as examples below to describe the process of determining the load control label respectively.

As an implementation manner, the load data includes the first type of load data, and the first type of load data represents the consumption of computer hardware resources in the process of the GPU performing the first rendering operation.

Accordingly, step 103 may include:

Based on the first type of load data instructing the GPU to perform the first rendering operation, the consumption of computer hardware resources satisfies the first condition, and determining the first load control tag corresponding to the first load data, and the first load control tag instructs the GPU to perform the first rendering operation on the GPU. 2. The operating parameters of the computer hardware are adjusted during the rendering operation.

The process of determining the first load control tag corresponding to the first load data may be to select one load control tag as the first load control tag from one or more known load control tags.

Correspondingly, the target solution is used to adjust the operating parameters of the computer hardware. The embodiments of the present application do not specifically limit the content of the target solution.

It can be understood that, for different first type of load data, the first condition and the first load control label may be different.

For example, if the first load data includes the CPU frequency, the first condition may be that the CPU frequency is greater than the preset frequency; correspondingly, if the CPU frequency is greater than the preset frequency, a first load control label is generated. The first load control label may include "CPU Bound", where "CPU Bound" indicates that the currently running game application is computationally intensive; the first load control label may also include "Bound intensity", where "Bound intensity" indicates that the calculation is intensive degree.

Exemplarily, as shown in FIG. 4 , the first load control label indicates to adjust the frequency point of the CPU, the GPU or the internal memory in the process of performing the second rendering operation on the GPU.

Correspondingly, the form of the first load control label can be as shown in Table 1 below, which specifically includes two fields of frequency modulation object and frequency point, that is, it indicates that the frequency point of the frequency modulation object is adjusted; wherein, the frequency modulation object can be CPU, GPU or internal frequency modulation object. memory.

Table 1

It should be noted that the embodiment of the present application uses frequency points as an example to describe the process of determining the first load control label, and other first type of load data can be specifically understood by referring to the above description.

As an implementation manner, the load data includes a second type of load data, and the second type of load data represents information of a mesh model rendered by the GPU by performing the first rendering operation.

Accordingly, step 103 may include:

Based on the information of the mesh model rendered by the first type of load data indicating that the GPU performs the first rendering operation and meeting the second condition, a second load control label corresponding to the second load data is determined, and the second load control label instructs the GPU to perform the first rendering operation on the GPU. The Mesh model rendered during the second rendering operation is simplified.

The process of determining the second load control tag corresponding to the second load data may be to select one load control tag as the second load control tag from one or more known load control tags.

It can be understood that, if the information of the mesh model rendered by the GPU performing the first rendering operation satisfies the second condition, it is considered necessary to perform model simplification. Among them, model simplification can be understood as reducing the detail accuracy of the Mesh model; specifically, based on the foregoing description, it can be seen that the Mesh model is composed of triangular meshes, so fewer triangular meshes can be used to represent the Mesh model to simplify the Mesh model and The purpose of reducing the load.

Correspondingly, the target solution is used to simplify the Mesh model rendered during the GPU performing the second rendering operation. The embodiments of the present application do not specifically limit the content of the target solution.

It can be understood that, for different second types of load data, the second condition and the second load control label may be different.

Exemplarily, the second type of load data includes the number of vertices and the number of Mesh models in each frame of image rendered by the first rendering operation, and correspondingly, the second condition includes the number of vertices in each frame of image rendered by the first rendering operation. The number is greater than the first threshold, or the number of Mesh models in each frame of image rendered by the first rendering operation is greater than the second threshold.

Taking the number of vertices in each frame of image as an example, you can first define multiple levels of the number of vertices in each frame of image. For example, the number of vertices in each frame of image is less than or equal to 500K is the first level, the number of vertices in each frame of image is greater than 500K and less than 2000K is the second level, the number of vertices in each frame image is greater than or equal to 2000K is the third level . The first threshold can be set to 500K or 2000K. Among them, K represents thousands.

Exemplarily, the form of the second load control label can be as shown in Table 2, that is, the characteristics of the Mesh model to be simplified in the process of instructing the GPU to perform the second rendering operation, the simplified algorithm applicable to the Mesh model to be simplified, and the characteristics of the simplified algorithm. parameter.

Table 2

It can be understood that the information of the mesh model rendered by the GPU performing the first rendering operation satisfies the second condition, and it can only be determined that model simplification is necessary, but not all Mesh models can be simplified. Simplified Mesh model. Therefore, the second load control label in the embodiment of the present application indicates the characteristics of the Mesh model to be simplified.

The features of the Mesh model to be simplified may include various features, which are not specifically limited in this embodiment of the present application. Exemplarily, as shown in FIG. 5 , the features of the Mesh model to be simplified may include shape features, the scale of the number of vertices, the rendering frequency, and the distance relative to the screen.

Among them, the shape feature can include umbrella body, closure body and extension body; the structure of the umbrella body is shown in Figure 6, in the game scene, the shape feature of the Mesh model of the tree is usually the umbrella body; The structure is shown in Figure 7. In the game scene, the shape features of Mesh models such as cars and characters are usually closed bodies; the structure of the extended body is shown in Figure 8. In the game scene, the shape features of Mesh models such as mountains and rivers are Usually an extension.

The scale of the number of vertices can include large-scale, medium-scale and small-scale; for example, a mesh model with a number of vertices greater than or equal to 50K can be called large-scale, and a mesh model with a number of vertices greater than 5K and less than 50K can be called medium-scale, and the number of vertices can be called medium-scale. Mesh models smaller than 5K can be called small scale. It should be noted that the representation of the scale of the number of vertices is not limited to the above.

Rendering frequency can include low frequency, mid frequency and high frequency. For example, the rendering frequency of the Mesh model can be classified according to the number of times the same Mesh model is rendered in 1000 frames of images. It should be noted that the representation of the rendering frequency is not limited to the above-mentioned manner.

The distance relative to the screen may include a short distance and a long distance; specifically, it may be specifically determined according to the distance of the Mesh model relative to the screen, which is not repeated in this embodiment of the present application.

Based on the foregoing description, it can be known that the second load label indicates the Mesh model that needs to be simplified. Exemplarily, the second load control tag indicates that among the Mesh models rendered by the first rendering operation, the number of vertices is greater than the third threshold, the number of repetitions is greater than the fourth threshold, or the distance from the screen is greater than the fifth threshold. The threshold mesh model is simplified.

It can be understood that when the number of vertices is greater than the third threshold, it means that the geometry of the Mesh model is complex, that is, rendering the Mesh model will bring a larger load. Therefore, it is necessary to simplify the Mesh model, that is, the second load label may indicate a Mesh model whose number of vertices is greater than the third threshold.

When the number of repetitions is greater than the fourth threshold, it means that the number of repetitions of the Mesh model is too large; even if the number of vertices contained in the Mesh model is small, the total number of vertices in all the Mesh models is also large. Therefore, it is necessary to simplify the Mesh model, that is, the second load label may indicate that the Mesh model whose repetition number is greater than the fourth threshold is to be simplified.

In the process of image rendering, for Mesh models with different distances from the screen, the same level of detail (LOD) is usually selected for rendering; while for Mesh models farther from the screen, there are usually different levels of detail in the image. It is important, so it is wasteful to choose a higher LOD for rendering, that is, the Mesh model that is far away from the screen can be simplified. Therefore, the second load label may indicate that the Mesh model whose distance relative to the screen is greater than the fifth threshold is simplified.

In the above example, the specific process of step 103 may be as shown in FIG. 9 . Specifically, the necessity of simplifying the Mesh model (that is, satisfying the second condition) is determined by the number of vertices in each frame of the image rendered by the first rendering operation and the number of Mesh models, and the image rendered by the GPU in the process of performing the first rendering operation is determined. The number of vertices of each Mesh model, the number of repetitions of the Mesh model in each frame of image rendered by the GPU performing the first rendering operation, and the distance of each Mesh model rendered by the GPU during the first rendering operation relative to the screen are determined The list of mesh models to be simplified. The list of mesh models to be simplified can be understood as the second load control label, which specifically indicates the characteristics of the mesh model to be simplified, the simplified algorithm applicable to the mesh model to be simplified, and the parameters of the simplified algorithm.

As an implementation manner, the load data includes a third type of load data, and the third type of load data represents information of at least one first instruction.

Accordingly, step 103 includes:

Based on the third type of load data indicating that the instruction information of the at least one first instruction satisfies the third condition, a third load control tag corresponding to the third load data is determined, and the third load control tag instructs to process the at least one second instruction.

The process of determining the third load control tag corresponding to the third load data may be to select one load control tag as the third load control tag from one or more known load control tags.

Correspondingly, the target solution is used to process the at least one second instruction. The embodiments of the present application do not specifically limit the content of the target solution.

It can be understood that, for different third types of load data, the third condition and the third load control label may be different.

For example, when the third load data includes the number of DCs, the third condition may be that the number of DCs is multiple, and two or more DCs are used to render the same object. Specifically, the third load data includes 100 DCs that render the same character. Since the DCs in the third load data satisfy the third condition, the 100 DCs that render the same character can be combined into one DC.

For another example, when the third load data includes the number of rendering channels, the size of the frame buffer, and the reference relationship between the rendering channels and the frame buffer, the third condition may be that the number of rendering channels is multiple, two or more. The reference relationship between the rendering pass pair and the frame buffer is adjacent, and the frame buffers corresponding to two or more rendering passes have the same size.

Specifically, the third load data includes three rendering channels, the size of the frame buffer, and the reference relationship between the three rendering channels and the frame buffer. Among them, the first rendering pass is used to render Mesh models such as mountains and trees, and the rendering results of the first rendering pass are output to the first frame buffer; the second rendering pass is used to render houses based on the rendering results of the first rendering pass. Waiting for the Mesh model of the building, the rendering result of the second rendering pass is output to the second frame buffer; the third rendering pass is used to render the Mesh model of the character on the basis of the rendering result of the second rendering pass, and the second rendering pass The rendering result is output to the third framebuffer. Since the second rendering pass needs to be rendered on the basis of the rendering result of the first rendering pass, the reference relationship between the second rendering pass and the second frame buffer is close to the reference relationship between the first rendering pass and the first frame buffer; the same Ground, the reference relationship between the third rendering pass and the third frame buffer is adjacent to the reference relationship between the second rendering pass and the second frame buffer. In addition, if the sizes of the first frame buffer, the second frame buffer and the third frame buffer are the same, the first rendering pass, the second rendering pass and the third rendering pass can be merged, and only one frame buffer is required for the merged rendering pass.

Likewise, when the third load data includes the number of DCs, the size of the frame buffer, and the reference relationship between the DC and the frame buffer, the third condition can be understood with reference to the third condition in the foregoing example.

In the above example, the process of determining the third load control tag is shown in Figure 10, that is, according to the API load (including the number of DCs, the size of the frame buffer, the number of rendering channels) and the relationship (including the reference relationship between the DC and the frame buffer, The reference relationship between the rendering pass and the frame buffer) determines the third load control label.

In addition, the API and scene changes in adjacent frame images can also be analyzed to obtain a third load control tag indicating that the image is rendered every other frame. For example, in the adjacent frame images, the scene becomes more and more complex, and the number of Mesh models in the image is more and more, it can be considered that the load is getting bigger and bigger; based on this, the corresponding load control label can be determined, and then the The corresponding target scheme to reduce the load.

Based on the foregoing description, if the third load control tag instructs to process multiple second instructions, after step 104, at least one second instruction acquired in step 104 is first processed according to the target scheme, and then the processed of at least one second instruction passed to the graphics API.

Specifically, in step 104, 8 second instructions are obtained, and these 8 second instructions constitute 2 DCs, after the 2 DCs constituted by these 8 second instructions are merged, the merged DCs are sent to pass down.

Based on the above description, exemplarily, the number of second instructions is multiple; as shown in FIG. 11 , the third load control label indicates at least one of the following contents: combining multiple DCs formed by multiple second instructions, Combine multiple rendering passes corresponding to multiple second instructions.

Exemplarily, the third load control tag may also instruct to merge multiple instances; for example, multiple second instructions are used to render 50 trees, and each 5 trees are rendered as a group (that is, one instance Instance), Then multiple second instructions for rendering 50 trees can be combined.

Illustratively, the third load control tag may also indicate a vibration isolation rendered image.

In this embodiment of the present application, the form of the third load control tag may be as shown in Table 3 below, where the third load tag indicates the name of the second instruction to be processed, the characteristics of the second instruction, and the scheme encoding and processing for processing the second instruction Scheme parameters of the second command.

table 3

After the above description of the load control method provided by the embodiment of the present application, the flow of the load control shown in FIG. 3 may be shown in FIG. 12 . Specifically, the acquired data related to the first instruction includes at least one first instruction, time-consuming management, running data and resource data.

Among them, the time-consuming management is used to determine the calculation time in the first type of load data, and the operation data is used to determine the CPU's operating parameters (such as frequency points), the GPU's operating parameters (such as frequency points) in the first type of load data, and Operating parameters of the internal memory (eg frequency points).

The load data determined according to the signature database includes the first type of load data, the second type of load data and the third type of load data.

According to the above three types of load data, a load control label can be determined, and according to the load control label, a target solution can be matched from the load control solution library, and the target solution can be used to adjust the operating parameters, simplify the mesh model, and process at least one second instruction.

Referring to FIG. 13 , an embodiment of the present application further provides an embodiment of an apparatus for controlling a load, which can be applied to a CPU, including:

an instruction acquisition module 201, configured to acquire at least one first instruction, where the at least one first instruction is used to call an application program interface API, so that the graphics processor GPU performs the first rendering operation;

A load data determination module 202, configured to determine load data of a process in which the GPU performs the first rendering operation based on at least one first instruction;

a target solution determination module 203, configured to determine a target solution for controlling the load based on the load data;

The instruction acquisition module 201 is further configured to acquire at least one second instruction, and the at least one second instruction is used to call the API, so that the GPU performs the second rendering operation;

The control module 204 is configured to control the load in the process of performing the second rendering operation on the GPU based on the target scheme.

As an implementation manner, the load data determination module 202 is configured to obtain at least one kind of load data of a process in which the GPU performs the first rendering operation based on at least one first instruction and at least one load characteristic factor, and each load characteristic factor corresponds to one type of load data. load data.

As an implementation manner, the at least one load characteristic factor includes a second type of load characteristic factor; the load data determination module 202 is configured to determine, based on the parameters carried by the at least one first instruction, the mesh rendered by the GPU by performing the first rendering operation Data of the model; based on the data of the mesh model rendered by the GPU performing the first rendering operation, obtain the load data corresponding to the second type of load characteristic factor in the process of the GPU performing the first rendering operation.

As an implementation manner, the target solution determination module 203 is configured to determine a load control label based on the load data, where the load control label indicates the content of the load control; and determine the target solution for controlling the load based on the load control label.

As an implementation manner, the load data includes the first type of load data, and the first type of load data represents the consumption of computer hardware resources in the process of the GPU performing the first rendering operation; the target solution determination module 203 is configured to, based on the first type of load, The data indicates that the consumption of computer hardware resources in the process of executing the first rendering operation on the GPU satisfies the first condition, and a first load control label is determined, and the first load control label indicates the running parameters of the computer hardware in the process of executing the second rendering operation on the GPU Make adjustments.

As an implementation manner, the load data includes the second type of load data, and the second type of load data represents the information of the mesh model rendered by the GPU performing the first rendering operation; the target solution determination module 203 is configured to, based on the first type of load, The data indicates that the information of the mesh model rendered by the GPU performing the first rendering operation satisfies the second condition, and a second load control label is determined, and the second load control label indicates the Mesh model rendered in the process of performing the second rendering operation on the GPU to simplify.

As an implementation manner, the load data includes a third type of load data, and the third type of load data represents information of at least one first instruction; the target solution determination module 203 is configured to indicate the at least one first instruction based on the third type of load data. When the instruction information satisfies the third condition, a third load control tag is determined, and the third load control tag instructs to process at least one second instruction.

As an implementation manner, the number of second instructions is multiple; the third load control label indicates at least one of the following contents: combining multiple DCs formed by multiple second instructions, and combining multiple DCs corresponding to multiple second instructions Combine multiple Render Passes.

This embodiment of the present application also provides another terminal device, as shown in FIG. 14 . For the convenience of description, only the part related to the embodiment of the present application is shown. If the specific technical details are not disclosed, please refer to the method of the embodiment of the present application. part. The terminal can be any terminal device including mobile phone, tablet computer, personal digital assistant (full name in English: Personal Digital Assistant, English abbreviation: PDA), sales terminal (full name in English: Point of Sales, English abbreviation: POS), vehicle computer, etc. Take the terminal as a mobile phone as an example:

FIG. 14 is a block diagram showing a partial structure of a mobile phone related to a terminal provided by an embodiment of the present application. Referring to FIG. 14 , the mobile phone includes: a radio frequency (full name in English: Radio Frequency, English abbreviation: RF) circuit 1010, a memory 1020, an input unit 1030, a display unit 1040, a sensor 1050, an audio circuit 1060, a wireless fidelity (full name in English: wireless fidelity) , English abbreviation: WiFi) module 1070, central processing unit CPU1080, graphics processing unit GPU1090, power supply and other components. Those skilled in the art can understand that the structure of the mobile phone shown in FIG. 14 does not constitute a limitation on the mobile phone, and may include more or less components than the one shown, or combine some components, or arrange different components.

The following is a detailed introduction to each component of the mobile phone with reference to Figure 14:

The RF circuit 1010 can be used for receiving and sending signals during sending and receiving of information or during a call. In particular, after receiving the downlink information of the base station, it is processed by the CPU 1080; in addition, it sends the designed uplink data to the base station. Generally, the RF circuit 1010 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (full name in English: Low Noise Amplifier, English abbreviation: LNA), a duplexer, and the like. In addition, the RF circuit 1010 can also communicate with networks and other devices via wireless communication. The above wireless communication can use any communication standard or protocol, including but not limited to Global System for Mobile Communication (English full name: Global System of Mobile communication, English abbreviation: GSM), General Packet Radio Service (English full name: General Packet Radio Service, GPRS ), Code Division Multiple Access (English full name: Code Division Multiple Access, English abbreviation: CDMA), Wideband Code Division Multiple Access (English full name: Wideband Code Division Multiple Access, English abbreviation: WCDMA), Long Term Evolution (English full name: Long Term Evolution, English abbreviation: LTE), e-mail, short message service (full name in English: Short Messaging Service, SMS) and so on.

The memory 1020 can be used to store software programs and modules, and the CPU 1080 executes various functional applications and data processing of the mobile phone by running the software programs and modules stored in the memory 1020 . The memory 1020 may mainly include a stored program area and a stored data area, wherein the stored program area may store an operating system, an application program required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; Data created by the use of the mobile phone (such as audio data, phone book, etc.), etc. Additionally, memory 1020 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.

The input unit 1030 can be used for receiving inputted number or character information, and generating key signal input related to user setting and function control of the mobile phone. Specifically, the input unit 1030 may include a touch panel 1031 and other input devices 1032 . The touch panel 1031, also referred to as a touch screen, can collect the user's touch operations on or near it (such as the user's finger, stylus, etc., any suitable object or accessory on or near the touch panel 1031). operation), and drive the corresponding connection device according to the preset program. Optionally, the touch panel 1031 may include two parts, a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and then sends it to the touch controller. To the CPU1080, and can receive the command sent by the CPU1080 and execute it. In addition, the touch panel 1031 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves. Besides the touch panel 1031 , the input unit 1030 may also include other input devices 1032 . Specifically, other input devices 1032 may include, but are not limited to, one or more of physical keyboards, function keys (such as volume control keys, switch keys, etc.), trackballs, mice, joysticks, and the like.

The display unit 1040 may be used to display information input by the user or information provided to the user and various menus of the mobile phone. The display unit 1040 may include a display panel 1041. Optionally, a liquid crystal display (English full name: Liquid Crystal Display, English abbreviation: LCD), an organic light-emitting diode (English full name: Organic Light-Emitting Diode, English abbreviation: OLED), etc. form to configure the display panel 1041. Further, the touch panel 1031 can cover the display panel 1041. When the touch panel 1031 detects a touch operation on or near it, it transmits it to the CPU 1080 to determine the type of the touch event, and then the CPU 1080 displays the touch event on the display panel according to the type of the touch event. The corresponding visual output is provided on 1041. Although in FIG. 14, the touch panel 1031 and the display panel 1041 are used as two independent components to realize the input and input functions of the mobile phone, in some embodiments, the touch panel 1031 and the display panel 1041 can be integrated to form Realize the input and output functions of the mobile phone.

The cell phone may also include at least one sensor 1050, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 1041 according to the brightness of the ambient light, and the proximity sensor may turn off the display panel 1041 and/or when the mobile phone is moved to the ear. or backlight. As a kind of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (usually three axes), and can detect the magnitude and direction of gravity when it is stationary. games, magnetometer attitude calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors such as gyroscope, barometer, hygrometer, thermometer, infrared sensor, etc. Repeat.

The audio circuit 1060, the speaker 1061, and the microphone 1062 can provide an audio interface between the user and the mobile phone. The audio circuit 1060 can convert the received audio data into an electrical signal, and transmit it to the speaker 1061, and the speaker 1061 converts it into a sound signal and outputs it; After receiving, it is converted into audio data, and then the audio data is output to the CPU 1080 for processing, and then sent to, for example, another mobile phone via the RF circuit 1010, or the audio data is output to the memory 1020 for further processing.

WiFi is a short-distance wireless transmission technology. The mobile phone can help users to send and receive emails, browse web pages, and access streaming media through the WiFi module 1070, which provides users with wireless broadband Internet access. Although FIG. 14 shows the WiFi module 1070, it can be understood that it is not a necessary component of the mobile phone, and can be completely omitted as required within the scope of not changing the essence of the invention.

The CPU 1080 is the control center of the mobile phone, and uses various interfaces and lines to connect various parts of the entire mobile phone. Various functions and processing data for overall monitoring of the mobile phone. Optionally, the CPU 1080 may include one or more processing units; preferably, the CPU 1080 may integrate an application processor and a modulation and demodulation processor, wherein the application processor mainly processes the operating system, user interface, and application programs, etc. The processor mainly handles wireless communication. It can be understood that, the above-mentioned modulation and demodulation processor may not be integrated into the CPU 1080.

The GPU 1090 is mainly used to render the data in the memory 1020 into images, which are displayed in the display unit 1040 .

The mobile phone also includes a power source (such as a battery) for supplying power to various components. Preferably, the power source can be logically connected to the CPU 1080 through a power management system, so as to manage charging, discharging, and power consumption management functions through the power management system.

Although not shown, the mobile phone may also include a camera, a Bluetooth module, and the like, which will not be repeated here.

In this embodiment of the present application, the CPU 1080 included in the terminal also has the following functions:

Acquiring at least one first instruction, where the at least one first instruction is used to call an application program interface API, so that the graphics processor GPU performs the first rendering operation;

determining load data of a process in which the GPU performs the first rendering operation based on the at least one first instruction;

Determine a target scheme for controlling the load based on the load data;

Acquiring at least one second instruction, where the at least one second instruction is used to call the API, so that the GPU performs the second rendering operation;

The load during the execution of the second rendering operation by the GPU is controlled based on the target scheme.

In this embodiment of the present application, the GPU 1090 included in the terminal also has the following functions:

A first rendering operation is performed based on at least one first instruction, and a second rendering operation is performed based on at least one second instruction.

Embodiments of the present application further provide a chip including one or more processors. Part or all of the processor is used to read and execute the computer program stored in the memory, so as to execute the method of each embodiment corresponding to FIG. 2 .

Optionally, the chip includes a memory, and the memory and the processor are connected to the memory through a circuit or a wire. Further optionally, the chip further includes a communication interface, and the processor is connected to the communication interface. The communication interface is used for receiving data and/or information to be processed, the processor obtains the data and/or information from the communication interface, processes the data and/or information, and outputs the processing result through the communication interface. The communication interface may be an input-output interface.

Embodiments of the present application also provide a computer storage medium, where the computer storage medium is used for storing computer software instructions used by the above-mentioned computer device, which includes a program for executing a program designed for the computer device.

The computer device can be an apparatus for running an application program as described in the aforementioned FIG. 13 .

Embodiments of the present application further provide a computer program product, where the computer program product includes computer software instructions, and the computer software instructions can be loaded by a processor to implement the process in the method shown in FIG. 2 .

Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, device and unit described above may refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

In the several embodiments provided in this application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the present application can be embodied in the form of software products in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, and the computer software products are stored in a storage medium , including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, Read-Only Memory (ROM, Read-Only Memory), Random Access Memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes .

Claims

A method for controlling a load, comprising:

Acquiring at least one first instruction, where the at least one first instruction is used to call an application program interface API, so that the graphics processor GPU performs a first rendering operation;

determining load data of a process in which the GPU performs the first rendering operation based on the at least one first instruction;

determining a target solution for controlling the load based on the load data;

Acquiring at least one second instruction, where the at least one second instruction is used to call the API, so that the GPU performs a second rendering operation;

The load during the execution of the second rendering operation by the GPU is controlled based on the target scheme.
The method according to claim 1, wherein the determining, based on the at least one first instruction, the load data of the process in which the GPU performs the first rendering operation comprises:

At least one kind of load data of a process in which the GPU performs the first rendering operation is acquired based on the at least one first instruction and at least one load characteristic factor, and each load characteristic factor corresponds to one kind of load data.
The method of claim 2, wherein the at least one load characteristic factor comprises at least one of the following three types of load characteristic factors: a first type of load characteristic factor, a second type of load characteristic factor, and a third type of load characteristic factor load characteristic factor;

The first type of load characteristic factor is used to represent the consumption of computer hardware resources in the process of executing the first rendering operation by the GPU;

The second type of load characteristic factor is used to represent the information of the mesh Mesh model rendered by the GPU in the process of performing the first rendering operation;

The third type of load characteristic factor is used to represent the information of the at least one first instruction.
The method according to claim 3, wherein the first type of load characteristic factor comprises at least one of the following: calculation time in the process of the GPU performing the first rendering operation, a central processing unit (CPU) The operating parameters of the GPU, the operating parameters of the GPU and the operating parameters of the internal memory;

The second type of load characteristic factor includes at least one of the following: the number of vertices of each Mesh model rendered in the process of the GPU performing the first rendering operation, the process of the GPU performing the first rendering operation The number of vertices in each frame of image to be rendered, the number of Mesh models in each frame of image rendered by the process of the GPU performing the first rendering operation, and the process of the GPU performing the first rendering operation. The number of repetitions of the Mesh model in each frame of image and the distance of each Mesh model rendered by the GPU in the process of performing the first rendering operation relative to the screen; or

The third type of load characteristic factor includes at least one of the following: the number of drawing commands DC formed by the at least one first instruction, the number of rendering channels Render Pass corresponding to the at least one first instruction, the frame buffer Frame Buffer size, the reference relationship between the rendering pass and the frame buffer, and the reference relationship between the DC and the frame buffer.
The method according to claim 3 or 4, wherein, if the at least one load characteristic factor includes the second type of load characteristic factor, the at least one load characteristic factor is based on the at least one first instruction and the at least one load characteristic factor. Acquiring at least one kind of load data of the process in which the GPU performs the first rendering operation includes:

determining data of the mesh model rendered by the GPU by performing the first rendering operation based on the parameters carried by the at least one first instruction;

Based on the data of the mesh model rendered by the GPU performing the first rendering operation, the load data corresponding to the second type of load characteristic factor in the process of the GPU performing the first rendering operation is acquired.
The method according to any one of claims 1 to 5, wherein the determining a target scheme for controlling the load based on the load data comprises:

determining a load control tag based on the load data, the load control tag indicating the content of the load control;

One load control scheme is selected from at least one load control scheme based on the load control tag as a target scheme for controlling the load.
The method according to claim 6, wherein the load data comprises a first type of load data, and the first type of load data represents the consumption of computer hardware resources in the process of the GPU performing the first rendering operation Condition;

The determining of the load control tag based on the load data includes:

The first load control tag corresponding to the first load data is determined based on the first type of load data indicating that the consumption of computer hardware resources by the GPU in the process of performing the first rendering operation satisfies the first condition, and the The first load control tag instructs to adjust the operating parameters of the computer hardware in the process that the GPU performs the second rendering operation.
The method according to claim 7, wherein the first load control tag instructs to adjust the frequency of the CPU, the GPU or the internal memory during the process of the GPU performing the second rendering operation.
The method according to any one of claims 6 to 8, wherein the load data comprises a second type of load data, and the second type of load data represents the rendering by the GPU by performing the first rendering operation The information of the mesh Mesh model;

The determining of the load control tag based on the load data includes:

Based on the information that the first type of load data indicates that the mesh model rendered by the GPU performing the first rendering operation satisfies the second condition, the second load control label corresponding to the second load data is determined, and the The second load control tag indicates to simplify the Mesh model rendered in the process of the GPU performing the second rendering operation.
The method according to claim 9, wherein the second load control label indicates the characteristics of the mesh model to be simplified in the process of the GPU performing the second rendering operation, and the mesh model to be simplified is applicable and the parameters of the simplified algorithm.
The method according to claim 9 or 10, wherein the second type of load data includes the number of vertices and the number of Mesh models in each frame of image rendered by the first rendering operation;

The second condition includes that the number of vertices in each frame of image rendered by the first rendering operation is greater than a first threshold, or the number of Mesh models in each frame of image rendered by the first rendering operation is greater than a second threshold. .
The method according to any one of claims 9 to 11, wherein the second load control tag indicates a Mesh model with a number of vertices greater than a third threshold among the Mesh models rendered by the first rendering operation , Simplify the Mesh model whose number of repetitions is greater than the fourth threshold or the Mesh model whose distance from the screen is greater than the fifth threshold.
The method according to any one of claims 6 to 12, wherein the load data comprises a third type of load data, and the third type of load data represents information of the at least one first instruction;

The determining of the load control tag based on the load data includes:

Based on the third type of load data indicating that the instruction information of the at least one first instruction satisfies a third condition, a third load control tag corresponding to the third load data is determined, and the third load control tag indicates that the At least one second instruction is processed.
The method according to claim 13, wherein the number of the second instructions is multiple;

The third load control tag indicates at least one of the following: merging multiple DCs formed by multiple second instructions, and merging multiple rendering channels corresponding to multiple second instructions.
A device for controlling a load, comprising:

an instruction acquisition module, configured to acquire at least one first instruction, where the at least one first instruction is used to call an application program interface API, so that the graphics processor GPU performs the first rendering operation;

a load data determination module, configured to determine load data of a process in which the GPU performs the first rendering operation based on the at least one first instruction;

a target solution determination module, configured to determine a target solution for controlling the load based on the load data;

an instruction acquisition module, further configured to acquire at least one second instruction, where the at least one second instruction is used to call the API, so that the GPU performs a second rendering operation;

A control module, configured to control the load in the process of executing the second rendering operation by the GPU based on the target scheme.
A terminal device, comprising: one or more processors and a memory; wherein, the memory stores computer-readable instructions;

The computer readable instructions are read by the one or more processors to cause the computer device to implement the method of any of claims 1 to 14.
A computer-readable storage medium, characterized by comprising computer-readable instructions, which, when executed on a computer, cause the computer to execute the method according to any one of claims 1 to 14 .
A computer program product comprising computer readable instructions which, when executed on a computer, cause the computer to perform the method of any one of claims 1 to 14.