WO2023024649A1 - 图形渲染方法、装置及存储介质 - Google Patents

图形渲染方法、装置及存储介质 Download PDF

Info

Publication number
WO2023024649A1
WO2023024649A1 PCT/CN2022/097714 CN2022097714W WO2023024649A1 WO 2023024649 A1 WO2023024649 A1 WO 2023024649A1 CN 2022097714 W CN2022097714 W CN 2022097714W WO 2023024649 A1 WO2023024649 A1 WO 2023024649A1
Authority
WO
WIPO (PCT)
Prior art keywords
target
shader
terminal device
simplified
information
Prior art date
Application number
PCT/CN2022/097714
Other languages
English (en)
French (fr)
Inventor
孙黎
尹朝阳
李卫
刘金晓
Original Assignee
荣耀终端有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 荣耀终端有限公司 filed Critical 荣耀终端有限公司
Priority to EP22859996.5A priority Critical patent/EP4358028A4/en
Publication of WO2023024649A1 publication Critical patent/WO2023024649A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/505—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/485—Task life-cycle, e.g. stopping, restarting, resuming execution
    • G06F9/4856—Task life-cycle, e.g. stopping, restarting, resuming execution resumption being on a different machine, e.g. task migration, virtual machine migration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—3D [Three Dimensional] image rendering

Definitions

  • Shaders are editable programs that can be used to implement graphics rendering.
  • Graphics rendering is the process by which a computer converts shapes stored in memory into images displayed on the screen.
  • the shader may consume too much calculation of the shader computing unit (for example: shader core) in the graphics processing unit (graphic processing unit, GPU), making The GPU cannot complete the rendering within one frame time, causing the terminal device to freeze.
  • shader computing unit for example: shader core
  • graphics processing unit graphics processing unit, GPU
  • Embodiments of the present application provide a graphics rendering method, device, and storage medium, which relate to the technical field of image processing, and help to keep applications in terminal devices running smoothly even in complex scenarios.
  • the embodiment of the present application provides a graphics rendering method, the method includes: the terminal device uses the target shader to render the graphics of the application program; when the terminal device determines that the application program runs to a complex scene, the terminal device obtains the target simplified coloring
  • a complex scene is a scene in which the number of instructions required by the application program to run is greater than the first preset threshold, and the target simplified shader occupies less computing resources of the terminal device than the target shader occupies; the terminal device utilizes Targeted simplification shaders for graphics rendering of applications in complex scenes.
  • the terminal device when the terminal device determines that the application program runs to a complex scene, it obtains the target simplified shader, and uses the target simplified shader to render the graphics of the application program in the complex scene.
  • the occupancy is smaller than the occupancy of the computing resources of the terminal device by the target shader, thus helping to make the application program in the terminal device run smoothly even in complex scenes.
  • the terminal device obtaining the target simplified shader includes: the terminal device obtains the target simplified shader according to multiple pre-stored correspondences; the multiple correspondences include shader information, device hardware information, and Simplify the correspondence between shaders; shader information includes any one of shaders or shader identifiers.
  • the above-mentioned terminal device obtains the information of the target shader according to multiple pre-stored correspondences, including: the terminal equipment obtains the information of the target shader and the hardware information of the terminal device in the multiple correspondences
  • the corresponding target simplifies the shader.
  • the terminal device can acquire a simplified shader that helps improve the operating performance of the terminal device when performing graphics rendering according to the information of the shader and the hardware information of the device.
  • the above-mentioned corresponding relationship also includes a load level; the load level is a quantized value corresponding to the load of the terminal device; the above-mentioned terminal device obtains the information of the target shader according to multiple pre-stored corresponding relationships, including: terminal The device obtains the number of rendering instructions in complex scenes; the terminal device obtains the target load level according to the number of rendering instructions; the terminal device obtains the information of the target shader, the target load level, and the hardware information of the terminal device in multiple correspondences.
  • the target simplifies the shader. In this way, the terminal device can obtain a simplified shader that helps improve the operating performance of the terminal device when performing graphics rendering in the complex scene according to the information of the shader, the target load level, and the hardware information of the device.
  • the method further includes: the terminal device simplifies the target shader according to a first preset rule to obtain multiple simplified shaders; the first preset rule includes code movement, expression deletion, or syntax simplification At least one of: a plurality of simplified shaders includes a target simplified shader; the terminal device collects the status information of the terminal device when the terminal device uses multiple simplified shaders to perform graphics rendering of the application program in a complex scene; the status information of the terminal device includes The load of the terminal device, the power consumption of the terminal device, the temperature of the terminal device, the rendering time, and the image quality of the picture rendered by each simplified shader; the rendering time is the graphic rendering of the application program performed by each simplified shader in a complex scene The time required; the terminal device obtains the information of the target shader, the hardware information of the terminal device, the target load level and the corresponding relationship between the target simplified shader according to the state information.
  • a plurality of simplified shaders includes a target simplified shader
  • the terminal device collects the status information of the terminal device when the
  • the terminal device can analyze the hardware information of the terminal device corresponding to the multiple simplified shaders corresponding to the target shader in advance. According to the state information of the terminal device, a target simplified shader with good performance can be obtained.
  • the above-mentioned terminal device obtains the information of the target shader, the hardware information of the terminal device, the target load level, and the corresponding relationship between the target simplified shader according to the status information, including: the terminal device obtains the corresponding relationship between the target shader from the status information Obtain multiple target rendering durations, and the multiple target rendering durations are all less than the second preset threshold; the terminal device obtains the target state information from the state information corresponding to the multiple target rendering durations; the image quality of the target state information is greater than the third preset Threshold; the target state information is the state information of the terminal device when the target simplified shader performs graphics rendering of the application program in a complex scene; the terminal device obtains the target load level according to the load of the terminal device in the target state information; the terminal device establishes the target shader information, the hardware information of the end device, the target load level, and the correspondence between target simplified shaders.
  • the terminal device can first filter according to the rendering time, and then filter the target simplified shader according to the quality of rendering, and establish the information of the target shader, the hardware information of the terminal device, the target load level and the target simplified shader Correspondence between.
  • the above-mentioned terminal device obtains the information of the target shader, the hardware information of the terminal device, the target load level and the corresponding relationship between the target simplified shader according to the status information, including: the terminal device obtains multiple The target image quality, multiple target image qualities are greater than the fourth preset threshold; the terminal device obtains the target state information from the state information corresponding to the multiple target image quality; the rendering time of the target state information is less than the fifth preset threshold ;Target status information is the status information of the terminal device when the target simplified shader performs graphics rendering of the application program in a complex scene; the terminal device obtains the target load level according to the load of the terminal device in the target status information; the terminal device establishes the information of the target shader , the hardware information of the terminal device, the target load level, and the corresponding relationship between the target simplified shader.
  • the terminal device can first filter according to the image quality obtained by rendering, and then filter according to the rendering time to obtain the target simplified shader, and establish the information of the target shader, the hardware information of the terminal device, the target load level and the target simplified shader Correspondence between shaders.
  • the method further includes: the terminal device receives multiple correspondences sent by other devices.
  • other devices simplify the target shader according to a first preset rule to obtain multiple simplified shaders;
  • the first preset rule includes at least one of code movement, expression deletion, or syntax simplification;
  • Multiple simplified shaders include target simplified shaders;
  • other devices collect state information of other devices when other devices use multiple simplified shaders to perform graphics rendering of applications in complex scenes;
  • state information of other devices includes loads of other devices, The power consumption of other devices, the temperature of other devices, the rendering time, and the image quality of the picture rendered by each simplified shader;
  • the rendering time is the time required for each simplified shader to render the graphics of the application in a complex scene;
  • Other devices obtain the corresponding relationship between the information of the target shader, the hardware information of other devices, the target load level and the target simplified shader according to the state information, and send the acquired corresponding relationship to the terminal device.
  • the above-mentioned other devices obtain the information of the target shader, the hardware information of other devices, the target load level, and the corresponding relationship between the target simplified shaders according to the status information, including: other devices obtain the information from the status information Obtain multiple target rendering durations, and the multiple target rendering durations are all less than the second preset threshold; other devices obtain target status information from the status information corresponding to multiple target rendering durations; the image quality of the target status information is greater than the third preset Threshold; the target state information is the state information of other devices when the target simplified shader renders the graphics of the application in a complex scene; other devices obtain the target load level according to the load of other devices in the target state information; other devices establish the target shader information, hardware information of other devices, target load class, and the correspondence between target simplified shaders.
  • other devices can first filter according to the rendering time, and then filter the target simplified shader according to the rendering quality, and establish the information of the target shader, hardware information of other devices, target load level and target simplified shader Correspondence between.
  • the above-mentioned other devices obtain the information of the target shader, the hardware information of other devices, the target load level, and the corresponding relationship between the target simplified shaders according to the status information, including: other devices obtain the information from the status information Obtain multiple target picture quality, and the multiple target picture quality is greater than the fourth preset threshold; other devices obtain the target state information from the state information corresponding to the multiple target picture quality; the rendering time of the target state information is less than the fifth Preset threshold; the target state information is the state information of other devices when the target simplifies the shader's application graphics rendering in complex scenes; other devices obtain the target load level according to the load of other devices in the target state information; other devices establish target shader Information about shaders, hardware information of other devices, target load levels, and the correspondence between target simplified shaders.
  • the terminal device obtaining the target simplified shader includes: the terminal device simplifies the target shader according to a first preset rule to obtain the target simplified shader; the first preset rule includes code movement, expression At least one of deletion or syntax simplification.
  • the terminal device when the terminal device determines that the application program runs to a complex scene, the terminal device obtains the target simplified shader, including: when the terminal device determines that the application program runs to a complex scene, and the code of the target shader is complex When the degree is greater than the sixth preset threshold, or the number of times the target shader is called is greater than the seventh preset threshold, the terminal device acquires the target simplified shader.
  • the embodiment of the present application provides a terminal device, including a processing module.
  • the processing module is used to use the target shader to perform graphics rendering of the application program; when it is determined that the application program runs to a complex scene, obtain the target simplified shader; the number of instructions required for the operation of the application program in a complex scene is greater than the first preset threshold In the scene, the target simplified shader occupies less computing resources of the terminal device than the target shader occupies; the target simplified shader is used to render the graphics of the application in complex scenes.
  • the processing module is specifically used to: obtain the target simplified shader according to multiple pre-stored correspondences; the multiple correspondences include shader information, device hardware information, and correspondence between simplified shaders; the shader The information includes either the shader or the shader's ID.
  • the processing module is specifically configured to: acquire the information of the target shader and the target simplified shader corresponding to the hardware information of the terminal device among the multiple correspondences.
  • the above corresponding relationship also includes a load level; the load level is a quantized value corresponding to the load of the device; the processing module is specifically used to: obtain the number of drawing instructions in a complex scene; obtain the target load level according to the number of drawing instructions ; Among the multiple correspondences, obtain the information of the target shader, the target load level, and the target simplified shader corresponding to the hardware information of the terminal device.
  • the processing module is further configured to: simplify the target shader according to a first preset rule to obtain multiple simplified shaders; the first preset rule includes at least one of code movement, expression deletion, or syntax simplification; multiple A simplified shader includes a target simplified shader; when the terminal device uses multiple simplified shaders to render the graphics of the application program in a complex scene, the state information of the terminal device is collected; the state information of the terminal device includes the load of the terminal device, the state information of the terminal device Power consumption, temperature of the terminal device, rendering time, and the image quality of the picture rendered by each simplified shader; the rendering time is the time required for each simplified shader to render the graphics of the application in a complex scene; according to the state information Obtain the correspondence between target shader information, terminal device hardware information, target load level, and target simplified shaders.
  • the processing module is specifically configured to obtain multiple target rendering durations from the state information, and the multiple target rendering durations are all less than the second preset threshold; obtain target state information from the state information corresponding to the multiple target rendering durations; the target The picture quality of the state information is greater than the third preset threshold; the target state information is the state information of the terminal device when the target simplified shader performs graphics rendering of the application program in a complex scene; the target load is obtained according to the load of the terminal device in the target state information Level; establish the correspondence between target shader information, terminal device hardware information, target load level, and target simplified shader.
  • the processing module is specifically configured to: obtain multiple target image qualities from the state information, and the multiple target image qualities are all greater than the fourth preset threshold; obtain the target image quality from the state information corresponding to the multiple target image qualities. State information; the rendering duration of the target state information is less than the fifth preset threshold; the target state information is the state information of the terminal device when the target simplified shader performs graphics rendering of the application program in a complex scene; according to the load of the terminal device in the target state information Obtain the target load level; establish the correspondence between target shader information, terminal device hardware information, target load level, and target simplified shader.
  • the terminal device further includes a communication module: configured to receive multiple correspondences sent by other devices.
  • the processing module is specifically configured to: simplify the target shader according to a first preset rule to obtain a target simplified shader; the first preset rule includes at least one of code movement, expression deletion, or syntax simplification.
  • the processing module is specifically configured to: determine that the application program runs into a complex scene, and the code complexity of the target shader is greater than the sixth preset threshold, or the target shader When the number of calls is greater than the seventh preset threshold, the target simplified shader is obtained.
  • the embodiment of the present application provides a server, including a processing module.
  • the processing module is used to: simplify the target shader according to the first preset rule to obtain multiple simplified shaders; the first preset rule includes at least one of code movement, expression deletion, or syntax simplification; the multiple simplified shaders include Target simplified shader; collect terminal device status information when using multiple simplified shaders to render application graphics in complex scenes; terminal device status information includes terminal device load, terminal device power consumption, terminal device The temperature, rendering time, and the quality of the picture rendered by each simplified shader; the rendering time is the time required for each simplified shader to render the graphics of the application in a complex scene; get the target shader according to the status information information, the hardware information of the end device, the target load level, and the correspondence between target simplified shaders.
  • the processing module is specifically configured to obtain multiple target rendering durations from the state information, and the multiple target rendering durations are all less than the second preset threshold; obtain target state information from the state information corresponding to the multiple target rendering durations; the target The picture quality of the state information is greater than the third preset threshold; the target state information is the state information of the terminal device when the target simplified shader performs graphics rendering of the application program in a complex scene; the target load is obtained according to the load of the terminal device in the target state information Level; establish the correspondence between target shader information, terminal device hardware information, target load level, and target simplified shader.
  • the processing module is specifically configured to: obtain multiple target image qualities from the state information, and the multiple target image qualities are all greater than the fourth preset threshold; obtain the target image quality from the state information corresponding to the multiple target image qualities. State information; the rendering duration of the target state information is less than the fifth preset threshold; the target state information is the state information of the terminal device when the target simplified shader performs graphics rendering of the application program in a complex scene; according to the load of the terminal device in the target state information Obtain the target load level; establish the correspondence between target shader information, terminal device hardware information, target load level, and target simplified shader.
  • the terminal device further includes a communication module: configured to send multiple correspondences to the terminal device.
  • an embodiment of the present application provides a computer-readable storage medium, in which a computer program or instruction is stored, and when the computer program or instruction is run on a computer, the computer executes the first aspect or the first aspect.
  • the embodiment of the present application provides a computer program product including a computer program.
  • the computer program When the computer program is run on the computer, the computer executes the graphics described in the first aspect or any possible implementation of the first aspect. rendering method.
  • the present application provides a chip or a chip system, the chip or chip system includes at least one processor and a communication interface, the communication interface and the at least one processor are interconnected through lines, and the at least one processor is used to run computer programs or instructions, To execute the graphics rendering method described in the first aspect or any possible implementation manner of the first aspect.
  • the communication interface in the chip may be an input/output interface, a pin or a circuit, and the like.
  • the chip or the chip system described above in the present application further includes at least one memory, and instructions are stored in the at least one memory.
  • the memory may be a storage unit inside the chip, such as a register, a cache, etc., or a storage unit of the chip (eg, a read-only memory, a random access memory, etc.).
  • FIG. 1 is a schematic diagram of the corresponding relationship between the frame rate and the running time applicable to the embodiment of the present application;
  • FIG. 2 is a schematic structural diagram of a simplified platform applicable to the graphics rendering method provided by the embodiment of the present application;
  • FIG. 3 is a schematic structural diagram of a graphics rendering system applicable to the graphics rendering method provided in the embodiment of the present application;
  • FIG. 4 is a schematic structural diagram of an electronic device to which the graphics rendering method provided in the embodiment of the present application is applicable;
  • FIG. 5 is a software structural block diagram of an electronic device applicable to an embodiment of the present application.
  • FIG. 6 is a schematic flow diagram of a learning process of multiple object correspondences in a graphics rendering method provided in an embodiment of the present application
  • FIG. 7 is a schematic diagram of a scene of an application program applicable to an embodiment of the present application.
  • FIG. 8A is a schematic diagram of a shader and its corresponding simplified shader provided by the embodiment of the present application.
  • FIG. 8B is a schematic flowchart of another method for obtaining a simplified shader provided by the embodiment of the present application.
  • FIG. 9 is a schematic flow diagram describing the learning process of multiple target correspondences from another perspective provided by the embodiment of the present application.
  • FIG. 10 is a schematic flowchart of using multiple target correspondences to simplify the shader in the running process of the application program in a graphics rendering method provided by an embodiment of the present application;
  • FIG. 11 is a schematic flowchart of another graphics rendering method provided by the embodiment of the present application.
  • FIG. 12 is a schematic flowchart of another graphics rendering method provided by the embodiment of the present application.
  • FIG. 13 is a schematic structural diagram of a terminal device provided in an embodiment of the present application.
  • FIG. 14 is a schematic structural diagram of a server provided by an embodiment of the present application.
  • FIG. 15 is a schematic structural diagram of a chip provided by an embodiment of the present application.
  • a frame refers to a single picture that is the smallest unit in the interface display.
  • a frame can be understood as a still picture, and displaying multiple connected frames in rapid succession can form the illusion of object motion.
  • the frame rate refers to the number of frames that refresh the picture in 1 second, and can also be understood as the number of times the GPU in the terminal device refreshes the picture per second.
  • a high frame rate results in smoother and more realistic animations. The more frames per second, the smoother the displayed motion will be.
  • Frame drawing refers to picture drawing of the display interface.
  • the display interface can be composed of one or more views. Each view can be drawn by the visual control of the view system. Each view is composed of subviews. A subview corresponds to a widget in the view. For example, one of the subviews corresponds to the image view. A symbol in .
  • Frame rendering is to perform coloring operations or add 3D effects to the drawn view.
  • the 3D effect may be a lighting effect, a shadow effect, a texture effect, and the like.
  • Shaders are editable programs that can run on the GPU to achieve graphics rendering. Shaders include vertex shaders, pixel shaders, fragment shaders, geometry shaders, and compute shaders.
  • the hardware information may include at least one of chip type, chip performance, such as: central processing unit (central processing unit, CPU) frequency or GPU frequency, and the like.
  • Scene complexity can be characterized by the number of draw calls in the scene.
  • the drawing instruction is a calling command of the CPU to the underlying graphics drawing interface.
  • Rendering instructions are used by terminal devices to perform tasks such as detecting rendering status, submitting rendering data, and submitting rendering status.
  • words such as “first” and “second” are used to distinguish the same or similar items with basically the same function and effect.
  • the first chip and the second chip are only used to distinguish different chips, and their sequence is not limited.
  • words such as “first” and “second” do not limit the number and execution order, and words such as “first” and “second” do not necessarily limit the difference.
  • “at least one” means one or more, and “multiple” means two or more.
  • “And/or” describes the association relationship of associated objects, indicating that there may be three types of relationships, for example, A and/or B, which can mean: A exists alone, A and B exist simultaneously, and B exists alone, where A, B can be singular or plural.
  • the character “/” generally indicates that the contextual objects are an “or” relationship.
  • “At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items.
  • At least one item (piece) of a, b, or c can represent: a, b, c, a-b, a--c, b-c, or a-b-c, where a, b, c can be single or is multiple.
  • the terminal device may freeze (also known as frame drop phenomenon).
  • FIG. 1 when the application program is running in the terminal device, it is a graph corresponding to the frame rate of the application program and the running time of the application program.
  • the horizontal axis is the running time of the application
  • the vertical axis is the frame rate when the application is running.
  • the frame rate of the application is set to 60fps.
  • the frame rate of the application is only 4 minutes and 7 seconds. It is 31.34fps, and it is displayed on the terminal device at this time.
  • the terminal device freezes.
  • the complex scene may refer to a scene in which the number of rendering instructions is greater than a first preset threshold.
  • the drawing instruction is a calling command of the CPU to the underlying graphics drawing interface.
  • Rendering instructions are used by terminal devices to perform tasks such as detecting rendering status, submitting rendering data, and submitting rendering status.
  • factors affecting the rendering time may also include: the scene, the shader used in the application program, and the hardware information of the terminal device carrying the application program.
  • the scenarios may include simple scenarios and complex scenarios.
  • a simple scene may refer to a scene in which the number of rendering instructions is less than or equal to a first preset threshold.
  • the hardware information of the terminal device may include: at least one of the type of chip used by the terminal device or the performance of the chip in the terminal device, such as: a central processing unit (central processing unit, CPU) frequency or a GPU frequency.
  • the hardware information of the terminal device has been fixed, and the scene is not suitable for changing. Therefore, it is helpful to simplify the shader used to render the scene in the terminal device to reduce the load of the GPU. In order to make the application program in the terminal device run in a complex scene, it can still run smoothly.
  • the embodiment of the present application provides a graphics rendering method, which can simplify the shader into a simplified shader when the terminal device runs the application program, because the simplified shader occupies less workload of the computing unit in the GPU when it is running. Small, it can effectively improve the stuttering phenomenon of the terminal device caused by the GPU occupation of the shader.
  • the terminal device simplifies the shader into a simplified shader that can include two implementations:
  • the target correspondences include shader information, simplified shader information, and terminal devices.
  • the terminal device can obtain the target from the target correspondence based on the scene complexity of the scene in the application program, the shader information of the shader used by the application program, and its own hardware information. Simplify the shader information and use the simplified shader to run the application, because the simplified shader occupies less workload of the computing unit in the GPU when it is running, which can effectively improve the terminal device stuttering phenomenon caused by the GPU occupation of the shader .
  • the graphics rendering method provided in the embodiment of the present application may be applied to a simplified platform.
  • the simplification platform can be used to provide unified shader simplification functions for different applications and terminal devices with different hardware information.
  • FIG. 2 is a schematic structural diagram of a simplified platform applicable to the graphics rendering method provided by the embodiment of the present application.
  • the simplified platform may include a shader simplification module and a shader replacement module.
  • the shader simplification module is used to obtain the above-mentioned multiple target correspondences.
  • the shader replacement module is used by the terminal device to replace the shader with a simplified shader during the running of the application. It can be understood that the shader simplification module and the shader replacement module can run on different devices, and can also implement their respective functions at different time periods, which is not limited in this embodiment of the present application.
  • the shader simplification module may include a simplification subunit, an application program and hardware information collection unit, and a shader screening unit.
  • the simplification subunit is used for: collecting the shaders (for example: vertex shader and/or fragment shader, etc.) Simplify, get multiple simplified shaders, and store the multiple simplified shaders.
  • shaders for example: vertex shader and/or fragment shader, etc.
  • the application program and hardware information collection unit is used to: collect scene data (also known as status information) during the running of the application program (such as the simplified application program described above), and collect hardware information, such as the CPU frequency and/or GPU frequency of the terminal device at least one of the information.
  • scene data also known as status information
  • hardware information such as the CPU frequency and/or GPU frequency of the terminal device at least one of the information.
  • the shader filtering unit is used to: filter the simplified shaders according to the collected scene data and hardware information to obtain multiple target correspondences.
  • the rendering time also known as the rendering time
  • a simplified shader whose image quality is greater than the second preset threshold is selected (also known as simplified model).
  • the image quality of the picture rendered in the scene data is segmented, and a simplified shader whose rendering time is shorter than a third preset threshold is selected in each segment category.
  • the shader replacement module may include: a shader list management unit, an application instruction interception unit, a shader comparison unit, a scene recognition unit, a shader decision selection unit and a shader replacement unit.
  • the shader list (that is, the multiple target correspondences) management unit is configured to: organize and store the multiple target correspondences obtained above according to a preset data structure.
  • the preset data structure may be preset by the code.
  • shader list management unit can be integrated in the shader filter unit.
  • all or part of the functions in the shader list management unit can be integrated in the decision selection unit.
  • This embodiment of the present application does not limit it.
  • the function of obtaining the shader list can be integrated in the shader filtering unit.
  • the instruction interception unit is used for: intercepting the calling instruction to the shader in the application program.
  • the shader comparison unit is used to: find the intercepted shader in the shader list. If the search fails (that is, the intercepted shader does not exist in the shader list), the shader in the shader call instruction will not be replaced, and the original process in the application program will continue to be executed to the GPU for the next rendering.
  • the scene identification unit is used for: if the shader comparison is successful (that is, there is an intercepted shader in the shader list), then identify the complexity of the scene. For example: identifying the scene complexity of the scene corresponding to the shader called by the intercepted call instruction.
  • the decision and selection unit is configured to: obtain target simplified shader information among multiple target correspondences based on the scene complexity of the scene in the application program, the shader information used by the application program, and the hardware information of the terminal device.
  • the shader replacement unit is used to: replace the shader in the interception instruction, compile the replaced shader, and send it to the GPU for rendering.
  • the application program (application, APP) in FIG. 2 is used to represent the application program running in the terminal device.
  • FIG. 3 is a schematic structural diagram of a graphics rendering system to which the graphics rendering method provided in the embodiment of the present application is applicable.
  • FIG. 3 includes a terminal device 301 and a server 302 .
  • the terminal device 301 may be a smart phone, a desktop computer, a notebook computer, a tablet computer, a virtual reality (virtual reality, VR) device, an augmented reality (augmented reality, AR) device, or a smart wearable device.
  • the server 302 may be a server, a server cluster, or a cloud server. This embodiment of the present application does not limit it.
  • the terminal device 301 in FIG. 3 may be used to implement the functions of one or more units in FIG. 2 above.
  • the functions of other units are implemented by the server 302 .
  • the terminal device 301 may be used to realize the function of the shader replacement module in FIG. 2 above, and the server 302 may be used to realize the function of the shader simplification module.
  • the terminal device 301 may be used to implement the functions of the instruction interception unit, the scene recognition unit, and the shader replacement unit.
  • the server 302 can be used to implement the functions of the shader simplification module, the shader list management unit, the shader comparison unit, the decision selection unit, and the like.
  • FIG. 4 is a schematic structural diagram of an electronic device to which the graphics rendering method provided by the embodiment of the present application is applicable.
  • the electronic device 40 shown in FIG. 4 may include: a processor 401, a memory 402, a display screen 403, and a communication module 404.
  • the structure shown in this embodiment does not constitute a specific limitation on the electronic device 40 .
  • the electronic device 40 may include more or fewer components than shown in the illustration, or combine certain components, or separate certain components, or arrange different components.
  • the illustrated components can be implemented in hardware, software, or a combination of software and hardware.
  • the processor 401 may include one or more processing units, for example: the processor 401 may include an application processor (application processor, AP), a modem processor, a GPU, an image signal processor (image signal processor, ISP), a control device, video codec, digital signal processor (digital signal processor, DSP), baseband processor, display processing unit (display process unit, DPU), and/or neural network processor (neural-network processing unit, NPU) wait.
  • application processor application processor
  • ISP image signal processor
  • control device video codec
  • digital signal processor digital signal processor
  • DSP digital signal processor
  • baseband processor display processing unit
  • display processing unit display process unit, DPU
  • NPU neural network processing unit
  • different processing units can be independent devices, and can also be integrated in one or more processors.
  • the electronic device 40 may also include one or more processors 401 .
  • the processor may be the nerve center and command center of the electronic device 40 .
  • the processor can generate an operation control signal according to the instruction opcode and the timing signal, and complete the control of fetching and executing the instruction.
  • a memory may also be provided in the processor 401 for storing instructions and data.
  • the memory in processor 401 is a cache memory. The memory may hold instructions or data used or recycled by the processor 401 . If the processor 401 needs to use the instruction or data again, it can be called directly from the memory. This avoids repeated access, reduces the waiting time of the processor 401, and thus improves the efficiency of the electronic device 40.
  • the communication module 404 of the electronic device 40 can use any device such as a transceiver for communicating with other devices or communication networks, such as wide area network (wide area network, WAN), local area network (local area networks, LAN) and so on.
  • a transceiver for communicating with other devices or communication networks, such as wide area network (wide area network, WAN), local area network (local area networks, LAN) and so on.
  • the electronic device 40 can implement a display function through a GPU, a display screen 403 , and an application processor.
  • Application processors may include NPUs and DPUs.
  • the GPU is a microprocessor for image processing, and is connected to the display screen 403 and the application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering.
  • Processor 401 may include one or more GPUs that execute instructions to generate or change display information.
  • the NPU is a neural-network (NN) computing processor.
  • NN neural-network
  • DPU is also called display sub-system (display sub-system, DSS), DPU is used to adjust the color of display screen 403, DPU can adjust the color of display screen through three-dimensional look up table (3D look up table, 3D LUT) .
  • the DPU can also perform image scaling, noise reduction, contrast enhancement, backlight brightness management, hdr processing, display parameter Gamma adjustment and other processing.
  • the display screen 403 is used to display images, videos and the like.
  • the display screen 403 includes a display panel.
  • the display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active matrix organic light emitting diode or an active matrix organic light emitting diode (active-matrix organic light emitting diode, AMOLED), flexible light-emitting diode (flex light-emitting diode, FLED), Miniled, MicroLed, Micro-oLed, quantum dot light emitting diodes (quantum dot light emitting diodes, QLED), etc.
  • the electronic device 40 may include 1 or N display screens 403, where N is a positive integer greater than 1.
  • Memory 402 may be used to store one or more computer programs comprising instructions.
  • the processor 401 may execute the above-mentioned instructions stored in the memory 402 to make the electronic device 40 execute various functional applications and data processing.
  • the memory 402 may include an area for storing programs and an area for storing data.
  • the stored program area can store an operating system; the stored program area can also store one or more application programs (such as a gallery, contacts, etc.) and the like.
  • the data storage area can store data (such as photos, contacts, etc.) created during the use of the electronic device 40 .
  • the memory 402 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, flash memory device, universal flash storage (universal flash storage, UFS) and the like.
  • the processor 401 can cause the electronic device 40 to execute various functional applications and data processing by executing instructions stored in the memory 402 and/or instructions stored in the memory provided in the processor 401 .
  • the memory 402 is used to store multiple object correspondences in the embodiment of the present application.
  • the display screen 403 is used to display images, videos and the like.
  • the display screen 403 includes a display panel.
  • the display panel can adopt liquid crystal display (liquid crystal display, LCD), organic light-emitting diode (organic light-emitting diode, OLED), active-matrix organic light-emitting diode or active-matrix organic light-emitting diode (active-matrix organic light emitting diode, AMOLED), flexible light-emitting diode (flex light-emitting diode, FLED), Miniled, MicroLed, Micro-oLed, quantum dot light emitting diodes (quantum dot light emitting diodes, QLED), etc.
  • the electronic device 100 may include 1 or N display screens 403 , where N is a positive integer greater than 1.
  • the software system of the electronic device 40 may adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture.
  • the embodiment of the present application takes the Android system with a layered architecture as an example to illustrate the software structure of the electronic device 40 .
  • FIG. 5 is a software structural block diagram of an electronic device applicable to an embodiment of the present application.
  • the layered architecture divides the software system of the electronic device 40 into several layers, and each layer has a clear role and division of labor. Layers communicate through software interfaces.
  • the Android system can be divided into five layers, namely applications, application framework, Android runtime, system library, and hardware abstraction layer. layer, HAL) and the kernel layer (kernel).
  • the application program layer may include a series of application program packages, and the application program layer runs the application program by calling an application program interface (application programming interface, API) provided by the application program framework layer.
  • application program interface application programming interface, API
  • the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and games.
  • the application framework layer provides API and programming framework for applications in the application layer.
  • the application framework layer includes some predefined functions. As shown in Figure 5, the application framework layer can include window manager, content provider, view system, phone manager, resource manager, notification manager, database, etc.
  • a window manager is used to manage window programs.
  • the window manager can get the size of the display screen, determine whether there is a status bar, lock the screen, capture the screen, etc.
  • Content providers are used to store and retrieve data and make it accessible to applications. Data can include videos, images, audio, calls made and received, browsing history and bookmarks, phonebook, etc.
  • the view system includes visual controls, such as controls for displaying text, controls for displaying pictures, and so on.
  • the view system can be used to build applications.
  • a display interface can consist of one or more views. For example, a display interface including a text message notification icon may include a view for displaying text and a view for displaying pictures.
  • the phone manager is used to provide communication functions of the electronic device 40 .
  • the resource manager provides various resources for the application, such as localized strings, icons, pictures, layout files, video files, etc.
  • the notification manager enables the application to display notification information in the status bar, which can be used to convey notification-type messages, and can automatically disappear after a short stay without user interaction.
  • the notification manager is used to notify the download completion, message reminder, etc.
  • the notification manager can also be a notification that appears on the top status bar of the system in the form of a chart or scroll bar text, such as a notification of an application running in the background, or a notification that appears on the screen in the form of a dialog window. For example, text information is prompted in the status bar, a prompt sound is issued, the electronic device 40 vibrates, and the indicator light flashes, etc.
  • Databases can be used to organize, store and manage data in terms of data structures.
  • the Android runtime includes core libraries and a virtual machine.
  • the Android runtime is responsible for the scheduling and management of the Android system.
  • the core library consists of two parts: one part is the function function that the java language needs to call, and the other part is the core library of Android.
  • the application layer and the application framework layer run in virtual machines.
  • the virtual machine executes the java files of the application program layer and the application program framework layer as binary files.
  • the virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.
  • a system library can include multiple function modules. For example: surface manager (surface manager), media library (media libraries), three-dimensional graphics processing library, such as: open graphics library (open graphics library, OpenGL), 2D graphics engine (for example: SGL), etc.
  • surface manager surface manager
  • media library media libraries
  • three-dimensional graphics processing library such as: open graphics library (open graphics library, OpenGL), 2D graphics engine (for example: SGL), etc.
  • the surface manager is used to manage the display subsystem and provides the fusion of 2D and 3D layers for multiple applications.
  • the media library supports playback and recording of various commonly used audio and video formats, as well as still image files, etc.
  • the media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.
  • the 3D graphics processing library is used to implement 3D graphics drawing, graphics rendering, synthesis and layer processing, etc.
  • 2D graphics engine is a drawing engine for 2D drawing.
  • the hardware abstraction layer can contain multiple library modules, and the library modules can include, for example, sensor library modules.
  • the Android system can load corresponding library modules for the device hardware, and then realize the purpose of the application framework layer accessing the device hardware.
  • the device hardware may include, for example, a temperature sensor in the electronic device 40 and the like.
  • the kernel layer is the layer between hardware and software.
  • the kernel layer is used to drive the hardware and make the hardware work.
  • the kernel layer includes at least a display driver, etc., which is not limited in this embodiment of the present application.
  • a graphics rendering method provided by an embodiment of the present application includes a process of learning multiple object correspondences and using the multiple object correspondences to simplify a shader process during the running of an application program.
  • FIG. 6 is a schematic flow diagram of a learning process of multiple target correspondences in a graphics rendering method provided in an embodiment of the present application. As shown in FIG. 6 , the following steps may be included:
  • S600 the terminal device captures the shader used in the running process of the application program.
  • the terminal device determines a shader call instruction according to a preset keyword, and intercepts the shader call instruction, where the call instruction includes at least one of a shader identifier or a shader. In the case that the shader is not included in the calling instruction, the terminal device obtains the shader according to the shader identifier.
  • the preset keyword may be specified by a code.
  • the preset keyword is ShaderSource
  • the terminal device acquires the shader calling instruction when it determines that the ShaderSource is included in the instruction.
  • the shader calling instruction obtained by the terminal device is glShaderSource(vertexShader, 1, &vertexShaderSource, NULL).
  • vertexShaderSource is the shader identifier.
  • the terminal device obtains the shader indicated by this identifier according to the vertexShaderSource.
  • the shader indicated by vertexShaderSource is used to render color for the target object in the target scene as shown in Figure 7 .
  • the target object is the soldier 701 in the scene shown in FIG. 7 , and the soldier 701 is drawn by the visual control of the view system in the terminal device.
  • the terminal device acquires a heavy-duty shader among the collected shaders according to at least one of the code complexity of the shader or the number of calls of the shader.
  • code complexity includes time complexity or space complexity, etc.
  • Time complexity is the change trend of code execution time with the increase of data size, also known as asymptotic time complexity.
  • the space complexity is asymptotic space complexity, and the space complexity represents the growth relationship between the storage space of the algorithm and the data scale.
  • the number of times the shader is called is the number of times the shader is called in the scene rendered by the shader. The more times a shader is called, the greater the load on that shader.
  • the terminal device determines a shader whose code complexity is greater than a fourth preset threshold as a heavy-load shader. Or, the terminal device determines a shader whose code complexity is greater than a fourth preset threshold and whose number of calls is greater than the fifth preset threshold as a heavy-load shader. The shader is determined to be a heavy duty shader.
  • the terminal device can obtain the rendering time of the shader.
  • the terminal device can use the shaders with the preset ratio as heavy load shaders in order of rendering time.
  • the terminal device sorts the acquired rendering durations of the shaders in descending order, and uses the shaders corresponding to the top 80% of the rendering durations as heavy-duty shaders.
  • the terminal device simplifies heavy-load shaders according to a first preset rule to obtain multiple simplified shaders.
  • the first preset rule includes at least one of code movement, expression deletion, or syntax simplification.
  • Code movement is to move the fragments in the fragment shader to the corresponding vertex shader, and use the target execution result in the fragment shader.
  • the target execution result is the execution result of the vertex shader executing the fragment.
  • a vertex shader that renders the same object corresponds to a fragment shader.
  • the expression is deleted.
  • the expression is also called a function.
  • the functions in the shader are all used to generate special effects or complete calculations. Therefore, the functions will have a certain impact on the final rendering result.
  • the preset influence value can be set by the code according to experience.
  • Syntax simplification is to simplify the code in the shader and reduce the amount of code.
  • the encapsulated package is used to replace the code that realizes the same function.
  • the terminal device selects at least one variable or expression from the first shader as a moving target, and the terminal device extracts a statement related to the calculation of the moving target in the first shader as a target segment, and moves the target segment to
  • the first shader is a fragment shader
  • the second shader is a vertex shader corresponding to the first shader.
  • the terminal device deletes the target fragment in the first shader to obtain a simplified shader of the first shader.
  • the simplified shader uses the execution result of the target fragment in the vertex shader.
  • the terminal device performs code movement on shader 1 to obtain simplified shader No. 1, the identifier of simplified shader No. 1 is 1-1, and the terminal device performs syntax simplification and expression on shader 1 Delete to get Simplified Shader No. 2, the logo of Simplified Shader No. 2 is 1-2, the terminal device performs code movement, expression deletion and syntax simplification on Shader 1 to get Simplified Shader No. 3, and the logo of Simplified Shader No. 3 for 1-3.
  • the terminal device extracts shaders in multiple application programs (such as: application program APP1 to application program APP3 in FIG. 8B ) according to the simplified model, and identifies the shaders in the extracted shaders.
  • the heavy load shader The terminal device simplifies the heavy-load shader according to the first preset rule to obtain a shader simplified model set. Among them, the simplified model is used to realize shader extraction, shader identification and shader simplification.
  • the simplified model extracts the shaders in the application programs APP1 to APP3 as shown in FIG. 8B .
  • the simplified model performs shader recognition on the extracted shaders to obtain the heavy-duty shaders in the application program APP1, including shader 1, shader 2 and shader 3.
  • the heavy duty shaders in the application APP2 include shader4, shader5 and shader6.
  • the heavy load shaders in the application APP3 include shader7, shader8 and shader9.
  • S601 the description in S601 above, and details are not repeated here.
  • the simplified model simplifies shader 1 according to different first preset rules to obtain three simplified shaders identified as 1-1 to 1-3.
  • the simplification of shader 2 to shader 9 by the simplified model is similar to this, and will not be repeated here.
  • the terminal device replaces the heavy-load shader in the application with different simplified shaders obtained by simplifying the heavy-load shader, to obtain multiple simplified applications.
  • the different simplified shaders are shaders obtained by simplifying the heavy load shader according to different first preset rules.
  • the terminal device replaces shader 1 in the application program APP1 with a simplified shader identified as 1-1 to obtain the first simplified application program, and the terminal device replaces shader 1 in the application program APP1 with the identification
  • the simplified shader of 1-2 obtains the second simplified application program, and the terminal device replaces shader 1 in the application program APP1 with the simplified shader identified by 1-3 to obtain the third simplified application program.
  • the terminal device collects status information of the plurality of simplified application programs running on the target device and hardware information of the target device.
  • the status information includes information such as the rendering time required by the simplified shader, the image quality of the rendered image, the load of the target device, the power consumption of the target device, or the temperature of the target device.
  • the hardware information includes at least one of chip type and chip performance such as central processing unit (central processing unit, CPU) frequency or GPU frequency.
  • the terminal device filters and simplifies the shaders according to the collected status information and hardware information, and obtains multiple target correspondences.
  • the target correspondence includes the correspondence of shader information (ie heavy load shader information), target hardware information, target load level and simplified shader information.
  • the target hardware information is hardware information included in the target device.
  • the target load level is the load level corresponding to the load of the target device when the simplified application program is run on the target device.
  • the terminal device segments the rendering duration in the collected state information according to the collected state information and the hardware information of the terminal device, and selects the target image quality corresponding to the segment in the segment , and determine a number of first simplified shaders.
  • the terminal device obtains multiple target correspondences according to the heavy-load shader information, the plurality of first simplified shader information, the hardware information of the terminal device, and the load level of the terminal device.
  • the target image quality is greater than the second preset threshold
  • the first simplified shader is a simplified shader in a simplified application corresponding to the target image quality.
  • the load level of the terminal device can be obtained according to the load of the terminal device in the state information.
  • the terminal device may quantify the load of the terminal device, and obtain a load level corresponding to the load of the terminal device as a fixed value.
  • the fixed value is the quantized value corresponding to the load of the terminal device. For example 1, a fixed value of 1 is used to characterize high load. A fixed value of 0 is used to characterize low load.
  • the terminal device segments the image quality in the collected state information according to the collected state information and the hardware information of the terminal device, and selects an image corresponding to the segment in each segment.
  • the terminal device obtains multiple target correspondences according to the heavy-load shader information, the plurality of second simplified shader information, the hardware information of the terminal device, and the load level of the terminal device.
  • the target rendering duration is less than the third preset threshold
  • the target simplified shader is a simplified shader in a simplified application corresponding to the target rendering duration.
  • shader files used by different applications are quite different, and different application manufacturers write and generate shaders in different ways, when the application runs on the terminal device, it is impossible to ensure that each application can run smoothly and stably. run.
  • terminal devices use many types of chips, and the GPU performance of the chips varies greatly.
  • the rendering time of the same shader file is different.
  • the simplified shaders corresponding to different hardware information and load levels of the shader can be obtained by the above method, so as to adapt to different applications and terminal devices with different hardware information .
  • a terminal device may validate a specific type of target correspondence in response to a configuration instruction.
  • the configuration instruction may be a user's operation instruction received by the terminal device.
  • FIG. 9 is a schematic flow diagram describing the learning process of multiple target correspondences from another perspective provided by the embodiment of the present application.
  • the heavy-duty shader in the application APP1 includes shader 1, shader 2 and 3, wherein the shader variants of shader 1 include shaders labeled 1-1 to 1-3, the shader variants of shader 2 include shaders labeled 2-1 to 2-3, and shader 3
  • the shader variants for include shaders labeled 3-1 and 3-2.
  • the terminal device replaces each variant (that is, shader variant) in the application program, that is, replaces shader 1 in the application program with its variant shader. Get multiple replaced applications.
  • This step corresponds to the description in S603 in the above embodiment, and will not be repeated here.
  • Run the replaced application on a device that includes different hardware information (such as different chips) and the terminal device records the runtime status of the application, and records the image quality, rendering time, CPU&GPU load, device temperature and other information of the rendered image .
  • This step corresponds to the description in S604 in the above embodiment, and will not be repeated here.
  • the terminal device divides the image quality of the rendered image, selects simplified models with different image quality, and obtains a simplified list of shaders. This step corresponds to the description in S901 in the above embodiment, and will not be repeated here.
  • FIG. 10 is a schematic flow diagram of using multiple target correspondences to simplify the shader in the running process of an application in a graphics rendering method provided in an embodiment of the present application. As shown in FIG. 10 , the following steps may be included:
  • the terminal device acquires a calling instruction of a source shader.
  • the source shader is used to achieve the rendering of the destination object.
  • the target object is an object in a target scene, and the target scene is a scene in an application program run on a terminal device.
  • the call instruction of the source shader includes source shader information.
  • the source shader information may include at least one of an identifier of the source shader or the source shader.
  • the terminal device determines the calling instruction of the source shader according to the preset keyword, and intercepts the calling instruction of the source shader.
  • the preset keyword may be specified by a code.
  • the preset keyword is ShaderSource
  • the terminal device acquires the source shader calling instruction when it determines that the instruction includes ShaderSource.
  • the source shader information included in the calling instruction of the source shader acquired by the terminal device is SShader.
  • the shader represented by SShader is used to render colors for the target object in the target scene as shown in FIG. 7 .
  • the target object is the soldier 701 in the scene shown in FIG. 7 , and the soldier 701 is drawn for the visual control of the view system in the terminal device.
  • S1001 The terminal device determines that source shader information is included in multiple pre-stored object correspondences.
  • the multiple target correspondences are the multiple target correspondences obtained in the above embodiments, and will not be repeated here.
  • the terminal device traverses the shader information in the pre-stored multiple target correspondences, and determines that the multiple target correspondences include the source shader information.
  • the terminal device determines that the multiple target correspondences include the SShader.
  • the terminal device traverses the shader information in the specific type of target correspondence, and determines that the specific type of target correspondence includes the source shader information.
  • a specific type of target correspondence is a target correspondence that takes effect on a terminal device according to a configuration instruction.
  • the terminal device acquires its own hardware information and the scene complexity of the target scene.
  • the terminal device acquires its hardware information, and calculates the number of rendering instructions in the target scene.
  • the terminal device determines the scene complexity of the target scene according to the number of rendering instructions.
  • the drawing instruction is a calling command of the CPU to the underlying graphics drawing interface.
  • the rendering instruction is used by the terminal device to perform tasks such as detecting the rendering state, submitting the rendering data, and submitting the rendering state.
  • the hardware information obtained by the terminal device includes that the main frequency of the CPU is 3.0 GHz, and the frequency of the GPU is 850 MHz. If the terminal device calculates that the number of rendering instructions in the target scene is 600, the first preset threshold is 300, and the number of rendering instructions calculated by the terminal device is greater than the first preset threshold, then the terminal device converts the scene The complexity is determined as a fixed value such as 1. 1 is used to represent high complexity of the scene.
  • the scene complexity corresponds to the load level in the foregoing embodiments. If the hardware information of the terminal devices is the same, the higher the complexity of the scene, the higher the load of the terminal device when it runs to the scene.
  • the terminal device acquires source shader information, hardware information of the terminal device, and target simplified shader information corresponding to the scene complexity of the target scene from the target correspondence including the source shader information.
  • the target correspondence including the source shader information is the target correspondence determined in S1001 including the source shader information.
  • the terminal device acquires source shader information, hardware information of the terminal device, and target simplified shader information corresponding to the scene complexity of the target scene from the target correspondence relationship including the source shader information.
  • the shader information SShader, the hardware information GPU frequency is 850MHz, the CPU main frequency is 3.0GHz, and the simplified shader information corresponding to scene complexity 1 is shader No. 1.
  • the shader information is SShader, the hardware information GPU frequency is 2330MHZ, the CPU main frequency is 3.6GHz, and the simplified shader information corresponding to scene complexity 1 is shader No. 2.
  • the shader information is VShader, the hardware information GPU frequency is 1000MHz, the CPU main frequency is 3.8GHz, and the simplified shader information corresponding to scene complexity 0 is shader number three. Wherein, the scene complexity represented by 0 is lower than the scene complexity represented by 1.
  • the terminal device obtains shader information from multiple target correspondences shown in Table 1.
  • the shader information is SShader
  • the CPU main frequency is 3.0GHz
  • the GPU frequency is 850MHz
  • the scene complexity of the target scene is 1.
  • the target simplified shader information is shader one.
  • the terminal device may also obtain source shader information and target simplified shader information corresponding to the hardware information of the terminal device from the target correspondence including the source shader information.
  • S1004 The terminal device replaces the source shader information in the calling instruction of the source shader with the target simplified shader information, and compiles the target simplified shader.
  • the terminal device When the source shader information is the source shader, the terminal device replaces the source shader with the target simplified shader, and converts the target simplified shader into a shader in a binary language that can be recognized by the terminal device.
  • the source shader and the target simplified shader are shaders written in the shader language.
  • both the source shader and the target simplified shader are shaders written in the OpenGL shading language.
  • the terminal device replaces the source shader indicated by the identification of the source shader with the target simplified shader, and converts the target simplified shader to be recognized by the terminal device Shaders for the binary language.
  • the terminal device transmits the compiled target simplified shader to the GPU to implement rendering of the target object.
  • the terminal device transmits the compiled target simplified shader to the GPU, and the GPU executes the target simplified shader to implement rendering of the target object.
  • the target object is a group of soldiers
  • the GPU executes the target simplified shader to render the same color for the armor worn by the group of soldiers.
  • the terminal device learns the simplified shaders corresponding to the shaders in different application programs in the terminal device with different hardware information, and obtains multiple target correspondences.
  • the terminal device can obtain target simplified shader information in the target correspondence based on the scene complexity of the scene in the application, the shader used by the application, and its own hardware information , and use the simplified shader to run the application, because the simplified shader occupies less workload of the computing unit in the GPU when it is running, which can effectively improve the terminal device stuttering phenomenon caused by the GPU occupation of the shader.
  • FIG. 11 is a schematic flowchart of another graphics rendering method provided by the embodiment of the present application, which is applied to the graphics rendering system shown in FIG. 3 , and FIG. 11 includes:
  • S1100 The server collects shaders used in the application program, identifies heavy-load shaders in the collected shaders, and simplifies the heavy-load shaders according to a first preset rule to obtain multiple simplified shaders.
  • the server collects hardware information and simplifies status information of applications running on terminal devices with different hardware information, and analyzes the collected information to obtain multiple target correspondences.
  • the simplified application program is an application program after the shaders in the application program are replaced with the simplified shaders.
  • the target correspondence is the target correspondence of shader information, hardware information, scene complexity, and simplified shader information.
  • S1102 The server sends multiple target correspondences to the terminal device.
  • the terminal device acquires a calling instruction of the source shader.
  • the source shader is used to achieve the rendering of the destination object.
  • the target object is an object in a target scene, and the target scene is a scene in an application program run on a terminal device.
  • the call instruction of the source shader includes source shader information.
  • the source shader information may include at least one of an identifier of the source shader or the source shader.
  • S1104 The terminal device determines that the multiple pre-stored target correspondences include the information of the source shader.
  • S1105 The terminal device acquires its own hardware information and the scene complexity of the target scene.
  • the terminal device acquires source shader information, hardware information of the terminal device, and target simplified shader information corresponding to the scene complexity of the target scene according to the multiple target correspondences.
  • the server learns simplified shaders corresponding to shaders in different application programs in terminal devices with different hardware information, and obtains multiple target correspondences.
  • the server sends the multiple target correspondences obtained to the terminal device, so that when the application program is run on the terminal device, the terminal device can use the scene complexity of the scene in the application program, the shader information used by the application program, and its own hardware Get target simplified shader information in multiple target correspondences, and use the simplified shader to run the application, because the simplified shader occupies less workload of the computing unit in the GPU when running, which can effectively improve the performance of the shader Terminal device freeze caused by GPU occupation.
  • FIG. 12 is a schematic flowchart of another graphics rendering method provided by the embodiment of the present application.
  • FIG. 12 includes:
  • the terminal device acquires a calling instruction of a source shader.
  • the source shader is used to achieve the rendering of the destination object.
  • the target object is an object in a target scene, and the target scene is a scene in an application program run on a terminal device.
  • the call instruction of the source shader includes source shader information.
  • the source shader information may include at least one of an identifier of the source shader or the source shader.
  • the terminal device obtains that the source shader is a heavy-duty shader according to at least one of the code complexity of the source shader or the number of calls of the source shader.
  • the terminal device simplifies the source shader according to the first preset rule to obtain the target simplified shader.
  • the first preset rule includes at least one of code movement, expression deletion, or syntax simplification.
  • S1203 The terminal device replaces the source shader information in the calling instruction of the source shader with the target simplified shader information, and compiles the target simplified shader.
  • the terminal device transmits the compiled target simplified shader to the GPU to implement rendering of the target object.
  • the terminal device when the application program is running on the terminal device, the terminal device can simplify the heavy load shader in the application program in real time, and use the simplified shader to run the application program, because the simplified shader is in When running, the workload of the computing unit in the GPU is small, which can effectively improve the terminal device stuttering phenomenon caused by the GPU occupation by the shader.
  • the embodiment of the present application can divide the functional modules of the device implementing the method for determining the intensity of ambient light according to the above method example, for example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module middle.
  • the functions of the target application program, the drawing interface and the display engine are integrated into the display control unit.
  • the above-mentioned integrated modules can be implemented in the form of hardware or in the form of software function modules. It should be noted that the division of modules in the embodiment of the present application is schematic, and is only a logical function division, and there may be other division methods in actual implementation.
  • FIG. 13 it is a schematic structural diagram of a terminal device provided by an embodiment of the present application.
  • the terminal device 130 shown in FIG. When the application program runs to a complex scene, obtain the target simplified shader; the complex scene is a scene in which the number of instructions required for the application program to run is greater than the first preset threshold, and the target simplified shader occupies less than the target computing resource of the terminal device 130 The occupancy of the computing resources of the terminal device 130 by the shader; the graphics rendering of the application program in a complex scene is performed by using the target simplified shader.
  • the processing module 1301 may be used to execute S600-S605, and referring to FIG. 10, the processing module 1301 may be used to execute S1000-S1005.
  • the processing module 1301 may be used to execute S1103 - S1108
  • the processing module may be used to execute S1200 - S1204 .
  • the processing module 1301 is specifically configured to: obtain the target simplified shader according to multiple pre-stored correspondences; the multiple correspondences include shader information, device hardware information, and correspondence between simplified shaders;
  • the shader information includes either a shader or a shader ID.
  • the processing module 1301 is specifically configured to: acquire the target shader information and the target simplified shader corresponding to the hardware information of the terminal device 130 among multiple correspondences.
  • the above correspondence relationship also includes a load level; the load level is a quantized value corresponding to the load of the terminal device 130; the processing module 1301 is specifically used to: acquire the number of rendering instructions in a complex scene; acquire according to the number of rendering instructions Target load level; among the multiple correspondences, acquire target shader information, target load level, and target simplified shader corresponding to hardware information of the terminal device 130 .
  • the processing module 1301 is further configured to: simplify the target shader according to a first preset rule to obtain multiple simplified shaders; the first preset rule includes at least one of code movement, expression deletion, or syntax simplification; A plurality of simplified shaders includes a target simplified shader; the state information of the terminal device 130 is collected when the terminal device 130 utilizes a plurality of simplified shaders to perform graphics rendering of an application program in a complex scene; the state information of the terminal device 130 includes the state information of the terminal device 130 Load, power consumption of the terminal device 130, temperature of the terminal device 130, rendering time, and image quality of the picture rendered by each simplified shader; the rendering time is when each simplified shader performs graphics rendering of the application program in a complex scene The required duration; according to the state information, obtain the corresponding relationship between the target shader information, the hardware information of the terminal device 130, the target load level and the target simplified shader.
  • the processing module 1301 is specifically configured to acquire multiple target rendering durations from the status information, and the multiple target rendering durations are all less than a second preset threshold; acquire the target status information from the status information corresponding to the multiple target rendering durations; The image quality of the target state information is greater than the third preset threshold; the target state information is the state information of the terminal device 130 when the target simplified shader performs graphics rendering of the application program in a complex scene; according to the load of the terminal device 130 in the target state information Obtaining the target load level; establishing a correspondence between the target shader information, the hardware information of the terminal device 130, the target load level, and the target simplified shader.
  • the processing module 1301 is specifically configured to: acquire a plurality of target image qualities from the state information, where the multiple target image qualities are all greater than a fourth preset threshold; acquire from state information corresponding to the plurality of target image qualities Target state information; the rendering duration of the target state information is less than the fifth preset threshold; the target state information is the state information of the terminal device 130 when the target simplified shader performs graphics rendering of the application program in a complex scene; according to the target state information in the terminal device
  • the load at 130 acquires the target load level; establishes the correspondence between the target shader information, the hardware information of the terminal device 130, the target load level, and the target simplified shader.
  • the terminal device 130 further includes a communication module 1302: configured to receive multiple correspondences sent by other devices.
  • the communication module 1302 may be used to execute the receiving step in S1102.
  • the processing module 1301 is specifically configured to: simplify the target shader according to a first preset rule to obtain a target simplified shader; the first preset rule includes at least one of code movement, expression deletion, or syntax simplification.
  • the processing module 1301 when the terminal device 130 determines that the application program runs into a complex scene, the processing module 1301 is specifically configured to: determine that the application program runs into a complex scene, and the code complexity of the target shader is greater than the sixth preset threshold, or the target When the number of times the shader is called is greater than the seventh preset threshold, the target simplified shader is acquired.
  • the function of the above-mentioned processing module 1301 can be implemented by the processor 401 in FIG. 4 calling a computer program in the memory 402 .
  • the function of the communication module 1302 can be realized by the communication module 404 in FIG. 4 .
  • FIG. 14 is a schematic structural diagram of a server provided by an embodiment of the present application.
  • the server 140 shown in FIG. 14 includes a processing module 1401 .
  • the processing module 1401 is used to: simplify the target shader according to the first preset rule to obtain multiple simplified shaders; the first preset rule includes at least one of code movement, expression deletion, or syntax simplification; multiple simplified shaders Including target simplification shaders; collect terminal device status information when using multiple simplified shaders to render application graphics in complex scenes; terminal device status information includes terminal device load, terminal device power consumption, terminal The temperature of the device, the rendering time, and the image quality of the picture rendered by each simplified shader; the rendering time is the time required for each simplified shader to render the graphics of the application in a complex scene; the target shader is obtained according to the state information , the hardware information of the end device, the target load level, and the correspondence between target simplified shaders.
  • the processing module 1401 may be used to execute S1100-S1101.
  • the processing module 1401 is specifically configured to acquire multiple target rendering durations from the status information, and the multiple target rendering durations are all less than a second preset threshold; acquire the target status information from the status information corresponding to the multiple target rendering durations; The image quality of the target state information is greater than the third preset threshold; the target state information is the state information of the terminal device when the target simplified shader performs graphics rendering of the application program in a complex scene; the target is obtained according to the load of the terminal device in the target state information Load level; establish the correspondence between target shader information, terminal device hardware information, target load level, and target simplified shader.
  • the processing module 1401 is specifically configured to: acquire a plurality of target image qualities from the status information, where the multiple target image qualities are all greater than a fourth preset threshold; acquire from the status information corresponding to the plurality of target image qualities Target status information; the rendering duration of the target status information is less than the fifth preset threshold; the target status information is the status information of the terminal device when the target simplified shader performs graphics rendering of the application program in a complex scene; according to the terminal device in the target status information The load obtains the target load level; establishes the correspondence between the information of the target shader, the hardware information of the terminal device, the target load level, and the target simplified shader.
  • the server 140 further includes a communication module 1402: configured to send multiple correspondences to the terminal device.
  • the communication module 1401 may be used to execute the sending step in S1102.
  • the functions of the above processing module 1401 may be implemented by the processor 401 in FIG. 4 calling a computer program in the memory 402 .
  • the function of the communication module 1402 can be realized by the communication module 404 in FIG. 4 .
  • FIG. 15 is a schematic structural diagram of a chip provided by an embodiment of the present application.
  • Chip 150 includes one or more than two (including two) processors 1501 , communication lines 1502 , communication interface 1503 and memory 1504 .
  • the memory 1504 stores the following elements: executable modules or data structures, or subsets thereof, or extensions thereof.
  • the methods described in the foregoing embodiments of the present application may be applied to the processor 1501 or implemented by the processor 1501 .
  • the processor 1501 may be an integrated circuit chip with signal processing capabilities.
  • each step of the above method may be implemented by an integrated logic circuit of hardware in the processor 1501 or instructions in the form of software.
  • the above-mentioned processor 1201 may be a general-purpose processor (for example, a microprocessor or a conventional processor), a digital signal processor (digital signal processing, DSP), an application specific integrated circuit (application specific integrated circuit, ASIC), an off-the-shelf programmable gate Array (field-programmable gate array, FPGA) or other programmable logic devices, discrete gates, transistor logic devices or discrete hardware components, the processor 1501 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application .
  • DSP digital signal processing
  • ASIC application specific integrated circuit
  • FPGA field-programmable gate array
  • the steps of the method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor.
  • the software module may be located in a mature storage medium in the field such as random access memory, read-only memory, programmable read-only memory, or electrically erasable programmable read only memory (EEPROM).
  • the storage medium is located in the memory 1504, and the processor 1501 reads the information in the memory 1504, and completes the steps of the above method in combination with its hardware.
  • the processor 1501 , the memory 1504 and the communication interface 1503 may communicate through the communication line 1502 .
  • the instructions stored in the memory for execution by the processor may be implemented in the form of computer program products.
  • the computer program product may be written in the memory in advance, or may be downloaded and installed in the memory in the form of software.
  • the embodiment of the present application also provides a computer program product including one or more computer instructions.
  • the computer can be a general purpose computer, special purpose computer, computer network, or other programmable apparatus.
  • Computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, e.g. Coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL) or wireless (such as infrared, wireless, microwave, etc.) transmission to another website site, computer, server or data center.
  • Computer readable storage medium can be Any available media capable of being stored by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media.
  • available media may include magnetic media (e.g., floppy disks, hard disks, or tapes), optical media (e.g., A digital versatile disc (digital versatile disc, DVD)), or a semiconductor medium (for example, a solid state disk (solid state disk, SSD)), etc.
  • Computer-readable media may include computer storage media and communication media, and may include any medium that can transfer a computer program from one place to another.
  • a storage media may be any target media that can be accessed by a computer.
  • the computer-readable medium may include compact disc read-only memory (compact disc read-only memory, CD-ROM), RAM, ROM, EEPROM or other optical disc storage; the computer-readable medium may include a magnetic disk memory or other disk storage devices.
  • any connected cord is properly termed a computer-readable medium.
  • Disk and disc includes compact disc (CD), laser disc, compact disc, digital versatile disc (DVD), floppy disk and Blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Reproduce data.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Graphics (AREA)
  • Image Generation (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

本申请实施例提供一种图形渲染方法、装置及存储介质，涉及图像处理技术领域，该方法包括终端设备利用目标着色器进行应用程序的图形渲染；在终端设备确定应用程序运行至复杂场景时，终端设备获取目标简化着色器；复杂场景为应用程序运行所需的指令的个数大于第一预设阈值的场景，目标简化着色器对终端设备计算资源的占用小于目标着色器对终端设备计算资源的占用；终端设备利用目标简化着色器在复杂场景中进行应用程序的图形渲染，由于目标简化着色器对终端设备计算资源的占用小于目标着色器对终端设备计算资源的占用，因而有助于使得终端设备中的应用程序运行至复杂场景下时仍然能够保持流畅运行。

Description

图形渲染方法、装置及存储介质
本申请要求于2021年08月24日提交中国国家知识产权局、申请号为202110977666.2、申请名称为“图形渲染方法、装置及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
背景技术
随着计算机图形学技术和计算机动画技术的发展，为了更真实和精细化的绘制应用程序场景，应用程序厂商会使用复杂着色器来提升画质。着色器为可编辑的程序，可以用于实现图形渲染。图形渲染是计算机将存储在内存中的形状转换成在屏幕上显示的图像的过程。
在一些终端设备中的应用程序运行至复杂的场景下时，着色器可能会消耗图形处理器(graphic processing unit，GPU)中过多的着色器计算单元(例如：shader core)的计算量，使得GPU无法在一帧时间内完成渲染，导致终端设备发生卡顿现象。
发明内容
本申请实施例提供一种图形渲染方法、装置及存储介质，涉及图像处理技术领域，有助于使得终端设备中的应用程序运行至复杂的场景下时，仍然能够保持流畅运行。
第一方面，本申请实施例提供一种图形渲染方法，该方法包括：终端设备利用目标着色器进行应用程序的图形渲染；在终端设备确定应用程序运行至复杂场景时，终端设备获取目标简化着色器；复杂场景为应用程序运行所需的指令的个数大于第一预设阈值的场景，目标简化着色器对终端设备计算资源的占用小于目标着色器对终端设备计算资源的占用；终端设备利用目标简化着色器在复杂场景中进行应用程序的图形渲染。
本申请实施例中，终端设备确定应用程序运行至复杂场景时，获取目标简化着色器，利用目标简化着色器在复杂场景中进行应用程序的图形渲染，由于目标简化着色器对终端设备计算资源的占用小于目标着色器对终端设备计算资源的占用，因而有助于使得终端设备中的应用程序运行至复杂场景下时仍然能够保持流畅运行。
一种可能的实现方式中，上述终端设备获取目标简化着色器，包括：终端设备根据预存的多个对应关系，获取目标简化着色器；多个对应关系包括着色器的信息、设备的硬件信息以及简化着色器之间的对应关系；着色器的信息包括着色器或着色器的标识中的任意一种。
另一种可能的实现方式中，上述终端设备根据预存的多个对应关系，获取目标着色器的信息，包括：终端设备在多个对应关系中，获取目标着色器的信息以及终端设备的硬件信息对应的目标简化着色器。这样，终端设备根据着色器的信息以及设备的硬件信息可以获取到有助于在进行图形渲染时，提升该终端设备运行性能的简化着色器。
另一种可能的实现方式中上述对应关系中还包括负载等级；负载等级为终端设备的负载对应的量化值；上述终端设备根据预存的多个对应关系，获取目标着色器的信息，包括： 终端设备获取复杂场景中描绘指令的个数；终端设备根据描绘指令的个数获取目标负载等级；终端设备在多个对应关系中，获取目标着色器的信息、目标负载等级以及终端设备的硬件信息对应的目标简化着色器。这样，终端设备根据着色器的信息、目标负载等级以及设备的硬件信息可以获取到有助于在该复杂场景进行图形渲染时，提升该终端设备运行性能的简化着色器。
另一种可能的实现方式中，该方法还包括：终端设备根据第一预设规则简化目标着色器，得到多个简化着色器；第一预设规则包括代码移动、表达式删除或语法精简中的至少一种；多个简化着色器包括目标简化着色器；终端设备采集终端设备利用多个简化着色器在复杂场景中进行应用程序的图形渲染时终端设备的状态信息；终端设备的状态信息包括终端设备的负载、终端设备的功耗、终端设备的温度、渲染时长、以及各简化着色器渲染得到的画面的画质质量；渲染时长为各简化着色器在复杂场景中进行应用程序的图形渲染时所需时长；终端设备根据状态信息得到目标着色器的信息、终端设备的硬件信息、目标负载等级以及目标简化着色器之间的对应关系。
这样，终端设备可以提前分析目标着色器对应的多个简化着色器在该终端设备对应的终端设备的硬件信息时，多个简化着色器在复杂场景中进行该应用程序的图形渲染时该终端设备的状态信息，根据该终端设备的状态信息可以得到性能良好的目标简化着色器。
另一种可能的实现方式中，上述终端设备根据状态信息得到目标着色器的信息、终端设备的硬件信息、目标负载等级以及目标简化着色器之间的对应关系，包括：终端设备从状态信息中获取多个目标渲染时长，多个目标渲染时长均小于第二预设阈值；终端设备从多个目标渲染时长对应的状态信息中获取目标状态信息；目标状态信息的画质质量大于第三预设阈值；目标状态信息为目标简化着色器在复杂场景中进行应用程序的图形渲染时终端设备的状态信息；终端设备根据目标状态信息中终端设备的负载获取目标负载等级；终端设备建立目标着色器的信息、终端设备的硬件信息、目标负载等级以及目标简化着色器之间的对应关系。
这样，终端设备可以首先根据渲染时长进行筛选，然后再根据渲染得到的画质质量筛选得到目标简化着色器，并建立目标着色器的信息、终端设备的硬件信息、目标负载等级以及目标简化着色器之间的对应关系。
可能的实现方式中，上述终端设备根据状态信息得到目标着色器的信息、终端设备的硬件信息、目标负载等级以及目标简化着色器之间的对应关系，包括：终端设备从状态信息中获取多个目标画质质量，多个目标画质质量均大于第四预设阈值；终端设备从多个目标画质质量对应的状态信息中获取目标状态信息；目标状态信息的渲染时长小于第五预设阈值；目标状态信息为目标简化着色器在复杂场景中进行应用程序的图形渲染时终端设备的状态信息；终端设备根据目标状态信息中终端设备的负载获取目标负载等级；终端设备建立目标着色器的信息、终端设备的硬件信息、目标负载等级以及目标简化着色器之间的对应关系。
这样，终端设备可以首先根据渲染得到的画质质量进行筛选，然后再根据渲染时长进行筛选，得到目标简化着色器，并建立目标着色器的信息、终端设备的硬件信息、目标负载等级以及目标简化着色器之间的对应关系。
另一种可能的实现方式中，该方法还包括：终端设备接收其他设备发送的多个对应关 系。
另一种可能的实现方式中，其他设备根据第一预设规则简化目标着色器，得到多个简化着色器；第一预设规则包括代码移动、表达式删除或语法精简中的至少一种；多个简化着色器包括目标简化着色器；其他设备采集其他设备利用多个简化着色器在复杂场景中进行应用程序的图形渲染时其他设备的状态信息；其他设备的状态信息包括其他设备的负载、其他设备的功耗、其他设备的温度、渲染时长、以及各简化着色器渲染得到的画面的画质质量；渲染时长为各简化着色器在复杂场景中进行应用程序的图形渲染时所需时长；其他设备根据状态信息得到目标着色器的信息、其他设备的硬件信息、目标负载等级以及目标简化着色器之间的对应关系，其他设备向终端设备发送获取的对应关系。
另一种可能的实现方式中，上述其他设备根据状态信息得到目标着色器的信息、其他设备的硬件信息、目标负载等级以及目标简化着色器之间的对应关系，包括：其他设备从状态信息中获取多个目标渲染时长，多个目标渲染时长均小于第二预设阈值；其他设备从多个目标渲染时长对应的状态信息中获取目标状态信息；目标状态信息的画质质量大于第三预设阈值；目标状态信息为目标简化着色器在复杂场景中进行应用程序的图形渲染时其他设备的状态信息；其他设备根据目标状态信息中其他设备的负载获取目标负载等级；其他设备建立目标着色器的信息、其他设备的硬件信息、目标负载等级以及目标简化着色器之间的对应关系。
这样，其他设备可以首先根据渲染时长进行筛选，然后再根据渲染得到的画质质量筛选得到目标简化着色器，并建立目标着色器的信息、其他设备的硬件信息、目标负载等级以及目标简化着色器之间的对应关系。
另一种可能的实现方式中，上述其他设备根据状态信息得到目标着色器的信息、其他设备的硬件信息、目标负载等级以及目标简化着色器之间的对应关系，包括：其他设备从状态信息中获取多个目标画质质量，多个目标画质质量均大于第四预设阈值；其他设备从多个目标画质质量对应的状态信息中获取目标状态信息；目标状态信息的渲染时长小于第五预设阈值；目标状态信息为目标简化着色器在复杂场景中进行应用程序的图形渲染时其他设备的状态信息；其他设备根据目标状态信息中其他设备的负载获取目标负载等级；其他设备建立目标着色器的信息、其他设备的硬件信息、目标负载等级以及目标简化着色器之间的对应关系。
另一种可能的实现方式中，上述终端设备获取目标简化着色器，包括：终端设备根据第一预设规则简化目标着色器，得到目标简化着色器；第一预设规则包括代码移动、表达式删除或语法精简中的至少一种。
另一种可能的实现方式中，在终端设备确定应用程序运行至复杂场景时，上述终端设备获取目标简化着色器，包括：在终端设备确定应用程序运行至复杂场景，以及目标着色器的代码复杂度大于第六预设阈值，或者，目标着色器的被调用次数大于第七预设阈值时，终端设备获取目标简化着色器。
第二方面，本申请实施例提供一种终端设备，包括处理模块。处理模块用于利用目标着色器进行应用程序的图形渲染；在确定应用程序运行至复杂场景时，获取目标简化着色器；复杂场景为应用程序运行所需的指令的个数大于第一预设阈值的场景，目标简化着色器对终端设备计算资源的占用小于目标着色器对终端设备计算资源的占用；利用目标简化 着色器在复杂场景中进行应用程序的图形渲染。
可选的，处理模块具体用于：根据预存的多个对应关系，获取目标简化着色器；多个对应关系包括着色器的信息、设备的硬件信息以及简化着色器之间的对应关系；着色器的信息包括着色器或着色器的标识中的任意一种。
可选的，处理模块具体用于：在多个对应关系中，获取目标着色器的信息以及终端设备的硬件信息对应的目标简化着色器。
可选的，上述对应关系中还包括负载等级；负载等级为设备的负载对应的量化值；处理模块具体用于：获取复杂场景中描绘指令的个数；根据描绘指令的个数获取目标负载等级；在多个对应关系中，获取目标着色器的信息、目标负载等级以及终端设备的硬件信息对应的目标简化着色器。
可选的，处理模块还用于：根据第一预设规则简化目标着色器，得到多个简化着色器；第一预设规则包括代码移动、表达式删除或语法精简中的至少一种；多个简化着色器包括目标简化着色器；采集终端设备利用多个简化着色器在复杂场景中进行应用程序的图形渲染时终端设备的状态信息；终端设备的状态信息包括终端设备的负载、终端设备的功耗、终端设备的温度、渲染时长、以及各简化着色器渲染得到的画面的画质质量；渲染时长为各简化着色器在复杂场景中进行应用程序的图形渲染时所需时长；根据状态信息得到目标着色器的信息、终端设备的硬件信息、目标负载等级以及目标简化着色器之间的对应关系。
可选的，处理模块具体用于从状态信息中获取多个目标渲染时长，多个目标渲染时长均小于第二预设阈值；从多个目标渲染时长对应的状态信息中获取目标状态信息；目标状态信息的画质质量大于第三预设阈值；目标状态信息为目标简化着色器在复杂场景中进行应用程序的图形渲染时终端设备的状态信息；根据目标状态信息中终端设备的负载获取目标负载等级；建立目标着色器的信息、终端设备的硬件信息、目标负载等级以及目标简化着色器之间的对应关系。
可选的，处理模块具体用于：从状态信息中获取多个目标画质质量，多个目标画质质量均大于第四预设阈值；从多个目标画质质量对应的状态信息中获取目标状态信息；目标状态信息的渲染时长小于第五预设阈值；目标状态信息为目标简化着色器在复杂场景中进行应用程序的图形渲染时终端设备的状态信息；根据目标状态信息中终端设备的负载获取目标负载等级；建立目标着色器的信息、终端设备的硬件信息、目标负载等级以及目标简化着色器之间的对应关系。
可选的，终端设备还包括通信模块：用于接收其他设备发送的多个对应关系。
可选的，处理模块具体用于：根据第一预设规则简化目标着色器，得到目标简化着色器；第一预设规则包括代码移动、表达式删除或语法精简中的至少一种。
可选的，在终端设备确定应用程序运行至复杂场景时，处理模块具体用于：确定应用程序运行至复杂场景，以及目标着色器的代码复杂度大于第六预设阈值，或者，目标着色器的被调用次数大于第七预设阈值时，获取目标简化着色器。
第三方面，本申请实施例提供一种服务器，包括处理模块。处理模块用于：根据第一预设规则简化目标着色器，得到多个简化着色器；第一预设规则包括代码移动、表达式删除或语法精简中的至少一种；多个简化着色器包括目标简化着色器；采集终端设备利用多个简化着色器在复杂场景中进行应用程序的图形渲染时终端设备的状态信息；终端设备的 状态信息包括终端设备的负载、终端设备的功耗、终端设备的温度、渲染时长、以及各简化着色器渲染得到的画面的画质质量；渲染时长为各简化着色器在复杂场景中进行应用程序的图形渲染时所需时长；根据状态信息得到目标着色器的信息、终端设备的硬件信息、目标负载等级以及目标简化着色器之间的对应关系。
可选的，处理模块具体用于从状态信息中获取多个目标渲染时长，多个目标渲染时长均小于第二预设阈值；从多个目标渲染时长对应的状态信息中获取目标状态信息；目标状态信息的画质质量大于第三预设阈值；目标状态信息为目标简化着色器在复杂场景中进行应用程序的图形渲染时终端设备的状态信息；根据目标状态信息中终端设备的负载获取目标负载等级；建立目标着色器的信息、终端设备的硬件信息、目标负载等级以及目标简化着色器之间的对应关系。
可选的，处理模块具体用于：从状态信息中获取多个目标画质质量，多个目标画质质量均大于第四预设阈值；从多个目标画质质量对应的状态信息中获取目标状态信息；目标状态信息的渲染时长小于第五预设阈值；目标状态信息为目标简化着色器在复杂场景中进行应用程序的图形渲染时终端设备的状态信息；根据目标状态信息中终端设备的负载获取目标负载等级；建立目标着色器的信息、终端设备的硬件信息、目标负载等级以及目标简化着色器之间的对应关系。
可选的，终端设备还包括通信模块：用于向终端设备发送多个对应关系。
第四方面，本申请实施例提供一种计算机可读存储介质，计算机可读存储介质中存储有计算机程序或指令，当计算机程序或指令在计算机上运行时，使得计算机执行第一方面或第一方面的任意一种可能的实现方式中描述的图形渲染方法。
第五方面，本申请实施例提供一种包括计算机程序的计算机程序产品，当计算机程序在计算机上运行时，使得计算机执行第一方面或第一方面的任意一种可能的实现方式中描述的图形渲染方法。
第六方面，本申请提供一种芯片或者芯片系统，该芯片或者芯片系统包括至少一个处理器和通信接口，通信接口和至少一个处理器通过线路互联，至少一个处理器用于运行计算机程序或指令，以执行第一方面或第一方面的任意一种可能的实现方式中描述的图形渲染方法。其中，芯片中的通信接口可以为输入/输出接口、管脚或电路等。
在一种可能的实现中，本申请中上述描述的芯片或者芯片系统还包括至少一个存储器，该至少一个存储器中存储有指令。该存储器可以为芯片内部的存储单元，例如，寄存器、缓存等，也可以是该芯片的存储单元(例如，只读存储器、随机存取存储器等)。
应当理解的是，本申请的第二方面至第六方面与本申请的第一方面的技术方案相对应，各方面及对应的可行实施方式所取得的有益效果相似，不再赘述。
附图说明
图1为本申请实施例所适用的帧率与运行时间的对应关系的示意图；
图2为本申请实施例所提供的图形渲染方法所适用的一种简化平台的结构示意图；
图3为本申请实施例提供的图形渲染方法所适用的一种图形渲染系统的结构示意图；
图4为本申请实施例提供的图形渲染方法所适用的电子设备的结构示意图；
图5为本申请实施例适用的电子设备的一种软件结构框图；
图6为本申请实施例提供的一种图形渲染方法中多个目标对应关系的学习流程的流程示意图；
图7为本申请实施例所适用的应用程序的场景的示意图；
图8A为本申请实施例提供的着色器及其对应简化着色器的示意图；
图8B为本申请实施例提供的另一种获取简化着色器的方法的流程示意图；
图9为本申请实施例提供的从另一个角度描述多个目标对应关系的学习流程的流程示意图；
图10为本申请实施例提供的一种图形渲染方法中使用多个目标对应关系简化应用程序运行过程中的着色器的流程示意图；
图11为本申请实施例提供的另一种图形渲染方法的流程示意图；
图12为本申请实施例提供的另一种图形渲染方法的流程示意图；
图13为本申请实施例提供的一种终端设备的结构示意图；
图14为本申请实施例提供的一种服务器的结构示意图；
图15为本申请实施例提供的一种芯片的结构示意图。
具体实施方式
为了便于清楚描述本申请实施例的技术方案，以下，对本申请实施例中所涉及的部分术语和技术进行简单介绍：
1)帧、帧率
帧是指界面显示中最小单位的单幅画面。一帧可以理解为一副静止的画面，快速连续地显示多个相连的帧可以形成物体运动的假象。
帧率是指在1秒钟时间里刷新图片的帧数，也可以理解为终端设备中GPU每秒钟刷新画面的次数。高的帧率可以得到更流畅和更逼真的动画。每秒钟帧数越多，所显示的动作就会越流畅。
需要说明的是，界面显示帧前通常需要经过绘制、渲染、合成等过程。
2)帧绘制
帧绘制是指显示界面的图片绘制。显示界面可以由一个或多个视图组成，各个视图可以由视图系统的可视控件绘制，各个视图由子视图组成，一个子视图对应视图中的一个小部件，例如，其中的一个子视图对应图片视图中的一个符号。
3)帧渲染(又称图形渲染)
帧渲染是将绘制后的视图进行着色操作或增加3D效果等。例如：3D效果可以是灯光效果、阴影效果和纹理效果等。
4)着色器
着色器为可编辑程序，着色器可以运行在GPU上以实现图形渲染。着色器包括顶点着色器(vertex shader)、像素着色器(pixel shader)、片段着色器(fragment shader)、几何着色器(geometry shader)以及计算着色器(compute shader)等。
5)硬件信息、场景复杂度
本申请实施例中硬件信息可以包括芯片种类、芯片的性能如：中央处理器(central processing unit，CPU)频率或GPU频率等中的至少一种信息。
场景复杂度可以用场景中描绘指令(draw call)的个数来表征。描绘指令为CPU对底层图形绘制接口的调用命令。描绘指令用于终端设备执行：检测渲染状态、提交渲染数据以及提交渲染状态等任务。
6)其他术语
在本申请的实施例中，采用了“第一”、“第二”等字样对功能和作用基本相同的相同项或相似项进行区分。例如，第一芯片和第二芯片仅仅是为了区分不同的芯片，并不对其先后顺序进行限定。本领域技术人员可以理解“第一”、“第二”等字样并不对数量和执行次序进行限定，并且“第一”、“第二”等字样也并不限定一定不同。
需要说明的是，本申请实施例中，“示例性的”或者“例如”等词用于表示作例子、例证或说明。本申请中被描述为“示例性的”或者“例如”的任何实施例或设计方案不应被解释为比其他实施例或设计方案更优选或更具优势。确切而言，使用“示例性的”或者“例如”等词旨在以具体方式呈现相关概念。
本申请实施例中，“至少一个”是指一个或者多个，“多个”是指两个或两个以上。“和/或”，描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B的情况，其中A，B可以是单数或者复数。字符“/”一般表示前后关联对象是一种“或”的关系。“以下至少一项(个)”或其类似表达，是指的这些项中的任意组合，包括单项(个)或复数项(个)的任意组合。例如，a，b，或c中的至少一项(个)，可以表示：a，b，c，a-b，a--c，b-c，或a-b-c，其中a，b，c可以是单个，也可以是多个。
用户使用终端设备运行应用程序时，当应用程序运行至复杂的场景，终端设备有可能发生卡顿现象(又称掉帧现象)。如图1所示为终端设备中运行应用程序时该应用程序的帧率与该应用程序的运行时间的对应关系图。图1中横轴为应用程序的运行时间，纵轴为应用程序运行时的帧率，该应用程序的帧率设置为60fps，图1中该应用程序在运行至4分钟7秒时帧率仅为31.34fps，此时表现在终端设备上，该应用程序在4分钟7秒所对应场景时，终端设备发生卡顿现象。其中，复杂场景可以指场景中描绘指令的个数大于第一预设阈值的场景。描绘指令为CPU对底层图形绘制接口的调用命令。描绘指令用于终端设备执行：检测渲染状态、提交渲染数据以及提交渲染状态等任务。
这是因为，当应用程序运行至复杂场景时，着色器消耗GPU中计算单元的工作量较大，可能达到90％以上，当着色器消耗GPU中着色器计算单元90％以上的工作量时，一帧图像的渲染时长过长，导致GPU无法在一帧时间内完成渲染，从而导致应用程序发生卡顿现象。
另外，影响渲染时长的因素还可以包括：场景、应用程序中所使用的着色器以及承载应用程序运行的终端设备的硬件信息。其中，场景可以包括简单场景和复杂场景。简单场景可以指场景中描绘指令的个数小于或者等于第一预设阈值的场景。终端设备的硬件信息可以包括：终端设备所使用的芯片种类或终端设备中芯片的性能如：中央处理器(central processing unit，CPU)频率或GPU频率等中的至少一种信息。
当应用程序在终端设备上运行时，终端设备的硬件信息已经固定，而场景不适合变动，因此，在终端设备中对用于渲染场景的着色器进行合理的简化以降低GPU的负载，有助于使得终端设备中的应用程序运行至复杂场景下时，仍然能够保持流畅运行。
基于此，本申请实施例提供图形渲染方法，该方法可以在终端设备运行应用程序时，将着色器简化为简化着色器，因为简化着色器在运行时，对GPU中计算单元工作量的占用较小，可以有效改善因着色器对GPU占用导致的终端设备卡顿现象。
其中，终端设备将着色器简化为简化着色器可以包括两种实现：
实现一：终端设备在着色器运行时，可以基于一定的规则实时对着色器进行简化，得到简化着色器。
实现二：可以先学习不同应用程序中的着色器在不同硬件信息的终端设备中对应的简化着色器，得到多个目标对应关系，目标对应关系中包括着色器信息、简化着色器信息、终端设备的硬件信息与场景的场景复杂度的关系。这样，后续在终端设备中运行应用程序时，终端设备可以基于该应用程序中场景的场景复杂度、该应用程序使用的着色器的着色器信息、以及自身的硬件信息在目标对应关系中得到目标简化着色器信息，并使用简化着色器运行应用程序，因为简化着色器在运行时，对GPU中计算单元工作量的占用较小，可以有效改善因着色器对GPU占用导致的终端设备卡顿现象。
示例性的，针对实现二，本申请实施例所提供的图形渲染方法可以适用于简化平台。该简化平台可以用于为不同应用程序以及不同硬件信息的终端设备提供统一的着色器简化功能。如图2所示为本申请实施例所提供的图形渲染方法所适用的一种简化平台的结构示意图，该简化平台可以包括着色器简化模块和着色器替换模块。
其中，着色器简化模块用于获取上述多个目标对应关系。着色器替换模块用于终端设备在应用程序运行过程中将着色器替换为简化着色器。可以理解的是，着色器简化模块和着色器替换模块可以运行在不同的设备，也可以在不同的时段各自实现各自的功能，本申请实施例不作限定。
示例性的，如图2所示，着色器简化模块可以包括简化子单元、应用程序及硬件信息采集单元和着色器筛选单元。
其中，简化子单元用于：采集应用程序运行过程中使用到的着色器(例如：顶点着色器和/或片段着色器等)，并根据预设的第一预设规则对采集的着色器进行简化，得到多个简化着色器，并存储该多个简化着色器。
应用程序及硬件信息采集单元用于：采集应用程序(如上述简化应用程序)运行过程中的场景数据(又称状态信息)，并采集硬件信息，如，终端设备的CPU频率和/或GPU频率中的至少一种信息。
着色器筛选单元用于：根据采集到的场景数据以及硬件信息对简化着色器进行筛选，得到多个目标对应关系。例如，对场景数据中的渲染时间(又称渲染时长)从高到低进行分段，在每个分段类别中选择渲染得到的画面的画质质量大于第二预设阈值的的简化着色器(又称简化模型)。又例如，对场景数据中渲染得到的画面的画质质量进行分段，在每个分段类别中选择渲染时长小于第三预设阈值的简化着色器。
示例性的，着色器替换模块可以包括：着色器列表管理单元、应用程序指令拦截单元、着色器比对单元、场景识别单元、着色器决策选择单元和着色器替换单元。
着色器列表(即上述多个目标对应关系)管理单元用于：按照预设的数据结构来组织和存储上述得到的多个目标对应关系。其中，预设的数据结构可以是代码预设的。
可以理解的是，着色器列表管理单元中的全部功能或部分功能可以集成在着色器筛选 单元中。或者，着色器列表管理单元中的全部功能或部分功能可以集成在决策选择单元中。本申请实施例对此不进行限定。示例性的，根据场景复杂度、场景数据以及硬件信息，获取着色器列表的功能可以集成在着色器筛选单元中。
指令拦截单元用于：拦截应用程序中对着色器的调用指令。
着色器比对单元用于：在着色器列表中查找拦截到的着色器。若查找失败(即在着色器列表中不存在拦截到的着色器)则不对着色器的调用指令中的着色器进行替换，继续执行应用程序中的原始流程，到GPU进行下一步渲染。
场景识别单元用于：若着色器比对成功(即在着色器列表中存在拦截到的着色器)，则进行场景复杂度的识别。例如：识别拦截到的调用指令所调用着色器对应的场景的场景复杂度。
决策选择单元用于：基于应用程序中场景的场景复杂度、该应用程序使用的着色器信息、以及终端设备的硬件信息，在多个目标对应关系中获取目标简化着色器信息。
着色器替换单元用于：对拦截指令中的着色器进行替换，编译替换后的着色器，再送入GPU中进行渲染。
图2中应用程序(application，APP)用于表征终端设备中运行的应用程序。
可以理解的是，本申请实施例中简化平台的功能可以由一个终端设备实现，也可以由终端设备与服务器组成的图形渲染系统实现。如图3所示为本申请实施例提供的图形渲染方法所适用的一种图形渲染系统的结构示意图。图3中包括终端设备301和服务器302。其中，终端设备301可以为智能手机、台式电脑、笔记本电脑、平板电脑、虚拟现实(virtual reality，VR)设备、增强现实(augmented reality，AR)设备或智能可穿戴设备等。服务器302可以为一个服务器、服务器集群或云服务器等。本申请实施例对此不进行限定。
图3中终端设备301可以用于实现上述图2中一个单元或多个单元的功能。其余单元的功能由服务器302实现。示例性的，终端设备301可以用于实现上述图2中着色器替换模块的功能，服务器302可以用于实现着色器简化模块的功能。或者，终端设备301可以用于实现指令拦截单元的功能、场景识别单元的功能以及着色器替换单元的功能。服务器302可以用于实现着色器简化模块的功能、着色器列表管理单元的功能、着色器比对单元的功能以及决策选择单元的功能等。
上述终端设备301或服务器302的功能均可以由图4所示的电子设备实现。图4所示为本申请实施例提供的图形渲染方法所适用的电子设备的结构示意图，图4所示的电子设备40可以包括：处理器401，存储器402，显示屏403以及通信模块404等。
可以理解的是，本实施例示意的结构并不构成对电子设备40的具体限定。在本申请另一些实施例中，电子设备40可以包括比图示更多或更少的部件，或者组合某些部件，或者拆分某些部件，或者不同的部件布置。图示的部件可以以硬件，软件，或软件和硬件的组合实现。
处理器401可以包括一个或多个处理单元，例如：处理器401可以包括应用处理器(application processor，AP)，调制解调处理器，GPU，图像信号处理器(image signal processor，ISP)，控制器，视频编解码器，数字信号处理器(digital signal processor，DSP)，基带处理器，显示处理单元(display process unit，DPU)，和/或神经网络处理器(neural-network processing unit，NPU)等。其中，不同的处理单元可以是独立的器件，也可以集成在一个 或多个处理器中。
在一些实施例中，电子设备40也可以包括一个或多个处理器401。其中，处理器可以是电子设备40的神经中枢和指挥中心。处理器可以根据指令操作码和时序信号，产生操作控制信号，完成取指令和执行指令的控制。处理器401中还可以设置存储器，用于存储指令和数据。在一些实施例中，处理器401中的存储器为高速缓冲存储器。该存储器可以保存处理器401用过或循环使用的指令或数据。如果处理器401需要再次使用该指令或数据，可从所述存储器中直接调用。这就避免了重复存取，减少了处理器401的等待时间，因而提高了电子设备40的效率。
电子设备40的通信模块404可以使用任何收发器一类的装置，用于与其他设备或通信网络通信，如广域网(wide area network，WAN)，局域网(local area networks，LAN)等。
电子设备40通过GPU，显示屏403，以及应用处理器等可以实现显示功能。应用处理器可以包括NPU、DPU。GPU为图像处理的微处理器，连接显示屏403和应用处理器。GPU用于执行数学和几何计算，用于图形渲染。处理器401可包括一个或多个GPU，其执行指令以生成或改变显示信息。NPU为神经网络(neural-network，NN)计算处理器，通过借鉴生物神经网络结构，例如借鉴人脑神经元之间传递模式，对输入信息快速处理，还可以不断的自学习。通过NPU可以实现电子设备40的智能认知等应用，例如：图像识别，人脸识别，语音识别，文本理解等。DPU也称为显示子系统(display sub-system，DSS)，DPU用于对显示屏403的色彩进行调整，DPU可以通过三维查找表(3D look up table，3D LUT)对显示屏的色彩进行调整。DPU还可以对画面进行缩放、降噪、对比度增强、背光亮度管理、hdr处理、显示屏参数Gamma调整等处理。
显示屏403用于显示图像，视频等。显示屏403包括显示面板。显示面板可以采用液晶显示屏(liquid crystal display，LCD)，有机发光二极管(organic light-emitting diode，OLED)，有源矩阵有机发光二极体或主动矩阵有机发光二极体(active-matrix organic light emitting diode的，AMOLED)，柔性发光二极管(flex light-emitting diode，FLED)，Miniled，MicroLed，Micro-oLed，量子点发光二极管(quantum dot light emitting diodes，QLED)等。在一些实施例中，电子设备40可以包括1个或N个显示屏403，N为大于1的正整数。
存储器402可以用于存储一个或多个计算机程序，该一个或多个计算机程序包括指令。处理器401可以通过运行存储在存储器402的上述指令，从而使得电子设备40执行各种功能应用以及数据处理等。存储器402可以包括存储程序区和存储数据区。其中，存储程序区可存储操作系统；该存储程序区还可以存储一个或多个应用程序(比如图库、联系人等)等。存储数据区可存储电子设备40使用过程中所创建的数据(比如照片，联系人等)等。此外，存储器402可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件，闪存器件，通用闪存存储器(universal flash storage，UFS)等。在一些实施例中，处理器401可以通过运行存储在存储器402的指令，和/或存储在设置于处理器401中的存储器的指令，来使得电子设备40执行各种功能应用及数据处理。
存储器402用于存储本申请实施例中多个目标对应关系。
显示屏403用于显示图像，视频等。显示屏403包括显示面板。显示面板可以采用液晶显示屏(liquid crystal display，LCD)，有机发光二极管(organic light-emitting diode，OLED)， 有源矩阵有机发光二极体或主动矩阵有机发光二极体(active-matrix organic light emitting diode的，AMOLED)，柔性发光二极管(flex light-emitting diode，FLED)，Miniled，MicroLed，Micro-oLed，量子点发光二极管(quantum dot light emitting diodes，QLED)等。在一些实施例中，电子设备100可以包括1个或N个显示屏403，N为大于1的正整数。
电子设备40的软件系统可以采用分层架构，事件驱动架构，微核架构，微服务架构，或云架构。本申请实施例以分层架构的Android系统为例，示例性说明电子设备40的软件结构。图5为本申请实施例适用的电子设备的一种软件结构框图。分层架构将电子设备40的软件系统分成若干个层，每一层都有清晰的角色和分工。层与层之间通过软件接口通信。在一些实施例中，可以将Android系统分为五层，分别为应用程序层(applications)、应用程序框架层(application framework)、安卓运行时(Android runtime)和系统库、硬件抽象层(hardware abstract layer，HAL)以及内核层(kernel)。
应用程序层可以包括一系列应用程序包，应用程序层通过调用应用程序框架层所提供的应用程序接口(application programming interface，API)运行应用程序。如图5所示，应用程序包可以包括相机，图库，日历，通话，地图，导航，WLAN，蓝牙，音乐，视频，游戏等应用程序。
应用程序框架层为应用程序层的应用程序提供API和编程框架。应用程序框架层包括一些预先定义的函数。如图5所示，应用程序框架层可以包括窗口管理器，内容提供器，视图系统，电话管理器，资源管理器，通知管理器，数据库等。
窗口管理器用于管理窗口程序。窗口管理器可以获取显示屏大小，判断是否有状态栏，锁定屏幕，截取屏幕等。内容提供器用来存放和获取数据，并使这些数据可以被应用程序访问。数据可以包括视频，图像，音频，拨打和接听的电话，浏览历史和书签，电话簿等。视图系统包括可视控件，例如显示文字的控件，显示图片的控件等。视图系统可用于构建应用程序。显示界面可以由一个或多个视图组成的。例如，包括短信通知图标的显示界面，可以包括显示文字的视图以及显示图片的视图。电话管理器用于提供电子设备40的通信功能。例如通话状态的管理(包括接通，挂断等)。资源管理器为应用程序提供各种资源，比如本地化字符串，图标，图片，布局文件，视频文件等。通知管理器使应用程序可以在状态栏中显示通知信息，可以用于传达告知类型的消息，可以短暂停留后自动消失，无需用户交互。比如通知管理器被用于告知下载完成，消息提醒等。通知管理器还可以是以图表或者滚动条文本形式出现在系统顶部状态栏的通知，例如后台运行的应用程序的通知，还可以是以对话窗口形式出现在屏幕上的通知。例如在状态栏提示文本信息，发出提示音，电子设备40振动，指示灯闪烁等。数据库可以用于按照数据结构组织、存储和管理数据。
安卓运行时包括核心库和虚拟机。安卓运行时负责安卓系统的调度和管理。核心库包含两部分：一部分是java语言需要调用的功能函数，另一部分是安卓的核心库。应用程序层和应用程序框架层运行在虚拟机中。虚拟机将应用程序层和应用程序框架层的java文件执行为二进制文件。虚拟机用于执行对象生命周期的管理，堆栈管理，线程管理，安全和异常的管理，以及垃圾回收等功能。
系统库可以包括多个功能模块。例如：表面管理器(surface manager)，媒体库(media libraries)，三维图形处理库，例如：开放图形库(open graphics library，OpenGL)，2D图形引擎(例如：SGL)等。
表面管理器用于对显示子系统进行管理，并且为多个应用程序提供了2D和3D图层的融合。媒体库支持多种常用的音频，视频格式回放和录制，以及静态图像文件等。媒体库可以支持多种音视频编码格式，例如：MPEG4，H.264，MP3，AAC，AMR，JPG，PNG等。三维图形处理库用于实现三维图形绘图，图形渲染，合成和图层处理等。2D图形引擎是2D绘图的绘图引擎。
硬件抽象层，可以包含多个库模块，库模块如可以包括传感器库模块等。Android系统可以为设备硬件加载相应的库模块，进而实现应用程序框架层访问设备硬件的目的。设备硬件可以包括如电子设备40中的温度传感器等。
内核层是硬件和软件之间的层。内核层用于驱动硬件，使得硬件工作。内核层至少包含显示驱动等，本申请实施例对此不做限制。
本申请实施例提供的一种图形渲染方法包括多个目标对应关系的学习流程以及使用该多个目标对应关系简化应用程序运行过程中的着色器的流程。
图6为本申请实施例提供的一种图形渲染方法中多个目标对应关系的学习流程的流程示意图，如图6所示，可以包括以下步骤：
S600：终端设备采集应用程序运行过程中所使用的着色器。
可能的实现方式中，终端设备根据预设关键词确定着色器的调用指令，并拦截着色器的调用指令，该调用指令中包括着色器标识或着色器中的至少一种。在调用指令中未包括着色器的情况下，终端设备根据着色器标识获取着色器。其中，预设关键词可以是代码指定的。
示例性的，预设关键词为ShaderSource，终端设备确定指令中包括ShaderSource的情况下，获取该着色器调用指令。终端设备获取的着色器调用指令为glShaderSource(vertexShader，1，&vertexShaderSource，NULL)。其中，vertexShaderSource为着色器标识。终端设备根据vertexShaderSource获取该标识所指示的着色器。vertexShaderSource所指示的着色器用于为如图7所示目标场景中的目标对象渲染颜色。目标对象为图7所示场景中士兵701，士兵701为终端设备中视图系统的可视控件绘制的。
S601：终端设备根据着色器的代码复杂度或者着色器被调用次数中的至少一种，获取采集的着色器中的重负载着色器。
其中，代码复杂度包括时间复杂度或空间复杂度等，时间复杂度为代码执行时间随数据规模增长的变化趋势，又称作渐进时间复杂度。空间复杂度为渐进空间复杂度，空间复杂度表示算法的存储空间与数据规模之间的增长关系。
着色器被调用次数为该着色器所渲染场景中该着色器被调用的次数。着色器被调用的次数越多该着色器的负载越大。
可能的实现方式中，终端设备将代码复杂度大于第四预设阈值的着色器确定为重负载着色器。或者，终端设备将被调用次数大于第五预设阈值的着色器确定为重负载着色器，或者，终端设备将代码复杂度大于第四预设阈值，且被调用次数大于第五预设阈值的着色器确定为重负载着色器。
可以理解的是，着色器的代码复杂度越大，调用该着色器进行图形渲染所需渲染时长越长，因此，终端设备可以获取着色器的渲染时长。终端设备可以按照渲染时长的大小顺序将前预设比重数量的着色器作为重负载着色器。
示例性的，终端设备对获取的着色器的渲染时长进行降序排序，并将排名前80％的渲染时长对应的着色器作为重负载着色器。
S602：终端设备根据第一预设规则简化重负载着色器，得到多个简化着色器。其中，第一预设规则包括代码移动、表达式删除、或语法精简中的至少一种。
代码移动是将片段着色器中的片段移动至对应的顶点着色器中，在片段着色器中使用目标执行结果。该目标执行结果为顶点着色器执行该片段的执行结果。渲染同一个对象的顶点着色器与片段着色器相对应。
表达式删除，本申请实施例中表达式又称作函数。着色器中的函数都是为了产生特效或者完成计算，因此，函数都会对最终的渲染结果产生一定的影响，这些影响有大有小，终端设备可以获取函数的影响值，并将影响值小于预设影响值的函数删除。其中，预设影响值可以根据经验由代码设定。
语法精简是将着色器中的代码进行精简，减少代码量。示例性的，利用封装好的包替换实现相同功能的代码。
可能的实现方式中，终端设备从第一着色器中选择至少一个变量或表达式作为移动目标，终端设备提取第一着色器中与计算移动目标相关的语句作为目标片段，并将目标片段移动至第二着色器中，第一着色器为片段着色器，第二着色器为与第一着色器相对应的顶点着色器。终端设备将第一着色器中的目标片段删除，得到第一着色器简化后的着色器。简化后的着色器使用目标片段在顶点着色器中的执行结果。
在一个例子中，如图8A所示终端设备对着色器1进行代码移动得到简化着色器一号，简化着色器一号的标识为1-1，终端设备对着色器1进行语法精简和表达式删除得到简化着色器二号，简化着色器二号的标识为1-2，终端设备对着色器1进行代码移动、表达式删除以及语法精简得到简化着色器三号，简化着色器三号的标识为1-3。
在另一个例子中，如图8B所示，终端设备根据简化模型提取多个应用程序(如：图8B中应用程序APP1至应用程序APP3等)中的着色器，并识别提取得到的着色器中的重负载着色器。终端设备根据第一预设规则对重负载着色器进行简化，得到着色器简化模型集合。其中，简化模型用于实现着色器提取、着色器识别以及着色器简化。
示例性的，简化模型提取如图8B中的应用程序APP1至应用程序APP3中的着色器。可能的实现方式参考上述S600中的描述，不再赘述。简化模型对提取的着色器进行着色器识别得到应用程序APP1中的重负载着色器包括着色器1、着色器2和着色器3。应用程序APP2中的重负载着色器包括着色器4、着色器5和着色器6。应用程序APP3中的重负载着色器包括着色器7、着色器8和着色器9。可能的实现方式参考上述S601中的描述，不再赘述。简化模型根据不同的第一预设规则，对着色器1进行简化得到标识为1-1至1-3的三个简化着色器。简化模型对着色器2至着色器9进行的简化与此类似，不再赘述。可能的实现方式参考上述S602中的描述，不再赘述。
S603：终端设备将应用程序中的重负载着色器替换为该重负载着色器简化得到的不同简化着色器，得到多个简化应用程序。其中，不同简化着色器为根据不同的第一预设规则对重负载着色器进行简化得到的着色器。
基于图8B的示例，终端设备将应用程序APP1中的着色器1替换为标识为1-1的简化着色器，得到第一简化应用程序，终端设备将应用程序APP1中的着色器1替换为标识 1-2的简化着色器，得到第二简化应用程序，终端设备将应用程序APP1中的着色器1替换为标识1-3的简化着色器，得到第三简化应用程序。
S604：终端设备采集该多个简化应用程序在目标设备上运行时的状态信息以及目标设备的硬件信息。其中，状态信息包括简化着色器所需渲染时长、渲染得到的画面的画质质量、目标设备的负载、目标设备的功耗或目标设备的温度等信息。硬件信息包括芯片种类、芯片的性能如：中央处理器(central processing unit，CPU)频率或GPU频率等中的至少一种信息。
S605：终端设备根据采集的状态信息以及硬件信息筛选简化着色器，得到多个目标对应关系。目标对应关系包括着色器信息(即重负载着色器信息)、目标硬件信息、目标负载等级以及简化着色器信息的对应关系。其中，目标硬件信息为目标设备所包括的硬件信息。目标负载等级为目标设备上运行简化应用程序时，目标设备的负载对应的负载等级。
一种可能的实现方式中，终端设备根据采集的状态信息以及该终端设备的硬件信息，对采集的状态信息中的渲染时长进行分段，在分段中选择对应该分段的目标画质质量，并确定多个第一简化着色器。终端设备根据重负载着色器信息、多个第一简化着色器信息、该终端设备的硬件信息以及该终端设备的负载等级得到多个目标对应关系。其中，目标画质质量大于第二预设阈值，第一简化着色器为目标画质质量对应的简化应用程序中的简化着色器。终端设备的负载等级可以根据状态信息中终端设备的负载得到。
示例性的，终端设备的负载大于第六预设阈值的情况下，终端设备可以将该终端设备的负载进行量化，得到该终端设备的负载对应的负载等级为固定值。该固定值即为该终端设备的负载对应的量化值。例如1，固定值1用于表征高负载。固定值0用于表征低负载。
另一种可能的实现方式中，终端设备根据采集的状态信息以及该终端设备的硬件信息，对采集的状态信息中的画质质量进行分段，在每个分段中选择对应该分段的目标渲染时长，并确定第二简化着色器。终端设备根据重负载着色器信息、多个第二简化着色器信息、该终端设备的硬件信息以及该终端设备的负载等级得到多个目标对应关系。其中，目标渲染时长小于第三预设阈值，目标简化着色器为目标渲染时长对应的简化应用程序中的简化着色器。
由于不同的应用程序使用的着色器文件差异较大，不同应用程序厂商对着色器的编写和生成方式不同，当应用程序运行在终端设备上时，无法做到每个应用程序都能流畅稳定的运行。而终端设备使用的芯片种类较多，且芯片的GPU性能差异较大，当应用程序运行在不同的手机芯片上时，同样的着色器文件的渲染时长不同。对于不同应用程序中的不同着色器，均可以通过上述方法得到该着色器对应不同硬件信息以及负载等级时对应该着色器的简化着色器，以适配不同的应用程序及不同硬件信息的终端设备。
可以理解的是，上述两种可能的实现方式可以结合，得到两种不同类型的目标对应关系，终端设备中分类存储该两种不同类型的目标对应关系。终端设备可以响应于配置指令生效特定类型的目标对应关系。配置指令可以是终端设备接收的用户的操作指令。
图9所示为本申请实施例提供的从另一个角度描述多个目标对应关系的学习流程的流程示意图，图9中应用程序APP1中的重负载着色器包括着色器1、着色器2以及着色器3，其中，着色器1的着色器变种包括标号为1-1至1-3的着色器，着色器2的着色器变种包括标号为2-1至2-3的着色器，着色器3的着色器变种包括标号为3-1和3-2的着 色器。
终端设备将每个变种(即着色器变种)在应用程序中进行替换，即将应用程序中的着色器1替换为其变种着色器。得到多个替换后的应用程序。该步骤对应上述实施例中S603中的描述，不再赘述。在包括不同硬件信息(如：不同芯片)的设备上运行替换后的应用程序，终端设备记录应用程序运行时状态，记录渲染得到的画面的画质质量、渲染时长、CPU&GPU负载，设备温度等信息。该步骤对应上述实施例中S604中的描述，不再赘述。终端设备对渲染得到的画面的画质质量进行划分，挑选不同画质质量的简化模型，得到着色器简化列表。该步骤对应上述实施例中S901中的描述，不再赘述。
图10为本申请实施例提供的一种图形渲染方法中使用多个目标对应关系简化应用程序运行过程中的着色器的流程示意图，如图10所示，可以包括以下步骤：
S1000：终端设备获取源着色器的调用指令。源着色器用于实现目标对象的渲染。目标对象为目标场景中的对象，目标场景为终端设备所运行应用程序中的场景。源着色器的调用指令包括源着色器信息。其中，源着色器信息可以包括源着色器的标识或源着色器中的至少一种。
可能的实现方式中，终端设备根据预设关键词确定源着色器的调用指令，并拦截源着色器的调用指令。其中，预设关键词可以是代码指定的。
示例性的，预设关键词为ShaderSource，终端设备确定指令中包括ShaderSource的情况下，获取该源着色器调用指令。终端设备获取的源着色器的调用指令包括的源着色器信息为SShader。SShader所表征的着色器用于为如图7所示目标场景中的目标对象渲染颜色，目标对象为图7所示场景中士兵701，士兵701为终端设备中视图系统的可视控件绘制的。
S1001：终端设备确定预存的多个目标对应关系中包括源着色器信息。
本申请实施例中，多个目标对应关系为上述实施例得到的多个目标对应关系，不再赘述。
一种可能的实现方式中，终端设备遍历预存的多个目标对应关系中的着色器信息，确定多个目标对应关系中包括源着色器信息。
基于S1000中示例，终端设备确定多个目标对应关系中包括SShader。
另一种可能的实现方式中，终端设备遍历特定类型的目标对应关系中的着色器信息，并确定该特定类型的目标对应关系中包括源着色器信息。特定类型的目标对应关系为终端设备根据配置指令生效的目标对应关系。
S1002：终端设备获取自身硬件信息以及目标场景的场景复杂度。
可能的实现方式中，终端设备获取其硬件信息，并计算目标场景中描绘指令的个数。终端设备根据描绘指令的个数确定目标场景的场景复杂度。其中，描绘指令为CPU对底层图形绘制接口的调用命令。描绘指令用于终端设备执行检测渲染状态、提交渲染数据以及提交渲染状态等任务。
示例性的，终端设备获取的硬件信息包括CPU主频为3.0GHz，GPU频率为850MHz。如果终端设备计算得到目标场景中描绘指令的个数为600，第一预设阈值为300，终端设备计算得到的描绘指令的个数大于第一预设阈值，那么，终端设备将目标场景的场景复杂度确定为固定值例如1。1用于表征场景复杂度高。
需要说明的是，场景复杂度与上述实施例中负载等级相对应。如果终端设备的硬件信 息相同，那么，场景复杂度越高的情况下，在终端设备运行至该场景时终端设备的负载也越高。
S1003：终端设备从包括源着色器信息的目标对应关系中，获取源着色器信息、终端设备的硬件信息以及目标场景的场景复杂度对应的目标简化着色器信息。
本申请实施例中，包括源着色器信息的目标对应关系为S1001中确定的包括源着色器信息的目标对应关系。
可能的实现方式中，终端设备从包括源着色器信息的目标对应关系中获取源着色器信息、终端设备的硬件信息以及目标场景的场景复杂度对应的目标简化着色器信息。
示例性的，多个目标对应关系如下表1所示：
表1
Figure PCTCN2022097714-appb-000001
表1中，着色器信息SShader，硬件信息GPU频率为850MHz，CPU主频3.0GHz，场景复杂度1对应的简化着色器信息为着色器一号。着色器信息SShader，硬件信息GPU频率为2330MHZ，CPU主频3.6GHz，场景复杂度1对应的简化着色器信息为着色器二号。着色器信息VShader，硬件信息GPU频率为1000MHz，CPU主频3.8GHz，场景复杂度0对应的简化着色器信息为着色器三号。其中，0表征的场景复杂度低于1表征的场景复杂度。
基于S1001以及S1002中的示例，终端设备从表1所示多个目标对应关系中获取着色器信息为SShader、CPU主频为3.0GHz、GPU频率为850MHz，目标场景的场景复杂度为1对应的目标简化着色器信息为着色器一号。
可以理解的是，终端设备也可以从包括源着色器信息的目标对应关系中，获取源着色器信息以及终端设备的硬件信息对应的目标简化着色器信息。
S1004：终端设备将源着色器的调用指令中的源着色器信息替换为目标简化着色器信息，并编译目标简化着色器。
在源着色器信息为源着色器的情况下，终端设备将源着色器替换为目标简化着色器，并将目标简化着色器转换为可以被终端设备所识别的二进制语言的着色器。其中，源着色器以及目标简化着色器均为着色器语言编写的着色器。示例性的，源着色器以及目标简化着色器均为OpenGL着色语言编写的着色器。
在源着色器信息为源着色器的标识的情况下，终端设备将源着色器的标识所指示的源着色器替换为目标简化着色器，并将目标简化着色器转换为可以被终端设备所识别的二进制语言的着色器。
S1005：终端设备将编译后的目标简化着色器传送给GPU以实现目标对象的渲染。
可能的实现方式中，终端设备将编译后的目标简化着色器传送给GPU，GPU执行该目标简化着色器以实现目标对象的渲染。
基于S1000中的示例，目标对象为一群士兵，GPU执行目标简化着色器为这群士兵 所穿铠甲渲染相同的颜色。
本申请实施例中，终端设备学习不同应用程序中的着色器在不同硬件信息的终端设备中对应的简化着色器，得到多个目标对应关系。这样，在终端设备中运行该应用程序时，终端设备可以基于该应用程序中场景的场景复杂度、该应用程序使用的着色器、以及自身的硬件信息在目标对应关系中得到目标简化着色器信息，并使用简化着色器运行应用程序，因为简化着色器在运行时，对GPU中计算单元工作量的占用较小，可以有效改善因着色器对GPU占用导致的终端设备卡顿现象。
可以理解的是上述实施例中的各个步骤可以由不同的设备执行，从而得到一个新的实施例。示例性的，如图11所示为本申请实施例提供的另一种图形渲染方法的流程示意图，应用于图3所示的图形渲染系统，图11包括：
S1100：服务器采集应用程序中所使用的着色器，并识别采集的着色器中的重负载着色器，根据第一预设规则对重负载着色器进行简化，得到多个简化着色器。
可能的实现方式，参考上述S600～S602中的描述，不再赘述。
S1101：服务器采集硬件信息以及简化应用程序在不同硬件信息的终端设备上运行时的状态信息，并分析采集的信息，得到多个目标对应关系。其中，简化应用程序为将应用程序中的着色器替换为简化着色器后的应用程序。目标对应关系为着色器信息、硬件信息、场景复杂度以及简化着色器信息的目标对应关系。
可能的实现方式，参考上述S603～S605中的描述，不再赘述。
S1102：服务器向终端设备发送多个目标对应关系。
S1103：终端设备获取源着色器的调用指令。源着色器用于实现目标对象的渲染。目标对象为目标场景中的对象，目标场景为终端设备所运行应用程序中的场景。源着色器的调用指令包括源着色器信息。其中，源着色器信息可以包括源着色器的标识或源着色器中的至少一种。
可能的实现方式及示例，参考上述S1000中的描述，不再赘述。
S1104：终端设备确定预存的多个目标对应关系中包括源着色器的信息。
可能的实现方式及示例，参考上述S1001中的描述，不再赘述。
S1105：终端设备获取自身硬件信息以及目标场景的场景复杂度。
可能的实现方式及示例，参考上述S1002中的描述，不再赘述。
S1106：终端设备根据多个目标对应关系，获取源着色器信息、终端设备的硬件信息以及目标场景的场景复杂度对应的目标简化着色器信息。
可能的实现方式及示例，参考上述S1003中的描述，不再赘述。
S1107～S1108参考上述S1004～S1005中的描述，不再赘述。
本申请实施例中，服务器学习不同应用程序中的着色器在不同硬件信息的终端设备中对应的简化着色器，得到多个目标对应关系。服务器将得到的多个目标对应关系发送给终端设备，这样，在终端设备中运行应用程序时，终端设备可以基于应用程序中场景的场景复杂度、应用程序使用的着色器信息、以及自身的硬件信息在多个目标对应关系中得到目标简化着色器信息，并使用简化着色器运行应用程序，因为简化着色器在运行时，对GPU中计算单元工作量的占用较小，可以有效改善因着色器对GPU占用导致的终端设备卡顿现象。
图12所示为本申请实施例提供的另一种图形渲染方法的流程示意图，图12包括：
S1200：终端设备获取源着色器的调用指令。源着色器用于实现目标对象的渲染。目标对象为目标场景中的对象，目标场景为终端设备所运行应用程序中的场景。源着色器的调用指令包括源着色器信息。其中，源着色器信息可以包括源着色器的标识或源着色器中的至少一种。
可能的实现方式及示例，参考上述S1000中的描述，不再赘述。
S1201：终端设备根据源着色器的代码复杂度或者源着色器被调用次数中的至少一种，得到源着色器为重负载着色器。
可能的实现方式及示例，参考上述S601中的描述，不再赘述。
S1202：终端设备根据第一预设规则简化源着色器，得到目标简化着色器。其中，第一预设规则包括代码移动、表达式删除、或语法精简中的至少一种。
可能的实现方式及示例，参考上述S602中的描述，不再赘述。
S1203：终端设备将源着色器的调用指令中的源着色器信息替换为目标简化着色器信息，并编译目标简化着色器。
可能的实现方式及示例，参考上述S1004中的描述，不再赘述。
S1204：终端设备将编译后的目标简化着色器传送给GPU以实现目标对象的渲染。
可能的实现方式及示例，参考上述S1005中的描述，不再赘述。
本申请实施例中，在终端设备中运行应用程序时，终端设备可以实时的对应用程序中的重负载着色器进行简化，并使用简化后的着色器运行应用程序，因为简化后的着色器在运行时，对GPU中计算单元工作量的占用较小，可以有效改善因着色器对GPU占用导致的终端设备卡顿现象。
上述主要从方法的角度对本申请实施例提供的方案进行了介绍。为了实现上述功能，其包含了执行各个功能相应的硬件结构和/或软件模块。本领域技术人员应该很容易意识到，结合本文中所公开的实施例描述的各示例的方法步骤，本申请能够以硬件或硬件和计算机软件的结合形式来实现。某个功能究竟以硬件还是计算机软件驱动硬件的方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
本申请实施例可以根据上述方法示例对实现环境光照强度确定方法的装置进行功能模块的划分，例如可以对应各个功能划分各个功能模块，也可以将两个或两个以上的功能集成在一个处理模块中。示例性的，将目标应用程序、画图接口以及显示引擎的功能集成在显示控制单元中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。需要说明的是，本申请实施例中对模块的划分是示意性的，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式。
如图13所示为本申请实施例提供的一种终端设备的结构示意图，图13所示终端设备130包括处理模块1301，处理模块1301用于利用目标着色器进行应用程序的图形渲染；在确定应用程序运行至复杂场景时，获取目标简化着色器；复杂场景为应用程序运行所需的指令的个数大于第一预设阈值的场景，目标简化着色器对终端设备130计算资源的占用小于目标着色器对终端设备130计算资源的占用；利用目标简化着色器在复杂场景中进行应用程序的图形渲染。例如：结合图6，处理模块1301可以用于执行S600～S605，结合图 10，处理模块1301可以用于执行S1000～S1005。结合图11，处理模块1301可以用于执行S1103～S1108，结合图12，处理模块可以用于执行S1200～S1204。
可选的，处理模块1301具体用于：根据预存的多个对应关系，获取目标简化着色器；多个对应关系包括着色器的信息、设备的硬件信息以及简化着色器之间的对应关系；着色器的信息包括着色器或着色器的标识中的任意一种。
可选的，处理模块1301具体用于：在多个对应关系中，获取目标着色器的信息以及终端设备130的硬件信息对应的目标简化着色器。
可选的，上述对应关系中还包括负载等级；负载等级为终端设备130的负载对应的量化值；处理模块1301具体用于：获取复杂场景中描绘指令的个数；根据描绘指令的个数获取目标负载等级；在多个对应关系中，获取目标着色器的信息、目标负载等级以及终端设备130的硬件信息对应的目标简化着色器。
可选的，处理模块1301还用于：根据第一预设规则简化目标着色器，得到多个简化着色器；第一预设规则包括代码移动、表达式删除或语法精简中的至少一种；多个简化着色器包括目标简化着色器；采集终端设备130利用多个简化着色器在复杂场景中进行应用程序的图形渲染时终端设备130的状态信息；终端设备130的状态信息包括终端设备130的负载、终端设备130的功耗、终端设备130的温度、渲染时长、以及各简化着色器渲染得到的画面的画质质量；渲染时长为各简化着色器在复杂场景中进行应用程序的图形渲染时所需时长；根据状态信息得到目标着色器的信息、终端设备130的硬件信息、目标负载等级以及目标简化着色器之间的对应关系。
可选的，处理模块1301具体用于从状态信息中获取多个目标渲染时长，多个目标渲染时长均小于第二预设阈值；从多个目标渲染时长对应的状态信息中获取目标状态信息；目标状态信息的画质质量大于第三预设阈值；目标状态信息为目标简化着色器在复杂场景中进行应用程序的图形渲染时终端设备130的状态信息；根据目标状态信息中终端设备130的负载获取目标负载等级；建立目标着色器的信息、终端设备130的硬件信息、目标负载等级以及目标简化着色器之间的对应关系。
可选的，处理模块1301具体用于：从状态信息中获取多个目标画质质量，多个目标画质质量均大于第四预设阈值；从多个目标画质质量对应的状态信息中获取目标状态信息；目标状态信息的渲染时长小于第五预设阈值；目标状态信息为目标简化着色器在复杂场景中进行应用程序的图形渲染时终端设备130的状态信息；根据目标状态信息中终端设备130的负载获取目标负载等级；建立目标着色器的信息、终端设备130的硬件信息、目标负载等级以及目标简化着色器之间的对应关系。
可选的，终端设备130还包括通信模块1302：用于接收其他设备发送的多个对应关系。例如，结合图11，通信模块1302可以用于执行S1102中的接收步骤。
可选的，处理模块1301具体用于：根据第一预设规则简化目标着色器，得到目标简化着色器；第一预设规则包括代码移动、表达式删除或语法精简中的至少一种。
可选的，在终端设备130确定应用程序运行至复杂场景时，处理模块1301具体用于：确定应用程序运行至复杂场景，以及目标着色器的代码复杂度大于第六预设阈值，或者，目标着色器的被调用次数大于第七预设阈值时，获取目标简化着色器。在一个例子中，结合图4，上述处理模块1301的功能可以由图4中的处理器401调用存储器402中的计算 机程序实现。通信模块1302的功能可以由图4中通信模块404实现。
如图14所示为本申请实施例提供的一种服务器的结构示意图，图14所示服务器140包括处理模块1401。处理模块1401用于：根据第一预设规则简化目标着色器，得到多个简化着色器；第一预设规则包括代码移动、表达式删除或语法精简中的至少一种；多个简化着色器包括目标简化着色器；采集终端设备利用多个简化着色器在复杂场景中进行应用程序的图形渲染时终端设备的状态信息；终端设备的状态信息包括终端设备的负载、终端设备的功耗、终端设备的温度、渲染时长、以及各简化着色器渲染得到的画面的画质质量；渲染时长为各简化着色器在复杂场景中进行应用程序的图形渲染时所需时长；根据状态信息得到目标着色器的信息、终端设备的硬件信息、目标负载等级以及目标简化着色器之间的对应关系。例如，结合图11，处理模块1401可以用于执行S1100～S1101。
可选的，处理模块1401具体用于从状态信息中获取多个目标渲染时长，多个目标渲染时长均小于第二预设阈值；从多个目标渲染时长对应的状态信息中获取目标状态信息；目标状态信息的画质质量大于第三预设阈值；目标状态信息为目标简化着色器在复杂场景中进行应用程序的图形渲染时终端设备的状态信息；根据目标状态信息中终端设备的负载获取目标负载等级；建立目标着色器的信息、终端设备的硬件信息、目标负载等级以及目标简化着色器之间的对应关系。
可选的，处理模块1401具体用于：从状态信息中获取多个目标画质质量，多个目标画质质量均大于第四预设阈值；从多个目标画质质量对应的状态信息中获取目标状态信息；目标状态信息的渲染时长小于第五预设阈值；目标状态信息为目标简化着色器在复杂场景中进行应用程序的图形渲染时终端设备的状态信息；根据目标状态信息中终端设备的负载获取目标负载等级；建立目标着色器的信息、终端设备的硬件信息、目标负载等级以及目标简化着色器之间的对应关系。
可选的，服务器140还包括通信模块1402：用于向终端设备发送多个对应关系。例如，结合图11，通信模块1401可以用于执行S1102中的发送步骤。
在一个例子中，结合图4，上述处理模块1401的功能可以由图4中的处理器401调用存储器402中的计算机程序实现。通信模块1402的功能可以由图4中通信模块404实现。
如图15所示为本申请实施例提供的一种芯片的结构示意图。芯片150包括一个或两个以上(包括两个)处理器1501、通信线路1502、通信接口1503和存储器1504。
在一些实施方式中，存储器1504存储了如下的元素：可执行模块或者数据结构，或者他们的子集，或者他们的扩展集。
上述本申请实施例描述的方法可以应用于处理器1501中，或者由处理器1501实现。处理器1501可能是一种集成电路芯片，具有信号的处理能力。在实现过程中，上述方法的各步骤可以通过处理器1501中的硬件的集成逻辑电路或者软件形式的指令完成。上述的处理器1201可以是通用处理器(例如，微处理器或常规处理器)、数字信号处理器(digital signal processing，DSP)、专用集成电路(application specific integrated circuit，ASIC)、现成可编程门阵列(field-programmable gate array，FPGA)或者其他可编程逻辑器件、分立门、晶体管逻辑器件或分立硬件组件，处理器1501可以实现或者执行本申请实施例中的公开的各方法、步骤及逻辑框图。
结合本申请实施例所公开的方法的步骤可以直接体现为硬件译码处理器执行完成，或者用译码处理器中的硬件及软件模块组合执行完成。其中，软件模块可以位于随机存储器、只读存储器、可编程只读存储器或带电可擦写可编程存储器(electrically erasable programmable read only memory，EEPROM)等本领域成熟的存储介质中。该存储介质位于存储器1504，处理器1501读取存储器1504中的信息，结合其硬件完成上述方法的步骤。
处理器1501、存储器1504以及通信接口1503之间可以通过通信线路1502进行通信。
在上述实施例中，存储器存储的供处理器执行的指令可以以计算机程序产品的形式实现。其中，计算机程序产品可以是事先写入在存储器中，也可以是以软件形式下载并安装在存储器中。
本申请实施例还提供一种计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行计算机程序指令时，全部或部分地产生按照本申请实施例的流程或功能。计算机可以是通用计算机、专用计算机、计算机网络或者其他可编程装置。计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一计算机可读存储介质传输，例如，计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(digital subscriber line，DSL)或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。计算机可读存储介质可以是计算机能够存储的任何可用介质或者是包括一个或多个可用介质集成的服务器、数据中心等数据存储设备。例如，可用介质可以包括磁性介质(例如，软盘、硬盘或磁带)、光介质(例如，数字通用光盘(digital versatile disc，DVD))、或者半导体介质(例如，固态硬盘(solid state disk，SSD))等。
本申请实施例还提供一种计算机可读存储介质。上述实施例中描述的方法可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。计算机可读介质可以包括计算机存储介质和通信介质，还可以包括任何可以将计算机程序从一个地方传送到另一个地方的介质。存储介质可以是可由计算机访问的任何目标介质。
作为一种可能的设计，计算机可读介质可以包括紧凑型光盘只读储存器(compact disc read-only memory，CD-ROM)、RAM、ROM、EEPROM或其它光盘存储器；计算机可读介质可以包括磁盘存储器或其它磁盘存储设备。而且，任何连接线也可以被适当地称为计算机可读介质。例如，如果使用同轴电缆，光纤电缆，双绞线，DSL或无线技术(如红外，无线电和微波)从网站，服务器或其它远程源传输软件，则同轴电缆，光纤电缆，双绞线，DSL或诸如红外，无线电和微波之类的无线技术包括在介质的定义中。如本文所使用的磁盘和光盘包括光盘(CD)，激光盘，光盘，数字通用光盘(digital versatile disc，DVD)，软盘和蓝光盘，其中磁盘通常以磁性方式再现数据，而光盘利用激光光学地再现数据。
上述的组合也应包括在计算机可读介质的范围内。以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以权利要求的保护范围为准。

Claims (12)

  1. 一种图形渲染方法，其特征在于，所述方法包括：
    终端设备利用目标着色器进行应用程序的图形渲染；
    在所述终端设备确定所述应用程序运行至复杂场景时，所述终端设备获取目标简化着色器；所述复杂场景为所述应用程序运行所需的指令的个数大于第一预设阈值的场景，所述目标简化着色器对所述终端设备计算资源的占用小于所述目标着色器对所述终端设备计算资源的占用；
    所述终端设备利用所述目标简化着色器在所述复杂场景中进行所述应用程序的图形渲染。
  2. 根据权利要求1所述的方法，其特征在于，所述终端设备获取目标简化着色器，包括：
    所述终端设备根据预存的多个对应关系，获取所述目标简化着色器；所述多个对应关系包括着色器的信息、设备的硬件信息以及简化着色器之间的对应关系；所述着色器的信息包括所述着色器或所述着色器的标识中的任意一种。
  3. 根据权利要求2所述的方法，其特征在于，所述终端设备根据预存的多个对应关系，获取所述目标着色器的信息，包括：
    所述终端设备在所述多个对应关系中，获取所述目标着色器的信息以及所述终端设备的硬件信息对应的所述目标简化着色器。
  4. 根据权利要求2所述的方法，其特征在于，所述对应关系中还包括负载等级；所述负载等级为所述终端设备的负载对应的量化值；所述终端设备根据预存的多个对应关系，获取所述目标着色器的信息，包括：
    所述终端设备获取所述复杂场景中描绘指令的个数；
    所述终端设备根据所述描绘指令的个数获取目标负载等级；
    所述终端设备在所述多个对应关系中，获取所述目标着色器的信息、所述目标负载等级以及所述终端设备的硬件信息对应的所述目标简化着色器。
  5. 根据权利要求2-4任一项所述的方法，其特征在于，所述方法还包括：
    所述终端设备根据第一预设规则简化所述目标着色器，得到多个简化着色器；所述第一预设规则包括代码移动、表达式删除或语法精简中的至少一种；所述多个简化着色器包括所述目标简化着色器；
    所述终端设备采集所述终端设备利用所述多个简化着色器在所述复杂场景中进行所述应用程序的图形渲染时所述终端设备的状态信息；所述终端设备的状态信息包括所述终端设备的负载、所述终端设备的功耗、所述终端设备的温度、渲染时长以及各简化着色器渲染得到的画面的画质质量；所述渲染时长为所述各简化着色器在所述复杂场景中进行所述应用程序的图形渲染时所需时长；
    所述终端设备根据所述状态信息得到所述目标着色器的信息、所述终端设备的硬件信息、所述目标负载等级以及所述目标简化着色器之间的对应关系。
  6. 根据权利要求5所述的方法，其特征在于，所述终端设备根据所述状态信息得到所述目标着色器的信息、所述终端设备的硬件信息、所述目标负载等级以及所述目标简化着 色器之间的对应关系，包括：
    所述终端设备从所述状态信息中获取多个目标渲染时长，所述多个目标渲染时长均小于第二预设阈值；
    所述终端设备从所述多个目标渲染时长对应的所述状态信息中获取目标状态信息；所述目标状态信息的画质质量大于第三预设阈值；所述目标状态信息为所述目标简化着色器在所述复杂场景中进行所述应用程序的图形渲染时所述终端设备的状态信息；
    所述终端设备根据所述目标状态信息中所述终端设备的负载获取所述目标负载等级；
    所述终端设备建立所述目标着色器的信息、所述终端设备的硬件信息、所述目标负载等级以及所述目标简化着色器之间的对应关系。
  7. 根据权利要求5所述的方法，其特征在于，所述终端设备根据所述状态信息得到所述目标着色器的信息、所述终端设备的硬件信息、所述目标负载等级以及所述目标简化着色器之间的对应关系，包括：
    所述终端设备从所述状态信息中获取多个目标画质质量，所述多个目标画质质量均大于第四预设阈值；
    所述终端设备从所述多个目标画质质量对应的所述状态信息中获取目标状态信息；所述目标状态信息的渲染时长小于第五预设阈值；所述目标状态信息为所述目标简化着色器在所述复杂场景中进行所述应用程序的图形渲染时所述终端设备的状态信息；
    所述终端设备根据所述目标状态信息中所述终端设备的负载获取所述目标负载等级；
    所述终端设备建立所述目标着色器的信息、所述终端设备的硬件信息、所述目标负载等级以及所述目标简化着色器之间的对应关系。
  8. 根据权利要求2-4任一项所述的方法，其特征在于，所述方法还包括：
    所述终端设备接收其他设备发送的所述多个对应关系。
  9. 根据权利要求1所述的方法，其特征在于，所述终端设备获取目标简化着色器，包括：
    所述终端设备根据第一预设规则简化所述目标着色器，得到所述目标简化着色器；所述第一预设规则包括代码移动、表达式删除或语法精简中的至少一种。
  10. 根据权利要求1-9任一项所述的方法，其特征在于，在所述终端设备确定所述应用程序运行至复杂场景时，所述终端设备获取目标简化着色器，包括：
    在所述终端设备确定所述应用程序运行至所述复杂场景，以及所述目标着色器的代码复杂度大于第六预设阈值，或者，所述目标着色器的被调用次数大于第七预设阈值时，所述终端设备获取所述目标简化着色器。
  11. 一种电子设备，其特征在于，包括：存储器和处理器，所述存储器用于存储计算机程序，所述处理器用于执行所述计算机程序，以执行如权利要求1至10任一项所述的图形渲染方法。
  12. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质存储有指令，当所述指令被执行时，使得计算机执行如权利要求1至10任一项所述的图形渲染方法。
PCT/CN2022/097714 2021-08-24 2022-06-08 图形渲染方法、装置及存储介质 WO2023024649A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP22859996.5A EP4358028A4 (en) 2021-08-24 2022-06-08 METHOD AND DEVICE FOR GRAPHICAL REPRESENTATION AND STORAGE MEDIUM

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110977666.2 2021-08-24
CN202110977666.2A CN115937380A (zh) 2021-08-24 2021-08-24 图形渲染方法、装置及存储介质

Publications (1)

Publication Number Publication Date
WO2023024649A1 true WO2023024649A1 (zh) 2023-03-02

Family

ID=85321394

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/097714 WO2023024649A1 (zh) 2021-08-24 2022-06-08 图形渲染方法、装置及存储介质

Country Status (3)

Country Link
EP (1) EP4358028A4 (zh)
CN (1) CN115937380A (zh)
WO (1) WO2023024649A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116450363B (zh) * 2023-06-13 2023-11-14 荣耀终端有限公司 一种资源调度方法及电子设备
CN116909407B (zh) * 2023-09-12 2024-01-12 深圳康荣电子有限公司 基于虚拟现实的触控显示屏全景交互方法及控制系统

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130127858A1 (en) * 2009-05-29 2013-05-23 Luc Leroy Interception of Graphics API Calls for Optimization of Rendering
CN108765534A (zh) * 2018-05-24 2018-11-06 武汉斗鱼网络科技有限公司 一种图像渲染方法、装置、设备及存储介质
CN109285211A (zh) * 2018-10-29 2019-01-29 Oppo广东移动通信有限公司 画面渲染方法、装置、终端及存储介质
CN109389663A (zh) * 2018-10-31 2019-02-26 Oppo广东移动通信有限公司 画面渲染方法、装置、终端及存储介质
US20200082491A1 (en) * 2018-09-10 2020-03-12 Arm Limited Instruction execution in graphics processor shader programs
CN111179150A (zh) * 2019-12-27 2020-05-19 浙江大学 一种基于绘制指令流的着色器自动简化方法和系统
CN111417978A (zh) * 2018-04-10 2020-07-14 谷歌有限责任公司 游戏渲染中的内存管理
CN114443189A (zh) * 2021-08-20 2022-05-06 荣耀终端有限公司 一种图像处理方法和电子设备

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107832108B (zh) * 2016-09-14 2021-05-07 阿里巴巴集团控股有限公司 3D canvas网页元素的渲染方法、装置及电子设备
US11321134B2 (en) * 2019-05-31 2022-05-03 Apple Inc. Normalizing target utilization rates of a cross-application table of concurrently executing applications to schedule work on a command queue of a graphics processors

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130127858A1 (en) * 2009-05-29 2013-05-23 Luc Leroy Interception of Graphics API Calls for Optimization of Rendering
CN111417978A (zh) * 2018-04-10 2020-07-14 谷歌有限责任公司 游戏渲染中的内存管理
CN108765534A (zh) * 2018-05-24 2018-11-06 武汉斗鱼网络科技有限公司 一种图像渲染方法、装置、设备及存储介质
US20200082491A1 (en) * 2018-09-10 2020-03-12 Arm Limited Instruction execution in graphics processor shader programs
CN109285211A (zh) * 2018-10-29 2019-01-29 Oppo广东移动通信有限公司 画面渲染方法、装置、终端及存储介质
CN109389663A (zh) * 2018-10-31 2019-02-26 Oppo广东移动通信有限公司 画面渲染方法、装置、终端及存储介质
CN111179150A (zh) * 2019-12-27 2020-05-19 浙江大学 一种基于绘制指令流的着色器自动简化方法和系统
CN114443189A (zh) * 2021-08-20 2022-05-06 荣耀终端有限公司 一种图像处理方法和电子设备

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4358028A4

Also Published As

Publication number Publication date
EP4358028A1 (en) 2024-04-24
EP4358028A4 (en) 2024-04-24
CN115937380A (zh) 2023-04-07

Similar Documents

Publication Publication Date Title
WO2023024649A1 (zh) 图形渲染方法、装置及存储介质
CN103502936B (zh) 基于图像的自动化系统和方法
US10437541B2 (en) Graphics engine and environment for efficient real time rendering of graphics that are not pre-known
US9196075B2 (en) Animation of computer-generated display components of user interfaces and content items
CN113453040B (zh) 短视频的生成方法、装置、相关设备及介质
CN113763856B (zh) 环境光照强度确定方法、装置及存储介质
CN109976735B (zh) 一种基于web可视化的知识图谱算法应用平台
KR20230019975A (ko) 이미지 프로세싱 sdk에서 사용하기 위한 인터페이스 캐러셀
KR20230021113A (ko) 이미지 프로세싱을 위한 소프트웨어 개발 키트
CN114669047B (zh) 一种图像处理方法、电子设备及存储介质
CN106030663A (zh) 具有共享数据信道的着色器管线
US11887229B2 (en) Method and system for populating a digital environment using a semantic map
US20160357532A1 (en) Graphics Engine And Environment For Encapsulating Graphics Libraries and Hardware
US11520473B2 (en) Switch control for animations
CN109416828A (zh) 将帧缓冲映射到逻辑显示器的装置及方法
US10417327B2 (en) Interactive and dynamically animated 3D fonts
WO2023005751A1 (zh) 渲染方法及电子设备
CN116091329A (zh) 图像处理方法、装置、设备及存储介质
CN114863432A (zh) 一种终端设备、对比度调整方法、装置及介质
US11145024B2 (en) Adaptive dispatch for acceleration of deep neural networks on graphic processing units
US20130321445A1 (en) Colorizing user interfaces
US20240233597A9 (en) Ambient Illumination Intensity Determining Method and Apparatus, and Storage Medium
WO2023280241A1 (zh) 图像画面渲染方法及电子设备
WO2024045701A9 (zh) 数据处理方法、装置、设备及存储介质
US20230215074A1 (en) Rendering workload management for extended reality

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22859996

Country of ref document: EP

Kind code of ref document: A1

DPE1 Request for preliminary examination filed after expiration of 19th month from priority date (pct application filed from 20040101)
WWE Wipo information: entry into national phase

Ref document number: 2022859996

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2022859996

Country of ref document: EP

Effective date: 20240115

NENP Non-entry into the national phase

Ref country code: DE