CN116912093A - Picture rendering method and device, electronic equipment and storage medium - Google Patents

Picture rendering method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN116912093A
CN116912093A CN202310929710.1A CN202310929710A CN116912093A CN 116912093 A CN116912093 A CN 116912093A CN 202310929710 A CN202310929710 A CN 202310929710A CN 116912093 A CN116912093 A CN 116912093A
Authority
CN
China
Prior art keywords
rendering
shader program
picture
image quality
content
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310929710.1A
Other languages
Chinese (zh)
Inventor
周攀
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Vivo Mobile Communication Co Ltd
Original Assignee
Vivo Mobile Communication Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Vivo Mobile Communication Co Ltd filed Critical Vivo Mobile Communication Co Ltd
Priority to CN202310929710.1A priority Critical patent/CN116912093A/en
Publication of CN116912093A publication Critical patent/CN116912093A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformation in the plane of the image
    • G06T3/40—Scaling the whole image or part thereof
    • G06T3/4053—Super resolution, i.e. output image resolution higher than sensor resolution
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/50—Controlling the output signals based on the game progress
    • A63F13/52—Controlling the output signals based on the game progress involving aspects of the displayed game scene
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—3D [Three Dimensional] image rendering
    • G06T15/10—Geometric effects
    • G06T15/20—Perspective computation
    • G06T15/205—Image-based rendering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—3D [Three Dimensional] image rendering
    • G06T15/50—Lighting effects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformation in the plane of the image
    • G06T3/40—Scaling the whole image or part thereof
    • G06T3/4046—Scaling the whole image or part thereof using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration by the use of more than one image, e.g. averaging, subtraction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440263—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the spatial resolution, e.g. for displaying on a connected PDA
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

The application discloses a picture rendering method, a device, electronic equipment and a storage medium, wherein the rendering method comprises the following steps: under the condition that the rendering of the first picture content is completed, carrying out image quality enhancement processing on the first picture content to obtain the first picture content with enhanced image quality; superposing second picture content on the first picture content with enhanced picture quality to obtain a target image frame; the target image frame is displayed.

Description

Picture rendering method and device, electronic equipment and storage medium
Technical Field
The application belongs to the technical field of image quality enhancement, and particularly relates to a picture rendering method, a picture rendering device, electronic equipment and a storage medium.
Background
In the related art, in order to improve image quality in an image rendering process, an image quality enhancement technique is generally used to improve a program picture for games or other programs or software having a high demand for image quality.
Common image quality enhancement technologies include an image super-resolution technology, which aims to perform a super-value algorithm or a deep learning super-resolution algorithm on a picture before screen printing to improve the resolution of an image.
The rendered image includes not only a picture of the body but also image contents superimposed on the body image such as text or icons, however, the image quality enhancement technique such as the super resolution technique employed in the related art may cause the image contents superimposed on the body image to become blurred.
Disclosure of Invention
The embodiment of the application aims to provide a method, a device, electronic equipment and a storage medium for rendering pictures, which can solve the problem of blurring of the pictures obtained by rendering in the related technology.
In a first aspect, an embodiment of the present application provides a method for rendering a picture, where the rendering method includes:
under the condition that the rendering of the first picture content is completed, carrying out image quality enhancement processing on the first picture content to obtain the first picture content with enhanced image quality;
superposing second picture content on the first picture content with enhanced picture quality to obtain a target image frame;
the target image frame is displayed.
In a second aspect, an embodiment of the present application provides a rendering apparatus for a picture, including:
the image quality enhancement module is used for carrying out image quality enhancement processing on the first picture content under the condition that the first picture content is completely rendered, so as to obtain the first picture content with enhanced image quality;
The content superposition module is used for superposing second picture content on the first picture content with enhanced picture quality to obtain a target image frame;
and the display module is used for displaying the target image frames.
In a third aspect, embodiments of the present application provide an electronic device comprising a processor and a memory storing a program or instructions executable on the processor, the program or instructions implementing the steps of the method as in the first aspect when executed by the processor.
In a fourth aspect, embodiments of the present application provide a readable storage medium having stored thereon a program or instructions which when executed by a processor perform the steps of the method as in the first aspect.
In a fifth aspect, embodiments of the present application provide a chip comprising a processor and a communication interface coupled to the processor for running a program or instructions implementing the steps of the method as in the first aspect.
In a sixth aspect, embodiments of the present application provide a computer program product stored in a storage medium, the program product being executable by at least one processor to implement a method as in the first aspect.
In the embodiment of the application, after the first picture content is rendered, the first picture content is subjected to image quality enhancement processing, and after the first picture content is subjected to image quality enhancement, the second picture content is superimposed on the first picture content subjected to image quality enhancement, and the target image frame is obtained and displayed, so that the image quality enhancement processing only takes effect on the first picture content and does not act on the second picture content, and the second picture content is prevented from becoming blurred under the condition that the first picture content enhancement processing is realized.
Drawings
FIG. 1 illustrates a flow chart of a method of rendering a picture in accordance with some embodiments of the application;
FIG. 2 illustrates a schematic diagram of a rendering flow of a picture in accordance with some embodiments of the application;
FIG. 3 illustrates a schematic diagram of a rendering pipeline of a picture in accordance with some embodiments of the application;
FIG. 4 illustrates a block diagram of a rendering device of a picture according to some embodiments of the application;
FIG. 5 shows a block diagram of an electronic device according to an embodiment of the application;
fig. 6 is a schematic diagram of a hardware structure of an electronic device implementing an embodiment of the present application.
Detailed Description
The technical solutions of the embodiments of the present application will be clearly described below with reference to the drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which are obtained by a person skilled in the art based on the embodiments of the present application, fall within the scope of protection of the present application.
The terms first, second and the like in the description and in the claims, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the terms so used are interchangeable under appropriate circumstances such that the embodiments of the application are capable of operation in sequences other than those illustrated or otherwise described herein, and that the objects identified by "first," "second," etc. are generally of a type not limited to the number of objects, for example, the first object may be one or more. Furthermore, in the description and claims, "and/or" means at least one of the connected objects, and the character "/", generally means that the associated object is an "or" relationship.
The method, the device, the electronic equipment and the storage medium for rendering the picture provided by the embodiment of the application are described in detail through specific embodiments and application scenes thereof with reference to the accompanying drawings.
In some embodiments of the present application, there is provided a method of rendering a picture, fig. 1 shows a flowchart of a method of rendering a picture of some embodiments of the present application, as shown in fig. 1, the method of rendering includes:
step 102, when the rendering of the first picture content is completed, performing image quality enhancement processing on the first picture content to obtain the first picture content with enhanced image quality.
In the embodiment of the present application, the screen may be a screen of an application program, for example, a screen of a game program, and the screen may be a screen of a 3D scene or a 2D screen.
The first screen content is the main screen content of the screen of the program, i.e., the main screen. The second screen content is screen content such as characters, icons, buttons, and the like superimposed on the main screen.
After the first picture content is rendered, performing picture quality enhancement processing on the rendered first picture content. The image quality enhancement processing includes super-resolution algorithm processing, such as adding pixel number interpolation algorithm by bilinear interpolation, trilinear interpolation, etc., or super-resolution processing by neural network model obtained by network training of deep learning.
The image quality enhancement processing can improve the resolution of the first picture content, and make the texture of the first picture content clearer and the sharpness higher, thereby improving the image quality of the main picture.
Step 104, superposing a second picture content on the first picture content with enhanced picture quality to obtain a target image frame;
in the embodiment of the application, the second picture content is obtained through common rendering and is not subjected to image quality enhancement processing.
Step 106, displaying the target image frame.
In the embodiment of the application, the second picture content is overlapped on the first picture with enhanced picture quality, so that the final target image frame comprising the complete picture content is obtained and displayed.
After the first picture content is rendered, before the second picture content is superimposed, the first picture content is subjected to image quality enhancement processing, and then the second picture content is superimposed on the first picture content with the enhanced image quality, so that the image quality enhancement processing is only effective on the main picture content, and does not act on the second picture content, and the second picture content is prevented from becoming blurred under the condition of enhancing the first picture content.
In some embodiments of the present application, performing image quality enhancement processing on first picture content includes:
The first picture content is image quality enhanced in an image quality enhanced rendering pipeline.
In the embodiment of the application, the pictures of the program are rendered by a rendering pipeline. The rendering pipeline mainly comprises stages of input assembly, vertex processing, geometry processing, illumination processing, pixel processing and the like, wherein each stage can be subjected to customization processing through a shader.
Wherein the first and second picture content may be rendered by different rendering pipelines. And adding an image quality enhancement rendering pipeline between the rendering pipeline of the first picture content and the rendering pipeline of the second picture content, and performing image quality enhancement processing on the rendered first picture content through the image quality enhancement rendering pipeline.
Specifically, taking a game screen as an example, fig. 2 shows a schematic diagram of a screen rendering flow according to some embodiments of the present application, as shown in fig. 2, in a rendering pipeline of the game screen, the game screen is generated by processing a game program together with a GPU (Graphics Processing Unit, graphics processor) through a CPU (Central Processing Unit), wherein the CPU is responsible for processing drawing instructions and drawing data of the game screen, and transmitting the drawing instructions and drawing data to the GPU for drawing an image. The GPU manages drawing of game pictures through a rendering pipeline.
In the related art, after the final game picture is rendered, the image quality enhancement processing is performed on the image frame before the screen is displayed by the CPU, so that the characters, the icons and the like are amplified proportionally, and the characters and the icons are blurred.
According to the application, the image quality enhancement rendering pipeline is added on the GPU, and the image quality enhancement task of the first picture content is processed through the image quality enhancement rendering pipeline, so that the task which is originally processed through the CPU can be transferred to be processed through the GPU, the operation pressure of the CPU can be effectively reduced, and the performance of the GPU is fully utilized.
In some embodiments of the present application, before performing the image enhancement processing on the first picture content in the image enhancement rendering pipeline, the rendering method further includes:
determining a first shader program; wherein the first shader program is for rendering at least part of the first screen content or for rendering at least part of the second screen content;
performing image quality enhancement processing on the first picture content in an image quality enhancement rendering pipeline, comprising:
according to the running state of the first shader program, a second shader program in the image quality enhancement rendering pipeline is run to perform image quality enhancement processing on the first picture content by the second shader program.
In the embodiment of the present application, a Shader, that is, a loader, is a program implemented using a shading language, and the loader runs on a GPU for fine control of graphics rendering. Depending on where in the rendering pipeline, the shaders mainly include Vertex shaders (Vertex shaders) and pixel shaders (Fragment shaders).
Vertex shaders are typically used to process vertex information, such as position, color, and map coordinates, for a 3D model, and perform coordinate conversion. The pixel shader is used to calculate and configure the color and transparency of each pixel.
The Shader Program, the loader Program, is used to manage the Shader loader. The shader program is typically a complete set of rendering flows, for example, in a rendering pipeline that renders a main picture of a game, the drawing of a game character is performed by the shader program responsible for character drawing.
The first shader is a selected general-purpose shader, and is used as a matching target shader when rendering a game screen, and the first shader program is a shader program for managing the first shader. After positioning the first shader program, a second shader program for image quality enhancement processing is inserted at the first shader program.
Wherein the second shader program is a shader program in a picture quality enhancement rendering pipeline. The first shader program may be the last shader program to render the main picture of the game picture or the first shader program to render the user interface picture.
Taking a rendered game picture as an example, the game picture specifically comprises a main picture and a user interface picture, wherein the main picture specifically comprises scene pictures of a game, such as characters, buildings, vegetation and the like, and the user interface picture specifically comprises buttons, navigation maps, characters and the like for providing a user man-machine interaction function.
The first screen content specifically includes screen content of a main screen, and the second screen content specifically includes screen content of a user interface screen.
The embodiment of the application inserts a second shader program for performing image quality enhancement processing on the first picture content, namely the main picture of the game, between a rendering pipeline for rendering the first picture content and a rendering pipeline for rendering the second picture content, and runs the second shader program to perform image quality enhancement processing on the first picture content of the main picture of the game based on the running state of the first shader program.
Because the first picture content only comprises the picture content of the game main picture and does not comprise the picture content of the user interface picture, the image enhancement processing, such as the pixel number interpolation algorithm added by bilinear interpolation, trilinear interpolation and other interpolation algorithms, or the super resolution processing by the neural network model obtained by the network training of deep learning, can only take effect on the game main picture, and cannot influence the characters, the icon configuration, the buttons and the like in the UI picture.
According to the embodiment of the application, the second shader program is operated according to the operation state of the first shader, the rendered program main picture is enhanced by the second shader program, and the UI picture is overlapped on the program main picture before and after the image quality is enhanced, so that the image quality enhancement process is effective only for the main picture content, the original proportion of the UI picture is maintained, and the user interface picture content such as characters, icons and the like can be kept clearly displayed under the condition of effectively enhancing the program main picture content.
In some embodiments of the application, the picture comprises a plurality of image frames, each of the plurality of image frames comprising a first object in its image content, the first shader program being for rendering the first object.
In the embodiment of the present application, the first shader program includes a first shader, and the first shader is specifically a general-purpose shader, and the general-purpose shader is specifically a shader that is used each time a game picture is drawn, so that the first shader program is also a shader program that is used each time a picture is rendered. For example, fig. 3 shows a schematic diagram of a rendering pipeline of a picture of some embodiments of the present application, and as shown in fig. 3, the rendering pipeline specifically includes a rendering pipeline for rendering a game main picture, a rendering pipeline for image quality enhancement, and a rendering pipeline for rendering a game UI.
The method comprises the steps of capturing a drawing pipeline of a game through a frame capturing tool, analyzing a Shader program in the drawing process of the game, finding a Shader program of a main picture of the last drawing game and a Shader program of a first superposition game UI, and finding a loader which is used for drawing the picture of the game each time, namely a first Shader.
As shown in fig. 3, the first shader may be a ShaderA in the game UI drawing pipeline for rendering buttons, where the first object is a button (the button may be a virtual rocker button, an attack button, or a skill release button, etc.). The first Shader may also be a loader 6 in the game master rendering pipeline for rendering vegetation, where the first object is vegetation.
According to the embodiment of the application, the shaders which are used when the game picture is rendered each time are marked as the universal shaders (namely the first Shader), so that the Shader program of the last drawn game main picture or the Shader program of the first overlapped user interface picture can be accurately identified, the time for running the picture quality enhancement Shader program, namely the second Shader program, is accurately identified, the picture quality enhancement is ensured to be only acted on the game main picture and not acted on the UI picture, the original proportion of the UI picture is ensured not to be changed, and characters, icons and the like in the UI picture can be clearly displayed under the condition of effectively enhancing the game main picture.
In some embodiments of the present application, the first shader program is configured to render at least part of the first screen content, where the rendering pipeline corresponding to the first screen content is a first rendering pipeline, and the first rendering pipeline includes at least two third shader programs, where the at least two third shader programs are configured to render at least part of the first screen content in a first rendering order;
determining a first shader program, comprising:
determining a last shader program of the at least two third shader programs ordered in the first rendering order as a first shader program;
running a second shader program in the image quality enhancement rendering pipeline according to the running state of the first shader program, comprising:
after the first shader program is run, a second shader program in the image quality enhancement rendering pipeline is run.
In the embodiment of the present application, taking a picture as a game picture as an example, as shown in fig. 3, the first rendering pipeline in this embodiment is a game main picture rendering pipeline shown in fig. 3, that is, a rendering pipeline for rendering the first picture content, where the first rendering pipeline includes at least two third Shader programs, and characters (loader Program), buildings (loader Program) and vegetation (loader Program) shown in fig. 3 all belong to the third Shader programs.
And according to the first rendering sequence written by the game, the game main picture rendering pipeline, namely the first rendering pipeline, is used for controlling the three third shader programs to render the game main picture according to the rendering sequence of the first character, the building and the last vegetation.
It will be appreciated that the characters, buildings and vegetation are all part of the first visual content.
The first Shader Program is specifically the last Shader Program to render the main game screen, i.e., the last Shader Program in the first rendering order, and takes the example shown in fig. 3 as an example, the first Shader Program is vegetation (loader Program).
Taking the first Shader as the example of the loader 6 shown in fig. 3, after marking the loader 6, each time a loader Program runs, it is matched whether the loader in the currently running loader Program is consistent with the loader 6, and if so, the current loader Program is marked as the first Shader Program.
After marking the first shader program, each time the first shader program is finished, before the game UI rendering pipeline works, the second shader program is operated, and the image quality of the first picture content of the game main picture is enhanced through the second shader program.
The second Shader program may include a plurality of shaders (loader_1 and loader_2 shown in fig. 3) for implementing image quality enhancement, and the image quality enhancement may use super-resolution technology or sharpening technology, so as to improve the definition and image quality of the main game screen.
Illustratively, the super-resolution technique refers to an image processing technique that generates a high-resolution image by processing and optimizing a low-resolution image.
Illustratively, the sharpening technique is an image processing technique that can enhance the sharpness and quality of an image by highlighting the edges and details of the image, making the image more vivid, real and realistic. This technique generally increases the contrast and sharpness of the image and reduces blurring and distortion of the image, thereby making the image appear more sharp and vivid.
According to the embodiment of the application, the first shader program for rendering the main game picture is marked, and after the first shader program is operated, the second shader program for enhancing the picture quality is operated, so that the user interface picture of the game can not be influenced under the condition of improving the picture quality of the main game picture, and the blurring of characters and icons is prevented. Meanwhile, the image quality enhancement is completed in the rendering pipeline, so that GPU resources can be more effectively and fully utilized, the load and pressure of a CPU are reduced, and the game performance is improved.
In some embodiments of the present application, the first shader program is configured to render at least part of the second screen content, the rendering pipeline corresponding to the second screen content is a second rendering pipeline, the second rendering pipeline includes at least two fourth shader programs, and the at least two fourth shader programs are configured to render at least part of the second screen content in a second rendering order;
determining a first shader program, comprising:
determining a first shader program of the at least two fourth shader programs ordered in the second rendering order as the first shader program;
running a second shader program in the image quality enhancement rendering pipeline according to the running state of the first shader program, comprising:
a second shader program in the image quality enhancement rendering pipeline is run before the first shader program is run.
In the embodiment of the present application, taking a picture as a game picture as an example, as shown in fig. 3, the second rendering pipeline in the present embodiment is a game UI rendering pipeline shown in fig. 3, where the second rendering pipeline includes at least two fourth Shader programs, and buttons (loader Program), characters (loader Program) and a navigation map (loader Program) shown in fig. 3 all belong to the fourth Shader programs.
And according to the second rendering sequence written by the game, the game UI picture rendering pipeline, namely the second rendering pipeline, controls the three fourth shader programs to render the game UI pictures according to the rendering sequence of the first buttons, the second characters and the final navigation map.
It can be appreciated that buttons, text, and navigational maps are all part of the second screen content.
The first Shader Program is specifically a first Shader Program that renders the game UI screen, that is, the first Shader Program in the second rendering order, and the example shown in fig. 3 is exemplified as a button (loader Program).
Taking the first Shader as the example of the loader a shown in fig. 3, after marking the loader a, each time a loader Program runs, it is matched whether the loader in the currently running loader Program is consistent with the loader a, and if so, the current loader Program is marked as the first Shader Program.
After marking the first shader program, the second shader program is operated each time before the first shader program is operated, namely after the operation of the game main picture rendering pipeline is completed, and the image quality of the first picture content of the game main picture is enhanced by the second shader program.
The second Shader program may include a plurality of shaders (loader_1 and loader_2 shown in fig. 3) for implementing image quality enhancement, and the image quality enhancement may use super-resolution technology or sharpening technology, so as to improve the definition and image quality of the main game screen.
Illustratively, the super-resolution technique refers to an image processing technique that generates a high-resolution image by processing and optimizing a low-resolution image.
Illustratively, the sharpening technique is an image processing technique that can enhance the sharpness and quality of an image by highlighting the edges and details of the image, making the image more vivid, real and realistic. This technique generally increases the contrast and sharpness of the image and reduces blurring and distortion of the image, thereby making the image appear more sharp and vivid.
The embodiment of the application can not influence the user interface picture of the game under the condition of improving the picture quality of the main picture of the game by marking the first shader program for rendering the UI picture of the game and running the second shader program for enhancing the picture quality just before running the first shader program, prevent words and icons from being blurred, and simultaneously can more effectively and fully utilize GPU resources by completing picture quality enhancement in a rendering pipeline, thereby reducing the load and pressure of a CPU and improving the game performance.
According to the game picture rendering method provided by the embodiment of the application, the execution main body can be a game picture rendering device. In the embodiment of the present application, a method for executing the rendering of a game frame by a game frame rendering device is taken as an example, and the game frame rendering device provided in the embodiment of the present application is described.
In some embodiments of the present application, there is provided a picture rendering apparatus, fig. 4 shows a block diagram of a structure of a picture rendering apparatus of some embodiments of the present application, and as shown in fig. 4, a picture rendering apparatus 400 includes:
an image quality enhancement module 402, configured to perform image quality enhancement processing on the first picture content when the first picture content is rendered, so as to obtain the first picture content with enhanced image quality;
a content superimposing module 404, configured to superimpose the second picture content on the first picture content after image quality enhancement, so as to obtain a target image frame;
and a display module 406 for displaying the target image frame.
After the first picture content is rendered, before the second picture content is superimposed, the first picture content is subjected to image quality enhancement processing, and then the second picture content is superimposed on the first picture content with the enhanced image quality, so that the image quality enhancement processing is only effective on the main picture content, and does not act on the second picture content, and the second picture content is prevented from becoming blurred under the condition of enhancing the first picture content.
In some embodiments of the present application, the image quality enhancement module is specifically configured to perform image quality enhancement processing on the first picture content in the image quality enhancement rendering pipeline.
According to the application, the image quality enhancement rendering pipeline is added on the GPU, and the image quality enhancement task of the first picture content is processed through the image quality enhancement rendering pipeline, so that the task which is originally processed through the CPU can be transferred to be processed through the GPU, the operation pressure of the CPU can be effectively reduced, and the performance of the GPU is fully utilized.
In some embodiments of the present application, the rendering apparatus further includes:
a determination module for determining a first shader program; wherein the first shader program is for rendering at least part of the first screen content or for rendering at least part of the second screen content;
and the operation module is used for operating a second shader program in the image quality enhancement rendering pipeline according to the operation state of the first shader program so as to carry out image quality enhancement processing on the first picture content through the second shader program.
According to the embodiment of the application, the second shader program is operated according to the operation state of the first shader, the rendered program main picture is enhanced by the second shader program, and the UI picture is overlapped on the program main picture before and after the image quality is enhanced, so that the image quality enhancement process is effective only for the main picture content, the original proportion of the UI picture is maintained, and the user interface picture content such as characters, icons and the like can be kept clearly displayed under the condition of effectively enhancing the program main picture content.
In some embodiments of the application, the picture comprises a plurality of image frames, each of the plurality of image frames comprising a first object in its image content, the first shader program being for rendering the first object.
According to the embodiment of the application, the shaders which are used when the game picture is rendered each time are marked as the universal shaders (namely the first Shader), so that the Shader program of the last drawn game main picture or the Shader program of the first overlapped user interface picture can be accurately identified, the time for running the picture quality enhancement Shader program, namely the second Shader program, is accurately identified, the picture quality enhancement is ensured to be only acted on the game main picture and not acted on the UI picture, the original proportion of the UI picture is ensured not to be changed, and characters, icons and the like in the UI picture can be clearly displayed under the condition of effectively enhancing the game main picture.
In some embodiments of the present application, the first shader program is configured to render at least part of the first screen content, where the rendering pipeline corresponding to the first screen content is a first rendering pipeline, and the first rendering pipeline includes at least two third shader programs, where the at least two third shader programs are configured to render at least part of the first screen content in a first rendering order;
A determining module, specifically configured to determine a last shader program of the at least two third shader programs ordered according to the first rendering order as a first shader program;
the operation module is specifically configured to operate a second shader program in the image quality enhancement rendering pipeline after the first shader program is operated.
According to the embodiment of the application, the first shader program for rendering the main game picture is marked, and after the first shader program is operated, the second shader program for enhancing the picture quality is operated, so that the user interface picture of the game can not be influenced under the condition of improving the picture quality of the main game picture, and the blurring of characters and icons is prevented. Meanwhile, the image quality enhancement is completed in the rendering pipeline, so that GPU resources can be more effectively and fully utilized, the load and pressure of a CPU are reduced, and the game performance is improved.
In some embodiments of the present application, the first shader program is configured to render at least part of the second screen content, the rendering pipeline corresponding to the second screen content is a second rendering pipeline, the second rendering pipeline includes at least two fourth shader programs, and the at least two fourth shader programs are configured to render at least part of the second screen content in a second rendering order;
A determining module, specifically configured to determine a first shader program of the at least two fourth shader programs ordered according to the second rendering order as a first shader program;
the execution module is specifically configured to execute a second shader program in the image quality enhancement rendering pipeline before the first shader program is executed.
The embodiment of the application can not influence the user interface picture of the game under the condition of improving the picture quality of the main picture of the game by marking the first shader program for rendering the UI picture of the game and running the second shader program for enhancing the picture quality just before running the first shader program, can more effectively and fully utilize GPU resources by completing the picture quality enhancement in the rendering pipeline while preventing the blurring of UI characters and icons, thereby reducing the load and pressure of a CPU and improving the game performance.
The rendering device of the picture in the embodiment of the application can be an electronic device or a component in the electronic device, such as an integrated circuit or a chip. The electronic device may be a terminal, or may be other devices than a terminal. By way of example, the electronic device may be a mobile phone, tablet computer, notebook computer, palm computer, vehicle-mounted electronic device, mobile internet appliance (Mobile Internet Device, MID), augmented reality (augmented reality, AR)/Virtual Reality (VR) device, robot, wearable device, ultra-mobile personal computer, UMPC, netbook or personal digital assistant (personal digital assistant, PDA), etc., but may also be a server, network attached storage (Network Attached Storage, NAS), personal computer (personal computer, PC), television (TV), teller machine or self-service machine, etc., and the embodiments of the present application are not limited in particular.
The rendering device of the picture in the embodiment of the application can be a device with an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, and the embodiment of the present application is not limited specifically.
The device for rendering the picture provided by the embodiment of the application can realize each process realized by the embodiment of the method and can achieve the same technical effect, and in order to avoid repetition, the description is omitted.
Optionally, an embodiment of the present application further provides an electronic device, fig. 5 shows a block diagram of an electronic device according to an embodiment of the present application, as shown in fig. 5, an electronic device 500 includes a processor 502, a memory 504, and a program or an instruction stored in the memory 504 and capable of running on the processor 502, where the program or the instruction implements each process of the foregoing method embodiment when executed by the processor 502, and the same technical effects are achieved, and are not repeated herein.
The electronic device in the embodiment of the application includes the mobile electronic device and the non-mobile electronic device.
Fig. 6 is a schematic diagram of a hardware structure of an electronic device implementing an embodiment of the present application.
The electronic device 600 includes, but is not limited to: radio frequency unit 601, network module 602, audio output unit 603, input unit 604, sensor 605, display unit 606, user input unit 607, interface unit 608, memory 609, and processor 610.
Those skilled in the art will appreciate that the electronic device 600 may further include a power source (e.g., a battery) for powering the various components, which may be logically connected to the processor 610 by a power management system to perform functions such as managing charge, discharge, and power consumption by the power management system. The electronic device structure shown in fig. 6 does not constitute a limitation of the electronic device, and the electronic device may include more or less components than shown, or may combine certain components, or may be arranged in different components, which are not described in detail herein.
The processor 610 is configured to perform image quality enhancement processing on the first image content to obtain the first image content with enhanced image quality when the first image content is rendered; superposing second picture content on the first picture content with enhanced picture quality to obtain a target image frame;
and a display unit 606 for displaying the target image frame.
After the first picture content is rendered, before the second picture content is superimposed, the first picture content is subjected to image quality enhancement processing, and then the second picture content is superimposed on the first picture content with the enhanced image quality, so that the image quality enhancement processing is only effective on the main picture content, and does not act on the second picture content, and the second picture content is prevented from becoming blurred under the condition of enhancing the first picture content.
Optionally, the processor 610 is further configured to perform image quality enhancement processing on the first picture content in the image quality enhancement rendering pipeline.
According to the application, the image quality enhancement rendering pipeline is added on the GPU, and the image quality enhancement task of the first picture content is processed through the image quality enhancement rendering pipeline, so that the task which is originally processed through the CPU can be transferred to be processed through the GPU, the operation pressure of the CPU can be effectively reduced, and the performance of the GPU is fully utilized.
Optionally, the processor 610 is further configured to determine a first shader program; wherein the first shader program is for rendering at least part of the first screen content or for rendering at least part of the second screen content; according to the running state of the first shader program, a second shader program in the image quality enhancement rendering pipeline is run to perform image quality enhancement processing on the first picture content by the second shader program.
According to the embodiment of the application, the second shader program is operated according to the operation state of the first shader, the rendered program main picture is enhanced by the second shader program, and the UI picture is overlapped on the program main picture before and after the image quality is enhanced, so that the image quality enhancement process is effective only for the main picture content, the original proportion of the UI picture is maintained, and the user interface picture content such as characters, icons and the like can be kept clearly displayed under the condition of effectively enhancing the program main picture content.
Optionally, the first shader program is configured to render at least part of the first frame content, the rendering pipeline corresponding to the first frame content is a first rendering pipeline, the first rendering pipeline includes at least two third shader programs, and the at least two third shader programs are configured to render at least part of the first frame content in a first rendering order;
the processor 610 is further configured to determine a last shader program of the at least two third shader programs ordered in the first rendering order as a first shader program; after the first shader program is run, a second shader program in the image quality enhancement rendering pipeline is run.
According to the embodiment of the application, the first shader program for rendering the main game picture is marked, and after the first shader program is operated, the second shader program for enhancing the picture quality is operated, so that the user interface picture of the game can not be influenced under the condition of improving the picture quality of the main game picture, and the blurring of UI characters and icons is prevented. Meanwhile, the image quality enhancement is completed in the rendering pipeline, so that GPU resources can be more effectively and fully utilized, the load and pressure of a CPU are reduced, and the game performance is improved.
Optionally, the first shader program is configured to render at least part of the second frame content, the rendering pipeline corresponding to the second frame content is a second rendering pipeline, and the second rendering pipeline includes at least two fourth shader programs, and the at least two fourth shader programs are configured to render at least part of the second frame content in a second rendering order;
the processor 610 is further configured to determine a first shader program of the at least two fourth shader programs ordered in the second rendering order as the first shader program; a second shader program in the image quality enhancement rendering pipeline is run before the first shader program is run.
The embodiment of the application can not influence the user interface picture of the game under the condition of improving the picture quality of the main picture of the game by marking the first shader program for rendering the UI picture of the game and running the second shader program for enhancing the picture quality just before running the first shader program, can more effectively and fully utilize GPU resources by completing the picture quality enhancement in the rendering pipeline while preventing the blurring of UI characters and icons, thereby reducing the load and pressure of a CPU and improving the game performance.
It should be understood that in an embodiment of the present application, the input unit 604 may include a graphics processor (Graphics Processing Unit, GPU) 6041 and a microphone 6042, and the graphics processor 6041 processes image data of still pictures or video obtained by an image capturing apparatus (e.g., a camera) in a video capturing mode or an image capturing mode. The display unit 606 may include a display panel 6061, and the display panel 6061 may be configured in the form of a liquid crystal display, an organic light emitting diode, or the like. The user input unit 607 includes at least one of a touch panel 6071 and other input devices 6072. The touch panel 6071 is also called a touch screen. The touch panel 6071 may include two parts of a touch detection device and a touch controller. Other input devices 6072 may include, but are not limited to, a physical keyboard, function keys (e.g., volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and so forth, which are not described in detail herein.
The memory 609 may be used to store software programs as well as various data. The memory 609 may mainly include a first storage area storing programs or instructions and a second storage area storing data, wherein the first storage area may store an operating system, application programs or instructions (such as a sound playing function, an image playing function, etc.) required for at least one function, and the like. Further, the memory 609 may include volatile memory or nonvolatile memory, or the memory 609 may include both volatile and nonvolatile memory. The nonvolatile Memory may be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable EPROM (EEPROM), or a flash Memory. The volatile memory may be random access memory (Random Access Memory, RAM), static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (ddr SDRAM), enhanced SDRAM (Enhanced SDRAM), synchronous DRAM (SLDRAM), and Direct RAM (DRRAM). Memory 609 in embodiments of the present application includes, but is not limited to, these and any other suitable types of memory.
The processor 610 may include one or more processing units; optionally, the processor 610 integrates an application processor that primarily processes operations involving an operating system, user interface, application programs, etc., and a modem processor that primarily processes wireless communication signals, such as a baseband processor. It will be appreciated that the modem processor described above may not be integrated into the processor 610.
The embodiment of the application also provides a readable storage medium, and the readable storage medium stores a program or an instruction, which when executed by a processor, implements each process of the above method embodiment, and can achieve the same technical effects, so that repetition is avoided, and no further description is provided herein.
The processor is a processor in the electronic device in the above embodiment. Readable storage media include computer readable storage media such as Read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), magnetic or optical disks, and the like.
The embodiment of the application further provides a chip, the chip comprises a processor and a communication interface, the communication interface is coupled with the processor, the processor is used for running programs or instructions, the processes of the embodiment of the method can be realized, the same technical effects can be achieved, and the repetition is avoided, and the description is omitted here.
It should be understood that the chips referred to in the embodiments of the present application may also be referred to as system-on-chip chips, chip systems, or system-on-chip chips, etc.
Embodiments of the present application provide a computer program product stored in a storage medium, where the program product is executed by at least one processor to implement the respective processes of the above method embodiments, and achieve the same technical effects, and for avoiding repetition, a detailed description is omitted herein.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element. Furthermore, it should be noted that the scope of the methods and apparatus in the embodiments of the present application is not limited to performing the functions in the order shown or discussed, but may also include performing the functions in a substantially simultaneous manner or in an opposite order depending on the functions involved, e.g., the described methods may be performed in an order different from that described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present application may be embodied essentially or in part in the form of a computer software product stored on a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) comprising instructions for causing a terminal (which may be a mobile phone, a computer, a server, or a network device, etc.) to perform the method of the embodiments of the present application.
The embodiments of the present application have been described above with reference to the accompanying drawings, but the present application is not limited to the above-described embodiments, which are merely illustrative and not restrictive, and many forms may be made by those having ordinary skill in the art without departing from the spirit of the present application and the scope of the claims, which are to be protected by the present application.

Claims (12)

1. A method of rendering a picture, the method comprising:
under the condition that the rendering of the first picture content is completed, carrying out image quality enhancement processing on the first picture content to obtain the first picture content with enhanced image quality;
superposing second picture content on the first picture content with the enhanced picture quality to obtain a target image frame;
the target image frame is displayed.
2. The rendering method according to claim 1, wherein said performing image quality enhancement processing on said first picture content comprises:
and performing image quality enhancement processing on the first picture content in an image quality enhancement rendering pipeline.
3. The rendering method according to claim 2, wherein before the image quality enhancement processing is performed on the first picture content in the image quality enhancement rendering pipeline, the rendering method further comprises:
determining a first shader program; wherein the first shader program is for rendering at least part of the first screen content or for rendering at least part of the second screen content;
the performing image enhancement processing on the first picture content in an image enhancement rendering pipeline includes:
And running a second shader program in the image quality enhancement rendering pipeline according to the running state of the first shader program so as to perform image quality enhancement processing on the first picture content through the second shader program.
4. A rendering method according to claim 3, wherein the first shader program is configured to render at least part of the first screen content, the rendering pipeline corresponding to the first screen content is a first rendering pipeline, the first rendering pipeline includes at least two third shader programs configured to render at least part of the first screen content in a first rendering order;
the determining a first shader program includes:
determining a last shader program of the at least two third shader programs ordered in the first rendering order as the first shader program;
the running a second shader program in the image quality enhancement rendering pipeline according to the running state of the first shader program includes:
after the first shader program is run, a second shader program in the image quality enhancement rendering pipeline is run.
5. A rendering method according to claim 3, wherein the first shader program is configured to render at least part of the second picture content, the rendering pipeline corresponding to the second picture content is a second rendering pipeline, the second rendering pipeline comprises at least two fourth shader programs, and the at least two fourth shader programs are configured to render at least part of the second picture content in a second rendering order;
the determining a first shader program includes:
determining a first shader program of the at least two fourth shader programs ordered in the second rendering order as the first shader program;
the running a second shader program in the image quality enhancement rendering pipeline according to the running state of the first shader program includes:
a second shader program in the image quality enhancement rendering pipeline is run before the first shader program is run.
6. A rendering apparatus of a picture, the rendering apparatus comprising:
the image quality enhancement module is used for carrying out image quality enhancement processing on the first picture content under the condition that the first picture content is completely rendered, so as to obtain the first picture content with enhanced image quality;
The content superposition module is used for superposing second picture content on the first picture content with the enhanced picture quality to obtain a target image frame;
and the display module is used for displaying the target image frame.
7. The rendering device of claim 6, wherein the image quality enhancement module is configured to perform image quality enhancement processing on the first picture content in an image quality enhancement rendering pipeline.
8. The rendering apparatus of claim 7, further comprising:
a determination module for determining a first shader program; wherein the first shader program is for rendering at least part of the first screen content or for rendering at least part of the second screen content;
and the operation module is used for operating a second shader program in the image quality enhancement rendering pipeline according to the operation state of the first shader program so as to carry out image quality enhancement processing on the first image content through the second shader program.
9. The rendering device of claim 8, wherein the first shader program is configured to render at least a portion of the first visual content, the rendering pipeline to which the first visual content corresponds is a first rendering pipeline, the first rendering pipeline including at least two third shader programs configured to render at least a portion of the first visual content in a first rendering order;
The determining module is specifically configured to determine a last shader program of the at least two third shader programs ordered according to the first rendering order as the first shader program;
the operation module is specifically configured to operate a second shader program in the image quality enhancement rendering pipeline after the first shader program is operated.
10. The rendering device of claim 8, wherein the first shader program is configured to render at least a portion of the second visual content, the rendering pipeline corresponding to the second visual content is a second rendering pipeline, the second rendering pipeline includes at least two fourth shader programs configured to render at least a portion of the second visual content in a second rendering order;
the determining module is specifically configured to determine a first shader program of the at least two fourth shader programs ordered according to the second rendering order as the first shader program;
the operation module is specifically configured to operate a second shader program in the image quality enhancement rendering pipeline before the first shader program is operated.
11. An electronic device comprising a processor and a memory storing a program or instructions executable on the processor, which when executed by the processor, implement the steps of the method of any one of claims 1 to 5.
12. A readable storage medium, characterized in that it stores thereon a program or instructions which, when executed by a processor, implement the steps of the method according to any of claims 1 to 5.
CN202310929710.1A 2023-07-26 2023-07-26 Picture rendering method and device, electronic equipment and storage medium Pending CN116912093A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310929710.1A CN116912093A (en) 2023-07-26 2023-07-26 Picture rendering method and device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310929710.1A CN116912093A (en) 2023-07-26 2023-07-26 Picture rendering method and device, electronic equipment and storage medium

Publications (1)

Publication Number Publication Date
CN116912093A true CN116912093A (en) 2023-10-20

Family

ID=88362800

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310929710.1A Pending CN116912093A (en) 2023-07-26 2023-07-26 Picture rendering method and device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN116912093A (en)

Similar Documents

Publication Publication Date Title
US10986330B2 (en) Method and system for 360 degree head-mounted display monitoring between software program modules using video or image texture sharing
EP2245598B1 (en) Multi-buffer support for off-screen surfaces in a graphics processing system
US9275493B2 (en) Rendering vector maps in a geographic information system
CN103810728A (en) System, method, and computer program product for inputting modified coverage data into a pixel shader
KR20100004119A (en) Post-render graphics overlays
CN104038807A (en) Layer mixing method and device based on open graphics library (OpenGL)
US8952981B2 (en) Subpixel compositing on transparent backgrounds
CN113126862B (en) Screen capture method and device, electronic equipment and readable storage medium
CN111754607A (en) Picture processing method and device, electronic equipment and computer readable storage medium
US9524573B2 (en) Systems, methods, and computer-readable media for manipulating and mapping tiles of graphical object data
CN112541960A (en) Three-dimensional scene rendering method and device and electronic equipment
CN111258693B (en) Remote display method and device
CN107491289B (en) Window rendering method and device
CN109065001B (en) Image down-sampling method and device, terminal equipment and medium
CN109598672B (en) Map road rendering method and device
CN105719335A (en) Map image rendering method and device, and vehicle-mounted terminal
CN111158840B (en) Image carousel method and device
CN111107264A (en) Image processing method, image processing device, storage medium and terminal
CN116912093A (en) Picture rendering method and device, electronic equipment and storage medium
CN113192173B (en) Image processing method and device of three-dimensional scene and electronic equipment
CN111054072B (en) Method, device, equipment and storage medium for role model tailing
CN112001995B (en) Rendering apparatus, method, electronic device, and readable storage medium
CN114998504A (en) Two-dimensional image illumination rendering method, device and system and electronic device
CN114972466A (en) Image processing method, image processing device, electronic equipment and readable storage medium
CN110941389A (en) Method and device for triggering AR information points by focus

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination