WO2023115408A1 - 图像处理装置和方法 - Google Patents

图像处理装置和方法 Download PDF

Info

Publication number
WO2023115408A1
WO2023115408A1 PCT/CN2021/140588 CN2021140588W WO2023115408A1 WO 2023115408 A1 WO2023115408 A1 WO 2023115408A1 CN 2021140588 W CN2021140588 W CN 2021140588W WO 2023115408 A1 WO2023115408 A1 WO 2023115408A1
Authority
WO
WIPO (PCT)
Prior art keywords
depth
rendered
pixel block
depth value
target
Prior art date
Application number
PCT/CN2021/140588
Other languages
English (en)
French (fr)
Inventor
蒋垚
钱伟锋
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to PCT/CN2021/140588 priority Critical patent/WO2023115408A1/zh
Priority to CN202180096777.0A priority patent/CN117203661A/zh
Publication of WO2023115408A1 publication Critical patent/WO2023115408A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/20—Processor architectures; Processor configuration, e.g. pipelining

Definitions

  • the present application relates to the field of computer technology, in particular to an image processing device and method.
  • Rendering is the process of two-dimensional projection of the model in the three-dimensional scene according to the set environment, lighting, material and rendering parameters to obtain a digital image.
  • the graphics processing unit GPU
  • the GPU can only remove a small amount of data to be rendered that does not affect the rendering result in the rendering stage, and cannot achieve a high removal rate.
  • the present application provides an image processing device and method, which can increase the rejection rate of data to be rendered, thereby reducing the consumption of computing resources and increasing the rendering speed.
  • the present application provides an image processing device, which includes a rendering pipeline, where a first rasterizer and a first depth testing module are arranged in the rendering pipeline.
  • the first rasterizer is used to convert the primitives input into the rendering pipeline into fragments to be rendered.
  • the first depth test module is configured to perform a first depth test on the segment to be rendered according to global depth information, and if the segment to be rendered fails the first depth test, reject the segment to be rendered, and the global
  • the depth information includes depth information of a plurality of pixel blocks updated according to all primitives to be rendered.
  • the image processing device provided by the present application can eliminate fragments to be rendered through the global depth information. Since the global depth information is determined according to all primitives to be rendered, and the fragments to be rendered are converted from primitives to be rendered, so Global depth information is equivalent to being generated based on all fragments to be rendered.
  • the embodiment of the present application uses the depth information of all the fragments to be rendered to perform fragment culling. Compared with related technologies that only use the depth information of some of the fragments to be rendered to cull the fragments, the method provided by the present application can increase the culling rate of the data to be rendered, thereby reducing the Calculate resource consumption to increase GPU rendering speed.
  • the first depth test module is specifically configured to: perform the first depth test on the segment to be rendered according to the depth information of the target pixel block in the global information and the depth information of the segment to be rendered.
  • the target pixel block is a pixel block among the plurality of pixel blocks that overlaps with the segment to be rendered.
  • the first depth testing module is specifically configured to: determine that the segment to be rendered fails to pass the first depth test.
  • the fragment to be rendered will be rendered on the target pixel block.
  • the depth information of the target pixel block in the global information and the depth information of the fragment to be rendered it can be judged that the fragment to be rendered will be visible on the target pixel block when the depth value of the fragment to be rendered is greater than the depth value of the target pixel block blocked by the fragment.
  • a pixel block (such as a target pixel block) may include a target area.
  • the depth information of the pixel block may include the maximum depth value of the target area of the pixel block, the maximum depth value of the non-target area of the pixel block, and the range of the target area of the pixel block.
  • the depth information of a fragment may include a depth value of the fragment.
  • the depth value of the fragment may be obtained through interpolation.
  • the target pixel block includes a target area
  • the depth value of the target pixel block includes a first depth value and a second depth value
  • the first depth test module is specifically used for:
  • the fragment to be rendered passes the first depth test.
  • the fragment to be rendered fails the first depth test.
  • the segment to be rendered passes the first depth test.
  • the segment to be rendered fails the first depth test.
  • the segment to be rendered is located in the target area of the target pixel block and the depth value of the segment to be rendered is greater than the first depth value, it means that the segment to be rendered will be blocked by visible segments in the target area of the target pixel block, so it can The segment to be rendered is eliminated to reduce the consumption of computing resources and increase the rendering speed of the GPU.
  • segment to be rendered is located in the target area of the non-target pixel block and the depth value of the segment to be rendered is greater than the second depth value, it means that the segment to be rendered will be blocked by visible segments in the non-target area of the target pixel block, so the segment to be rendered can be Render fragment culling to reduce computing resource consumption and improve GPU rendering speed.
  • the target pixel block is divided into the target area and the non-target area and the maximum depth value of the two areas is used to treat the rendering fragment Depth testing can increase the accuracy of depth testing and reduce the probability of false rejection.
  • the apparatus may further include a first caching module, where the first caching module is configured to cache the global depth information.
  • the device may further include a partition pipeline, where a second rasterizer and a second depth test module are disposed in the partition pipeline.
  • the second rasterizer is configured to project the graphics primitives to be rendered input into the partition pipeline to multiple pixel blocks to generate multiple projection areas.
  • the second depth module is configured to perform a second depth test on the graphics element to be rendered according to the plurality of projection areas, and if the graphics element to be rendered fails the second depth test, reject the image to be rendered Yuan.
  • the plurality of projection areas correspond to the plurality of pixel blocks one by one.
  • the device provided by this application can not only eliminate unrendered fragments to be rendered, but also unrequired unrendered primitives to be rendered.
  • the primitives to be rendered are eliminated, thereby further improving the elimination rate of the data to be rendered, further reducing the consumption of computing resources, and further improving the rendering speed of the GPU.
  • the second depth module is specifically configured to: determine whether the primitive to be rendered passes through the first pixel block according to the depth information of the first pixel block and the depth information of the first projection area.
  • the first pixel block is any pixel block among the plurality of pixel blocks
  • the first projection area is a projection area corresponding to the first pixel block. If the primitive to be rendered passes the depth test in the first pixel block, then determine that the primitive to be rendered passes the second depth test; if the primitive to be rendered passes the depth test in the plurality of pixel blocks If none of them pass the depth test, it is determined that the graphics element to be rendered fails the second depth test.
  • the second depth module is specifically configured to: if the minimum depth value of the first projection area is smaller than the depth value of the first pixel block, determine that the pixel to be rendered is Passing the depth test in the first pixel block.
  • the depth information of the projection area may include a maximum depth value of the projection area, a minimum depth value of the projection area, and a range of the projection area.
  • the first pixel block includes a target area
  • the depth value of the first pixel block includes a maximum depth value of the target area of the first pixel block and a maximum depth value of the first pixel block.
  • the maximum depth value of the non-target area, the second depth module is specifically used for:
  • the first projected area overlaps both the target area and the non-target area of the first pixel block and the minimum depth value of the first projected area is smaller than the first depth value or the second depth value, then it is determined that the primitive to be rendered is at the first pixel The block passes the test.
  • the first projection area overlaps both the target area and the non-target area of the first pixel block and the minimum depth value of the first projection area is greater than the first depth value and the second depth value, determine that the primitive to be rendered is Failed test in one pixel block
  • the first projection area only overlaps with the target area of the first pixel block and the minimum depth value of the first projection area is smaller than the first depth value, it is determined that the primitive to be rendered passes the test in the first pixel block.
  • the first projection area only overlaps with the target area of the first pixel block and the minimum depth value of the first projection area is greater than the first depth value, it is determined that the primitive to be rendered fails the test in the first pixel block.
  • the first projection area only overlaps with the non-target area of the first pixel block and the minimum depth value of the first projection area is smaller than the second depth value, it is determined that the primitive to be rendered passes the test in the first pixel block.
  • the first projection area only overlaps with the non-target area of the first pixel block and the minimum depth value of the first projection area is greater than the second depth value, it is determined that the primitive to be rendered fails the test in the first pixel block.
  • the second depth module is further configured to update the global depth information according to the multiple projection areas.
  • the second depth module is further specifically configured to: update the global depth information of the first pixel block in the global depth information according to the depth information of the first pixel block and the depth information of the first projection area.
  • the first pixel block is any pixel block in the plurality of pixel blocks
  • the first projection area is a projection area corresponding to the first pixel block.
  • the first pixel block includes a target area
  • the second depth module is further specifically configured to: according to the maximum depth value of the first projection area, the target area of the first pixel block The maximum depth value of the area, the maximum depth value of the non-target area of the first pixel block, and update the depth information of the first pixel block.
  • the second depth module is further specifically configured to:
  • the first projected area overlaps both the target area and the non-target area of the first pixel block and the maximum depth value of the first projected area is smaller than the first depth value or the second depth value, update the depth of the first pixel block information.
  • the first pixel block is not updated depth information.
  • the depth information of the first pixel block is updated.
  • the depth information of the first pixel block is not updated.
  • the depth information of the first pixel block is updated.
  • the depth information of the first pixel block is not updated.
  • the second depth module is further specifically configured to:
  • the first projected area completely overlaps with the target area of the first pixel block and the third depth value (that is, the maximum depth value of the projected area) is smaller than the first depth value
  • the target area of the first pixel block is updated to the non-target projected area and the first pixel block
  • the second depth value of is updated to the third depth value.
  • the target area of the first pixel block is updated to the first area and the second
  • the first depth value of a pixel block is updated to a first value.
  • the above-mentioned first condition is that the third depth value is smaller than the first depth value but larger than the second depth value or the third depth value is smaller than the second depth value but larger than the first depth value.
  • the above-mentioned second condition is that the first absolute value is smaller than the second absolute value
  • the above-mentioned first absolute value is the absolute value of the difference between the above-mentioned first depth value and the above-mentioned third depth value
  • the above-mentioned second absolute value is the difference between the above-mentioned second depth value and The absolute value of the difference between the above third depth values.
  • the above-mentioned first area is a union area of the target area and the target projection area
  • the first value is a larger value between the first depth value and the third depth value.
  • the target area of the first pixel block is updated to the second area and the second The second depth value of a pixel block is updated to a second value.
  • the above-mentioned third condition is that the first absolute value is greater than the second absolute value.
  • the above-mentioned second area is the intersection area of the target area and the non-target projection area.
  • the above-mentioned second value is a larger value between the second depth value and the third depth value.
  • the target area of the first pixel block is updated as Target projection area and change the first depth value of the first pixel block to the third depth value, and update the second depth value of the first pixel block to the third value.
  • the third value is a larger value between the first depth value and the second depth value.
  • the target projection area includes a first target projection area (ie CurMask0) and a second target projection area (ie CurMask0)
  • the first target projected area (ie CurMask0 ) may indicate the overlapping area of the projected area and the non-target area of the pixel block.
  • the second target projected area (ie CurMask0 ) may indicate the overlapping area of the projected area and the target area of the pixel block.
  • the target projection area (CurMask) may be a union of the first target projection area (CurMask0) and the second target projection area (CurMask0).
  • the rendering pipeline of the image processing device may further be provided with at least one of a pre-depth test module, a pixel shader, a post-depth test module, or a depth buffer module.
  • a front depth test module configured to perform a front depth test on the first target fragment according to the depth value in the depth buffer module.
  • the pixel shader is used to shade the second target fragment.
  • a post-depth test module configured to perform a post-depth test on the third target fragment according to the depth value in the depth buffer module.
  • the depth buffer module is used to cache the depth value.
  • the first target fragment is a fragment that passes a first depth test
  • the second target fragment is a fragment that passes a pre-depth test
  • the third target fragment is a fragment that has been shaded and rendered by a pixel shader.
  • the present application also provides another image processing device, which includes a rendering pipeline, and the rendering pipeline is provided with a rasterizer, a rendering coarse-grained depth test module, a front depth test module, a pixel shader, a depth Cache module and post depth test module.
  • the rasterizer is used to convert the primitives input into the rendering pipeline into fragments to be rendered; the rendering coarse-grained depth test module is used to perform a first depth test on the fragments to be rendered; the front depth The test module is used to perform a front depth test on the first target fragment according to the depth value in the depth buffer module; the pixel shader is used to perform color rendering on the second target fragment; the rear depth test module uses performing a post-depth test on the third target segment according to the depth value in the depth buffer module; the depth buffer module is used to cache the depth value.
  • the first target fragment is a fragment that passes a first depth test
  • the second target fragment is a fragment that passes a pre-depth test
  • the third target fragment is a fragment that has been shaded and rendered by a pixel shader.
  • the rendering pipeline of the image processing device is not only provided with a pre-depth test module and a post-depth test module, but also provided with a rendering coarse-grained depth test module.
  • the rendering coarse-grained depth testing module is specifically configured to: perform the first step on the segment to be rendered according to the depth information of the target pixel block in the global information and the depth information of the segment to be rendered.
  • a depth test the target pixel block is a pixel block that overlaps with the segment to be rendered among the plurality of pixel blocks
  • the global depth information includes the depths of the plurality of pixel blocks updated according to all primitives to be rendered information.
  • the apparatus further includes a coarse-grained depth buffer module, and the coarse-grained depth buffer module is configured to cache the global depth information.
  • the device further includes a partition pipeline, the partition pipeline is provided with a coarse-grained rasterizer and a partition coarse-grained depth test module; the coarse-grained rasterizer is configured to input the The graphics elements to be rendered in the partition pipeline are projected to multiple pixel blocks to generate multiple projection areas, and the multiple projection areas are in one-to-one correspondence with the multiple pixel blocks; the partition coarse-grained depth test module is used to A projection area performs a second depth test on the primitive to be rendered.
  • the partition coarse-grained depth testing module is further configured to: update global depth information according to the multiple projection areas, and the global depth information includes multiple The depth information of the pixel block.
  • the user can enter the scene of the rendering pipeline in the order of fragments in order of depth value from small to large, and turn off the rendering coarse-grained depth testing module in the rendering pipeline to reduce the amount of data processing in the rendering process.
  • scenes that do not involve depth such as planar scenes, 2D scenes, etc.
  • users can close all depth test modules in the rendering pipeline (ie, rendering coarse-grained depth test modules, pre-depth test modules and post-depth test modules) to make the data to be rendered Go directly to the pixel shader for shading rendering to reduce the amount of data processing in the rendering process.
  • the user can turn off the depth test module to reduce the amount of data processing in the rendering process.
  • the depth value of the fragment is affected by the pixel shader (that is, the depth value of the fragment will change before and after rendering by the pixel shader) and the direction of the change of the depth value of the fragment is not fixed (for example, the depth value of some increase after color rendering, and the depth value of another part of the fragment decreases after color rendering), and close the front depth test module to reduce the amount of data processing in the rendering process.
  • the present application also provides an image processing method, the method comprising: firstly converting graphics primitives input into a rendering pipeline into fragments to be rendered. Then perform a first depth test on the segment to be rendered according to the global depth information, and then remove the segment to be rendered if the segment to be rendered fails the first depth test.
  • the global depth information includes depth information of a plurality of pixel blocks updated according to all primitives to be rendered.
  • the global depth information includes depth information of multiple pixel blocks
  • performing the first depth test on the segment to be rendered according to the global depth information includes: according to the depth information of the target pixel block and performing the first depth test on the segment to be rendered with the depth information of the segment to be rendered, and the target pixel block is a pixel block overlapping with the segment to be rendered among the plurality of pixel blocks.
  • the performing the first depth test on the segment to be rendered according to the depth information of the target pixel block and the depth information of the segment to be rendered includes: If the depth value is greater than the depth value of the target pixel block, it is determined that the segment to be rendered fails the first depth test.
  • the target pixel block includes a target area
  • the depth value of the target pixel block includes a first depth value and a second depth value
  • the first depth value is a maximum value of the target area.
  • Depth value, the second depth value is the maximum depth value of the non-target area in the target pixel block.
  • a pixel block (such as a target pixel block) may include a target area.
  • the depth information of the pixel block may include the maximum depth value of the target area of the pixel block, the maximum depth value of the non-target area of the pixel block, and the range of the target area of the pixel block.
  • the depth information of a fragment may include a depth value of the fragment.
  • the depth value of the fragment may be obtained through interpolation.
  • the method further includes: projecting the graphics primitives to be rendered input into the partition pipeline to multiple pixel blocks to generate multiple projection areas, and the multiple projection areas are one-to-one with the multiple pixel blocks Correspondingly: performing a second depth test on the graphics entity to be rendered according to the plurality of projection areas; if the graphics entity to be rendered fails the second depth test, rejecting the graphics entity to be rendered.
  • the performing the second depth test on the primitive to be rendered according to the plurality of projection areas includes: determining according to the depth information of the first pixel block and the depth information of the first projection area Whether the primitive to be rendered passes the depth test in the first pixel block, the first pixel block is any pixel block in the plurality of pixel blocks, and the first projected area is the first The projection area corresponding to the pixel block; if the primitive to be rendered passes the depth test in the first pixel block, it is determined that the primitive to be rendered passes the second depth test; if the primitive to be rendered passes the depth test in the If none of the plurality of pixel blocks passes the depth test, it is determined that the graphics element to be rendered fails the second depth test.
  • the depth information of the projection area may include a maximum depth value of the projection area, a minimum depth value of the projection area, and a range of the projection area.
  • the determining whether the primitive to be rendered passes the depth test in the first pixel block according to the depth information of the first pixel block and the depth information of the first projection area includes: If the minimum depth value of the first projected area is smaller than the depth value of the first pixel block, it is determined that the primitive to be rendered passes the depth test in the first pixel block.
  • the first pixel block includes a target area
  • the depth value of the first pixel block includes a maximum depth value of the target area of the first pixel block and a maximum depth value of the first pixel block. The maximum depth value for non-target areas.
  • the method further includes: updating the global depth information according to the multiple projection areas.
  • the updating the global depth information according to the plurality of projection areas includes: updating the global depth information according to the depth information of the first pixel block and the depth information of the first projection area. Depth information of the first pixel block, where the first pixel block is any pixel block in the plurality of pixel blocks, and the first projection area is a projection area corresponding to the first pixel block.
  • the updating the depth information of the first pixel block in the global depth information according to the depth information of the first pixel block and the depth information of the first projection area includes: according to the first pixel block A maximum depth value of the projection area, a maximum depth value of the target area of the first pixel block, a maximum depth value of the non-target area of the first pixel block, and update the depth information of the first pixel block.
  • the method further includes: performing a pre-depth test on the fragments that pass the first depth test.
  • the method further includes: performing a post-depth test on the fragments that pass the first depth test.
  • the method further includes: performing a pre-depth test and a post-depth test on the fragments that pass the first depth test.
  • the present application also provides a graphics processor, the graphics processor includes the electronic device described in the above aspects or any possible implementation thereof, and the graphics processor is used to implement the above aspects or any possible implementation thereof The method described in the implementation of the .
  • the present application further provides an electronic device, which includes: a memory, at least one processor, a transceiver, and instructions stored in the memory and operable on the processor. Further, the memory, the processor and the communication interface communicate with each other through an internal connection path. Executing the instruction by the at least one processor enables the image processing apparatus to implement the methods described in the above aspects or any possible implementation thereof.
  • the electronic device is a mobile phone.
  • the present application further provides a computer-readable storage medium for storing a computer program, and the computer program includes a method for implementing the above aspects or any possible implementation thereof.
  • the present application further provides a computer program product containing instructions, which when run on a computer, enables the computer to implement the methods described in the above aspects or any possible implementation thereof.
  • the image processing method, image processing device, graphics processor, electronic equipment, computer storage medium, and computer program product provided in this embodiment are all used to execute the image processing method provided above. Therefore, the beneficial effects it can achieve can be Referring to the beneficial effects of the image processing method provided above, details will not be repeated here.
  • Figure 1 is a schematic diagram of the depth provided by the embodiment of the present application.
  • FIG. 2 is a schematic structural diagram of a rendering pipeline
  • FIG. 3 is a schematic structural diagram of an image processing device provided in an embodiment of the present application.
  • FIG. 4 is a schematic flow diagram of an image processing method provided in an embodiment of the present application.
  • FIG. 5 is a schematic flow diagram of another image processing method provided in the embodiment of the present application.
  • FIG. 6 is a schematic diagram of a projection process provided by an embodiment of the present application.
  • FIG. 7 is a schematic diagram of a pixel block provided by an embodiment of the present application.
  • FIG. 8 is a schematic diagram of another pixel block provided by the embodiment of the present application.
  • FIG. 9 is a schematic diagram of a depth value provided by an embodiment of the present application.
  • FIG. 10 is a schematic diagram of another pixel block provided by the embodiment of the present application.
  • FIG. 11 is a schematic diagram of another depth value provided by the embodiment of the present application.
  • FIG. 12 is a schematic diagram of another pixel block provided by the embodiment of the present application.
  • FIG. 13 is a schematic diagram of another pixel block provided by the embodiment of the present application.
  • FIG. 14 is a schematic diagram of another depth value provided by the embodiment of the present application.
  • FIG. 15 is a schematic diagram of another pixel block provided by the embodiment of the present application.
  • Fig. 16 is a schematic diagram of another depth value provided by the embodiment of the present application.
  • Fig. 17 is a schematic diagram of another pixel block provided by the embodiment of the present application.
  • Fig. 18 is a schematic diagram of another depth value provided by the embodiment of the present application.
  • FIG. 19 is a schematic diagram of another pixel block provided by the embodiment of the present application.
  • FIG. 20 is a schematic diagram of a picture including pixel blocks provided by an embodiment of the present application.
  • Fig. 21 is a schematic diagram of another pixel block provided by the embodiment of the present application.
  • FIG. 22 is a schematic structural diagram of another image processing device provided by an embodiment of the present application.
  • first and second in the specification and drawings of the present application are used to distinguish different objects, or to distinguish different processes for the same object, rather than to describe a specific sequence of objects.
  • Depth used to characterize the distance between the object and the camera. As shown in Figure 1, the distance is used to indicate the projection distance obtained by making a line segment according to the camera orientation.
  • Depth test the process of judging whether the depth value of the data to be rendered is greater than the depth threshold.
  • Pre-depth testing Depth testing of data (such as fragments) before the data is rendered without pixel shader shaders.
  • Post Depth Test A depth test performed on the data after it has been shaded and rendered by the pixel shader.
  • Primitive The basic unit of geometric representation in the graphics rendering pipeline.
  • the combination of geometric vertices is called a primitive (such as a point, a line segment, or a polygon, etc.).
  • Rasterization converts primitives into fragments, which hold information about pixels.
  • the rasterization process determines where all the pixels need to be drawn, and the rasterization process interpolates to determine the positions of all pixels between two vertices.
  • Rasterization can not only interpolate pixels, any output variable of the vertex shader and input variable of the fragment shader can be interpolated based on the corresponding pixel. Effects such as smooth color gradients and realistic lighting can be achieved through rasterization.
  • Binning pass Divide the screen into several regions (bins), convert all the primitives that need to be rendered to obtain the screen position of the primitives, and filter the primitives to the corresponding screen according to the screen positions of the primitives Regions, and finally each region (bin) is rendered independently, and only the collection of primitives that fall into this region during the partitioning process is rendered.
  • This screening process is the partitioning phase.
  • the GPU consumes a lot of computing resources during the rendering process. For this reason, the GPU can eliminate the data to be rendered that does not affect the rendering result in the rendering stage, so as to reduce the consumption of computing resources.
  • a depth test may be used to eliminate data to be rendered that does not affect a rendering result.
  • the pre-depth testing module in the rendering pipeline may perform early-detph testing (earlylate-detph- test).
  • early-detph-test early-detph testing
  • the depth value of the fragment is smaller than the depth value in the depth buffer
  • the pixel shader for shading rendering and update the depth value in the depth buffer to the depth value of the fragment
  • the depth value of the fragment is greater than the depth value in the depth buffer, it is determined that the fragment fails the previous depth test and the fragment is eliminated.
  • the depth value of the fragment will be affected by the pixel shader, and the depth test needs to be performed in the pixel shader. In this case, the late-detph-test mode is required for late-detph-test.
  • the depth value in the depth buffer is determined according to the parts to be rendered that have passed the test, that is, the related technology can only use the depth information of some parts to remove the parts to be rendered during the depth test, so it cannot achieve High rejection rate.
  • the fragments to be rendered include fragment 1 to be rendered, fragment 2 to be rendered, fragment 3 to be rendered, fragment 4 to be rendered and fragment 5 to be rendered, and the depth values are 50, 40, 30, 20, and 10 respectively. If the depth values of the 5 fragments to be rendered are used for depth testing at the same time, only the fragment 5 to be rendered with the smallest depth value will pass the depth test, and the remaining 4 fragments to be rendered will be eliminated.
  • the segment 3 to be rendered passes the depth test and enters the shader for shading rendering, and the depth value in the depth buffer Depth value updated to 30.
  • the depth value of the segment 4 to be rendered is 20 less than the depth value of 50 in the depth buffer, and the segment 3 to be rendered passes the depth test and enters the shader for rendering, and the depth value in the depth buffer Depth value updated to 20.
  • the fragment 3 to be rendered will pass the depth test and enter the shader for shading rendering.
  • the depth value is updated to 10.
  • the segment to be rendered 1, the segment to be rendered 2, the segment to be rendered 3, the segment to be rendered 4 and the segment to be rendered 5 can all pass the depth test, and the related technology only uses the depth information of some segments that pass the depth test Culling fragments to be rendered cannot achieve a high culling rate.
  • the embodiment of the present application provides an image processing method, which can improve the rejection rate of the data to be rendered.
  • the image processing method provided in the embodiment of the present application is applicable to an image processing device.
  • the image processing device may be provided in a graphics processor.
  • the image processing apparatus may be set in a graphics processor of an electronic device.
  • the electronic device may be a mobile phone, a tablet computer, a notebook computer, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook, a personal digital assistant (personal digital assistant, PDA) and the like.
  • the image processing device can be set in the graphics processor of the mobile phone.
  • Fig. 3 shows a schematic block diagram of the image processing device, which includes a partition pipeline, a global depth information buffer module and a rendering pipeline.
  • the screen space will be divided into multiple pixel blocks (pixel block), and each primitive (primitive) in the graphics programming interface (draw-call) called by each CPU will be rasterized (rasterization). Projected onto the plurality of pixel blocks mentioned above.
  • the screen space can be divided into N pixel blocks.
  • a coarse-grained rasterizer and a partitioned coarse-grained depth test module are set in the partition pipeline.
  • a coarse-grained rasterizer (also referred to as a second rasterizer) is configured to project the primitives input into the partition pipeline (that is, the primitives to be rendered) to multiple pixel blocks to generate multiple projection regions.
  • the second rasterizer may project primitives input into the partition pipeline into N pixel blocks to generate N projection regions.
  • a plurality of pixel blocks correspond to a plurality of projection areas one by one.
  • the partition coarse-grained depth test module (also called the second depth test module) is configured to perform partition coarse-grained depth test (also called the second depth test) on the primitive to be rendered according to multiple projection areas. If the primitive to be rendered fails the second depth test, the primitive to be rendered is eliminated. If the graphics primitive to be rendered fails the second depth test, the graphics primitive to be rendered passes the second depth test, and the graphics primitive to be rendered is input into the rendering pipeline.
  • the partition pipeline can also be configured with a polygon list generation (polygon list generation) module, and the polygon list generation module is used to store the primitives to be rendered that pass the second depth test through memory, and pass the second depth test.
  • the primitives to be rendered for the depth test are packed into a polygon list (PL) and input into the rendering pipeline in the form of a polygon stack (PL-Heap).
  • the second depth test module may update the depth information accumulated on each pixel block during the second depth test, and store the depth information of each pixel block in the global depth information cache module middle.
  • the global depth information caching module (may be referred to as the first caching module for short), is used for caching the depth information of the above-mentioned plurality of pixel blocks (may be referred to as the global depth information for short), and the depth information of the pixel block includes a pending area mask (pending mask, Pmask ), the pending destination z value (PZ) and the original target depth value (old Destination z value, OZ).
  • the first cache module caches global depth information, and the global depth information includes depth information of multiple (for example, N) pixel blocks obtained by updating all primitives to be rendered.
  • the global depth information cache module may be a cache inside the image processing device, or a cache of other devices (such as a cache of a CPU).
  • Pmask used to indicate the target area in the pixel block. Pmask can identify the range of the target area in the pixel block.
  • PZ is used to indicate the maximum depth value of the target area in the pixel block, that is, the depth values of all visible segments in the target area of the pixel block are smaller than the PZ value.
  • OZ is used to indicate the maximum depth value of the non-target area in the pixel block, that is, the depth values of all visible segments in the non-target area of the pixel block are smaller than the OZ value.
  • the rendering pipeline is set up with a rasterizer, a rendering coarse-grained depth test module, and a pixel shader.
  • a rasterizer (may be referred to as a first rasterizer), configured to convert primitives input into the rendering pipeline (that is, primitives that pass the second depth test) into fragments (that is, fragments to be rendered).
  • the rendering coarse-grained depth test module (may be referred to as the first depth test module) is configured to perform a first depth test on the segment to be rendered according to the above global depth information. If the segment to be rendered fails the first depth test, the segment to be rendered is eliminated. If the fragment to be rendered passes the first depth test, the fragment to be rendered is input to the pixel shader.
  • the rendering coarse-grained depth testing module may also use other information to perform the first depth test on the fragment to be rendered.
  • a pixel shader for shading rendering of fragments that are input to a pixel shader is
  • a pre-depth test module, a post-depth test module and a depth buffer module may also be set in the rendering pipeline.
  • the front depth test module is used for the front depth test of the fragment to be rendered.
  • the specific method of the pre-depth test may be processed by any method conceivable by those skilled in the art, which is not specifically limited in this embodiment of the present application.
  • the post-depth test module is used to perform post-depth tests on fragments after shader rendering.
  • the specific method of the post-depth test may be processed by any method conceivable by those skilled in the art, which is not specifically limited in this embodiment of the present application.
  • the second depth module is specifically configured to: determine whether the primitive to be rendered passes the depth test in the first pixel block according to the depth information of the first pixel block and the depth information of the first projection area.
  • a pixel block is any one of the plurality of pixel blocks, and the first projection area is the projection area corresponding to the first pixel block. If the primitive to be rendered passes the depth test in the first pixel block, it is determined that the primitive to be rendered passes the second depth test; if the primitive to be rendered fails the depth test in multiple pixel blocks, then it is determined that the primitive to be rendered passes the depth test Failed the second depth test.
  • the second depth module is specifically configured to: if the minimum depth value of the first projection area is smaller than the depth value of the first pixel block, determine that the pixel to be rendered is Passing the depth test in the first pixel block.
  • the depth information of the projection area may include a maximum depth value of the projection area, a minimum depth value of the projection area, and a range of the projection area.
  • the first pixel block includes a target area
  • the depth value of the first pixel block includes a maximum depth value of the target area of the first pixel block and a maximum depth value of the first pixel block.
  • the maximum depth value of the non-target area, the second depth module is specifically used for:
  • the first projected area overlaps both the target area and the non-target area of the first pixel block and the minimum depth value of the first projected area is smaller than the first depth value or the second depth value, then it is determined that the primitive to be rendered is at the first pixel The block passes the test.
  • the first projection area overlaps both the target area and the non-target area of the first pixel block and the minimum depth value of the first projection area is greater than the first depth value and the second depth value, determine that the primitive to be rendered is Failed test in one pixel block
  • the first projection area only overlaps with the target area of the first pixel block and the minimum depth value of the first projection area is smaller than the first depth value, it is determined that the primitive to be rendered passes the test in the first pixel block.
  • the first projection area only overlaps with the target area of the first pixel block and the minimum depth value of the first projection area is greater than the first depth value, it is determined that the primitive to be rendered fails the test in the first pixel block.
  • the first projection area only overlaps with the non-target area of the first pixel block and the minimum depth value of the first projection area is smaller than the second depth value, it is determined that the primitive to be rendered passes the test in the first pixel block.
  • the first projection area only overlaps with the non-target area of the first pixel block and the minimum depth value of the first projection area is greater than the second depth value, it is determined that the primitive to be rendered fails the test in the first pixel block.
  • the second depth module is further configured to update global depth information according to multiple projection areas.
  • the second depth module is further specifically configured to: update the depth information of the first pixel block in the global depth information according to the depth information of the first pixel block and the depth information of the first projection area, and the first pixel A block is any pixel block among the plurality of pixel blocks, and the first projection area is a projection area corresponding to the first pixel block.
  • the first pixel block includes the target area
  • the second depth module is further specifically configured to: according to the maximum depth value of the first projected area, the maximum depth value of the target area of the first pixel block, the first The maximum depth value of the non-target area of the pixel block is used to update the depth information of the first pixel block.
  • the second depth module is also specifically used for:
  • the first projected area overlaps both the target area and the non-target area of the first pixel block and the maximum depth value of the first projected area is smaller than the first depth value or the second depth value, update the depth of the first pixel block information.
  • the first pixel block is not updated depth information.
  • the depth information of the first pixel block is updated.
  • the depth information of the first pixel block is not updated.
  • the depth information of the first pixel block is updated.
  • the depth information of the first pixel block is not updated.
  • the second depth module is also specifically used for:
  • the first projected area completely overlaps with the target area of the first pixel block and the third depth value (that is, the maximum depth value of the projected area) is smaller than the first depth value
  • the target area of the first pixel block is updated to the non-target projected area and the first pixel block
  • the second depth value of is updated to the third depth value.
  • the target area of the first pixel block is updated to the first area and the second
  • the first depth value of a pixel block is updated to a first value.
  • the above-mentioned first condition is that the third depth value is smaller than the first depth value but larger than the second depth value or the third depth value is smaller than the second depth value but larger than the first depth value.
  • the above-mentioned second condition is that the first absolute value is smaller than the second absolute value
  • the above-mentioned first absolute value is the absolute value of the difference between the above-mentioned first depth value and the above-mentioned third depth value
  • the above-mentioned second absolute value is the difference between the above-mentioned second depth value and The absolute value of the difference between the above third depth values.
  • the above-mentioned first area is a union area of the target area and the target projection area
  • the first value is a larger value between the first depth value and the third depth value.
  • the target area of the first pixel block is updated to the second area and the second The second depth value of a pixel block is updated to a second value.
  • the above-mentioned third condition is that the first absolute value is greater than the second absolute value.
  • the above-mentioned second area is the intersection area of the target area and the non-target projection area.
  • the above-mentioned second value is a larger value between the second depth value and the third depth value.
  • the target area of the first pixel block is updated as Target projection area and change the first depth value of the first pixel block to the third depth value, and update the second depth value of the first pixel block to the third value.
  • the third value is a larger value between the first depth value and the second depth value.
  • the target projection area includes a first target projection area (ie CurMask0) and a second target projection area (ie CurMask0)
  • the first target projected area (ie CurMask0 ) may indicate the overlapping area of the projected area and the non-target area of the pixel block.
  • the second target projected area (ie CurMask0 ) may represent the overlapping area of the projected area and the target area of the pixel block.
  • the target projection area (CurMask) may be a union of the first target projection area (CurMask0) and the second target projection area (CurMask0).
  • the first depth test module is specifically configured to: perform a first depth test on the segment to be rendered according to the depth information of the target pixel block in the global information and the depth information of the segment to be rendered, and the target pixel block is a plurality of Among the pixel blocks, there are overlapping pixel blocks with the fragment to be rendered.
  • the first depth testing module is specifically configured to: determine that the segment to be rendered fails to pass the first depth test.
  • a pixel block (such as a target pixel block) may include a target area.
  • the depth information of the pixel block may include the maximum depth value of the target area of the pixel block, the maximum depth value of the non-target area of the pixel block, and the range of the target area of the pixel block.
  • the depth information of a fragment may include a depth value of the fragment.
  • the depth value of the fragment may be obtained through interpolation.
  • the target pixel block includes a target area
  • the depth value of the target pixel block includes a first depth value and a second depth value
  • the first depth test module is specifically used for:
  • the fragment to be rendered passes the first depth test.
  • the fragment to be rendered fails the first depth test.
  • the segment to be rendered passes the first depth test.
  • the segment to be rendered fails the first depth test.
  • the user can close all the depth test modules in the rendering pipeline of the image processing device (ie, the rendering coarse-grained depth test module, the front depth test module) in scenes that do not involve depth (such as flat scenes, two-dimensional scenes, etc. and post-depth test module) make the data to be rendered directly enter the pixel shader for shading rendering, so as to reduce the amount of data processing in the rendering process.
  • the user can enter the scene of the rendering pipeline in the order of fragments in order of depth value from small to large, and turn off the rendering coarse-grained depth testing module in the rendering pipeline to reduce the amount of data processing in the rendering process.
  • the user can turn off the depth test module to reduce the amount of data processing in the rendering process.
  • the user can use the scene where the depth value of the fragment is affected by the pixel shader (that is, the depth value of the fragment will change before and after the rendering of the pixel shader) and the direction of the depth value change of the fragment is not fixed (for example, the depth value of some fragments is shaded) Increase after rendering, and the depth value of another part of the fragment decreases after shading rendering), close the front depth test module to reduce the amount of data processing in the rendering process.
  • FIG. 4 shows a schematic flowchart of an image processing method provided by an embodiment of the present application, and the method may be executed by the image processing apparatus shown in FIG. 3 .
  • the image processing method includes:
  • the image processing device performs a second depth test on the graphics primitives to be rendered input into the partition pipeline, and rejects the graphics primitives to be rendered that fail the second depth test.
  • the image processing device first performs a second depth test on the first graphics primitive to be rendered, then performs a second depth test on the second graphics primitive, and then performs a second depth test on the second graphics primitive to be rendered, and then The 3rd, 4th, ..., Mth primitives to be rendered are subjected to the second depth test. If the graphics unit to be rendered fails the second depth test, the graphics unit to be rendered is rejected; if the graphics unit to be rendered passes the second test, the graphics unit to be rendered is input into the rendering pipeline.
  • the primitive to be rendered does not pass the second depth test, it means that the primitive to be rendered has a higher depth and will be blocked by other primitives to be rendered, so the primitive may not be rendered, but removed To reduce computing resource consumption.
  • the image processing device performing the second depth test on each primitive to be rendered may include: first projecting the primitive to be rendered to a plurality of pixels divided by screen space through a coarse-grained rasterizer The block generates multiple projection areas, and then performs a second depth test on the primitive to be rendered according to the multiple projection areas.
  • the above-mentioned multiple projection areas correspond one-to-one to the above-mentioned multiple pixel blocks.
  • the screen space can be divided into 16 pixel blocks (the white area in the figure), and the image processing device can respectively project the primitives to be rendered into these 16 pixel blocks to generate 16 pixel blocks One-to-one correspondence of 16 projection areas (black areas in the figure), and then a second depth test is performed on the primitives to be rendered based on these 16 projection areas.
  • the image processing device may determine whether the primitive to be rendered passes the depth test in each pixel block according to the depth information of each pixel block and the depth information of the projection area corresponding to each pixel block. If the primitive to be rendered passes the depth test in any pixel block, it is determined that the primitive to be rendered passes the second depth test; if the primitive to be rendered fails the depth test in the above-mentioned multiple pixel blocks, it is determined that The rendered primitive failed the second depth test.
  • the image processing device may determine that the primitives to be rendered are in the 16 pixel blocks in FIG. 6 according to the depth information of the 16 pixel blocks in FIG. Whether the depth test is passed in each pixel block. If the primitive to be rendered passes the depth test in any pixel block, it is determined that the primitive to be rendered passes the second depth test. If the primitive to be rendered does not pass the depth test in the above 16 pixel blocks, it is determined that the primitive to be rendered fails the second depth test.
  • the pixel block may include a target area
  • the depth information of the pixel block may include the maximum depth value of the target area of the pixel block (which may be referred to as the first depth value for short), the maximum depth value of the non-target area of the pixel block (which may be referred to as is the second depth value) and the range of the target area of the pixel block.
  • the depth information of the projection area may include a maximum depth value of the projection area and a minimum depth value of the projection area.
  • the image processing device may, according to the maximum depth value of the target area of each pixel block, the maximum depth value of the non-target area of each pixel block, and the minimum depth of the projection area corresponding to each pixel block The value determines whether the primitive to render passes the depth test in each texel.
  • FIG. 7 shows a pixel block and a projection area.
  • the gray area is the target area of the pixel block
  • the white area is the non-target area of the pixel block
  • the black area is the projection area.
  • Figure 7 (a) shows the first positional relationship, that is, the projection area overlaps with the target area and non-target area of the pixel block
  • Figure 7 (b) shows the second positional relationship, that is, the projection area There is only overlap with the target area of the pixel block
  • (c) in FIG. 7 shows the third positional relationship, that is, the projection area only overlaps with the non-target area of the pixel block.
  • the image processing device will introduce the maximum depth value of the target area of each pixel block, the maximum depth value of the non-target area of each pixel block, and the maximum depth value of each pixel block.
  • the minimum depth value of the projection area corresponding to each pixel block determines whether the primitive to be rendered passes the depth test in each pixel block.
  • the first pixel block is any one of the plurality of pixel blocks
  • the first projection area is the projection area corresponding to the first pixel block.
  • the image processing device may also update the global depth information according to the foregoing multiple projection areas.
  • the global depth information includes depth information of multiple pixel blocks divided by the screen space.
  • the image processing device may update the depth information of each pixel block in the global depth information according to the depth information of each pixel block (current depth information) and the depth information of the projection area corresponding to each pixel block.
  • the image processing device will introduce the maximum depth value of the first projected area and the maximum depth value of the target area of the first pixel block (ie, the first depth value). ), the maximum depth value (that is, the second depth value) of the non-target area of the first pixel block, and the specific process of updating the depth information of the first pixel block.
  • the first projected area when the first projected area only overlaps with the non-target area of the first pixel block and the minimum depth value of the first projected area is greater than the second depth value, the first projected area is not updated.
  • the depth information of the pixel block when the first projected area only overlaps with the non-target area of the first pixel block and the minimum depth value of the first projected area is greater than the second depth value, the first projected area is not updated. The depth information of the pixel block.
  • the image processing apparatus may update the depth information of the first pixel block according to the target area.
  • the target projection area includes a first target projection area (namely CurMask0) and a second target projection area (ie CurMask0).
  • CurMask0 In the overlapping area between the projected area of the primitive to be rendered and the non-target area of the pixel block (also called the OZ area), if the maximum depth of the projected area (which can be called the third depth value, can also be referred to as Zmax ) is less than the maximum depth value of the non-target area of the pixel block (that is, the second depth value, which can be referred to as OZ for short), then the depth values of all visible segments of the pixel block in the above overlapping area must be smaller than the Zmax value (otherwise it will be detected by the image element shading), then the OZ value of the pixel block is conditionally updated to the Zmax value of the projection area of the primitive to be rendered.
  • the third depth value which can also be referred to as Zmax
  • CurMask0 mask can be used to indicate the coverage of this part of the area. That is, in the case that the third depth value of the projected area is smaller than the second depth value of the pixel block, CurMask0 may represent an overlapping area between the projected area and the non-target area of the pixel block.
  • CurMask1 In the overlapping area between the projection area of the primitive to be rendered and the target area of the pixel block (also called the PZ area), if the maximum depth of the projection area is less than the maximum depth value of the target area of the pixel block (that is, the first depth value, which can be abbreviated as PZ), then the depth values of all visible segments of the pixel block in the overlapping area must be smaller than the Zmax value (otherwise it will be covered by the primitive), then the PZ value of the pixel block is conditionally updated as The Zmax value of the projection area of the primitive to be rendered.
  • the CurMask1 mask can be used to indicate the coverage of this part of the area. That is, in the case that the third depth value of the projected area is smaller than the first depth value of the pixel block, CurMask1 may indicate the overlapping area of the projected area and the target area of the pixel block.
  • CurMask the union of CurMask0 and CurMask1.
  • layer0Mask Indicates the area not indicated by CurMask0 in the original OZ area (that is, in the OZ area before update).
  • layer1Mask Indicates the area not indicated by CurMask1 in the original PZ area (that is, in the PZ area before update).
  • updating the depth information of the first pixel block by the image processing device may include:
  • the first projected area completely overlaps with the target area of the first pixel block and the third depth value (that is, the maximum depth value of the projected area) is smaller than the first depth value
  • FIG. 8 shows the target area (gray area) and non-target area (white area) before the update of the first pixel block
  • FIG. 8 shows the first projected area (black area) and the first pixel block. It can be seen from the diagram (b) in FIG. 8 that the first projected area completely overlaps with the target area of the first pixel block.
  • Figure 9 (a) shows the first depth value (PZ), the second depth value (OZ) and the depth range of the first projection area, the horizontal axis in Figure 9 represents the depth value, and the depth value increases from left to right . It can be seen from the diagram (a) in FIG.
  • the maximum depth value (Zmax) of the first projection area is smaller than the first depth value (PZ) and the second depth value (OZ), so the target projection area includes the entire first projection area.
  • the image processing device updates the target area of the first pixel block to the target projection area.
  • the image processing device updates the first depth value (PZ) of the first pixel block to the third depth value (Zmax).
  • the target area of the first pixel block is updated to the non-target projected area and the first pixel block
  • the second depth value of is updated to the third depth value.
  • the first projected area completely overlaps with the non-target area of the first pixel block.
  • the third depth value is smaller than the second depth value but greater than the first depth value, and the target projection area includes the overlapping area of the first projection area and the non-target area of the first pixel block ( i.e. the entire non-target area).
  • the image processing device since the non-target projection area is the same as the target area of the first pixel block, the image processing device does not update the target area.
  • the image processing device updates the second depth value (OZ) of the first pixel block to the third depth value (Zmax).
  • the first projected area completely overlaps with the non-target area of the first pixel block. If the maximum depth value (Zmax) of the first projection area is smaller than the first depth value (PZ) and the second depth value (OZ), the target projection area includes the entire first projection area. Then, as shown in Figure 12 (c), the image processing device updates the target area of the first pixel block to a non-target projection area (that is, the area in the first pixel block that does not overlap with the first projection area) and converts the first The second depth value (OZ) of the pixel block is updated to the third depth value (Zmax).
  • a non-target projection area that is, the area in the first pixel block that does not overlap with the first projection area
  • the target area of the first pixel block is updated to the first area and the second
  • the first depth value of a pixel block is updated to a first value.
  • the above-mentioned first condition is that the third depth value is smaller than the first depth value but larger than the second depth value or the third depth value is smaller than the second depth value but larger than the first depth value.
  • the above-mentioned second condition is that the first absolute value is smaller than the second absolute value
  • the above-mentioned first absolute value is the absolute value of the difference between the above-mentioned first depth value and the above-mentioned third depth value
  • the above-mentioned second absolute value is the difference between the above-mentioned second depth value and The absolute value of the difference between the above third depth values.
  • the above-mentioned first area is a union area of the target area and the target projection area
  • the first value is a larger value between the first depth value and the third depth value.
  • satisfying the first condition and the second condition means that the third depth value is between the first depth value and the second depth value, and the third depth value is closer to the first depth value.
  • the first projected area does not completely overlap with the target area and the non-target area of the first pixel block.
  • the third depth value (Zmax) is between the first depth value (PZ) and the second depth value (OZ), and the third depth value is closer to the first depth value. Since the third depth value is smaller than the second depth value but greater than the first depth value, the target projection area only includes the overlapping area of the first projection area and the non-target area, and the first area includes the target projection area and the current target area. As shown in (c) of FIG.
  • the image processing device updates the target area of the first pixel block to the first area. Since the third depth value is greater than the first depth value, the first value is the third depth value. As shown in (b) of FIG. 14 , the image processing device updates the first depth value (PZ) of the first pixel block to the third depth value (Zmax).
  • the target area of the first pixel block is updated to the second area and the second The second depth value of a pixel block is updated to a second value.
  • the above-mentioned third condition is that the first absolute value is greater than the second absolute value.
  • the above-mentioned second area is the intersection area of the target area and the non-target projection area.
  • the above-mentioned second value is a larger value between the second depth value and the third depth value.
  • satisfying the first condition and the third condition means that the third depth value is between the first depth value and the second depth value, and the third depth value is closer to the second depth value.
  • the first projected area does not completely overlap with the target area and the non-target area of the first pixel block.
  • the third depth (Zmax) value is between the first depth value (PZ) and the second depth value (OZ), and the third depth value is closer to the second depth value. Since the third depth value is smaller than the second depth value but greater than the first depth value, the target projection area only includes the overlapping area of the first projection area and the non-target area. The second area includes the intersection area of the non-target projection area and the current target area. As shown in (c) of FIG.
  • the image processing device since the second area is the same as the current target area, the image processing device does not update the target area. Since the second depth value is greater than the third depth value, the second value is the second depth value. As shown in (b) of FIG. 16 , since the second value is the same as the current second depth value (OZ), the image processing device does not update the second depth value (OZ).
  • the target area of the first pixel block is updated as Target projection area and change the first depth value of the first pixel block to the third depth value, and update the second depth value of the first pixel block to the third value.
  • the third value is a larger value between the first depth value and the second depth value.
  • the first projected area does not completely overlap with the target area and the non-target area of the first pixel block.
  • the third depth value (Zmax) is smaller than the first depth value (PZ) and the second depth value (OZ). Since the third depth value is smaller than the first depth value and the second depth value, the target projection area includes the entire first projection area.
  • the image processing device updates the target area of the first pixel block to the target projection area, that is, the entire first projection area. Since the first depth value is smaller than the second depth value, the third value is the second depth value.
  • the image processing device updates the first depth value of the first pixel block to the third depth value.
  • the image processing device does not update the second depth value of the first pixel block.
  • the image processing device may store the primitives to be rendered that pass the second depth test in polygon list generation (polygon list generation) in the partition pipeline.
  • the image processing device inputs the graphics primitives to be rendered that pass the second depth test into the rendering pipeline.
  • the polygon list generation in the image processing device may pass the second depth test
  • the primitives to be rendered are packed into multiple polygon lists (PL), and then input into the rendering pipeline in the form of polygon stacks (PL-Heap).
  • the image processing device converts graphics primitives input into the rendering pipeline into fragments to be rendered.
  • the image processing apparatus may convert the primitives input into the rendering pipeline (ie, the primitives passing the second depth test) into fragments to be rendered through a rasterizer in the rendering pipeline.
  • the image processing apparatus performs a first depth test on the fragments to be rendered and rejects the fragments to be rendered that fail the first depth test.
  • the image processing apparatus may perform a first depth test on the segment to be rendered according to the global depth information in the global depth information cache module.
  • the image processing apparatus may perform a first depth test on the segment to be rendered according to the depth information of the target pixel block in the global depth information and the depth information of the segment to be rendered.
  • the target pixel block is a pixel block that overlaps with the segment to be rendered among the plurality of pixel blocks.
  • FIG. 19 shows 9 pixel blocks and 1 fragment to be rendered (the black square in the figure), and it can be seen that the pixel block in the upper left corner overlaps with the fragment to be rendered.
  • the image processing device performs a first depth test on the segment to be rendered according to the depth information of the upper left pixel block in the global depth information and the depth information of the segment to be rendered.
  • the depth information of the segment may include a depth value of the segment.
  • the depth value of the fragment may be obtained through interpolation.
  • the image processing apparatus performs a first depth test on the segment to be rendered according to a depth value, a first depth value, or a second depth value of the segment to be rendered.
  • the image processing device may perform a first depth test on the fragment to be rendered according to the depth value, the first depth value or the second depth value of the fragment to be rendered, which may include:
  • the image processing device determines that the segment to be rendered passes the first depth test when the segment to be rendered is located in the target region of the target pixel block and the depth value of the segment to be rendered is less than or equal to the first depth value.
  • the image processing device determines that the segment to be rendered fails the first depth test when the segment to be rendered is located in the target region of the target pixel block and the depth value of the segment to be rendered is greater than the first depth value.
  • the image processing device determines that the segment to be rendered passes the first depth test when the segment to be rendered is not located in the target area of the target pixel block and the depth value of the segment to be rendered is less than or equal to the second depth value.
  • the image processing device determines that the segment to be rendered fails the first depth test when the segment to be rendered is not located in the target region of the target pixel block and the depth value of the segment to be rendered is greater than the second depth value.
  • the segment to be rendered is located in the target area of the target pixel block and the depth value of the segment to be rendered is greater than the first depth value, it means that the segment to be rendered will be blocked by visible segments in the target area of the target pixel block, so it can The segment to be rendered is eliminated to reduce the consumption of computing resources and increase the rendering speed of the GPU.
  • segment to be rendered is located in the target area of the non-target pixel block and the depth value of the segment to be rendered is greater than the second depth value, it means that the segment to be rendered will be blocked by visible segments in the non-target area of the target pixel block, so the segment to be rendered can be Render fragment culling to reduce computing resource consumption and improve GPU rendering speed.
  • the objects rendered in this pixel block include sky, roof and a triangle.
  • the gray area shown in (a) of FIG. 21 is the target area of the pixel block, and the black area is the non-target area of the pixel block.
  • the maximum depth value (PZ) of the target area of the pixel block is 0.1
  • the maximum depth value (OZ) of the non-target area is 1.0.
  • the method provided by the embodiment of the present application can eliminate fragments to be rendered through the global depth information. Since the global depth information is determined according to all primitives to be rendered, and the fragments to be rendered are converted from primitives to be rendered, so Global depth information is equivalent to being generated based on all fragments to be rendered.
  • the embodiment of the present application uses the depth information of all the fragments to be rendered to remove fragments. Compared with the related art, only the depth information of some fragments to be rendered is used to remove fragments, which can improve the rejection rate of the data to be rendered, thereby reducing the consumption of computing resources. To increase the rendering speed of the GPU.
  • the image processing apparatus inputs the fragment to be rendered that passes the first depth test into the pixel shader.
  • the image processing device may first perform a pre-depth test on the to-be-rendered fragments that pass the first depth test through the pre-depth test module, and then input the to-be-rendered fragments that pass the pre-depth test to the pixel shader.
  • the image processing apparatus performs color rendering on the fragment input to the pixel shader.
  • the image processing device may perform a post-depth test on the shaded and rendered fragment through the post-depth test module.
  • FIG. 5 shows a schematic flowchart of another image processing method provided by an embodiment of the present application, and the method may be executed by the image processing apparatus shown in FIG. 3 .
  • the image processing method includes:
  • the image processing device converts graphics primitives input into a rendering pipeline into fragments to be rendered.
  • the image processing device performs a first depth test on the segment to be rendered according to the global depth information
  • the global depth information includes depth information of a plurality of pixel blocks updated according to all primitives to be rendered.
  • the image processing device rejects the segment to be rendered.
  • the embodiment of the present application also provides a computer storage medium, the computer storage medium stores computer instructions, and when the computer instructions are run on the image processing device, the image processing device executes the above-mentioned related method steps to realize the image in the above-mentioned embodiment Approach.
  • An embodiment of the present application also provides a computer program product, which, when running on a computer, causes the computer to execute the above-mentioned related steps, so as to implement the image processing method in the above-mentioned embodiment.
  • the embodiment of the present application also provides an image processing device, and the device may specifically be a chip, an integrated circuit, a component, or a module.
  • the device may include a connected processor and a memory for storing instructions, or the device may include at least one processor for fetching instructions from an external memory.
  • the processor can execute instructions, so that the chip executes the image processing methods in the above method embodiments.
  • FIG. 22 is a schematic structural diagram of another image processing device provided by an embodiment of the present application.
  • the image processing apparatus 2000 includes: at least one CPU, GPU, memory, the type of memory may include, for example, SRAM and ROM, a microcontroller (Micro controller Unit, MCU), a WLAN subsystem, a bus, a transmission interface, and the like.
  • the image processing device 2000 may also include an application processor (Application Processor, AP), other dedicated processors such as NPU, and a power management subsystem, a clock management subsystem, and a power management subsystem, etc. other subsystems.
  • Application Processor Application Processor
  • the connectors include various interfaces, transmission lines or buses, etc. These interfaces are usually electrical communication interfaces, but may also be mechanical interfaces or other forms of interfaces. , which is not limited in this embodiment.
  • the CPU can be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor; optionally, the CPU can be a processor group composed of a plurality of processors, between which are coupled to each other by one or more buses.
  • the CPU implements any image processing method in the foregoing method embodiments by calling program instructions stored in the on-chip memory or off-chip memory.
  • the CPU and the MCU jointly implement any image processing method in the foregoing method embodiments, for example, the CPU completes some steps in the image processing method, and the MCU completes other steps in the image processing method.
  • the AP or other special-purpose processor implements any one of the image processing methods in the foregoing method embodiments by calling program instructions stored in the on-chip memory or off-chip memory.
  • the transmission interface can be an interface for receiving and sending data of the processor chip, and the transmission interface usually includes multiple interfaces.
  • the transmission interface can include an internal integrated circuit (Inter-Integrated Circuit, I2C) Interface, Serial Peripheral Interface (SPI), Universal asynchronous receiver-transmitter (UART) interface, General-purpose input/output (GPIO) interface, etc. It should be understood that these interfaces may implement different functions by multiplexing the same physical interface.
  • the transmission interface may also include a high-definition multimedia interface (High Definition Multimedia Interface, HDMI), a V-By-One interface, an embedded display port (Embedded Display Port, eDP), a mobile industry processing Interface (Mobile Industry Processor Interface, MIPI) or Display Port (DP), etc.
  • HDMI High Definition Multimedia Interface
  • V-By-One interface an embedded display port
  • eDP embedded Display Port
  • MIPI Mobile Industry Processor Interface
  • DP Display Port
  • the above-mentioned parts are integrated on the same chip; in another optional situation, the memory may be an independent chip.
  • a WLAN subsystem may include, for example, radio frequency circuits and a baseband.
  • the chip involved in the embodiment of this application is a system manufactured on the same semiconductor substrate by an integrated circuit process, also called a semiconductor chip, which can be fabricated on a substrate (usually a semiconductor such as silicon) by using an integrated circuit process.
  • the above-mentioned integrated circuit may include various functional devices, and each type of functional device includes transistors such as logic gate circuits, metal-oxide-semiconductor (Metal-Oxide-Semiconductor, MOS) transistors, bipolar transistors or diodes, and may also include capacitors, resistors or inductors and other components.
  • Each functional device can work independently or under the action of necessary driver software, and can realize various functions such as communication, calculation or storage.
  • the image processing device, computer storage medium, and computer program product provided in this embodiment are all used to execute the corresponding method provided above, therefore, the beneficial effects that it can achieve can refer to the corresponding method provided above The beneficial effects in the above will not be repeated here.
  • sequence numbers of the above-mentioned processes do not mean the order of execution, and the execution order of the processes should be determined by their functions and internal logic, and should not be used in the embodiments of the present application.
  • the implementation process constitutes any limitation.
  • the disclosed systems, devices and methods may be implemented in other ways.
  • the device embodiments described above are only illustrative.
  • the division of the above units is only a logical function division. In actual implementation, there may be other division methods.
  • multiple units or components can be combined or can be Integrate into another system, or some features may be ignored, or not implemented.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.
  • the units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.
  • the above functions are realized in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium.
  • the technical solution of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the above-mentioned methods in various embodiments of the present application.
  • the above-mentioned storage medium includes: various media that can store program codes such as U disk, mobile hard disk, read-only memory (Read Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Image Generation (AREA)

Abstract

本申请实施例提供的图像处理装置和方法，涉及计算机技术领域，能够提升对待渲染数据的剔除率，从而降低计算资源消耗量并提升渲染速度。该装置包括渲染管线，渲染管线中设置有第一光栅化器和第一深度测试模块。第一光栅化器，用于将输入渲染管线的图元转换为待渲染片段。第一深度测试模块，用于根据全局深度信息对待渲染片段进行第一深度测试，若待渲染片段未通过第一深度测试，则剔除待渲染片段，全局深度信息包括根据全部待渲染图元更新得到的多个像素块的深度信息。

Description

图像处理装置和方法 技术领域
本申请涉及计算机技术领域，尤其涉及图像处理装置和方法。
背景技术
渲染是将三维场景中的模型，按照设定好的环境、灯光、材质及渲染参数进行二维投影得到数字图像的过程。但在渲染过程中图形处理器(graphics processing unit，GPU)会消耗大量计算资源。为此，GPU可以在渲染阶段剔除不影响渲染结果的待渲染数据，以降低计算资源消耗量。
然而在相关技术中，GPU仅能在渲染阶段剔除少量不影响渲染结果的待渲染数据，无法达到很高的剔除率。
发明内容
本申请提供图像处理装置和方法，能够提升对待渲染数据的剔除率，从而降低计算资源消耗量并提升渲染速度。
第一方面，本申请提供了一种图像处理装置，该装置包括渲染管线，所述渲染管线中设置有第一光栅化器和第一深度测试模块。第一光栅化器，用于将输入所述渲染管线的图元转换为待渲染片段。第一深度测试模块，用于根据全局深度信息对所述待渲染片段进行第一深度测试，若所述待渲染片段未通过所述第一深度测试，则剔除所述待渲染片段，所述全局深度信息包括根据全部待渲染图元更新得到的多个像素块的深度信息。
可以看出，本申请提供的图像处理装置可以通过全局深度信息对待渲染片段进行剔除，由于全局深度信息是根据全部待渲染图元确定，而待渲染片段是由待渲染图元转换得到的，所以全局深度信息相当于是根据所有待渲染片段生成的。本申请实施例利用全部待渲染片段的深度信息进行片段剔除，相较于相关技术仅通过部分待渲染片段的深度信息进行片段剔除，本申请提供的方法能够提升对待渲染数据的剔除率，从而降低计算资源消耗量，以提升GPU的渲染速度。
在一种可能的实现方式中，第一深度测试模块具体用于：根据所述全局信息中目标像素块的深度信息和所述待渲染片段的深度信息对所述待渲染片段进行所述第一深度测试，所述目标像素块为所述多个像素块中与所述待渲染片段存在重叠的像素块。
在一种可能的实现方式中，第一深度测试模块具体用于：在所述待渲染片段的深度值大于所述目标像素块的深度值的情况下，确定所述待渲染片段未通过第一深度测试。
由于目标像素块和待渲染片段存在重叠，所以待渲染片段会在目标像素块上渲染。通过全局信息中目标像素块的深度信息和所述待渲染片段的深度信息，可以在待渲染片段的深度值大于目标像素块的深度值的情况下判断待渲染片段会被目标像素块上的可见片段所遮挡。
可选地，像素块(如目标像素块)可以包括目标区域。像素块的深度信息可以包括像素块的目标区域的最大深度值、像素块的非目标区域的最大深度值和像素块的目标区域的范围。
可选地，片段(如待渲染片段)的深度信息可以包括片段的深度值。其中，片段的深度值可以是通过插值得到的。
在一种可能的实现方式中，所述目标像素块包括目标区域，所述目标像素块的深度值包括第一深度值和第二深度值，所述第一深度测试模块具体用于：
在待渲染片段位于目标像素块的目标区域且待渲染片段的深度值小于或等于第一深度值的情况下，确定待渲染片段通过第一深度测试。
在待渲染片段位于目标像素块的目标区域且待渲染片段的深度值大于第一深度值的情况下，确定待渲染片段未通过第一深度测试。
在待渲染片段位于目标像素块的非目标区域且待渲染片段的深度值小于或等于第二深度值的情况下，确定待渲染片段通过第一深度测试。
在待渲染片段位于目标像素块的非目标区域且待渲染片段的深度值大于第二深度值的情况下，确定待渲染片段未通过第一深度测试。
需要说明的是，待渲染片段位于目标像素块的目标区域且待渲染片段的深度值大于第一深度值，说明该待渲染片段会被目标像素块的目标区域内的可见片段所遮挡，所以可以将该待渲染片段剔除，以降低计算资源消耗量，提升GPU的渲染速度。
待渲染片段位于非目标像素块的目标区域且待渲染片段的深度值大于第二深度值，说明该待渲染片段会被目标像素块的非目标区域内的可见片段所遮挡，所以可以将该待渲染片段剔除，以降低计算资源消耗量，提升GPU的渲染速度。
可以理解的是，相较于仅通过目标像素块的深度值对待渲染片段进行深度测试，将目标像素块分为目标区域和非目标区域并分别通过这两个区域上的最大深度值对待渲染片段进行深度测试，可以增加深度测试的准确性，降低错误剔除的概率。
在一种可能的实现方式中，该装置还可以包括第一缓存模块，所述第一缓存模块用于缓存所述全局深度信息。
在一种可能的实现方式中，该装置还可以包括分区管线，所述分区管线中设置有第二光栅化器和第二深度测试模块。第二光栅化器，用于将输入所述分区管道的待渲染图元投射至多个像素块生成多个投射区域。第二深度模块，用于根据所述多个投射区域对所述待渲染图元进行第二深度测试，若所述待渲染图元未通过所述第二深度测试，则剔除所述待渲染图元。其中，所述多个投射区域与所述多个像素块一一对应。
可以看出，本申请提供的装置不仅可以剔除无需渲染的待渲染片段，还可以剔除无需渲染的待渲染图元，相较于仅对无需渲染的待渲染片段进行剔除，本申请还对无需渲染的待渲染图元进行剔除，从而进一步提升对待渲染数据的剔除率，进一步降低计算资源消耗量，以进一步提升GPU的渲染速度。
在一种可能的实现方式中，第二深度模块具体用于：根据第一像素块的深度信息和第一投射区域的深度信息确定所述待渲染图元在所述第一像素块中是否通过深度测试，所述第一像素块为所述多个像素块中的任一像素块，所述第一投射区域为所述第一像素块对应的投射区域。若所述待渲染图元在所述第一像素块中通过深度测试，则确定所述待渲染图 元通过所述第二深度测试；若所述待渲染图元在所述多个像素块中均未通过深度测试，则确定所述待渲染图元未通过所述第二深度测试。
在一种可能的实现方式中，第二深度模块具体用于：在所述第一投射区域的最小深度值小于所述第一像素块的深度值的情况下，确定所述待渲染图元在所述第一像素块中通过所述深度测试。
可选地，投射区域(如第一投射区域)的深度信息可以包括投射区域的最大深度值、投射区域的最小深度值和投射区域的范围。
在一种可能的实现方式中，所述第一像素块包括目标区域，所述第一像素块的深度值包括所述第一像素块的目标区域的最大深度值和所述第一像素块的非目标区域的最大深度值，第二深度模块具体用于：
在第一投射区域与第一像素块的目标区域和非目标区域均存在重叠且第一投射区域的最小深度值小于第一深度值或第二深度值，则确定待渲染图元在第一像素块中通过测试。
在第一投射区域与第一像素块的目标区域和非目标区域均存在重叠且第一投射区域的最小深度值大于第一深度值和第二深度值的情况下，确定待渲染图元在第一像素块中未通过测试
在第一投射区域仅与第一像素块的目标区域存在重叠且第一投射区域的最小深度值小于第一深度值的情况下，确定待渲染图元在第一像素块中通过测试。
在第一投射区域仅与第一像素块的目标区域存在重叠且第一投射区域的最小深度值大于第一深度值的情况下，确定待渲染图元在第一像素块中未通过测试。
在第一投射区域仅与第一像素块的非目标区域存在重叠且第一投射区域的最小深度值小于第二深度值的情况下，确定待渲染图元在第一像素块中通过测试。
在第一投射区域仅与第一像素块的非目标区域存在重叠且第一投射区域的最小深度值大于第二深度值的情况下，确定待渲染图元在第一像素块中未通过测试。
在一种可能的实现方式中，所述第二深度模块还用于根据所述多个投射区域更新所述全局深度信息。
在一种可能的实现方式中，所述第二深度模块还具体用于：根据第一像素块的深度信息和第一投射区域的深度信息更新所述全局深度信息中所述第一像素块的深度信息，所述第一像素块为所述多个像素块中的任一像素块，所述第一投射区域为所述第一像素块对应的投射区域。
在一种可能的实现方式中，所述第一像素块包括目标区域，所述第二深度模块还具体用于：根据所述第一投射区域的最大深度值、所述第一像素块的目标区域的最大深度值、所述第一像素块的非目标区域的最大深度值，更新所述第一像素块的深度信息。
在一种可能的实现方式中，所述第二深度模块还具体用于：
在第一投射区域与第一像素块的目标区域和非目标区域均存在重叠且第一投射区域的最大深度值小于第一深度值或第二深度值的情况下，更新第一像素块的深度信息。
在第一投射区域与第一像素块的目标区域和非目标区域均存在重叠且第一投射区域的最大深度值大于第一深度值和第二深度值的情况下，不更新第一像素块的深度信息。
在第一投射区域仅与第一像素块的目标区域存在重叠且第一投射区域的最小深度值 小于第一深度值的情况下，更新第一像素块的深度信息。
在第一投射区域仅与第一像素块的目标区域存在重叠且第一投射区域的最小深度值大于第一深度值的情况下，不更新第一像素块的深度信息。
在第一投射区域仅与第一像素块的非目标区域存在重叠且第一投射区域的最小深度值小于第二深度值的情况下，更新第一像素块的深度信息。
在第一投射区域仅与第一像素块的非目标区域存在重叠且第一投射区域的最小深度值大于第二深度值的情况下，不更新第一像素块的深度信息。
在一种可能的实现方式中，所述第二深度模块还具体用于：
在第一投射区域与第一像素块的目标区域完全重合且第三深度值(即投射区域的最大深度值)小于第一深度值的情况下，将第一像素块的目标区域更新为目标投射区域且将第一像素块的第一深度值更新为第三深度值。
在第一投射区域与第一像素块的非目标区域完全重合且第三深度值小于第二深度值的情况下，将第一像素块的目标区域更新为非目标投射区域且将第一像素块的第二深度值更新为第三深度值。
在第一投射区域与第一像素块的目标区域和非目标区域均不完全重合且满足第一条件和第二条件的情况下，将第一像素块的目标区域更新为第一区域且将第一像素块的第一深度值更新为第一数值。其中，上述第一条件为第三深度值小于第一深度值但大于第二深度值或第三深度值小于第二深度值但大于第一深度值。上述第二条件为第一绝对值小于第二绝对值，上述第一绝对值为上述第一深度值与上述第三深度值之差的绝对值，上述第二绝对值为上述第二深度值与上述第三深度值之差的绝对值。上述第一区域为目标区域与目标投射区域的并集区域，第一数值为第一深度值与第三深度值之间的较大值。
在第一投射区域与第一像素块的目标区域和非目标区域均不完全重合且满足第一条件和第三条件的情况下，将第一像素块的目标区域更新为第二区域且将第一像素块的第二深度值更新为第二数值。其中，上述第三条件为第一绝对值大于第二绝对值。上述第二区域为目标区域与非目标投射区域的交集区域。上述第二数值为第二深度值与第三深度值之间的较大值。
在第一投射区域与第一像素块的目标区域和非目标区域均不完全重合且第三深度值小于第一深度值和第二深度值的情况下，将第一像素块的目标区域更新为目标投射区域且将第一像素块的第一深度值更为第三深度值，将第一像素块的第二深度值更新为第三数值。其中，第三数值为第一深度值与第二深度值之间的较大值。
可选地，目标投射区域(CurMask)包括第一目标投射区域(即CurMask0)和第二目标投射区域(即CurMask0)
在投射区域的第三深度值小于像素块的第二深度值的情况下，第一目标投射区域(即CurMask0)可以表示投射区域与像素块的非目标区域的重叠区域。
在投射区域的第三深度值小于像素块的第一深度值的情况下，第二目标投射区域(即CurMask0)可以表示投射区域与像素块的目标区域的重叠区域。
目标投射区域(CurMask)可以为第一目标投射区域(CurMask0)和第二目标投射区域(CurMask0)的并集。
在一种可能的实现方式中，图像处理装置的渲染管线还可以设置有前深度测试模块、 像素着色器、后深度测试模块或深度缓存模块中的至少一项。前深度测试模块，用于根据所述深度缓存模块中的深度值对第一目标片段进行前深度测试。像素着色器用于对第二目标片段进行着色渲染。后深度测试模块，用于根据所述深度缓存模块中的深度值对第三目标片段进行后深度测试。深度缓存模块，用于缓存深度值。所述第一目标片段为通过第一深度测试的片段，所述第二目标片段为通过前深度测试的片段，所述第三目标片段为经过像素着色器着色渲染后的片段。
第二方面，本申请还提供了另一种图像处理装置，该装置包括渲染管线，所述渲染管线中设置有光栅化器、渲染粗粒度深度测试模块、前深度测试模块、像素着色器、深度缓存模块和后深度测试模块。所述光栅化器，用于将输入所述渲染管线的图元转换为待渲染片段；所述渲染粗粒度深度测试模块，用于对所述待渲染片段进行第一深度测试；所述前深度测试模块，用于根据所述深度缓存模块中的深度值对第一目标片段进行前深度测试；所述像素着色器用于对所述第二目标片段进行着色渲染；所述后深度测试模块，用于根据所述深度缓存模块中的深度值对所述第三目标片段进行后深度测试；所述深度缓存模块，用于缓存深度值。所述第一目标片段为通过第一深度测试的片段，所述第二目标片段为通过前深度测试的片段，所述第三目标片段为经过像素着色器着色渲染后的片段。
相较于相关技术仅在渲染管线中设置前深度测试模块和后深度测试模块对片段进行剔除。本申请提供的图像处理装置的渲染管线中不仅设置前深度测试模块和后深度测试模块，还设置有渲染粗粒度深度测试模块。通过渲染粗粒度深度测试模块、前深度测试模块和后深度测试模共同对片段进行深度测试，可以能够提升对待渲染数据的剔除率，从而降低计算资源消耗量，以提升GPU的渲染速度。
在一种可能的实现方式中，所述渲染粗粒度深度测试模块具体用于：根据全局信息中目标像素块的深度信息和所述待渲染片段的深度信息对所述待渲染片段进行所述第一深度测试，所述目标像素块为所述多个像素块中与所述待渲染片段存在重叠的像素块，所述全局深度信息包括根据全部待渲染图元更新得到的多个像素块的深度信息。
在一种可能的实现方式中，所述装置还包括粗粒度深度缓存模块，所述粗粒度深度缓存模块用于缓存所述全局深度信息。
在一种可能的实现方式中，所述装置还包括分区管线，所述分区管线设置有粗粒度光栅化器和分区粗粒度深度测试模块；所述粗粒度光栅化器，用于将输入所述分区管道的待渲染图元投射至多个像素块生成多个投射区域，所述多个投射区域与所述多个像素块一一对应；所述分区粗粒度深度测试模块，用于根据所述多个投射区域对所述待渲染图元进行第二深度测试。
在一种可能的实现方式中，所述分区粗粒度深度测试模块还用于：根据所述多个投射区域更新全局深度信息，所述全局深度信息包括根据全部待渲染图元更新得到的多个像素块的深度信息。
用户可以在片段按深度值由小到大的顺序输入渲染管线的场景，关闭渲染管线中的渲染粗粒度深度测试模块，以减少渲染过程的数据处理量。
用户可以在不涉及深度的场景(如平面场景、二维场景等)关闭渲染管线中的全部深度测试模块(即渲染粗粒度深度测试模块、前深度测试模块和后深度测试模块)使待渲染数据直接进入像素着色器进行着色渲染，以减少渲染过程的数据处理量。
用户可以在片段的深度值不受像素着色器影响(即片段的深度值在像素着色器着色渲染前后不会发生变化)的场景，关闭后深度测试模块，以减少渲染过程的数据处理量。
用户可以在片段的深度值受像素着色器影响(即片段的深度值在像素着色器着色渲染前后会发生变化)且片段的深度值变化方向不固定的场景(例如，部分片段的深度值在作色渲染后增大，另一部分片段的深度值在着色渲染后减少)，关闭前深度测试模块，以减少渲染过程的数据处理量。
第三方面，本申请还提供了一种图像处理方法，该方法包括：首先将输入渲染管线的图元转换为待渲染片段。然后根据全局深度信息对所述待渲染片段进行第一深度测试，之后若所述待渲染片段未通过所述第一深度测试，则剔除所述待渲染片段。其中，所述全局深度信息包括根据全部待渲染图元更新得到的多个像素块的深度信息。
在一种可能的实现方式中，所述全局深度信息包括多个像素块的深度信息，所述根据全局深度信息对所述待渲染片段进行第一深度测试，包括：根据目标像素块的深度信息和所述待渲染片段的深度信息对所述待渲染片段进行所述第一深度测试，所述目标像素块为所述多个像素块中与所述待渲染片段存在重叠的像素块。
在一种可能的实现方式中，所述根据目标像素块的深度信息和所述待渲染片段的深度信息对所述待渲染片段进行所述第一深度测试，包括：在所述待渲染片段的深度值大于所述目标像素块的深度值的情况下，确定所述待渲染片段未通过第一深度测试。
在一种可能的实现方式中，所述目标像素块包括目标区域，所述目标像素块的深度值包括第一深度值和第二深度值，所述第一深度值为所述目标区域的最大深度值，所述第二深度值为所述目标像素块中非目标区域的最大深度值。
可选地，像素块(如目标像素块)可以包括目标区域。像素块的深度信息可以包括像素块的目标区域的最大深度值、像素块的非目标区域的最大深度值和像素块的目标区域的范围。
可选地，片段(如待渲染片段)的深度信息可以包括片段的深度值。其中，片段的深度值可以是通过插值得到的。
在一种可能的实现方式中，所述方法还包括：将输入分区管道的待渲染图元投射至多个像素块生成多个投射区域，所述多个投射区域与所述多个像素块一一对应；根据所述多个投射区域对所述待渲染图元进行第二深度测试；若所述待渲染图元未通过所述第二深度测试，则剔除所述待渲染图元。
在一种可能的实现方式中，所述根据所述多个投射区域对所述待渲染图元进行第二深度测试，包括：根据第一像素块的深度信息和第一投射区域的深度信息确定所述待渲染图元在所述第一像素块中是否通过深度测试，所述第一像素块为所述多个像素块中的任一像素块，所述第一投射区域为所述第一像素块对应的投射区域；若所述待渲染图元在所述第一像素块中通过深度测试，则确定所述待渲染图元通过所述第二深度测试；若所述待渲染图元在所述多个像素块中均未通过深度测试，则确定所述待渲染图元未通过所述第二深度测试。
可选地，投射区域(如第一投射区域)的深度信息可以包括投射区域的最大深度值、投射区域的最小深度值和投射区域的范围。
在一种可能的实现方式中，所述根据第一像素块的深度信息和第一投射区域的深度信 息确定所述待渲染图元在所述第一像素块中是否通过深度测试，包括：在所述第一投射区域的最小深度值小于所述第一像素块的深度值的情况下，确定所述待渲染图元在所述第一像素块中通过所述深度测试。
在一种可能的实现方式中，所述第一像素块包括目标区域，所述第一像素块的深度值包括所述第一像素块的目标区域的最大深度值和所述第一像素块的非目标区域的最大深度值。
在一种可能的实现方式中，所述方法还包括：根据所述多个投射区域更新所述全局深度信息。
在一种可能的实现方式中，所述根据所述多个投射区域更新所述全局深度信息，包括：根据第一像素块的深度信息和第一投射区域的深度信息更新所述全局深度信息中所述第一像素块的深度信息，所述第一像素块为所述多个像素块中的任一像素块，所述第一投射区域为所述第一像素块对应的投射区域。
在一种可能的实现方式中，所述根据第一像素块的深度信息和第一投射区域的深度信息更新所述全局深度信息中所述第一像素块的深度信息，包括：根据所述第一投射区域的最大深度值、所述第一像素块的目标区域的最大深度值、所述第一像素块的非目标区域的最大深度值，更新所述第一像素块的深度信息。
在一种可能的实现方式中，所述方法还包括：对通过第一深度测试的片段进行前深度测试。
在另一种可能的实现方式中，所述方法还包括：对通过第一深度测试的片段进行后深度测试。
在又一种可能的实现方式中，所述方法还包括：对通过第一深度测试的片段进行前深度测试和后深度测试。
第四方面，本申请还提供了一种图形处理器，该图形处理器包括述各方面或其任意可能的实现方式中所述的电子设备，该图形处理器用于实现上述各方面或其任意可能的实现方式中所述的方法。
第五方面，本申请还提供一种电子设备，该电子设备包括：存储器、至少一个处理器、收发器及存储在该存储器上并可在该处理器上运行的指令。进一步，该存储器、该处理器以及该通信接口之间通过内部连接通路互相通信。该至少一个处理器执行该指令使得该图像处理装置实现上述各方面或其任意可能的实现方式中所述的方法。
示例性地，该电子设备为手机。
第六方面，本申请还提供一种计算机可读存储介质，用于存储计算机程序，该计算机程序包括用于实现上述各方面或其任意可能的实现方式中所述的方法。
第七方面，本申请还提供一种包含指令的计算机程序产品，当其在计算机上运行时，使得计算机实现上述各个方面或其任意可能的实现方式中所述的方法。
本实施例提供的图像处理方法、图像处理装置、图形处理器、电子设备、计算机存储介质和计算机程序产品均用于执行上文所提供的图像处理方法，因此，其所能达到的有益效果可参考上文所提供的图像处理方法中的有益效果，此处不再赘述。
附图说明
图1为本申请实施例提供的一种深度示意图；
图2为一种渲染管线的结构示意图；
图3为本申请实施例提供的一种图像处理装置的结构示意图；
图4为本申请实施例提供的一种图像处理方法的流程示意图；
图5为本申请实施例提供的另一种图像处理方法的流程示意图；
图6为本申请实施例提供的一种投射过程的示意图；
图7为本申请实施例提供的一种像素块的示意图；
图8为本申请实施例提供的另一种像素块的示意图；
图9为本申请实施例提供的一种深度值的示意图；
图10为本申请实施例提供的又一种像素块的示意图；
图11为本申请实施例提供的另一种深度值的示意图；
图12为本申请实施例提供的又一种像素块的示意图；
图13为本申请实施例提供的又一种像素块的示意图；
图14为本申请实施例提供的又一种深度值的示意图；
图15为本申请实施例提供的又一种像素块的示意图；
图16为本申请实施例提供的又一种深度值的示意图；
图17为本申请实施例提供的又一种像素块的示意图；
图18为本申请实施例提供的又一种深度值的示意图；
图19为本申请实施例提供的又一种像素块的示意图；
图20为本申请实施例提供的一种包含像素块的画面的示意图；
图21为本申请实施例提供的又一种像素块的示意图；
图22为本申请实施例提供的另一种图像处理装置的结构示意图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
本申请中术语“和/或”，仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。
本申请的说明书以及附图中的术语“第一”和“第二”等是用于区别不同的对象，或者用于区别对同一对象的不同处理，而不是用于描述对象的特定顺序。
此外，本申请的描述中所提到的术语“包括”和“具有”以及它们的任何变形，意图在于覆盖不排他的包含。例如包含了一系列步骤或单元的过程、方法、系统、产品或设备没有限定于已列出的步骤或单元，而是可选地还包括其他没有列出的步骤或单元，或可选地还包括对于这些过程、方法、产品或设备固有的其他步骤或单元。
需要说明的是，本申请实施例的描述中，“示例性地”或者“例如”等词用于表示作例子、例证或说明。本申请实施例中被描述为“示例性地”或者“例如”的任何实施例或 设计方案不应被解释为比其他实施例或设计方案更优选或更具优势。确切而言，使用“示例性地”或者“例如”等词旨在以具体方式呈现相关概念。
在本申请的描述中，除非另有说明，“多个”的含义是指两个或两个以上。
首先，对本申请中的部分用语进行解释说明，以便于本领域技术人员理解。
深度：用于表征物体与摄像机之间远近程度。如图1所示，远近程度用于表示根据摄像机朝向做线段得到的投影距离。
深度测试：判断待渲染数据的深度值是否大于深度阈值的过程。
前深度测试：在数据(如片段)未经像素着色器着色渲染之前对数据进行的深度测试。
后深度测试：在数据经像素着色器着色渲染之后对数据进行的深度测试。
图元(primitive)：图形渲染管线中几何表示的基本单位，几何顶点的组合称为图元(如点、线段或多边形等)。
光栅化：光栅化可以将图元转化为片段，片段拥有关于像素的信息。光栅化过程确定了所有像素需要绘制的位置，光栅化过程会进行插值，以便确定两个顶点间的所有像素的位置。光栅化不仅可以对像素插值，任何顶点着色器的输出变量和片段着色器的输入变量都可以基于对应的像素插值。通过光栅化可以实现平滑的颜色渐变和真实光照等效果。
分区阶段(binning pass)：将屏幕分为若干区域(bin)，将所有需要进行渲染的图元进行视角变换之后得到该图元的屏幕位置，根据图元的屏幕位置将图元筛选到对应屏幕区域，最终每个区域(bin)独立进行渲染，只渲染分区过程中落到本区域的图元集合。这个筛选过程就是分区阶段。
渲染过程中GPU会消耗大量计算资源。为此，GPU可以在渲染阶段剔除不影响渲染结果的待渲染数据，以降低计算资源消耗量。
在相关技术中，可以采用深度测试剔除不影响渲染结果的待渲染数据。例如，在如图2所示的图像处理器的渲染管线中，渲染管线中的前深度测试模块可以根据深度缓存中的深度值对输入前深度测试模块的片段进行前深度测试(earlylate-detph-test)。当该片段的深度值小于深度缓存中的深度值时，确定该片段通过前深度测试之后将该片段输入像素着色器进行着色渲染并将深度缓存中的深度值更新为该片段的深度值；当该片段的深度值大于深度缓存中的深度值时，确定该片段未通过前深度测试并剔除该片段。在一部分场景中，比片段的深度值会受到像素着色器影响，深度测试需要在像素着色器，这时就需要后深度测试模式进行后深度测试(late-detph-test)。
可以看出相关技术中，深度缓存中的深度值是根据已通过测试的部分待渲染片段确定的，即相关技术在深度测试时仅能利用部分片段的深度信息对待渲染片段进行剔除，所以无法达到很高的剔除率。例如，待渲染片段包括待渲染片段1、待渲染片段2、待渲染片段3、待渲染片段4和待渲染片段5，深度值分别为50、40、30、20、10。如果同时利用这5个待渲染片段的深度值进行深度测试，则只有深度值最小的待渲染片段5会通过深度测试，其余4个待渲染片段则会被剔除。
如表1所示，通过现有技术对待渲染片段1进行深度测试时，深度缓存中未缓存有深度值，则待渲染片段1通过深度测试进入着色器中进行着色渲染，深度缓存中的深度值更新为50。通过现有技术对待渲染片段2进行深度测试时，待渲染片段2的深度值40小于深度缓存中的深度值50，则待渲染片段2通过深度测试进入着色器中进行着色渲染，深 度缓存中的深度值更新为40。
通过现有技术对待渲染片段3进行深度测试时，待渲染片段3的深度值30小于深度缓存中的深度值50，则待渲染片段3通过深度测试进入着色器中进行着色渲染，深度缓存中的深度值更新为30。
通过现有技术对待渲染片段4进行深度测试时，待渲染片段4的深度值20小于深度缓存中的深度值50，则待渲染片段3通过深度测试进入着色器中进行着色渲染，深度缓存中的深度值更新为20。
通过现有技术对待渲染片段5进行深度测试时，待渲染片段5的深度值10小于深度缓存中的深度值50，则待渲染片段3通过深度测试进入着色器中进行着色渲染，深度缓存中的深度值更新为10。
通过表1可以看出，待渲染片段1、待渲染片段2、待渲染片段3、待渲染片段4和待渲染片段5均能通过深度测试，相关技术仅利用通过深度测试的部分片段的深度信息对待渲染片段进行剔除无法达到较高的剔除率。
表1
Figure PCTCN2021140588-appb-000001
为此，本申请实施例提供了一种图像处理方法，能够提升对待渲染数据的剔除率。
本申请实施例提供的图像处理方法适用于图像处理装置。该图像处理装置可以设置在图形处理器中。
具体地，该图像处理装置可以设置于电子设备的图形处理器中。本申请实施例对电子设备的具体类型不作任何限制。例如，电子设备可以为手机、平板电脑、笔记本电脑、超级移动个人计算机(ultra-mobile personal computer，UMPC)、上网本、个人数字助理(personal digital assistant，PDA)等。
示例性地，图像处理装置可以设置在手机的图形处理器中。
图3示出了该图像处理装置的一种示意性框图，该图像处理装置包括分区管线、全局深度信息缓存模块和渲染管线。
在分区管线中，屏幕空间会被分为多个像素块(pixel block),每个CPU调用图形编程接口(draw-call)中的每一个图元(primitive)经过光栅化(rasterization)之后会被投射到上述多个像素块上。例如，屏幕空间可以分为N个像素块。
分区管线中设置有粗粒度光栅化器和分区粗粒度深度测试模块。
粗粒度光栅化器(也可称为第二光栅化器)，用于将输入分区管线中的图元(即待渲染图元)投射至多个像素块生成多个投射区域。例如，第二光栅化器可以将输入分区管线中的图元投射至N个像素块中生成N个投射区域。其中，多个像素块和多个投射区域一一对应。
分区粗粒度深度测试模块(也可称为第二深度测试模块)，用于根据多个投射区域对待渲染图元进行分区粗粒度深度测试(也可称为第二深度测试)。若待渲染图元未通过第二深度测试，则剔除待渲染图元。若待渲染图元未通过第二深度测试，则待渲染图元通过第二深度测试，则将待渲染图元输入渲染管线。
在一种可能的实现方式中，分区管线还可以设置有多边形列表生成(polygon list generation)模块，polygon list generation模块用于通过内存存储通过第二深度测试的待渲染图元，以及将通过第二深度测试的待渲染图元打包为多边形列表(PL)，以多边形堆栈(PL-Heap)的形式输入渲染管线。
在一种可能的实现方式中，第二深度测试模块可以在第二深度测试过程中更新每个像素块上所累积的深度信息，并将每个像素块的深度信息存储到全局深度信息缓存模块中。
全局深度信息缓存模块(可简称为第一缓存模块)，用于缓存上述多个像素块的深度信息(可简称为全局深度信息)，像素块的深度信息包括待定区掩膜(pending mask，Pmask)、待定目标深度值(pending destination z value，PZ)和原始目标深度值(old Destination z value，OZ)。例如，第一缓存模块缓存有全局深度信息，该全局深度信息包括通过全部待渲染图元更新得到的多个(如N个)像素块的深度信息。
全局深度信息缓存模块可以为图像处理装置内部的缓存，也可以是其他设备的缓存(如CPU的缓存)。
Pmask，用于指示像素块中的目标区域。Pmask可以标识像素块中目标区域的范围。
PZ，用于指示像素块中目标区域的最大深度值，即像素块的目标区域中所有可见片段的深度值都比PZ值小。
OZ，用于指示像素块中非目标区域的最大深度值，即像素块的非目标区域中所有可见片段的深度值都比OZ值小。
渲染管线设置有光栅化器、渲染粗粒度深度测试模块和像素着色器。
光栅化器(可称为第一光栅化器)，用于将输入渲染管线的图元(即通过第二深度测试的图元)转换为片段(即待渲染片段)。
渲染粗粒度深度测试模块(可称为第一深度测试模块)，用于根据上述全局深度信息对待渲染片段进行第一深度测试。若待渲染片段未通过第一深度测试，则剔除待渲染片段。若待渲染片段通过第一深度测试，则将待渲染片段输入像素着色器。
在一种可能的实现方式中，渲染粗粒度深度测试模块也可以通过其他信息对待渲染片段进行第一深度测试。
像素着色器，用于对输入像素着色器的片段进行着色渲染。
在一种可能的实现方式中，渲染管线中还可以设置有前深度测试模块、后深度测试模块和深度缓存模块。
前深度测试模块，用于对待渲染片段进行前深度测试。前深度测试的具体方法可以采用本领域技术人员能够想到的任何一种方法进行处理，本申请实施例对此不做具体限定。
后深度测试模块，用于对着色渲染后的片段进行后深度测试。后深度测试的具体方法可以采用本领域技术人员能够想到的任何一种方法进行处理，本申请实施例对此不做具体限定。
在一种可能的实现方式中，第二深度模块具体用于：根据第一像素块的深度信息和第 一投射区域的深度信息确定待渲染图元在第一像素块中是否通过深度测试，第一像素块为多个像素块中的任一像素块，第一投射区域为第一像素块对应的投射区域。若待渲染图元在第一像素块中通过深度测试，则确定待渲染图元通过第二深度测试；若待渲染图元在多个像素块中均未通过深度测试，则确定待渲染图元未通过第二深度测试。
在一种可能的实现方式中，第二深度模块具体用于：在所述第一投射区域的最小深度值小于所述第一像素块的深度值的情况下，确定所述待渲染图元在所述第一像素块中通过所述深度测试。
可选地，投射区域(如第一投射区域)的深度信息可以包括投射区域的最大深度值、投射区域的最小深度值和投射区域的范围。
在一种可能的实现方式中，所述第一像素块包括目标区域，所述第一像素块的深度值包括所述第一像素块的目标区域的最大深度值和所述第一像素块的非目标区域的最大深度值，第二深度模块具体用于：
在第一投射区域与第一像素块的目标区域和非目标区域均存在重叠且第一投射区域的最小深度值小于第一深度值或第二深度值，则确定待渲染图元在第一像素块中通过测试。
在第一投射区域与第一像素块的目标区域和非目标区域均存在重叠且第一投射区域的最小深度值大于第一深度值和第二深度值的情况下，确定待渲染图元在第一像素块中未通过测试
在第一投射区域仅与第一像素块的目标区域存在重叠且第一投射区域的最小深度值小于第一深度值的情况下，确定待渲染图元在第一像素块中通过测试。
在第一投射区域仅与第一像素块的目标区域存在重叠且第一投射区域的最小深度值大于第一深度值的情况下，确定待渲染图元在第一像素块中未通过测试。
在第一投射区域仅与第一像素块的非目标区域存在重叠且第一投射区域的最小深度值小于第二深度值的情况下，确定待渲染图元在第一像素块中通过测试。
在第一投射区域仅与第一像素块的非目标区域存在重叠且第一投射区域的最小深度值大于第二深度值的情况下，确定待渲染图元在第一像素块中未通过测试。
在一种可能的实现方式中，第二深度模块还用于根据多个投射区域更新全局深度信息。
在一种可能的实现方式中，第二深度模块还具体用于：根据第一像素块的深度信息和第一投射区域的深度信息更新全局深度信息中第一像素块的深度信息，第一像素块为多个像素块中的任一像素块，第一投射区域为第一像素块对应的投射区域。
在一种可能的实现方式中，第一像素块包括目标区域，第二深度模块还具体用于：根据第一投射区域的最大深度值、第一像素块的目标区域的最大深度值、第一像素块的非目标区域的最大深度值，更新第一像素块的深度信息。
在一种可能的实现方式中，第二深度模块还具体用于：
在第一投射区域与第一像素块的目标区域和非目标区域均存在重叠且第一投射区域的最大深度值小于第一深度值或第二深度值的情况下，更新第一像素块的深度信息。
在第一投射区域与第一像素块的目标区域和非目标区域均存在重叠且第一投射区域的最大深度值大于第一深度值和第二深度值的情况下，不更新第一像素块的深度信息。
在第一投射区域仅与第一像素块的目标区域存在重叠且第一投射区域的最小深度值小于第一深度值的情况下，更新第一像素块的深度信息。
在第一投射区域仅与第一像素块的目标区域存在重叠且第一投射区域的最小深度值大于第一深度值的情况下，不更新第一像素块的深度信息。
在第一投射区域仅与第一像素块的非目标区域存在重叠且第一投射区域的最小深度值小于第二深度值的情况下，更新第一像素块的深度信息。
在第一投射区域仅与第一像素块的非目标区域存在重叠且第一投射区域的最小深度值大于第二深度值的情况下，不更新第一像素块的深度信息。
在一种可能的实现方式中，第二深度模块还具体用于：
在第一投射区域与第一像素块的目标区域完全重合且第三深度值(即投射区域的最大深度值)小于第一深度值的情况下，将第一像素块的目标区域更新为目标投射区域且将第一像素块的第一深度值更新为第三深度值。
在第一投射区域与第一像素块的非目标区域完全重合且第三深度值小于第二深度值的情况下，将第一像素块的目标区域更新为非目标投射区域且将第一像素块的第二深度值更新为第三深度值。
在第一投射区域与第一像素块的目标区域和非目标区域均不完全重合且满足第一条件和第二条件的情况下，将第一像素块的目标区域更新为第一区域且将第一像素块的第一深度值更新为第一数值。其中，上述第一条件为第三深度值小于第一深度值但大于第二深度值或第三深度值小于第二深度值但大于第一深度值。上述第二条件为第一绝对值小于第二绝对值，上述第一绝对值为上述第一深度值与上述第三深度值之差的绝对值，上述第二绝对值为上述第二深度值与上述第三深度值之差的绝对值。上述第一区域为目标区域与目标投射区域的并集区域，第一数值为第一深度值与第三深度值之间的较大值。
在第一投射区域与第一像素块的目标区域和非目标区域均不完全重合且满足第一条件和第三条件的情况下，将第一像素块的目标区域更新为第二区域且将第一像素块的第二深度值更新为第二数值。其中，上述第三条件为第一绝对值大于第二绝对值。上述第二区域为目标区域与非目标投射区域的交集区域。上述第二数值为第二深度值与第三深度值之间的较大值。
在第一投射区域与第一像素块的目标区域和非目标区域均不完全重合且第三深度值小于第一深度值和第二深度值的情况下，将第一像素块的目标区域更新为目标投射区域且将第一像素块的第一深度值更为第三深度值，将第一像素块的第二深度值更新为第三数值。其中，第三数值为第一深度值与第二深度值之间的较大值。
可选地，目标投射区域(CurMask)包括第一目标投射区域(即CurMask0)和第二目标投射区域(即CurMask0)
在投射区域的第三深度值小于像素块的第二深度值的情况下，第一目标投射区域(即CurMask0)可以表示投射区域与像素块的非目标区域的重叠区域。
在投射区域的第三深度值小于像素块的第一深度值的情况下，第二目标投射区域(即CurMask0)可以表示投射区域与像素块的目标区域的重叠区域。
目标投射区域(CurMask)可以为第一目标投射区域(CurMask0)和第二目标投射区域(CurMask0)的并集。
在一种可能的实现方式中，第一深度测试模块具体用于：根据全局信息中目标像素块的深度信息和待渲染片段的深度信息对待渲染片段进行第一深度测试，目标像素块为多个像素块中与待渲染片段存在重叠的像素块。
在一种可能的实现方式中，第一深度测试模块具体用于：在所述待渲染片段的深度值大于所述目标像素块的深度值的情况下，确定所述待渲染片段未通过第一深度测试。
可选地，像素块(如目标像素块)可以包括目标区域。像素块的深度信息可以包括像素块的目标区域的最大深度值、像素块的非目标区域的最大深度值和像素块的目标区域的范围。
可选地，片段(如待渲染片段)的深度信息可以包括片段的深度值。其中，片段的深度值可以是通过插值得到的。
在一种可能的实现方式中，目标像素块包括目标区域，所述目标像素块的深度值包括第一深度值和第二深度值，第一深度测试模块具体用于：
在待渲染片段位于目标像素块的目标区域且待渲染片段的深度值小于或等于第一深度值的情况下，确定待渲染片段通过第一深度测试。
在待渲染片段位于目标像素块的目标区域且待渲染片段的深度值大于第一深度值的情况下，确定待渲染片段未通过第一深度测试。
在待渲染片段位于目标像素块的非目标区域且待渲染片段的深度值小于或等于第二深度值的情况下，确定待渲染片段通过第一深度测试。
在待渲染片段未位于目标像素块的非目标区域且待渲染片段的深度值大于第二深度值的情况下，确定待渲染片段未通过第一深度测试。
如图3所示，用户可以在不涉及深度的场景(如平面场景、二维场景等)关闭图像处理装置中渲染管线中的全部深度测试模块(即渲染粗粒度深度测试模块、前深度测试模块和后深度测试模块)使待渲染数据直接进入像素着色器进行着色渲染，以减少渲染过程的数据处理量。
用户可以在片段按深度值由小到大的顺序输入渲染管线的场景，关闭渲染管线中的渲染粗粒度深度测试模块，以减少渲染过程的数据处理量。
用户可以在片段的深度值不受像素着色器影响(即片段的深度值在像素着色器着色渲染前后不会发生变化)的场景，关闭后深度测试模块，以减少渲染过程的数据处理量。
用户可以在片段的深度值受像素着色器影响(即片段的深度值在像素着色器着色渲染前后会发生变化)且片段的深度值变化方向不固定的场景(例如，部分片段的深度值在着色渲染后增大，另一部分片段的深度值在着色渲染后减少)，关闭前深度测试模块，以减少渲染过程的数据处理量。
图4示出了本申请实施例提供的图像处理方法的示意性流程图，该方法可以由图3所示的图像处理装置执行。如图4所示，该图像处理方法包括：
S401、图像处理装置对输入分区管线的待渲染图元进行第二深度测试并剔除未通过第二深度测试的待渲染图元。
示例性地，以M个待渲染图元输入分区管线为例，则图像处理装置先对第1个待渲染图元进行第二深度测试，接着对第2个进行第二深度测试，然后对第3个、第4个、……、第M个待渲染图元进行第二深度测试。若待渲染图元未通过第二深度测试，则剔除该待 渲染图元；若该待渲染图元通过第二测试，则将该待渲染图元输入渲染管线。
需要说明的是，待渲染图元未通过第二深度测试说明该待渲染图元深度较高会被其他待渲染图元所遮挡，所以可以不对该图元进行渲染，而是将该图元剔除以降低计算资源消耗量。
在一种可能的实现方式中，图像处理装置对每个待渲染图元进行第二深度测试可以包括：先通过粗粒度光栅化器将该待渲染图元投射至由屏幕空间分成的多个像素块生成多个投射区域，然后根据多个投射区域对待渲染图元进行第二深度测试。其中，上述多个投射区域与上述多个像素块一一对应。
示例性地，如图6所示，屏幕空间可以分成16个像素块(图中白色区域)，图像处理装置可以将待渲染图元投射分别投射至这16个像素块中生成与16个像素块一一对应的16个投射区域(图中黑色区域)，然后根据这16个投射区域对待渲染图元进行第二深度测试。
在一种可能的实现方式中，图像处理装置可以根据每个像素块的深度信息和每个像素块对应的投射区域的深度信息确定待渲染图元在每个像素块中是否通过深度测试。若该待渲染图元在任一像素块中通过深度测试，则确定该待渲染图元通过第二深度测试；若该待渲染图元在上述多个像素块中均未通过深度测试，则确定待渲染图元未通过第二深度测试。
示例性地，如图6所示，图像处理装置可以根据图6中16个像素块的深度信息和图6中16个投射区域的深度信息确定待渲染图元在图6中16个像素块中每个像素块中是否通过深度测试。若该待渲染图元在任一像素块中通过深度测试，则确定该待渲染图元通过第二深度测试。若该待渲染图元在上述16个像素块中均未深度通过测试，则确定待渲染图元未通过第二深度测试。
可选地，像素块可以包括目标区域，像素块的深度信息可以包括像素块的目标区域的最大深度值(可简称为第一深度值)、像素块的非目标区域的最大深度值(可简称为第二深度值)和像素块的目标区域的范围。
可选地，投射区域的深度信息可以包括投射区域的最大深度值和投射区域的最小深度值。
在一种可能的实现方式中，图像处理装置可以根据每个像素块的目标区域的最大深度值、每个像素块的非目标区域的最大深度值和每个像素块对应的投射区域的最小深度值确定待渲染图元在每个像素块中是否通过深度测试。
图7示出了像素块和投射区域，图7中灰色区域为像素块的目标区域，白色区域为像素块的非目标区域，黑色区域为投射区域。通过图7可以看出像素块与投射区域一共存在三种位置关系。图7中(a)图示出了第一种位置关系即投射区域与像素块的目标区域和非目标区域均存在重叠；图7中(b)图示出了第二种位置关系即投射区域仅与像素块的目标区域存在重叠；图7中(c)图示出了第三种位置关系即投射区域仅与像素块的非目标区域存在重叠。
下面以结合图7，以第一像素块和第一投射区域为例，介绍图像处理装置根据每个像素块的目标区域的最大深度值、每个像素块的非目标区域的最大深度值和每个像素块对应的投射区域的最小深度值确定待渲染图元在每个像素块中是否通过深度测试的具体过程。 其中，第一像素块为多个像素块中的任一像素块，第一投射区域为第一像素块对应的投射区域。
如图7中(a)图所示，在第一投射区域与第一像素块的目标区域和非目标区域均存在重叠且第一投射区域的最小深度值小于第一深度值或第二深度值的情况下，确定待渲染图元在第一像素块中通过深度测试。
如图7中(a)图所示，在第一投射区域与第一像素块的目标区域和非目标区域均存在重叠且第一投射区域的最小深度值大于第一深度值和第二深度值的情况下，确定待渲染图元在第一像素块中未通过深度测试。
如图7中(b)图所示，在第一投射区域仅与第一像素块的目标区域存在重叠且第一投射区域的最小深度值小于第一深度值的情况下，确定待渲染图元在第一像素块中通过深度测试。
如图7中(b)图所示，在第一投射区域仅与第一像素块的目标区域存在重叠且第一投射区域的最小深度值大于第一深度值的情况下，确定待渲染图元在第一像素块中未通过深度测试。
如图7中(c)图所示，在第一投射区域仅与第一像素块的非目标区域存在重叠且第一投射区域的最小深度值小于第二深度值的情况下，确定待渲染图元在第一像素块中通过深度测试。
如图7中(c)图所示，在第一投射区域仅与第一像素块的非目标区域存在重叠且第一投射区域的最小深度值大于第二深度值情况下，确定待渲染图元在第一像素块中未深度通过测试。
在一种可能的实现方式中，图像处理装置还可以根据上述多个投射区域更新全局深度信息。其中，全局深度信息包括由屏幕空间分成的多个像素块的深度信息。例如，图像处理装置可以根据每个像素块的深度信息(当前的深度信息)和每个像素块对应的投射区域的深度信息更新全局深度信息中每个像素块的深度信息。
下面以结合图7，以第一投射区域和第一像素块为例，介绍图像处理装置根据第一投射区域的最大深度值、第一像素块的目标区域的最大深度值(即第一深度值)、第一像素块的非目标区域的最大深度值(即第二深度值)，更新第一像素块的深度信息的具体过程。
如图7中(a)图所示，在第一投射区域与第一像素块的目标区域和非目标区域均存在重叠且第一投射区域的最大深度值小于第一深度值或第二深度值的情况下，更新第一像素块的深度信息。
如图7中(a)图所示，在第一投射区域与第一像素块的目标区域和非目标区域均存在重叠且第一投射区域的最大深度值大于第一深度值和第二深度值的情况下，不更新第一像素块的深度信息。
如图7中(b)图所示，在第一投射区域仅与第一像素块的目标区域存在重叠且第一投射区域的最小深度值小于第一深度值的情况下，更新第一像素块的深度信息。
如图7中(b)图所示，在第一投射区域仅与第一像素块的目标区域存在重叠且第一投射区域的最小深度值大于第一深度值的情况下，不更新第一像素块的深度信息。
如图7中(c)图所示，在第一投射区域仅与第一像素块的非目标区域存在重叠且第一投射区域的最小深度值小于第二深度值的情况下，更新第一像素块的深度信息。
如图7中(c)图所示，在第一投射区域仅与第一像素块的非目标区域存在重叠且第一投射区域的最小深度值大于第二深度值的情况下，不更新第一像素块的深度信息。
在一种可能的实现方式中，图像处理装置可以根据目标区域更新第一像素块的深度信息。其中，目标投射区域(CurMask)包括第一目标投射区域(即CurMask0)和第二目标投射区域(即CurMask0)。
CurMask0：待渲染图元的投射区域与像素块的非目标区域(也可称为OZ区域)的重叠区域中，如果该投射区域的最大深度(可称为第三深度值，也可简称为Zmax)小于像素块的非目标区域的最大深度值(即第二深度值，可简称为OZ)，那么上述重叠区域中像素块的所有可见片段的深度值一定比Zmax值小(否则会被该图元遮蔽)，那么该像素块的OZ值就有条件被更新为该待渲染图元的投射区域的Zmax值。这种情况下可以用CurMask0掩码表示这部分区域的覆盖范围。即在投射区域的第三深度值小于像素块的第二深度值的情况下，CurMask0可以表示投射区域与像素块的非目标区域的重叠区域。
CurMask1：待渲染图元的投射区域与像素块的目标区域(也可称为PZ区域)的重叠区域中，如果该投射区域的最大深度小于像素块的目标区域的最大深度值(即第一深度值，可简称为PZ)，那么上述重叠区域中像素块的所有可见片段的深度值一定比Zmax值小(否则会被该图元遮蔽)，那么该像素块的PZ值就有条件被更新为该待渲染图元的投射区域的Zmax值。这种情况下可以用CurMask1掩码表示这部分区域的覆盖范围。即在投射区域的第三深度值小于像素块的第一深度值的情况下，CurMask1可以表示投射区域与像素块的目标区域的重叠区域。
CurMask：CurMask0和CurMask1的并集。
layer0Mask：表示原始OZ区域(即更新前的OZ区域中)中非CurMask0指示的区域。
layer1Mask：表示原始PZ区域(即更新前的PZ区域中)中非CurMask1指示的区域。
具体地，图像处理装置更新第一像素块的深度信息，可以包括：
在第一投射区域与第一像素块的目标区域完全重合且第三深度值(即投射区域的最大深度值)小于第一深度值的情况下，将第一像素块的目标区域更新为目标投射区域且将第一像素块的第一深度值更新为第三深度值。
示例性地，图8中(a)图示出了第一像素块更新前的目标区域(灰色区域)和非目标区域(白色区域)，图8中(b)图示出了第一投射区域(黑色区域)和第一像素块。通过图8中(b)图可以看出第一投射区域与第一像素块的目标区域完全重合。图9中(a)图示出了第一深度值(PZ)、第二深度值(OZ)以及第一投射区域的深度范围，图9中横轴表示深度值，从左往右深度值递增。通过图9中(a)图可以看出第一投射区域的最大深度值(Zmax)小于第一深度值(PZ)和第二深度值(OZ)，所以目标投射区域包括整个第一投射区域。相应地，如图8中(c)图所示，图像处理装置将第一像素块的目标区域更新为目标投射区域。如图9中(b)图所示，图像处理装置将第一像素块的第一深度值(PZ)更新为第三深度值(Zmax)。
在第一投射区域与第一像素块的非目标区域完全重合且第三深度值小于第二深度值的情况下，将第一像素块的目标区域更新为非目标投射区域且将第一像素块的第二深度值更新为第三深度值。
示例性地，参照图10中(a)和(b)图可以看出，第一投射区域与第一像素块的非 目标区域完全重合。参照图11中(a)图可以看出，第三深度值小于第二深度值但大于第一深度值，则目标投射区域包括第一投射区域与第一像素块的非目标区域的重合区域(即整个非目标区域)。如图8中(c)图所示，由于非目标投射区域与第一像素块的目标区域相同，图像处理装置则不更新目标区域。如图11中(b)图所示，图像处理装置将第一像素块的第二深度值(OZ)更新为第三深度值(Zmax)。
又示例性地，参照图12中(a)和(b)图可以看出，第一投射区域与第一像素块的非目标区域完全重合。若第一投射区域的最大深度值(Zmax)小于第一深度值(PZ)和第二深度值(OZ)，则目标投射区域包括整个第一投射区域。那么如图12中(c)图所示，图像处理装置将第一像素块的目标区域更新为非目标投射区域(即第一像素块中未与第一投影区域重叠的区域)并将第一像素块的第二深度值(OZ)更新为第三深度值(Zmax)。
在第一投射区域与第一像素块的目标区域和非目标区域均不完全重合且满足第一条件和第二条件的情况下，将第一像素块的目标区域更新为第一区域且将第一像素块的第一深度值更新为第一数值。其中，上述第一条件为第三深度值小于第一深度值但大于第二深度值或第三深度值小于第二深度值但大于第一深度值。上述第二条件为第一绝对值小于第二绝对值，上述第一绝对值为上述第一深度值与上述第三深度值之差的绝对值，上述第二绝对值为上述第二深度值与上述第三深度值之差的绝对值。上述第一区域为目标区域与目标投射区域的并集区域，第一数值为第一深度值与第三深度值之间的较大值。
需要说明的是，满足第一条件和第二条件，表示第三深度值处于第一深度值和第二深度值之间，且第三深度值距离第一深度值较近。
又示例性地，参照图13中(a)和(b)图可以看出，第一投射区域与第一像素块的目标区域和非目标区域均不完全重合。参照图14中(a)图可以看出第三深度值(Zmax)处于第一深度值(PZ)和第二深度值(OZ)之间，且第三深度值距离第一深度值较近。由于第三深度值小于第二深度值但大于第一深度值，所以目标投射区域仅包括第一投射区域与非目标区域的重叠区域，第一区域则包括目标投射区域和当前目标区域。如图13中(c)图所示，图像处理装置将第一像素块的目标区域更新为第一区域。由于第三深度值大于第一深度值，则第一数值为第三深度值。如图14中(b)图所示，图像处理装置将第一像素块的第一深度值(PZ)更新为第三深度值(Zmax)。
在第一投射区域与第一像素块的目标区域和非目标区域均不完全重合且满足第一条件和第三条件的情况下，将第一像素块的目标区域更新为第二区域且将第一像素块的第二深度值更新为第二数值。其中，上述第三条件为第一绝对值大于第二绝对值。上述第二区域为目标区域与非目标投射区域的交集区域。上述第二数值为第二深度值与第三深度值之间的较大值。
需要说明的是，满足第一条件和第三条件，表示第三深度值处于第一深度值和第二深度值之间，且第三深度值距离第二深度值较近。
又示例性地，参照图15中(a)和(b)图可以看出，第一投射区域与第一像素块的目标区域和非目标区域均不完全重合。参照图16中(a)图可以看出第三深度(Zmax)值处于第一深度值(PZ)和第二深度值(OZ)之间，且第三深度值距离第二深度值较近。由于第三深度值小于第二深度值但大于第一深度值，所以目标投射区域仅包括第一投射区域与非目标区域的重叠区域。第二区域则包括非目标投射区域和当前目标区域的交集区 域。如图15中(c)图所示，由于第二区域与当前目标区域相同，则图像处理装置则不更新目标区域。由于第二深度值大于第三深度值，则第二数值为第二深度值。如图16中(b)图所示，由于第二数值和当前第二深度值(OZ)相同，图像处理装置则不更新第二深度值(OZ)。
在第一投射区域与第一像素块的目标区域和非目标区域均不完全重合且第三深度值小于第一深度值和第二深度值的情况下，将第一像素块的目标区域更新为目标投射区域且将第一像素块的第一深度值更为第三深度值，将第一像素块的第二深度值更新为第三数值。其中，第三数值为第一深度值与第二深度值之间的较大值。
又示例性地，参照图17中(a)和(b)图可以看出，第一投射区域与第一像素块的目标区域和非目标区域均不完全重合。参照图18中(a)图可以看出第三深度值(Zmax)小于第一深度值(PZ)和第二深度值(OZ)。由于第三深度值小于第一深度值和第二深度值，所以目标投射区域包括整个第一投射区域。如图17中(c)图所示，图像处理装置将第一像素块的目标区域更新目标投射区域即整个第一投射区域。由于第一深度值小于第二深度值，所以第三数值为第二深度值。如图18中(a)图所示，图像处理装置将第一像素块的第一深度值更新为第三深度值。另外，由于第一像素块的当前第二深度值与第三数值相同，图像处理装置则不更新第一像素块的第二深度值。
可选地，图像处理装置可以将通过第二深度测试的待渲染图元存储在分区管线中的多边形列表生成(polygon list generation)。
S402、图像处理装置将通过第二深度测试的待渲染图元输入渲染管线。
示例性地，以M个待渲染图元输入分区管线的待渲染图元中的K个待渲染图元通过第二深度测试为例，图像处理装置中的polygon list generation可以将通过第二深度测试的待渲染图元打包成多个多边形列表(PL)，然后以多边形堆栈(PL-Heap)的形式输入渲染管线。
S403、图像处理装置将输入渲染管线的图元转换为待渲染片段。
示例性地，图像处理装置可以通过渲染管线中的光栅化器将输入渲染管线的图元(即通过第二深度测试的图元)转换为待渲染片段。
上述将图元转换为片段的具体方法可以采用本领域技术人员能够想到的任何一种方法进行处理，本申请实施例对此不做具体限定。
S404、图像处理装置对待渲染片段进行第一深度测试并剔除未通过第一深度测试的待渲染片段。
在一种可能的实现方式中，图像处理装置中可以根据全局深度信息缓存模块中的全局深度信息对待渲染片段进行第一深度测试。
具体地，图像处理装置中可以根据全局深度信息中目标像素块的深度信息和待渲染片段的深度信息对待渲染片段进行第一深度测试。其中，目标像素块为多个像素块中与待渲染片段存在重叠的像素块。
示例性地，图19示出了9个像素块和1个待渲染片段(图中黑色方块)，可以看出左上角的像素块与待渲染片段存在重叠。图像处理装置则根据全局深度信息中左上角的像素块的深度信息和待渲染片段的深度信息对待渲染片段进行第一深度测试。
可选地，片段的深度信息可以包括片段的深度值。其中，片段的深度值可以是通过插 值得到的。
在一种可能的实现方式中，图像处理装置根据待渲染片段的深度值、第一深度值或第二深度值对待渲染片段进行第一深度测试。
在一种可能的实现方式中，图像处理装置可以根据待渲染片段的深度值、第一深度值或第二深度值对待渲染片段进行第一深度测试，可以包括：
图像处理装置在待渲染片段位于目标像素块的目标区域内且该待渲染片段的深度值小于或等于第一深度值的情况下，确定该待渲染片段通过第一深度测试。
图像处理装置在待渲染片段位于目标像素块的目标区域内且该待渲染片段的深度值大于第一深度值的情况下，确定该待渲染片段未通过第一深度测试。
图像处理装置在待渲染片段未位于目标像素块的目标区域且该待渲染片段的深度值小于或等于第二深度值的情况下，确定该待渲染片段通过第一深度测试。
图像处理装置在待渲染片段未位于目标像素块的目标区域且该待渲染片段的深度值大于第二深度值的情况下，确定该待渲染片段未通过第一深度测试。
需要说明的是，待渲染片段位于目标像素块的目标区域且待渲染片段的深度值大于第一深度值，说明该待渲染片段会被目标像素块的目标区域内的可见片段所遮挡，所以可以将该待渲染片段剔除，以降低计算资源消耗量，提升GPU的渲染速度。
待渲染片段位于非目标像素块的目标区域且待渲染片段的深度值大于第二深度值，说明该待渲染片段会被目标像素块的非目标区域内的可见片段所遮挡，所以可以将该待渲染片段剔除，以降低计算资源消耗量，提升GPU的渲染速度。
示例性地，以图20所示的画面中位于左侧框内的一个4x4大小的像素块为例，可以看出该像素块中渲染的物体包括天空，屋顶以及一个的三角形。图21中(a)图所示的灰色区域为该像素块的目标区域，黑色区域为该像素块的非目标区域。全局深度信息中该像素块的目标区域的最大深度值(PZ)为0.1，非目标区域的最大深度值(OZ)为1.0。在对三角形进行光栅化后，通过图21中(b)图可以看出该三角形的两个片段(即图中的白色方格)位于目标区域，通过图21中(c)图可以看出该三角形的两个片段(即图中的浅灰色方格)位于非目标区域。如图21中(d)图所示，由于位于目标区域的两个片段插值出来的深度值大于PZ，所以位于目标区域的两个片段会被剔除；而位于非目标区域的两个片段插值出来的深度值小于OZ，所以位于非目标区域的两个片段不会被剔除，而是会输入像素着色器中进行着色渲染。
可以看出，本申请实施例提供的方法可以通过全局深度信息对待渲染片段进行剔除，由于全局深度信息是根据全部待渲染图元确定，而待渲染片段是由待渲染图元转换得到的，所以全局深度信息相当于是根据所有待渲染片段生成的。本申请实施例利用全部待渲染片段的深度信息进行片段剔除，相较于相关技术仅通过部分待渲染片段的深度信息进行片段剔除，能够提升对待渲染数据的剔除率，从而降低计算资源消耗量，以提升GPU的渲染速度。
S405、图像处理装置将通过第一深度测试的待渲染片段输入像素着色器。
在一种可能的实现方式中，图像处理装置可以先通过前深度测试模块对通过第一深度测试的待渲染片段进行前深度测试，然后将通过前深度测试的待渲染片段输入像素着色器。
S406、图像处理装置对输入像素着色器的片段进行着色渲染。
上述着色渲染的具体方法可以采用本领域技术人员能够想到的任何一种方法进行处理，本申请实施例对此不做具体限定。
在一种可能的实现方式中，图像处理装置可以通过后深度测试模块对着色渲染后的片段进行后深度测试。
图5示出了本申请实施例提供的另一种图像处理方法的示意性流程图，该方法可以由图3所示的图像处理装置执行。如图5所示，该图像处理方法包括：
S501、图像处理装置将输入渲染管线的图元转换为待渲染片段。
S502、图像处理装置根据全局深度信息对待渲染片段进行第一深度测试
其中，全局深度信息包括根据全部待渲染图元更新得到的多个像素块的深度信息。
需要说明的是，S502的具体实现方式可以参考上述S404的描述，在此就不再赘述。
S503、若待渲染片段未通过第一深度测试，图像处理装置则剔除所述待渲染片段。
本申请实施例还提供一种计算机存储介质，该计算机存储介质中存储有计算机指令，当该计算机指令在图像处理装置上运行时，使得图像处理装置执行上述相关方法步骤实现上述实施例中的图像处理方法。
本申请实施例还提供了一种计算机程序产品，当该计算机程序产品在计算机上运行时，使得计算机执行上述相关步骤，以实现上述实施例中的图像处理方法。
本申请实施例还提供一种图像处理装置，这个装置具体可以是芯片、集成电路、组件或模块。具体的，该装置可包括相连的处理器和用于存储指令的存储器，或者该装置包括至少一个处理器，用于从外部存储器获取指令。当装置运行时，处理器可执行指令，以使芯片执行上述各方法实施例中的图像处理方法。
请参阅图22，图22是本申请实施例提供的另一种图像处理装置的结构示意图。该图像处理装置2000包括：至少一个CPU，GPU，存储器，存储器的类型例如可以包括SRAM和ROM，微控制器(Micro controller Unit，MCU)、WLAN子系统、总线、传输接口等。虽然图22中未示出，该图像处理装置2000还可以包括应用处理器(Application Processor，AP)，NPU等其他专用处理器，以及电源管理子系统、时钟管理子系统和功耗管理子系统等其他子系统。
图像处理装置2000的上述各个部分通过连接器相耦合，示例性的，连接器包括各类接口、传输线或总线等，这些接口通常是电性通信接口，但是也可能是机械接口或其他形式的接口，本实施例对此不做限定。
可选地，CPU可以是一个单核(single-CPU)处理器或多核(multi-CPU)处理器；可选地，CPU可以是多个处理器构成的处理器组，多个处理器之间通过一个或多个总线彼此耦合。在一种可选地情况中，CPU通过调用片上述存储器或者片外存储器中存储的程序指令实现如上述方法实施例中的任一种图像处理方法。在一种可选地情况中，CPU和MCU共同实现如上述方法实施例中的任一种图像处理方法，例如CPU完成图像处理方法中的部分步骤，而MCU完成图像处理方法中的其他步骤。在一种可选地情况中，AP或者其他专用处理器通过调用片上述存储器或者片外存储器中存储的程序指令实现如上述方法实施例中的任一种图像处理方法。
该传输接口可以为处理器芯片的接收和发送数据的接口，该传输接口通常包括多种接 口，在一种可选地情况下，该传输接口可以包括内部整合电路(Inter-Integrated Circuit，I2C)接口、串行外设接口(Serial Peripheral Interface，SPI)、通用异步收发机(Universal asynchronous receiver-transmitter，UART)接口、通用输入输出(General-purpose input/output，GPIO)接口等。应当理解，这些接口可以是通过复用相同的物理接口来实现不同的功能。
在一种可选地情况中，传输接口还可以包括高清晰度多媒体接口(High Definition Multimedia Interface，HDMI)、V-By-One接口、嵌入式显示端口(Embedded Display Port，eDP)、移动产业处理器接口(Mobile Industry Processor Interface，MIPI)或Display Port(DP)等。
在一种可选地情况中，上述各部分集成在同一个芯片上；在另一种可选地情况中，存储器可以是独立存在的芯片。
WLAN子系统例如可以包括射频电路和基带。
在本申请实施例中涉及的芯片是以集成电路工艺制造在同一个半导体衬底上的系统，也叫半导体芯片，其可以是利用集成电路工艺制作在衬底(通常是例如硅一类的半导体材料)上形成的集成电路的集合，其外层通常被半导体封装材料封装。上述集成电路可以包括各类功能器件，每一类功能器件包括逻辑门电路、金属氧化物半导体(Metal-Oxide-Semiconductor，MOS)晶体管、双极晶体管或二极管等晶体管，也可包括电容、电阻或电感等其他部件。每个功能器件可以独立工作或者在必要的驱动软件的作用下工作，可以实现通信、运算或存储等各类功能。
其中，本实施例提供的图像处理装置、计算机存储介质和计算机程序产品均用于执行上文所提供的对应的方法，因此，其所能达到的有益效果可参考上文所提供的对应的方法中的有益效果，此处不再赘述。
应理解，在本申请的各种实施例中，上述各过程的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本申请实施例的实施过程构成任何限定。
本领域普通技术人员可以意识到，结合本申请中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统、装置和单元的具体工作过程，可以参考上述方法实施例中的对应过程，在此不再赘述。
在本申请所提供的几个实施例中，应该理解到，所揭露的系统、装置和方法，可以通过其他的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，上述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其他的形式。
上述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络 单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。
上述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例上述方法的全部或部分步骤。而上述的存储介质包括：U盘、移动硬盘、只读存储器(Read Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应所述以权利要求的保护范围为准。

Claims (33)

  1. 一种图像处理装置，其特征在于，包括：渲染管线，所述渲染管线中设置有第一光栅化器和第一深度测试模块；
    所述第一光栅化器，用于将输入所述渲染管线的图元转换为待渲染片段；
    所述第一深度测试模块，用于根据全局深度信息对所述待渲染片段进行第一深度测试；
    若所述待渲染片段未通过所述第一深度测试，则剔除所述待渲染片段，所述全局深度信息包括根据全部待渲染图元更新得到的多个像素块的深度信息。
  2. 根据权利要求1所述的装置，其特征在于，所述第一深度测试模块具体用于：
    根据所述全局信息中目标像素块的深度信息和所述待渲染片段的深度信息对所述待渲染片段进行所述第一深度测试，所述目标像素块为所述多个像素块中与所述待渲染片段存在重叠的像素块。
  3. 根据权利要求2所述的装置，其特征在于，所述第一深度测试模块具体用于：
    在所述待渲染片段的深度值大于所述目标像素块的深度值的情况下，确定所述待渲染片段未通过第一深度测试。
  4. 根据权利要求3所述的装置，其特征在于，所述目标像素块包括目标区域，所述目标像素块的深度值包括第一深度值和第二深度值，所述第一深度值为所述目标区域的最大深度值，所述第二深度值为所述目标像素块中非目标区域的最大深度值。
  5. 根据权利要求1至4中任一项所述的装置，其特征在于，所述装置还包括第一缓存模块，所述第一缓存模块用于缓存所述全局深度信息。
  6. 根据权利要求1至5中任一项所述的装置，其特征在于，所述装置还包括：分区管线，所述分区管线中设置有第二光栅化器和第二深度测试模块：
    所述第二光栅化器，用于将输入所述分区管道的待渲染图元投射至多个像素块生成多个投射区域，所述多个投射区域与所述多个像素块一一对应；
    所述第二深度模块，用于根据所述多个投射区域对所述待渲染图元进行第二深度测试；
    若所述待渲染图元未通过所述第二深度测试，则剔除所述待渲染图元。
  7. 根据权利要求6所述的装置，其特征在于，所述第二深度模块具体用于：
    根据第一像素块的深度信息和第一投射区域的深度信息确定所述待渲染图元在所述第一像素块中是否通过深度测试，所述第一像素块为所述多个像素块中的任一像素块，所述第一投射区域为所述第一像素块对应的投射区域；
    若所述待渲染图元在所述第一像素块中通过所述深度测试，则确定所述待渲染图元通过所述第二深度测试；
    若所述待渲染图元在所述多个像素块中均未通过所述深度测试，则确定所述待渲染图元未通过所述第二深度测试。
  8. 根据权利要求7所述的装置，其特征在于，所述第二深度模块具体用于：
    在所述第一投射区域的最小深度值小于所述第一像素块的深度值的情况下，确定所述待渲染图元在所述第一像素块中通过所述深度测试。
  9. 根据权利要求8所述的装置，其特征在于，所述第一像素块包括目标区域，所述第一像素块的深度值包括所述第一像素块的目标区域的最大深度值和所述第一像素块的非目标区域的最大深度值。
  10. 根据权利要求6至9中任一项所述的装置，其特征在于，所述第二深度模块还用于根据所述多个投射区域更新所述全局深度信息。
  11. 根据权利要求10所述的装置，其特征在于，所述第二深度模块还具体用于：
    根据第一像素块的深度信息和第一投射区域的深度信息更新所述全局深度信息中所述第一像素块的深度信息，所述第一像素块为所述多个像素块中的任一像素块，所述第一投射区域为所述第一像素块对应的投射区域。
  12. 根据权利要求11所述的装置，其特征在于，所述第一像素块包括目标区域，所述第二深度模块还具体用于：
    根据所述第一投射区域的最大深度值、所述第一像素块的目标区域的最大深度值、所述第一像素块的非目标区域的最大深度值，更新所述第一像素块的深度信息。
  13. 根据权利要求1至12中任一项所述的装置，其特征在于，所述渲染管线还设置有前深度模块、像素着色器、后深度测试模块或深度缓存模块中的至少一项；
    所述前深度测试模块，用于根据所述深度缓存模块中的深度值对第一目标片段进行前深度测试，所述第一目标片段为通过所述第一深度测试的片段；
    所述像素着色器，用于对第二目标片段进行着色渲染，所述第二目标片段为通过所述前深度测试的片段；
    所述后深度测试模块，用于根据所述深度缓存模块中的深度值第三目标片段进行后深度测试，所述第三目标片段为经过所述像素着色器着色渲染后的片段；
    所述深度缓存模块，用于缓存深度值。
  14. 一种图像处理装置，其特征在于，包括：渲染管线，所述渲染管线中设置有光栅化器、渲染粗粒度深度测试模块、前深度测试模块、像素着色器、深度缓存模块和后深度测试模块；
    所述光栅化器，用于将输入所述渲染管线的图元转换为待渲染片段；
    所述渲染粗粒度深度测试模块，用于对所述待渲染片段进行第一深度测试；
    所述前深度测试模块，用于根据所述深度缓存模块中的深度值对第一目标片段进行前深度测试，所述第一目标片段为通过所述第一深度测试的片段；
    所述像素着色器，用于对第二目标片段进行着色渲染，所述第二目标片段为通过所述前深度测试的片段；
    所述后深度测试模块，用于根据所述深度缓存模块中的深度值对第三目标片段进行后深度测试，所述第三目标片段为经过所述像素着色器着色渲染后的片段；
    所述深度缓存模块，用于缓存深度值。
  15. 根据权利要求14所述的装置，其特征在于，所述渲染粗粒度深度测试模块具体用于：
    根据全局信息中目标像素块的深度信息和所述待渲染片段的深度信息对所述待渲染片段进行所述第一深度测试，所述目标像素块为所述多个像素块中与所述待渲染片段存在重叠的像素块，所述全局深度信息包括根据全部待渲染图元更新得到的多个像素块的深度 信息。
  16. 根据权利要求15所述的装置，其特征在于，所述装置还包括粗粒度深度缓存模块，所述粗粒度深度缓存模块用于缓存所述全局深度信息。
  17. 根据权利要求14至16中任一项所述的装置，其特征在于，所述装置还包括分区管线，所述分区管线设置有粗粒度光栅化器和分区粗粒度深度测试模块；
    所述粗粒度光栅化器，用于将输入所述分区管道的待渲染图元投射至多个像素块生成多个投射区域，所述多个投射区域与所述多个像素块一一对应；
    所述分区粗粒度深度测试模块，用于根据所述多个投射区域对所述待渲染图元进行第二深度测试。
  18. 根据权利要求17所述的装置，其特征在于，所述分区粗粒度深度测试模块还用于：
    根据所述多个投射区域更新全局深度信息，所述全局深度信息包括根据全部待渲染图元更新得到的多个像素块的深度信息。
  19. 一种图像处理方法，其特征在于，包括：
    将输入渲染管线的图元转换为待渲染片段；
    根据全局深度信息对所述待渲染片段进行第一深度测试，所述全局深度信息包括根据全部待渲染图元更新得到的多个像素块的深度信息；
    若所述待渲染片段未通过所述第一深度测试，则剔除所述待渲染片段。
  20. 根据权利要求19所述的方法，其特征在于，所述全局深度信息包括多个像素块的深度信息，所述根据全局深度信息对所述待渲染片段进行第一深度测试，包括：
    根据目标像素块的深度信息和所述待渲染片段的深度信息对所述待渲染片段进行所述第一深度测试，所述目标像素块为所述多个像素块中与所述待渲染片段存在重叠的像素块。
  21. 根据权利要求20所述的方法，其特征在于，所述根据目标像素块的深度信息和所述待渲染片段的深度信息对所述待渲染片段进行所述第一深度测试，包括：
    在所述待渲染片段的深度值大于所述目标像素块的深度值的情况下，确定所述待渲染片段未通过第一深度测试。
  22. 根据权利要求21所述的方法，其特征在于，所述目标像素块包括目标区域，所述目标像素块的深度值包括第一深度值和第二深度值，所述第一深度值为所述目标区域的最大深度值，所述第二深度值为所述目标像素块中非目标区域的最大深度值。
  23. 根据权利要求19至22中任一项所述的方法，其特征在于，所述方法还包括：
    将输入分区管道的待渲染图元投射至多个像素块生成多个投射区域，所述多个投射区域与所述多个像素块一一对应；
    根据所述多个投射区域对所述待渲染图元进行第二深度测试；
    若所述待渲染图元未通过所述第二深度测试，则剔除所述待渲染图元。
  24. 根据权利要求23所述的方法，其特征在于，所述根据所述多个投射区域对所述待渲染图元进行第二深度测试，包括：
    根据第一像素块的深度信息和第一投射区域的深度信息确定所述待渲染图元在所述第一像素块中是否通过深度测试，所述第一像素块为所述多个像素块中的任一像素块，所述第一投射区域为所述第一像素块对应的投射区域；
    若所述待渲染图元在所述第一像素块中通过所述深度测试，则确定所述待渲染图元通过所述第二深度测试；
    若所述待渲染图元在所述多个像素块中均未通过所述深度测试，则确定所述待渲染图元未通过所述第二深度测试。
  25. 根据权利要求24所述的方法，其特征在于，所述根据第一像素块的深度信息和第一投射区域的深度信息确定所述待渲染图元在所述第一像素块中是否通过深度测试，包括：
    在所述第一投射区域的最小深度值小于所述第一像素块的深度值的情况下，确定所述待渲染图元在所述第一像素块中通过所述深度测试。
  26. 根据权利要求25所述的方法，其特征在于，所述第一像素块包括目标区域，所述第一像素块的深度值包括所述第一像素块的目标区域的最大深度值和所述第一像素块的非目标区域的最大深度值。
  27. 根据权利要求23至26中任一项所述的方法，其特征在于，所述方法还包括：
    根据所述多个投射区域更新所述全局深度信息。
  28. 根据权利要求27所述的方法，其特征在于，所述根据所述多个投射区域更新所述全局深度信息，包括：
    根据第一像素块的深度信息和第一投射区域的深度信息更新所述全局深度信息中所述第一像素块的深度信息，所述第一像素块为所述多个像素块中的任一像素块，所述第一投射区域为所述第一像素块对应的投射区域。
  29. 根据权利要求28所述的方法，其特征在于，所述根据第一像素块的深度信息和第一投射区域的深度信息更新所述全局深度信息中所述第一像素块的深度信息，包括：
    根据所述第一投射区域的最大深度值、所述第一像素块的目标区域的最大深度值、所述第一像素块的非目标区域的最大深度值，更新所述第一像素块的深度信息。
  30. 根据权利要求19至29中任一项所述的方法，其特征在于，所述方法还包括：
    对通过第一深度测试的片段进行前深度测试和/或后深度测试。
  31. 一种电子设备，包括至少一个处理器和接口电路，所述至少一个处理器和所述接口电路耦合，其特征在于，所述至少一个处理器执行存储在存储器中的程序或指令，以使得所述电子设备实现权利要求19至30中任一项所述的方法。
  32. 一种计算机可读存储介质，用于存储计算机程序，其特征在于，所述计算机程序包括用于实现上述权利要求19至30中任一项所述的方法的指令。
  33. 一种计算机程序产品，所述计算机程序产品中包含指令，其特征在于，当所述指令在计算机或处理器上运行时，使得所述计算机或所述处理器实现上述权利要求19至30中任一项所述的方法。
PCT/CN2021/140588 2021-12-22 2021-12-22 图像处理装置和方法 WO2023115408A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2021/140588 WO2023115408A1 (zh) 2021-12-22 2021-12-22 图像处理装置和方法
CN202180096777.0A CN117203661A (zh) 2021-12-22 2021-12-22 图像处理装置和方法

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2021/140588 WO2023115408A1 (zh) 2021-12-22 2021-12-22 图像处理装置和方法

Publications (1)

Publication Number Publication Date
WO2023115408A1 true WO2023115408A1 (zh) 2023-06-29

Family

ID=86901074

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/140588 WO2023115408A1 (zh) 2021-12-22 2021-12-22 图像处理装置和方法

Country Status (2)

Country Link
CN (1) CN117203661A (zh)
WO (1) WO2023115408A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117710502A (zh) * 2023-12-12 2024-03-15 摩尔线程智能科技(北京)有限责任公司 渲染方法、装置及存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140354640A1 (en) * 2013-05-31 2014-12-04 Arm Limited Hidden surface removal in graphics processing systems
CN109564695A (zh) * 2016-09-16 2019-04-02 英特尔公司 用于高效3d图形流水线的装置和方法
CN111127299A (zh) * 2020-03-26 2020-05-08 南京芯瞳半导体技术有限公司 一种加速光栅化遍历的方法、装置及计算机存储介质
CN112116518A (zh) * 2019-06-20 2020-12-22 三星电子株式会社 合并单元、选择覆盖范围合并方案的方法和深度测试系统
CN112933599A (zh) * 2021-04-08 2021-06-11 腾讯科技（深圳）有限公司 三维模型渲染方法、装置、设备及存储介质
CN113781289A (zh) * 2020-06-09 2021-12-10 Arm有限公司 图形处理

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140354640A1 (en) * 2013-05-31 2014-12-04 Arm Limited Hidden surface removal in graphics processing systems
CN109564695A (zh) * 2016-09-16 2019-04-02 英特尔公司 用于高效3d图形流水线的装置和方法
CN112116518A (zh) * 2019-06-20 2020-12-22 三星电子株式会社 合并单元、选择覆盖范围合并方案的方法和深度测试系统
CN111127299A (zh) * 2020-03-26 2020-05-08 南京芯瞳半导体技术有限公司 一种加速光栅化遍历的方法、装置及计算机存储介质
CN113781289A (zh) * 2020-06-09 2021-12-10 Arm有限公司 图形处理
CN112933599A (zh) * 2021-04-08 2021-06-11 腾讯科技（深圳）有限公司 三维模型渲染方法、装置、设备及存储介质

Also Published As

Publication number Publication date
CN117203661A (zh) 2023-12-08

Similar Documents

Publication Publication Date Title
US10210651B2 (en) Allocation of tiles to processing engines in a graphics processing system
US11880926B2 (en) Hybrid render with deferred primitive batch binning
TWI592902B (zh) 來自片段著色器程式之樣本遮罩的控制
US8773449B2 (en) Rendering of stereoscopic images with multithreaded rendering software pipeline
US9292965B2 (en) Accelerated data structure positioning based upon view orientation
US8248412B2 (en) Physical rendering with textured bounding volume primitive mapping
CN116050495A (zh) 用稀疏数据训练神经网络的系统和方法
JP2010507875A (ja) グラフィック処理ユニットにおける３次元クリッピング
WO2008118805A2 (en) Processor with adaptive multi-shader
US11954782B2 (en) Hybrid render with preferred primitive batch binning and sorting
US11562533B2 (en) Assembling primitive data into multi-view primitive blocks in a graphics processing system
CN115147579B (zh) 一种扩展图块边界的分块渲染模式图形处理方法及系统
CN111667542B (zh) 适用于人工神经网络的用于处理压缩数据的解压缩技术
US20230122999A1 (en) Data structures, methods and primitive block generators for storing primitives in a graphics processing system
US8068120B2 (en) Guard band clipping systems and methods
WO2023115408A1 (zh) 图像处理装置和方法
US6778189B1 (en) Two-sided stencil testing system and method
US20030122820A1 (en) Object culling in zone rendering
US10255655B1 (en) Serial pixel processing with storage for overlapping texel data
US10621782B1 (en) Sub-patch techniques for graphics tessellation
US8711156B1 (en) Method and system for remapping processing elements in a pipeline of a graphics processing unit
CN116630516B (zh) 一种基于3d特性的2d渲染排序方法、装置、设备及介质
CN113051491B (zh) 地图数据处理的方法、设备、存储介质及程序产品
US20240104845A1 (en) Mesh topology generation using parallel processing
WO2023029424A1 (zh) 一种对应用进行渲染的方法及相关装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21968557

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 202180096777.0

Country of ref document: CN