WO2023154109A1 - Procédés et systèmes de mise à l'échelle supérieure de graphiques vidéo - Google Patents

Procédés et systèmes de mise à l'échelle supérieure de graphiques vidéo Download PDF

Info

Publication number
WO2023154109A1
WO2023154109A1 PCT/US2022/052726 US2022052726W WO2023154109A1 WO 2023154109 A1 WO2023154109 A1 WO 2023154109A1 US 2022052726 W US2022052726 W US 2022052726W WO 2023154109 A1 WO2023154109 A1 WO 2023154109A1
Authority
WO
WIPO (PCT)
Prior art keywords
pixel
block
video
pixels
rows
Prior art date
Application number
PCT/US2022/052726
Other languages
English (en)
Inventor
Steven Jackson
Chen Li
Hongyu Sun
Original Assignee
Innopeak Technology, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Innopeak Technology, Inc. filed Critical Innopeak Technology, Inc.
Publication of WO2023154109A1 publication Critical patent/WO2023154109A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/36—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
    • G09G5/37—Details of the operation on graphic patterns
    • G09G5/373—Details of the operation on graphic patterns for modifying the size of the graphic pattern
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/36—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
    • G09G5/39—Control of the bit-mapped memory
    • G09G5/393—Arrangements for updating the contents of the bit-mapped memory
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2340/00—Aspects of display data processing
    • G09G2340/04—Changes in size, position or resolution of an image
    • G09G2340/045—Zooming at least part of an image, i.e. enlarging it or shrinking it
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2350/00—Solving problems of bandwidth in display systems
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects

Definitions

  • the present invention is directed to video processing methods and systems.
  • an input video characterized by a resolution of m rows and n columns is upscaled to an output video characterized by a resolution of 2m rows and 2n columns.
  • An intermediate image based on the input video is generated with m rows and n columns of blocks. Sampling is performed for a first pair of non-adjacent pixels of each block, and a second pair of non-adjacent pixels are calculated using the first pair of non-adjacent pixels and neighboring blocks.
  • Embodiments of the present invention can be implemented in conjunction with existing systems and processes.
  • the video processing methods and systems according to the present invention can be used in a wide variety of applications, including video streaming, client-side media players, and online media platforms.
  • various techniques according to the present invention can be adopted into existing systems via architectures that support multi-sample anti-aliasing (MSAA) textures and include extension ARB_sample_locations.
  • MSAA multi-sample anti-aliasing
  • a system of one or more computers can be configured to perform particular operations or actions by virtue of having software, firmware, hardware, or a combination of them installed on the system that in operation causes or cause the system to perform the actions.
  • One or more computer programs can be configured to perform particular operations or actions by virtue of including instructions that, when executed by a data processing apparatus, cause the apparatus to perform the actions.
  • One general aspect includes a method for video upscaling. The method includes receiving an input video, wherein the input video includes a first frame and a second frame, the first frame being characterized by a dimension of m rows by n columns of pixels. The method also includes storing the first frame and the second frame at a first buffer. The method also includes providing a first stencil image at a second buffer, the first stencil image including m rows by n columns of pixel blocks, the pixel blocks including a first block and a second block and a third block, the first block being adjacent to the second block, the first block including two top pixels and two bottom pixels.
  • the method also includes writing values to a first top pixel and a first bottom pixel of the first block, the first top pixel being diagonally positioned relative to the first bottom pixel.
  • the method also includes calculating a first pixel value of a second top pixel of the first block, the first pixel value being based at least on the first top pixel and the first bottom pixel.
  • the method also includes generating a first output image associated with the first stencil image, the first output image being characterized by a dimension of 2m rows by 2n columns of pixels.
  • Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods.
  • the method where the first buffer may include a frame buffer and the second buffer may include a stencil buffer.
  • the first pixel value is further based on adjacent pixels of the second block and the third block.
  • the method may include performing sampling to generate the values of the first top pixel and the first bottom pixel.
  • the method may include calculating a pixel value of a second bottom pixel of the first block, the pixel value being based at least on the first top pixel and the first bottom pixel.
  • the method may further include providing a second stencil image at the second buffer, the second stencil image including a fourth block positioned at a same location as the first block, the fourth block including a third top pixel and a third bottom pixel, the third top pixel being positioned at a same location of the second top pixel, the third bottom pixel being positioned at a same location of the second bottom pixel.
  • the method may also include calculating a second pixel value of a fourth top pixel of the second block, the second pixel value being based at least on the third top pixel and the third bottom pixel.
  • the method may additionally include generating a second output image associated with the second stencil image.
  • the method may include storing the first output image.
  • the method may include enabling a super sampling mode.
  • Implementations of the described techniques may include hardware, a method or process, or computer software on a computer-accessible medium.
  • One general aspect includes a method for processing video. The method includes receiving a request for an output video, the output video being characterized by a dimension of 2m rows by 2n columns of pixels. The method also includes providing an input video, the input video including a first frame and a second frame, the first frame being characterized by a dimension of m rows by n columns of pixels. The method also includes storing the first frame and the second frame at a first buffer.
  • the method also includes providing a first stencil image at a second buffer, the first stencil image including m rows by n columns of pixel blocks, the pixel blocks including a first block and a second block and a third block, the first block being adjacent to the second block and the second block, the first block including two top pixels and two bottom pixels.
  • the method also includes writing values to a first top pixel and a first bottom pixel of the first block, the first top pixel being diagonally positioned relative to the first bottom pixel.
  • the method also includes calculating a first pixel value of a second top pixel of the first block, the pixel value being based at least on the first top pixel and the first bottom pixel.
  • the method also includes generating a first output image associated with the first stencil image, the first output image being characterized by a dimension of 2m rows by 2n columns of pixels.
  • the method also includes providing the output video, wherein the output video may include the first output image.
  • Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods.
  • Implementations may include one or more of the following features.
  • the method where the first buffer may include a data structure for storing video data at a dimension of m rows by n columns of pixels.
  • the method further may include rendering the input video at a dimension of m rows by n columns of pixels.
  • the method further may include receiving a request to enable an upscaling process 125.
  • the method further may include performing sampling to obtain the values for the first top pixel and the first bottom pixel.
  • Implementations of the described techniques may include hardware, a method or process, or computer software on a computer-accessible medium.
  • the video processing system includes a first data buffer that may include a first data structure to store video at a dimension of m rows by n columns of pixels, the first data buffer being configured to store an input video, the input video including a first frame and a second frame, the first frame being characterized by a dimension of m rows by n columns of pixels.
  • the system also includes a second data buffer, that may include a second data structure, to store video at a dimension of 2m rows by 2n columns of pixels, the second data structure being configured to store a first stencil image at a second buffer, the first stencil image including m rows by n columns of pixel blocks, the pixel blocks including a first block and a second block and a third block, the first block being adjacent to the second block and the second block, the first block including two top pixels and two bottom pixels.
  • the system also includes a data bus.
  • the system also includes a processor coupled to the first data buffer and the second data buffer via a data bus.
  • the process is configured to write values to a first top pixel and a first bottom pixel of the first block, the first top pixel being diagonally positioned relative to the first bottom pixel.
  • the process is also configured to calculate a first pixel value of a second top pixel of the first block, the pixel value being based at least on the first top pixel and the first bottom pixel, and generate an output video, which may include the first stencil image, wherein the output video is characterized by a dimensions of 2m rows by 2n columns of pixels.
  • Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods.
  • the video processing system where the first data buffer may include a random access memory.
  • the video processing system may include a graphical processing unit configured to generate the input video.
  • the video processing system may include a communication interface coupled to a display, the display being characterized by a resolution of at least 2m rows by 2n columns of pixels.
  • the video processing system may include a storage configured to store instruction executable by the processor.
  • the video processing system may include interpolating the first stencil image.
  • embodiments of the present invention can reduce power consumption and increase frame rate while maintaining a desired resolution in video applications.
  • the present invention can also remove the maintenance workload for certain aspects of video processing from the application/game developer.
  • the present invention provides a new performance improvement technique via image up-scaling that can be implemented with application programming interfaces (APIs), such as OpenGL, and on mobile platforms.
  • APIs application programming interfaces
  • Embodiments of the present invention can be implemented in conjunction with existing systems and processes.
  • the video processing methods and systems according to the present invention can be used in a wide variety of applications, including video streaming, client-side media players, and online media platforms.
  • FIG. 1 is a simplified block diagram illustrating a video processing system according to embodiments of the present invention.
  • Figure 2 is a simplified block diagram illustrating software modules for video processing according to embodiments of the present invention.
  • Figure 3 is a simplified flow diagram illustrating a method of video processing according to embodiments of the present invention.
  • Figure 4 is a simplified diagram video texture blocks according to embodiments of the present invention.
  • Figure 5 is a simplified diagram illustrating a super sampling block according to embodiments of the present invention.
  • Figure 6 is a simplified diagram illustrating super sampling processing with pixel blocks according to embodiments of the present invention.
  • DETAILED DESCRIPTION OF THE INVENTION [0015]
  • the present invention is directed to video processing methods and systems. In a specific embodiment, an input video characterized by a resolution of m rows and n columns is upscaled to an output video characterized by a resolution of 2m rows and 2n columns.
  • An intermediate image based on the input video is generated with m rows and n columns of blocks. Sampling is performed for a first pair of non-adjacent pixels of each block, and a second pair of non-adjacent pixels are calculated using the first pair of non-adjacent pixels and neighboring blocks.
  • a system of one or more computers can be configured to perform particular operations or actions by virtue of having software, firmware, hardware, or a combination of them installed on the system that in operation causes or cause the system to perform the actions.
  • One or more computer programs can be configured to perform particular operations or actions by virtue of including instructions that, when executed by a data processing apparatus, cause the apparatus to perform the actions.
  • the present invention provides a CBSS mechanism, which is a method for increasing the performance of the rendering process on the GPU at a specified output resolution.
  • the process may be transparent to the application layer, and does not require any additional integration and maintenance by application developers.
  • CBSS includes five stages: (1) FBO Replace, (2) Checkerboard Stencil, (3) Main Color Pass, (4) Resolve, and (5) Upscale.
  • the process uses an MSAA texture for a scene render process to achieve the checkerboard render pattern. When scene rendering is complete, the checkerboard holes are then filled by calculating the most appropriate value.
  • the resolved image may be manually up-scaled from the MSAA texture to a non-MSAA texture for displaying of the frame.
  • any element in a claim that does not explicitly state “means for” performing a specified function, or “step for” performing a specific function, is not to be interpreted as a “means” or “step” clause as specified in 35 U.S.C. Section 112, Paragraph 6.
  • the use of “step of” or “act of” in the Claims herein is not intended to invoke the provisions of 35 U.S.C.112, Paragraph 6.
  • the labels left, right, front, back, top, bottom, forward, reverse, clockwise and counterclockwise have been used for convenience purposes only and are not intended to imply any particular fixed direction. Instead, they are used to reflect relative locations and/or directions between various portions of an object.
  • FIG. 1 is a simplified block diagram illustrating a video processing system 100 according to embodiments of the present invention. This diagram is merely an example, which should not unduly limit the scope of the claims. One of ordinary skill in the art would recognize many variations, alternatives, and modifications.
  • the video processing system 100 includes a first data buffer 110 and a second data buffer 120 coupled via a data bus 130 to a processor 140.
  • the elements of system 100 can be configured together to perform a video upscaling process on an input video to produce an output video for a higher resolution, as further described below.
  • the first data buffer 110 can include a first data structure 115 to store video at a dimension of m rows by n columns of pixels, where m and n are integers greater than zero.
  • This first data buffer 110 can be configured to store an input video at the first data structure 115, including at least a first frame and a second frame.
  • the first frame can be characterized by a dimension of m rows by n columns of pixels.
  • the first data buffer 110 includes a random access memory (RAM), or other similar memory storage device 142.
  • the second data buffer 120 includes a second data structure 122 to store video at a dimension of 2m rows by 2n columns of pixels.
  • This second data structure 122 can be configured to store a first stencil image at the second data structure, which includes m rows by n columns of pixel blocks.
  • the pixel blocks can include a first block and a second block and a third block.
  • the first block is spatially positioned adjacent to the second block and the third block, In a specific example, the first block includes two top pixels and two bottom pixels.
  • the system 100 can also generate a second stencil image based on the second frame. In other examples, the system 100 can generate a plurality of stencil images based on a plurality of frames of the input video. These stencil images can also be stored in the second data structure 122 of the second data buffer 120.
  • the system 100 can include a storage 142 (e.g., RAM, hard drive, flash drive, etc.) coupled to the processor 140 and configured to store instructions executable by the processor 140.
  • the processor 140 is configured to write values to a first top pixel and a first bottom pixel of the first block, calculate a first pixel value of a second top pixel of the first block based at least on the first top pixel and the first bottom pixel, and generate an output video, second data structure 122, being characterized by a dimension of 2m rows by 2n columns of pixels and including the first stencil image.
  • the first top pixel is diagonally positioned relative to the first bottom pixel.
  • This system 100 can also include a graphical processing unit (GPU) 150 and a communication interface 160 coupled to the processor 140.
  • the GPU 150 can be coupled to the first data buffer and configured to generate the input video.
  • the communication interface 160 can be coupled to a display 170 that is characterized by a resolution of at least 2m rows by 2n columns of pixels.
  • Other embodiments of this system include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods.
  • FIG. 2 is a simplified block diagram illustrating software modules for video processing according to embodiments of the present invention.
  • This diagram is merely an example, which should not unduly limit the scope of the claims.
  • an application/game software module 210 coupled together with an application programming interface (API) module 220 and a video output module 230.
  • API application programming interface
  • the API module 220 can be configured to enable services for video streaming, client-side video players, online media platforms, and the like.
  • the application/game module 210 can use these services to display desired media content at the video output module 230 through a render module that generates the media content to be displayed in a display device 250.
  • the block diagram includes a first render module 242 and a second render module 244 configured in separate pathways between the API module 220 and the video output module 230.
  • the API module 220 can choose between rendering an input video at the first render module 242 or the second render module 244.
  • the first render module 242 is configured as a full-resolution 3D render module
  • the second render module 244 is configured as a 1 ⁇ 2 render module with a checkerboard super-sampling (CBSS) mechanism.
  • CBSS checkerboard super-sampling
  • CBSS is a technique for up-scaling a half-resolution checkerboard frame to a standard resolution.
  • the CBSS functionality acts as a layer between the application and driver intercepting API calls (e.g., OpenGL, or the like) and modifying the actions performed.
  • CBSS can replace a main color pass frame buffer with a multi-sample anti-aliasing (MSAA) variant with half-width and half-height of the original.
  • MSAA multi-sample anti-aliasing
  • CBSS up-scales a half-resolution rendered frame to a standard resolution output frame without additional data. Further details of the CBSS functionality and other video processing techniques are discussed with reference to Figures 3- 6.
  • FIG. 3 is a simplified flow diagram illustrating a method 300 of video processing according to embodiments of the present invention.
  • This diagram is merely an example, which should not unduly limit the scope of the claims.
  • One of ordinary skill in the art would recognize many variations, alternatives, and modifications. For example, one or more steps may be added, removed, replaced, modified, rearranged, repeated, and/or overlapped, and they should limit the scope of claims.
  • the method 300 includes receiving a request for an output video, step 302.
  • This output video can be characterized by a dimension of 2m rows by 2n columns of pixels, where m and n are integers greater than zero.
  • the method includes providing an input video, step 304.
  • This input video can include at least a first frame and a second frame.
  • the first frame can be characterized by a dimension of m rows by n columns of pixels.
  • the first and the second frames can be stored at a first buffer.
  • the first buffer can include a data structure (e.g., a frame buffer, or the like) for storing video data at a dimension of m rows by n columns of pixels.
  • the method can also include rendering the input video at a dimension of m rows by n columns of pixels. This rendering process can be used to produce the first and second frames.
  • the input video can include a plurality of frames, and each such frame can be characterized by a dimension of m rows by n columns of pixels.
  • the method includes providing a first stencil image at a second buffer (e.g., a stencil buffer, or the like).
  • the first stencil image can include m rows by n columns of pixel blocks.
  • These pixel blocks can include at least a first block and a second block and a third block.
  • the first block can be spatially positioned adjacent to the second block and the third block.
  • the first block includes at least two top pixels and at least two bottom pixels.
  • a 4xMSAA frame buffer object may be introduced to replace the FBO for the main color pass process, where the non-MSAA will be the default main color pass texture.
  • FBO 4xMSAA frame buffer object
  • the use of MSAA frame buffer is advantageous to capture the additional texel samples over the default non- MSAA texture. Further details of a frame buffer object replacement process according to an example of the present invention are discussed with reference to Figure 4.
  • the method includes writing values to a first top pixel and a first bottom pixel of the first block. In this case, the first top pixel is diagonally positioned relative to the first bottom pixel.
  • the method can also include a sampling process to obtain the values for the first top pixel and the first bottom pixel.
  • the method includes calculating a first pixel value of a second top pixel of the first block. This pixel value can be based at least on the first top pixel and the first bottom pixel. In an example, the pixel value can be further based on adjacent pixels of the second block and the third block. Further, the method can include calculating a pixel value of a second bottom pixel of the first block. This pixel value can also be based at least on the first top pixel and the first bottom pixel. Further details of a resolve pass process according to an example of the present invention are discussed with reference to Figure 6.
  • the method can include providing a second stencil image at the second buffer.
  • This second stencil image can include a fourth block positioned at a same location as the first block, and this fourth block can include a third top pixel and a third bottom pixel.
  • the third top pixel can be spatially positioned at a same location of the second top pixel, and the third bottom pixel can be spatially positioned at a same location of the second bottom pixel.
  • the method can include calculating the second pixel value of a fourth top pixel of the second block. This second pixel value can be based at least on the third top pixel and the third bottom pixel.
  • the method includes generating a first output image associated with the first stencil image.
  • This first output image can be characterized by a dimension of 2m rows by 2n columns of pixels.
  • the method can include generating a second output image associated with the second stencil image.
  • additional stencil images can be provided and additional output images associated with these stencil images can be generated.
  • the method can also include storing the first output image, the second output image, or any additional output image in an output buffer, the video output module, or the like. Further details of an up-scale pass process according to an example of the present invention is discussed below. [0043]
  • the method can include enabling a super sampling mode. Following the previous steps, the method can include providing the output video comprising the first output image.
  • this output video can be sent by a GPU to a display device using a communication device (see Figure 1).
  • the output video can be sent to a video output module (see Figure 2) that is configured to transmit the video to a display device.
  • Implementations of the described techniques may include hardware, a method or process, or computer software on a computer-accessible medium.
  • Figure 4 is a simplified diagram of video texture blocks 400 according to embodiments of the present invention. This diagram is merely an example, which should not unduly limit the scope of the claims. One of ordinary skill in the art would recognize many variations, alternatives, and modifications.
  • the graph 401 on the left-side represents a 4xMSAA (640x360) video texture block
  • the graph 402 on the right-side represents a non-MSAA (1280x720) video texture block.
  • the CBSS mechanism can introduce an MSAA frame buffer object (FBO) to replace the FBO for the main color pass process (i.e., default frame render process).
  • This 4xMSAA FBO can contain a 4xMSAA texture 401 with half- width and half-height as the original main color pass texture used by an application/game.
  • the 4xMSAA texture 401 has the same number of samples (in this case, the total is 921,600) as the original main color pass texture.
  • FIG. 5 is a simplified diagram illustrating a super sampling block 500 according to embodiments of the present invention. This diagram is merely an example, which should not unduly limit the scope of the claims. One of ordinary skill in the art would recognize many variations, alternatives, and modifications.
  • the video texture block 510 is used in an example of the present invention in a checkerboard stencil pass process.
  • this checkerboard stencil pass technique is responsible for writing values to the stencil buffer in a checkerboard pattern.
  • samples opposite each other will be stenciled out.
  • the CBSS mechanism e.g., as illustrated in Figure 2 can be configured with at least two modes: (1) using the same checkerboard pattern for each frame, and (2) using an alternate checkerboard pattern for each frame.
  • a depth-based method may be employed for blocking out the checkerboard pattern. For example, a depth value may be contained at the time when a sample fragment is written and stored in a depth buffer. By comparing the depth value between the sample fragments, the samples to be blocked out can be determined. For instance, the sample that has a greater depth value may be discarded as it is further away from the viewer. [0047] Considering the previous example, outside of binding the 4xMSAA 401 render target instead of a standard non-MSAA 402 buffer, the main color pass process should otherwise remain unchanged.
  • Preparations made to the render target depth buffer (see method 300 in Figure 3) in the checkboard stencil pass 510 can allow the main color pass process to maintain standard functionality.
  • the present invention implements a resolve pass process that is responsible for filling the checkboard holes created by the checkboard stencil pass process 510 and the resulting stencil testing. Rather than performing a full-screen resolve by invoking every sample fragment, this example implementation involves preparing a depth test that matches the values written in the checkerboard stencil pass process 510. This implementation would only invoke sample fragments not modified during the main color pass process (i.e., sample fragments with image holes).
  • Figure 6 is a simplified diagram illustrating super sampling processing with pixel blocks 600 according to embodiments of the present invention.
  • the pixel block configuration 600 there are four pixel blocks arranged in a 2x2 configuration with the blocks denoted by T(x, y), T(x+1, y), T(x, y-1), and T(x+1, y- 1). Each of these pixel blocks includes four quadrants denoted by Q(0), Q(1), Q(2), and Q(3). In an example, resolving each quadrant Q([0 -3]) can use neighboring pixel data for the resolve process because the diagonal sample data is already rendered.
  • resolving the quadrants can include reading all available diagonal sample values, performing an interpolation process, and comparing the result to the rendered value from the previous frame to calculate the current frame value.
  • resolving the quadrants can include reading all available diagonal sample values, performing an interpolation process as the calculated value for the current frame.
  • the present invention implements an up-scale pass process that is responsible for scaling the half-resolution resolved frame to a target resolution. In a specific example, this scaling can be from a 4xMSAA 401 main color pass frame to a color back buffer frame.
  • each pixel in the 4xMSAA 401 texture has four samples which will be expanded to a 2x2 pixel region in the color back buffer.
  • the techniques described previously can apply to scaling from different frame sizes and blocks of different dimensions. Those of ordinary skill in the art will recognize variations, modifications, and alternatives to these methods and devices for video processing. [0052] It is to be appreciated that embodiments of the present invention provide many advantages over conventional techniques. Among other things, embodiments of the present invention can reduce power consumption and increase frame rate while maintaining a desired resolution in video applications. The present invention can also remove the maintenance workload for certain aspects of video processing from the application/game developer.
  • the present invention provides a new performance improvement technique via image up-scaling that can be implemented with application programming interfaces (APIs), such as OpenGL, Vulkan, and on mobile platforms.
  • APIs application programming interfaces

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Hardware Design (AREA)
  • Image Processing (AREA)

Abstract

La présente invention concerne des procédés et des systèmes de traitement vidéo. Dans un mode de réalisation spécifique, une vidéo d'entrée caractérisée par une résolution de m rangées et n colonnes est convertie par mise à l'échelle supérieure en une vidéo de sortie caractérisée par une résolution de 2m rangées et 2n colonnes. Une image intermédiaire basée sur la vidéo d'entrée est générée avec m rangées et n colonnes de blocs. Un échantillonnage est effectué pour une première paire de pixels non adjacents de chaque bloc, et une seconde paire de pixels non adjacents est calculée à l'aide de la première paire de pixels non adjacents et de blocs voisins. D'autres modes de réalisation existent également.
PCT/US2022/052726 2022-02-10 2022-12-13 Procédés et systèmes de mise à l'échelle supérieure de graphiques vidéo WO2023154109A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US202263308860P 2022-02-10 2022-02-10
US63/308,860 2022-02-10

Publications (1)

Publication Number Publication Date
WO2023154109A1 true WO2023154109A1 (fr) 2023-08-17

Family

ID=87564874

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2022/052726 WO2023154109A1 (fr) 2022-02-10 2022-12-13 Procédés et systèmes de mise à l'échelle supérieure de graphiques vidéo

Country Status (1)

Country Link
WO (1) WO2023154109A1 (fr)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100067818A1 (en) * 2008-09-15 2010-03-18 Sony Corporation, A Japanese Corporation System and method for high quality image and video upscaling
US9432616B1 (en) * 2011-01-18 2016-08-30 Dimension, Inc. Systems and methods for up-scaling video
US9747665B2 (en) * 2015-02-04 2017-08-29 Synaptics Japan Gk Device and method for divisional image scaling
US9792666B2 (en) * 2013-06-27 2017-10-17 Seiko Epson Corporation Image processing device, image display device, and method of controlling image processing device for reducing and enlarging an image size
US20180013978A1 (en) * 2015-09-24 2018-01-11 Boe Technology Group Co., Ltd. Video signal conversion method, video signal conversion device and display system
US10922785B2 (en) * 2016-08-01 2021-02-16 Beijing Baidu Netcom Science And Technology Co., Ltd. Processor and method for scaling image

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100067818A1 (en) * 2008-09-15 2010-03-18 Sony Corporation, A Japanese Corporation System and method for high quality image and video upscaling
US9432616B1 (en) * 2011-01-18 2016-08-30 Dimension, Inc. Systems and methods for up-scaling video
US9792666B2 (en) * 2013-06-27 2017-10-17 Seiko Epson Corporation Image processing device, image display device, and method of controlling image processing device for reducing and enlarging an image size
US9747665B2 (en) * 2015-02-04 2017-08-29 Synaptics Japan Gk Device and method for divisional image scaling
US20180013978A1 (en) * 2015-09-24 2018-01-11 Boe Technology Group Co., Ltd. Video signal conversion method, video signal conversion device and display system
US10922785B2 (en) * 2016-08-01 2021-02-16 Beijing Baidu Netcom Science And Technology Co., Ltd. Processor and method for scaling image

Similar Documents

Publication Publication Date Title
US10885607B2 (en) Storage for foveated rendering
US11367165B2 (en) Neural super-sampling for real-time rendering
KR102475212B1 (ko) 타일식 아키텍처들에서의 포비티드 렌더링
US7969444B1 (en) Distributed rendering of texture data
US9836810B2 (en) Optimized multi-pass rendering on tiled base architectures
US9565414B2 (en) Efficient stereo to multiview rendering using interleaved rendering
KR102247565B1 (ko) 렌더링 방법 및 장치
US7768516B1 (en) Image splitting to use multiple execution channels of a graphics processor to perform an operation on single-channel input
JP2022543729A (ja) フォービエイテッドレンダリングのためのシステムおよび方法
JP2018512644A (ja) 低品質タイルを使用してメモリ帯域幅を減らすためのシステムおよび方法
WO2020092110A1 (fr) Échantillonnage de texture parallèle
US20130229422A1 (en) Conversion of Contiguous Interleaved Image Data for CPU Readback
CN115880408A (zh) 一种分块渲染模式图形处理方法及系统
US8681154B1 (en) Adaptive rendering of indistinct objects
US20130063462A1 (en) Tile-based image processing using mipmaps
EP4115607A1 (fr) Procédés et appareil permettant un tramage à points de vue multiples à rendement élevé
WO2023154109A1 (fr) Procédés et systèmes de mise à l'échelle supérieure de graphiques vidéo
US11908079B2 (en) Variable rate tessellation
US9811945B2 (en) On-demand transformation aware shape tessellation
EP4196958A1 (fr) Anticrénelage pour rendu graphique de champ de distance

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22926299

Country of ref document: EP

Kind code of ref document: A1