Classifications

• • A—HUMAN NECESSITIES
  • A63—SPORTS; GAMES; AMUSEMENTS
  • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
  • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
  • A63F13/50—Controlling the output signals based on the game progress
  • A63F13/52—Controlling the output signals based on the game progress involving aspects of the displayed game scene
• • A—HUMAN NECESSITIES
  • A63—SPORTS; GAMES; AMUSEMENTS
  • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
  • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
  • A63F13/30—Interconnection arrangements between game servers and game devices; Interconnection arrangements between game devices; Interconnection arrangements between game servers
  • A63F13/35—Details of game servers
  • A63F13/355—Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T1/00—General purpose image data processing
  • G06T1/20—Processor architectures; Processor configuration, e.g. pipelining
• • G—PHYSICS
  • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
  • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
  • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
  • G09G5/36—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
  • G09G5/363—Graphics controllers
• • A—HUMAN NECESSITIES
  • A63—SPORTS; GAMES; AMUSEMENTS
  • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
  • A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
  • A63F2300/20—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterised by details of the game platform
  • A63F2300/203—Image generating hardware
• • A—HUMAN NECESSITIES
  • A63—SPORTS; GAMES; AMUSEMENTS
  • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
  • A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
  • A63F2300/50—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by details of game servers
  • A63F2300/55—Details of game data or player data management
  • A63F2300/5593—Details of game data or player data management involving scheduling aspects
• • G—PHYSICS
  • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
  • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
  • G09G2360/00—Aspects of the architecture of display systems
  • G09G2360/06—Use of more than one graphics processor to process data before displaying to one or more screens
• • G—PHYSICS
  • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
  • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
  • G09G2360/00—Aspects of the architecture of display systems
  • G09G2360/08—Power processing, i.e. workload management for processors involved in display operations, such as CPUs or GPUs
• • G—PHYSICS
  • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
  • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
  • G09G2360/00—Aspects of the architecture of display systems
  • G09G2360/12—Frame memory handling
  • G09G2360/122—Tiling

Definitions

• a non-transitory computer-readable medium for performing a method including program instructions for rendering graphics for an application using a plurality of graphics processing units (GPUs).
• the computer-readable medium including program instructions for dividing responsibility for the rendering of geometry of the graphics between the plurality of GPUs based on a plurality of screen regions, each GPU having a corresponding division of the responsibility which is known to the plurality of GPUs, wherein screen regions in the plurality of screen regions are interleaved.
• the computer readable medium including program instructions for assigning a GPU a piece of geometry of an image frame generated by an application for geometry pretesting.
• a non-transitory computer-readable medium for performing a method including program instructions for rendering graphics for an application using a plurality of graphics processing units (GPUs).
• the computer-readable medium including program instructions for dividing responsibility for the rendering of geometry of the graphics between the plurality of GPUs based on a plurality of screen regions, each GPU having a corresponding division of the responsibility which is known to the plurality of GPUs.
• the computer-readable medium including program instructions for performing geometry testing at a pretest GPU on a plurality of pieces of geometry of an image frame generated by an application to generate information regarding each piece of geometry and its relation to each of the plurality of screen regions.
• the computer-readable medium including program instructions for rendering the plurality of pieces of geometry at each of the plurality of GPUs using the information generated for each of the plurality of pieces of geometry, where using the information includes, for example, skipping rendering entirely if it has been determined that the piece of geometry does not overlap any screen region assigned to a given GPU.
• the computer- readable medium including program instructions for performing geometry testing at the plurality of GPUs on the plurality of pieces of geometry to generate information regarding each piece of geometry and its relation to each of the plurality of screen regions.
• the computer-readable medium including program instructions for setting a second state configuring the one or more shaders to perform rendering.
• the computer-readable medium including program instructions for rendering the plurality of pieces of geometry at each of the plurality of GPUs using the information generated for each of the plurality of pieces of geometry, where using the information includes, for example, skipping rendering entirely if it has been determined that the piece of geometry does not overlap any screen region assigned to a given GPU.
• a computer system including a processor and memory coupled to the processor and having stored therein instructions that, if executed by the computer system, cause the computer system to execute a method for graphics processing.
• FIG. 9A-9C illustrates various strategies for assigning screen regions to corresponding GPUs when multiple GPUs collaborate to render a single image, in accordance with one embodiment of the present disclosure.
• FIG. 1 illustrates the implementation of multi-GPU rendering of geometry between one or more cloud gaming servers of a cloud gaming system
• FIG. 1 illustrates the implementation of multi-GPU rendering of geometry between one or more cloud gaming servers of a cloud gaming system
• FIG. 1 illustrates the implementation of multi-GPU rendering of geometry between one or more cloud gaming servers of a cloud gaming system
• FIG. 1 illustrates the implementation of multi-GPU rendering of geometry between one or more cloud gaming servers of a cloud gaming system
• FIG. 1 illustrates the implementation of multi-GPU rendering of geometry between one or more cloud gaming servers of a cloud gaming system
• game server 160 and/or the game title processing engine 111 includes basic processor based functions for executing the game and services associated with the gaming application.
• game server 160 includes central processing unit (CPU) resources 163 and graphics processing unit (GPU) resources 365 that are configured for performing processor based functions include 2D or 3D rendering, physics simulation, scripting, audio, animation, graphics processing, lighting, shading, rasterization, ray tracing, shadowing, culling, transformation, artificial intelligence, etc.
• CPU central processing unit
• GPU graphics processing unit
• the CPU and GPU group may implement services for the gaming application, including, in part, memory management, multi thread management, quality of service (QoS), bandwidth testing, social networking, management of social friends, communication with social networks of friends, communication channels, texting, instant messaging, chat support, etc.
• one or more applications share a particular GPU resource.
• multiple GPU devices may be combined to perform graphics processing for a single application that is executing on a corresponding CPU.
• cloud game network 190 is a distributed game server system and/or architecture.
• a distributed game engine executing game logic is configured as a corresponding instance of a corresponding game.
• the GPU and GPU group may include CPU 163 and GPU resources 365, which are configured for performing processor based functions include 2D or 3D rendering, physics simulation, scripting, audio, animation, graphics processing, lighting, shading, rasterization, ray tracing, shadowing, culling, transformation, artificial intelligence, etc.
• GPU resources 365 are responsible and/or configured for rendering of objects (e.g. writing color or normal vector values for a pixel of the object to multiple render targets - MRTs) and for execution of synchronous compute kernels (e.g. full screen effects on the resulting MRTs); the synchronous compute to perform, and the objects to render are specified by commands contained in rendering command buffers 325 that the GPU will execute.
• GPU resources 365 is configured to render objects and perform synchronous compute (e.g. during the execution of synchronous compute kernels) when executing commands from the rendering command buffers 325, wherein commands and/or operations may be dependent on other operations such that they are performed in sequence.
• GPU resources 365 are configured to perform synchronous compute and/or rendering of objects using one or more rendering command buffers 325 (e.g. rendering command buffer 325a, rendering buffer 325b ... rendering command buffer 325n).
• Each GPU in the GPU resources 365 may have their own command buffers, in one embodiment.
• the GPUs in GPU resources 365 may use the same command buffer or the same set of command buffers.
• each of the GPUs in GPU resources 365 may support the ability for a command to be executed by one GPU, but not by another.
• the output of the pixel processing stage 430 includes processed fragments (e.g., texture and shading information) and is delivered to the output merger stage 440 in the next stage of the graphics pipeline 400.
• the output merger stage 440 generates a final color for the pixel, using the output of the pixel processing stage 430, as well as other data, such as a value already in memory.
• the output merger stage 440 may perform optional blending of values between fragments and/or pixels determined from the pixel processing stage 430, and values already written to an MRT for that pixel.
• flow diagram 500 of FIG. 5 illustrates a method for graphics processing when implementing multi-GPU rendering of geometry for an image frame generated by an application by pretesting the geometry against interleaved screen regions before rendering, in accordance with one embodiment of the present disclosure. In that manner, multiple GPU resources are used to efficiently perform rendering of objects when executing an application.
• Geometry pretesting is typically in embodiments performed simultaneously for all geometry of a corresponding image frame by the plurality of GPUs. That is, each GPU performs geometry pretesting for its portion of the geometry of a corresponding image frame. In that manner, geometry pretesting by the GPUs allows each GPU to know which pieces of geometry to render, and also which pieces of geometry to skip. In particular, when a corresponding GPU performs geometry pretesting, it tests its portion of the geometry against the screen regions of each of the plurality of GPUs used for rendering the image frame. For example, if there are four GPUs, then each GPU may perform geometry testing on a quarter of the geometry of the image frame, especially if the geometry is assigned evenly to the GPUs for purposes of geometry testing.
• the method includes using the information at each of the plurality of GPUs when rendering the piece of geometry (e.g. to include fully rendering the piece of geometry or skipping the rendering of that piece of geometry). That is, the information is used at each of the plurality of GPUs to render the piece of geometry, wherein test results (e.g. information) of the geometry are sent to other GPUs, such that the information is known to each of the GPUs.
• the geometry e.g. pieces of geometry
• the image frame is typically in embodiments rendered simultaneously by the plurality of GPUs.
• that GPU will render that piece of geometry based on the information.
• the piece of geometry does not overlap any screen region assigned to the corresponding GPU for object rendering
• that GPU can skip rendering of that piece of geometry based on the information.
• the information allows all GPUs to more efficiently render geometry in an image frame, and/or to avoid rendering that geometry altogether.
• the rendering may be performed by shaders in a corresponding command buffer as executed by the plurality of GPUs.
• the shaders may be configured to perform one or both of geometry testing and/or rendering, based on corresponding GPU configurations.
• one GPU e.g. a pretest GPU
• the dedicated GPU is not used for rendering objects (e.g. pieces of geometry) in the corresponding image frame.
• graphics for an application are rendered using a plurality of GPUs, as previously described.
• responsibility for rendering geometry of the graphics is divided between the plurality of GPUs based on a plurality of screen regions, which may be interleaved, wherein each GPU has a corresponding division of the responsibility which is known to the plurality of GPUs.
• each GPU must still process most or all of the geometry. For example, it may be difficult to check object bounding boxes against all of the regions that a GPU is responsible for. Also, even if bounding boxes can be checked in a timely manner, due to small regions size, the result will be that each GPU likely has to process most of the geometry because every object in an image overlaps at least one regions of each of the GPUs (e.g. a GPU processes an entire object even though only a portion of the object overlaps at least one region in a set of regions assigned to that GPU).
• FIG. 7A In the piece of the rendering command buffer 700A shown in FIG. 7 A that illustrates one phase, there are four objects to be rendered (e.g., object 0, object 1, object 2, and object 3), as is shown in FIG. 7B-1.
• objects to be rendered e.g., object 0, object 1, object 2, and object 3
• GPUs are used for rendering objects in the image shown in screen 700B.
• Screen 700B is divided more finely than by quadrants as shown in FIG. 6A, in an effort to balance pixel and vertex load between the GPUs.
• screen 700B is divided into regions, that may be interleaved.
• the interleaving includes multiple rows of regions.
• Each of rows 731 and 733 includes region A alternating with region B.
• Each of rows 732 and 734 includes region C alternating with region D. More particularly, rows including regions A and B alternate with rows including regions C and D, in a pattern.
• FIG. 7B-2 illustrates a table showing the rendering performed by each GPU when rendering the four objects of FIG. 7B-1, in accordance with one embodiment of the present disclosure.
• object 0 is rendered by GPU-B
• object 1 is rendered by GPU-C and GPU-D
• object 2 is rendered by GPU-A, GPU-B, and GPU-D
• object 3 is rendered by GPU-B, GPU-C, and GPU-D.
• GPU A needs to render object 2 only
• GPU D needs to render objects 1, 2 and 3.
• FIG. 8B illustrates testing of portions of an object against screen regions and/or screen sub-regions when multiple GPUs collaborate to render a single image frame, in accordance with one embodiment of the present disclosure.
• the pieces of geometry can be portions of objects.
• object 810 may be split into pieces, such that the geometry used by or generated by a draw call is subdivided into smaller pieces of geometry.
• the pieces of geometry are each roughly the size for which the position cache and/or parameter cache are allocated.
• the information e.g. hint or hints
• the information are generated for those smaller pieces of geometry during geometry testing, wherein the information is used by the rendering GPU, as previously described.
• FIG. 9C illustrates pattern 900C of screen regions for screen 910.
• Each of the screen regions is not uniform in size. That is, screen regions for which GPUs are assigned responsibility for rendering objects may not be uniform in size.
• screen 910 is divided such that each GPU is assigned to an identical number of pixels. For example, iff a 4K display (3840x2160) were to be divided equally into four regions vertically, then each region would be 520 pixels tall. However, typically GPUs perform many operations in 32x32 blocks of pixels, and 520 pixels is not a multiple of 32 pixels.
• pattern 900C may include blocks that are at a height of 512 pixels (a multiple of 32), and other blocks that are at a height of 544 pixels (also a multiple of 32), in one embodiment. Other embodiments may use differently sized blocks.
• Pattern 900C shows equal amounts of screen pixels assigned to each GPU, by using non-uniform screen regions.
• object 1 e.g. as specified to be rendered by commands 724 in the rendering command buffer 700A
• object 2 e.g. as specified to be rendered by commands 724 in the rendering command buffer 700A
• the pieces may be ordered (e.g., a-o) for purposes of distributing responsibility for geometry testing to the GPUs.
• Distribution 1010 (e.g. the ABCDABCDABCD... row) shows an even distribution of the responsibility for performing geometry testing between a plurality of GPUs.
• one GPU take the first quarter of the geometry (e.g. in a block, such as GPU A takes the first four pieces of the approximately sixteen total pieces including “a”, “b”, “c” and “d” for geometry testing), and the second GPU take the second quarter, etc., assignment to GPUs is interleaved. That is, successive pieces of geometry are assigned to different GPUs.
• FIG. 1 IB is a flow diagram 1100B illustrating a method for graphics processing including pretesting and rendering of geometry of a previous image frame by a plurality of GPUs, and the use of statistics collected during rendering to influence the assignment of pretesting of geometry of a current image frame to the plurality of GPUs in the current image frame, in accordance with one embodiment of the present disclosure.
• the diagram of FIG. 11 A illustrates the use of statistics in the method of flow diagram 1100B to determine the distribution of assignments of geometry (e.g. pieces of geometry) between the GPUs for an image frame.
• the method includes assigning based on the statistics a second plurality of pieces of geometry of a current image frame generated by the application to the plurality of GPUs for geometry testing. That is, those statistics may be used to assign the same, fewer, or more pieces of geometry for geometry testing to a particular GPU when rendering the next, or current image frame. In some cases, the statistics may indicate that the pieces in the second plurality of pieces of geometry should be assigned evenly to the plurality of GPUs when performing geometry testing.
• the portion of the command buffer depicted in 1200A can be implicitly executed twice, e.g. by using special commands to mark beginning and end of the portion to execute twice, and to implicitly set a different configuration (e.g. a register setting) for the first and second executions of the portion of the command buffer.
• a different configuration e.g. a register setting
• the commands in the portion of the command buffer 1200A are executed (e.g., commands that set state or commands that execute a shader), based on GPU state, the results of the commands are different (e.g. result in performing geometry pretesting vs. performing rendering). That is, the commands in the command buffer 1200A may be configured for geometry pretesting or rendering.
• shader 1 is configured to perform rendering of object 1
• shader 2 is configured to perform rendering of object 2
• shader 3 is configured to perform rendering of object 3.
• FIG. 12B is a flow diagram 1200B illustrating a method for graphics processing including performing both pretesting and rendering of geometry of an image frame using the same set of shaders in two passes through a portion of the command buffer, in accordance with one embodiment of the present disclosure.
• various architectures may include multiple GPUs collaborating to render a single image by performing multi-GPU rendering of geometry for an application, such as within one or more cloud gaming servers of a cloud gaming system, or within a stand-alone system, such as a personal computer or gaming console that includes a high-end graphics card having multiple GPUs, etc.
• FIGS. 13A-13B illustrate another strategy for processing rendering command buffers.
• executed via commands 1311 and 1313) are configured for performing geometry pretesting on a first set of pieces of geometry, wherein after geometry testing is performed those same shaders (e.g. executed by commands 1311 and 1313) are then configured for performing rendering.
• other shaders e.g. executed via commands 1315 and 1317
• the command buffer After rendering is performed on the first set of pieces of geometry, other shaders (e.g. executed via commands 1315 and 1317) in the command buffer are configured for performing geometry pretesting on a second set of pieces of geometry, wherein after geometry pretesting is performed those same shaders (e.g. executed via commands 1315 and 1317) are then configured for performing rendering, and rendering is performed using those commands on the second set of pieces of geometry.
• the benefit of this strategy is that imbalance between GPUs can be addressed dynamically, such as by using asymmetric interleaving of geometry testing throughout the rendering. An example of asymmetric interleaving of geometry testing was previously introduced in distribution 102 of FIG. 10.
• the second set of shaders is then used for geometry testing and rendering of the second set of pieces of geometry.
• the second set of shaders of a command buffer is configured to perform geometry pretesting on the second set of pieces of geometry, as previously described.
• geometry testing is performed at the plurality of GPUs on the second set of pieces of geometry to generate second information regarding each piece of geometry in the second set and its relation to each of the plurality of screen regions.
• the second set of shaders is configured to perform rendering of the second set of pieces of geometry, as previously described.
• rendering of the second set of pieces of geometry is performed at each of the plurality of GPUs using the second information.
• the information indicates which pieces of geometry overlap screen regions (e.g. of a corresponding set) assigned to a corresponding GPU for object rendering.
• Storage 1406 provides non-volatile storage and other computer readable media for applications and data and may include fixed disk drives, removable disk drives, flash memory devices, and
• Embodiments of the present disclosure may be practiced with various computer system configurations including hand-held devices, microprocessor systems, microprocessor- based or programmable consumer electronics, minicomputers, mainframe computers and the like. Embodiments of the present disclosure can also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a wire -based or wireless network.
• embodiments of the present disclosure can employ various computer-implemented operations involving data stored in computer systems. These operations are those requiring physical manipulation of physical quantities. Any of the operations described herein that form part of embodiments of the present disclosure are useful machine operations. Embodiments of the disclosure also relate to a device or an apparatus for performing these operations.
• the apparatus can be specially constructed for the required purpose, or the apparatus can be a general-purpose computer selectively activated or configured by a computer program stored in the computer.
• various general-purpose machines can be used with computer programs written in accordance with the teachings herein, or it may be more convenient to construct a more specialized apparatus to perform the required operations.
• the disclosure can also be embodied as computer readable code on a computer readable medium.
• the computer readable medium is any data storage device that can store data, which can be thereafter be read by a computer system. Examples of the computer readable medium include hard drives, network attached storage (NAS), read-only memory, random- access memory, CD-ROMs, CD-Rs, CD-RWs, magnetic tapes and other optical and non-optical data storage devices.
• the computer readable medium can include computer readable tangible medium distributed over a network-coupled computer system so that the computer readable code is stored and executed in a distributed fashion.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Theoretical Computer Science (AREA)
• Computer Graphics (AREA)
• Computer Hardware Design (AREA)
• Image Generation (AREA)
• Processing Or Creating Images (AREA)

Applications Claiming Priority (4)

Publications (1)

Family

ID=74701593

Family Applications (1)

Country Status (4)

Families Citing this family (2)

Family Cites Families (8)

• 2021
  • 2021-02-01 JP JP2022546703A patent/JP7334358B2/ja active Active
  • 2021-02-01 CN CN202180023019.6A patent/CN115298686B/zh active Active
  • 2021-02-01 WO PCT/US2021/016079 patent/WO2021158483A1/en unknown
  • 2021-02-01 EP EP21707864.1A patent/EP4100922A1/en active Pending
• 2023
  • 2023-08-16 JP JP2023132609A patent/JP7481556B2/ja active Active

Also Published As

Similar Documents

Legal Events