TW201719570A - Apparatus and method for pattern driven self-adaptive virtual graphics processor units - Google Patents

Abstract

An apparatus and method are described for a pattern driven self-adaptive graphics processing unit (GPU). For example, one embodiment of an apparatus comprises: a graphics processing unit (GPU) to process graphics commands and responsively render a plurality of image frames; a hypervisor to virtualize the GPU to share the GPU among a plurality of virtual machines (VMs), the hypervisor subdividing GPU processing for each VM into a plurality of quanta; and scheduling logic to monitor GPU utilization including GPU busy time and/or GPU idle time for each VM during its allocated quantum and to store utilization data reflecting the GPU utilization during each quantum; the scheduling logic to predict a cost of waiting within a given quantum of a first VM based on the utilization data and to further predict a cost of yielding to a second VM, the scheduling logic to yield the GPU to the second VM if the cost of waiting is greater than the cost of yielding to the second VM.

Description

Apparatus and method for pattern driven adaptive virtual drawing processing unit

The present invention generally relates to the field of computer processors. More specifically, the present invention relates to an apparatus and method for a pattern driven adaptive virtual graphics processing unit (vGPU).

Various methods have been developed for drawing virtualization. For example, one approach allows the power of the entire GPU to be directly assigned to a single user, passing native driver functionality via a hypervisor without any restrictions. The generic name for this version of drawing virtualization is the Direct Graphics Adaptor (vDGA). Another virtualization approach requires a virtual drawing driver in the virtual machine and uses API forwarding technology to interface with the drawing hardware. In a recent method of sharing GPUs between multiple concurrent users, each virtual desktop machine maintains a copy of the native graphics driver. Based on the time sliced, the agent in the hypervisor directly assigns the complete GPU resource to each virtual machine. Therefore, during its time segment, although the virtual machine is connected With a full dedicated GPU, from a system perspective, multiple virtual machines share a single GPU.

100‧‧‧Processing system

102‧‧‧Processor

104‧‧‧Cache memory

106‧‧‧Scratch file

107‧‧‧Processor core

108‧‧‧Drawing Processor

109‧‧‧Instruction Set

110‧‧‧Processor bus

112‧‧‧External graphics processor

116‧‧‧Memory Controller Hub

120‧‧‧ memory device

121‧‧‧ directive

122‧‧‧Information

124‧‧‧Data storage device

126‧‧‧Wireless transceiver

128‧‧‧ Firmware interface

130‧‧‧Input/Output (I/O) Controller Hub

134‧‧‧Network Controller

140‧‧‧Traditional I/O Controller

142‧‧‧Common Serial Bus (USB) Controller

144‧‧‧ keyboard and mouse

146‧‧‧ audio controller

200‧‧‧ processor

202A-202N‧‧‧ Processor Core

204A-204N‧‧‧Internal cache unit

206‧‧‧Shared cache unit

208‧‧‧Drawing processor

210‧‧‧System Agent Core

211‧‧‧ display controller

212‧‧‧Circular-based interconnect unit/ring interconnect

213‧‧‧I/O link

214‧‧‧ memory controller

216‧‧‧ Busbar Controller Unit

218‧‧‧ Embedded Memory Module

300‧‧‧Drawing Processor

302‧‧‧ display controller

304‧‧‧ Block Image Transfer (BLIT) Engine

306‧‧‧Video Codec Engine

310‧‧‧Drawing Processing Engine (GPE)

312‧‧3D pipeline

314‧‧‧ memory interface

315‧‧‧3D/media subsystem

316‧‧‧Media pipeline

320‧‧‧ display device

410‧‧‧Drawing Processing Engine

403‧‧‧Command Streamer

412‧‧‧Media pipeline

414‧‧‧Execution unit array

416‧‧‧Media pipeline

430‧‧‧Sampling engine

432‧‧‧To noise/deinterlacing module

434‧‧‧Sports estimation module

436‧‧‧Image scaling and filtering module

444‧‧‧Information埠

500‧‧‧Drawing Processor

502‧‧‧ Ring Interconnect

503‧‧‧Command Streamer

504‧‧‧ pipeline front end

534‧‧‧Video front end

536‧‧‧Geometric pipeline

530‧‧·Video Quality Engine (VQE)

533‧‧‧Multi-format encoding/decoding (MFX)

537‧‧‧Media Engine

550A-550N‧‧‧Subcore

552A-552N‧‧‧Execution unit

554A-554N‧‧‧Media/Texture Sampler

570A-570N‧‧‧Shared resources

560A-560N‧‧‧Subcore

562A-562N‧‧‧Execution unit

564A-564N‧‧‧ sampler

580A-580N‧‧‧ drawing core

600‧‧‧Thread Execution Logic

602‧‧‧ pixel shader

604‧‧‧Thread Dispatcher

606‧‧‧ instruction cache

608A-608N‧‧‧Execution unit

610‧‧‧sampler

612‧‧‧Information cache

614‧‧‧Information埠

800‧‧‧Drawing Processor

802‧‧‧ ring interconnect

803‧‧‧Command Streamer

805‧‧‧Vertex Extractor

807‧‧‧Vertex Shader

811‧‧‧Shell shader

813‧‧‧Diagram

817‧‧‧Domain Shader

819‧‧‧Geometry shader

820‧‧‧Drawing pipeline

823‧‧‧Stream output unit

829‧‧‧Cutter

830‧‧‧Media pipeline

834‧‧ ‧ video front end

837‧‧‧Media Engine

831‧‧‧Thread Dispatcher

840‧‧‧Display engine

841‧‧‧2D engine

843‧‧‧ display controller

850‧‧‧Thread Execution Logic

851‧‧‧L1 cache

852A, 852B‧‧‧ execution unit

854‧‧‧Texture and media sampler

856‧‧‧Information埠

858‧‧‧Texture/sampler cache

870‧‧‧ Rendering output pipeline

873‧‧‧Rasterization and depth test components

875‧‧‧L3 cache

877‧‧‧pixel operating elements

878‧‧‧ Rendering cache

879‧‧‧Deep cache

1000‧‧‧Data Processing System

1010‧‧‧3D drawing application

1012‧‧‧ Shader Instructions

1014‧‧‧executable instructions

1016‧‧‧ Drawing objects

1020‧‧‧ operating system

1022‧‧‧Drawing API

1024‧‧‧front-end shader compiler

1026‧‧‧User mode drawing driver

1027‧‧‧Backend shader compiler

1028‧‧‧Operating system core mode function

1029‧‧‧core mode drawing driver

1030‧‧‧ Processor

1032‧‧‧Drawing processor

1034‧‧‧General Processor Core

1050‧‧‧ system memory

1100‧‧‧IP Core Development System

1110‧‧‧Software simulation

1115‧‧‧RTL design

1120‧‧‧ hardware model

1130‧‧‧Design facilities

1140‧‧‧ Non-volatile memory

1150‧‧‧Wired connection

1160‧‧‧Wireless connection

1165‧‧‧ Manufacturing facilities

1200‧‧‧ system single chip integrated circuit

1205‧‧‧Application Processor

1210‧‧‧Drawing Processor

1215‧‧‧Image Processor

1220‧‧‧Video Processor

1225‧‧‧USB controller

1230‧‧‧UART controller

1235‧‧‧SPI/SDIO Controller

1240‧‧‧I ² S/I ² C controller

1245‧‧‧Display device

1250‧‧‧High-resolution multimedia interface (HDMI) controller

1255‧‧‧Mobile Industry Processor Interface (MIPI) display interface

1260‧‧‧Flash Memory Subsystem

1265‧‧‧ memory controller

1270‧‧‧ Embedded Security Engine

1400‧‧‧Exemplary embodiment

1410‧‧‧Super Manager

1412‧‧‧GPU scheduler

1470‧‧‧PDSAS logic

1418‧‧‧Command Profiler

1420‧‧‧GPU

1460A, 1460B‧‧‧Virtual GPU (vGPU)

1430‧‧‧VM

1440‧‧‧VM

A better understanding of the present invention can be obtained by the following detailed description of the accompanying drawings in which: FIG. 1 is a block diagram of an embodiment of a computer system having a processor and a graphics processor having one or more processor cores 2 is a block diagram of one embodiment of a processor having one or more processor cores, an integrated memory controller, and an integrated graphics processor; FIG. 3 is an embodiment of a graphics processor Block diagram, the graphics processor may be an individual graphics processing unit, or may be a graphics processor integrated with a plurality of processing cores; FIG. 4 is a block diagram of an embodiment of a graphics processing engine for a graphics processor; Figure 4 is a block diagram of another embodiment of a graphics processor; Figure 6 is a block diagram of thread execution logic including an array of processing elements; Figure 7 depicts a graphics processor execution unit instruction format in accordance with an embodiment; 8 is a block diagram of another embodiment of a graphics processor including a graphics pipeline, a media pipeline, a display engine, thread execution logic, and rendering (r Derender) output pipeline; 9A is a block diagram showing a command format of a graphics processor according to an embodiment; FIG. 9B is a block diagram showing a sequence of command of a graphics processor according to an embodiment; and FIG. 10 is a diagram showing a data processing system according to an embodiment; Exemplary Graphics Software Architecture; Figure 11 depicts an exemplary IP core development system in accordance with an embodiment that can be used to fabricate integrated circuitry to perform operations; Figure 12 depicts an exemplary system single wafer integrated circuitry in accordance with an embodiment , which may be fabricated using one or more IP cores; Figure 13 depicts an exemplary GPU resource utilization under a loop scheduler; Figure 14 depicts an embodiment of a GPU scheduler, the GPU scheduler implementation Sample Drive Adaptive Scheme (PDSAS); Figure 15 depicts an exemplary possible curve for idle from an exemplary 3D workload; Figure 16 depicts an exemplary distribution curve for possible latency; Figures 17A-B depict the invention in accordance with the present invention The method of one embodiment; FIG. 18 depicts the operation of one embodiment of the present invention for different time values, switching cost values, and thresholds; and FIG. 19 depicts an exemplary probability curve indicating the probability of different time values Take And include the balance factor.

SUMMARY OF THE INVENTION AND EMBODIMENT

In the following description, for the purpose of explanation, several specific details are proposed. To provide a thorough understanding of the embodiments of the invention described below. However, it is apparent to those skilled in the art that the embodiments of the invention may be practiced without the specific details. In other instances, well-known structures and devices are shown in block diagrams in order to avoid obscuring the basic principles of the embodiments of the invention.

Exemplary graphics processor architecture and data type System overview

FIG. 1 is a block diagram of a processing system 100 in accordance with an embodiment. In various embodiments, system 100 includes one or more processors 102 and one or more graphics processors 108, and can be a single processor desktop system, a multi-processor workstation system, or have a large number of processors 102 Or the server system of processor core 107. In one embodiment, system 100 is a processing platform incorporated into a system single-chip (SoC) integrated circuit for mobile, handheld, or embedded devices.

Embodiments of system 100 may include a server-based gaming platform, a gaming machine (including gaming and media consoles, a mobile gaming machine, a handheld gaming machine, or an online gaming machine), or incorporated within the foregoing. In some embodiments, system 100 is a mobile phone, a smart phone, a tablet computing device, or a mobile internet device. The data processing system 100 can also include, be coupled to, or be integrated into a wearable device (such as a smart watch wearable device, a smart eyewear device, an augmented reality device, or a virtual reality device). In some embodiments, data processing system 100 is a television or set-top box device having one or more processors 102 and produced by one or more graphics processors 108 The raw drawing interface.

In some embodiments, the one or more processors 102 each include one or more processor cores 107 to process instructions that, when executed, perform operations for the system and user software. In some embodiments, each of the one or more processor cores 107 is configured to process a particular set of instructions 109. In some embodiments, the set of instructions 109 may facilitate computation of complex instruction set calculations (CISC), reduced instruction set calculations (RISC), or through very long instruction words (VLIW). Multiple processor cores 107 may each process a different set of instructions 109, which may include instructions that facilitate simulation of other sets of instructions. Processor core 107 may also include other processing devices, such as a digital signal processor (DSP).

In some embodiments, processor 102 includes cache memory 104. Depending on the architecture, processor 102 can have a single internal cache or multiple internal caches. In some embodiments, cache memory is shared between various elements of processor 102. In some embodiments, processor 102 also uses an external cache (eg, a third order (L3) cache or a last order cache (LLC)) (not shown), which can use known cache coherency Sex technology is shared between processor cores 107. The processor 102 also includes a scratchpad file 106 that can include different types of registers for storing different types of data (eg, integer registers, floating point registers, status registers, and instructions) Indicator register). Some registers may be general purpose registers, while other registers may be dedicated to the design of processor 102.

In some embodiments, processor 102 is coupled to processor bus 110 for transmission between processor 102 and other components in system 100. Communication signals such as address, data, or control signals. In one embodiment, system 100 uses an exemplary "hub" system architecture, including a memory controller hub 116 and an input/output (I/O) controller hub 130. The memory controller hub 116 facilitates communication between the memory device and other components of the system 100, while the I/O controller hub (ICH) 130 provides a connection via the I/O bus to the I/O device. In one embodiment, the logic of the memory controller hub 116 is integrated within the processor.

The memory device 120 can be a dynamic random access memory (DRAM) device, a static random access memory (SRAM) device, a cache memory device, a phase change memory device, or some other suitable function as a processing memory. Other memory devices of the body. In one embodiment, memory device 120 is operative for system memory of system 100 to store data 122 and instructions 121 for use by one or more processors 102 when executing applications or processes. Memory controller hub 116 is also coupled to optional external graphics processor 112, which can communicate with one or more graphics processors 108 in processor 102 to perform graphics and media operations.

In some embodiments, ICH 130 enables peripheral devices to connect to memory device 120 and processor 102 via a high speed I/O bus. I/O peripherals include, but are not limited to, an audio controller 146, a firmware interface 128, a wireless transceiver 126 (eg, Wi-Fi, Bluetooth), a data storage device 124 (eg, a hard disk drive, cache) Memory, etc.), and legacy I/O controller 140 for coupling legacy (eg, Personal System 2 (PS/2)) devices to the system. One or more universal serial bus (USB) controllers 142 are connected An input device such as a combination of a keyboard and a mouse 144. Network controller 134 may also be coupled to ICH 130. In some embodiments, a high performance network controller (not shown) is coupled to the processor bus. It should be understood that the illustrated system 100 is illustrative and not limiting, as other types of data processing systems of different configurations may also be used. For example, I/O controller hub 130 can be integrated into one or more processors 102, or memory controller hub 116 and I/O controller hub 130 can be integrated into individual external graphics processors, such as External graphics processor 112.

2 is a block diagram of an embodiment of a processor 200 having one or more processor cores 202A-202N, an integrated memory controller 214, and an integrated graphics processor 208. The elements of Figure 2 having the same reference numbers (or names) as the elements of any other figures herein may operate or function in any manner similar to that described herein, but are not limited thereto. Processor 200 may include additional cores up to and including additional cores 202N represented by dashed boxes. Each of processor cores 202A-202N includes one or more internal cache units 204A-204N. In some embodiments, each processor core may also access one or more shared cache units 206.

Internal cache units 204A-204N and shared cache unit 206 represent cache memory levels within processor 200. The cache memory hierarchy can include at least one order of instruction and data caches within each processor core, and one or more orders of shared intermediate caches, such as second order (L2), third order ( L3), 4th order (L4), or other order cache, where the highest order cache before the external memory is classified as LLC. In some embodiments, cache coherency logic maintains between various cache units 206 and 204A-204N consistency.

In some embodiments, processor 200 can also include a set of one or more bus controller unit 216 and system agent core 210. One or more bus controller units 216 manage a set of peripheral busses, such as one or more peripheral component interconnect busses (eg, PCI, PCI Express). System agent core 210 provides management functions for various processor elements. In some embodiments, system agent core 210 includes one or more integrated memory controllers 214 to manage access to various external memory devices (not shown).

In some embodiments, one or more processor cores 202A-202N include support for simultaneous multi-threading. In such an embodiment, system agent core 210 includes elements for coordinating and operating cores 202A-202N during multi-threading processing. System agent core 210 may also include a power control unit (PCU) that includes logic and elements to condition the power states of processor cores 202A-202N and graphics processor 208.

In some embodiments, processor 200 also includes a graphics processor 208 for performing graphics processing operations. In some embodiments, the graphics processor 208 is coupled to the set of shared cache units 206 and to the system proxy core 210 that includes one or more integrated memory controllers 214. In some embodiments, display controller 211 is coupled to graphics processor 208 to drive the graphics processor output to one or more coupled displays. In some embodiments, display controller 211 can be a separate module coupled to the graphics processor via at least one interconnect, or can be integrated within graphics processor 208 or system proxy core 210.

In some embodiments, the ring-based interconnect unit 212 is used to couple internal components of the processor 200. However, alternative interconnect units may be used, such as point-to-point interconnects, switched interconnects, or other techniques, including techniques well known in the art. In some embodiments, graphics processor 208 is coupled to ring interconnect 212 via I/O link 213.

The exemplary I/O link 213 represents at least one of a variety of I/O interconnects, including facilitating communication between various processor elements and a high performance embedded memory module 218, such as an eDRAM module. The I/O interconnect on the package. In some embodiments, each of processor cores 202A-202N and graphics processor 208 uses embedded memory module 218 as a shared last-order cache memory.

In some embodiments, processor cores 202A-202N are homogeneous cores that implement the same instruction set architecture. In another embodiment, processor cores 202A-202N are heterogeneous with respect to an instruction set architecture (ISA), wherein one or more of processor cores 202A-N execute a first set of instructions, while in other cores At least one of them performs a subset of the first set of instructions or a different set of instructions. In one embodiment, in terms of microarchitecture, processor cores 202A-202N are heterogeneous, with one or more cores having relatively higher power consumption coupled to one or more cores having lower power consumption. Moreover, processor 200 can be implemented on one or more wafers or as an SoC integrated circuit with the elements shown, among other components.

3 is a block diagram of a graphics processor 300, which may be an individual graphics processing unit, or may be a graphics processor integrated with a plurality of processing cores. In some embodiments, the graphics processor is via The memory mapped I/O interface of the scratchpad on the graphics processor communicates with commands placed in the processor memory. In some embodiments, the graphics processor 300 includes a memory interface 314 for accessing memory. The memory interface 314 can be an interface to a region memory, one or more internal caches, one or more shared external caches, and/or system memory.

In some embodiments, the graphics processor 300 also includes a display controller 302 to drive display output data to the display device 320. Display controller 302 includes one or more stacked planar hardware for display and synthesis of multiple video layers or user interface elements. In some embodiments, the graphics processor 300 includes a video codec engine 306 for encoding media to one or more media encoding formats, decoding media from one or more media encoding formats, or encoding one or more media. Transcoding media between formats, including, but not limited to, an Animation Experts Group (MPEG) format such as MPEG-2, Advanced Video Coding (AVC) such as H.264/MPEG-4 AVC Format, and Society of Motion Picture and Television Engineers (SMPTE) 421M/VC-1, Joint Photographic Experts Group (JPEG) format such as JPEG, and Motion JPEG (MJPEG) format.

In some embodiments, graphics processor 300 includes a block image transfer (BLIT) engine 304 to perform a two-dimensional (2D) rasterization operation including, for example, bit-boundary block transfer. However, in one embodiment, one or more components of the graphics processing engine (GPE) 310 are used to perform a 2D drawing operation. In some embodiments, the drawing processing engine 310 is a computing engine for performing drawing operations including three-dimensional (3D) drawing operations and media operations.

In some embodiments, GPE 310 includes a 3D pipeline 312 for performing 3D operations, such as rendering a three-dimensional image and scene using a processing function that acts on a 3D primitive shape (eg, rectangle, triangle, etc.). The 3D pipeline 312 includes programmable and fixed function elements that perform various tasks within the component and/or spawn execution threads to the 3D/media subsystem 315. Although the 3D pipeline 312 can be used to perform media operations, the embodiment of the GPE 310 also includes a media pipeline 316 that is specifically used to perform media operations such as post-visual processing and image enhancement.

In some embodiments, media pipeline 316 includes fixed functions or programmable logic units to perform one or more dedicated media operations, such as video decoding acceleration, video de-interlacing, in place of or on behalf of video codec engine 306, And video encoding acceleration. In some embodiments, media pipeline 316 also includes a thread spawning unit to generate threads for execution on 3D/media subsystem 315. The generated threads perform calculations for media operations on one or more graphics execution units included in 3D/media subsystem 315.

In some embodiments, 3D/media subsystem 315 includes logic for executing threads generated by 3D pipeline 312 and media pipeline 316. In one embodiment, the pipeline delivery threads execute requests to the 3D/media subsystem 315, which includes thread dispatch logic for arbitrating and dispatching various requests to available thread execution resources. The execution resources include an array of graphics execution units for processing 3D and media threads. In some embodiments, 3D/media subsystem 315 includes one or more internal caches for thread instructions and material. In some embodiments, the subsystem also includes a shared note Recall, which includes a scratchpad and addressable memory for sharing data between threads and storing output data.

3D/media processing

4 is a block diagram of a graphics processing engine 410 of a graphics processor in accordance with some embodiments. In one embodiment, GPE 410 is the version of GPE 310 shown in FIG. The elements of Figure 4 having the same reference numbers (or names) as the elements of any other figures herein may operate or function in any manner similar to that described elsewhere herein, but are not limited thereto.

In some embodiments, GPE 410 is coupled to command streamer 403, which provides a command stream to GPE 3D and media pipelines 412, 416. In some embodiments, the command streamer 403 is coupled to a memory, which may be system memory, or one or more of internal cache memory and shared cache memory. In some embodiments, command streamer 403 receives commands from memory and transmits the commands to 3D pipeline 412 and/or media pipeline 416. The commands are directives extracted from a ring buffer that stores commands for the 3D and media pipelines 412, 416. In one embodiment, the ring buffer may also include a batch command buffer that stores a batch of multiple commands. The 3D and media pipelines 412, 416 process the commands by performing operations via logic within the individual pipelines or by dispatching one or more execution threads to the execution unit array 414. In some embodiments, the execution unit array 414 is expandable such that the array includes a variable number of execution units based on the target power and performance level of the GPE 410.

In some embodiments, the sampling engine 430 is coupled to a memory (eg, a cache or system memory) and an array of execution units 414. In some embodiments, the sampling engine 430 provides a memory access mechanism to the execution unit array 414 that allows the execution array 414 to read graphics and media material from memory. In some embodiments, the sampling engine 430 includes logic to perform specialized image sampling operations for the media.

In some embodiments, the specialized media sampling logic in the sampling engine 430 includes a de-noising/de-interlacing module 432, a motion estimation module 434, and an image scaling and filtering module 436. In some embodiments, the denoising/deinterlacing module 432 includes logic to perform one or more of a denoising or deinterlacing algorithm on the decoded video material. The de-interlacing logic combines alternating fields of interlaced video content into a single frame of video. The de-noising logic reduces or removes data noise from video and video data. In some embodiments, the de-noising logic and de-interlacing logic are dynamically adaptive and use spatial or temporal filtering based on the amount of motion detected in the video material. In some embodiments, the denoising/de-interlacing module 432 includes dedicated motion detection logic (eg, within the motion estimation engine 434).

In some embodiments, motion estimation engine 434 provides hardware acceleration for video operations by performing video acceleration functions such as motion vector estimation and prediction on video data. The motion estimation engine determines a motion vector that describes the transformation of the image data between consecutive video frames. In some embodiments, the graphics processor media codec uses video motion estimation engine 434 to perform video operations at the macroblock level, which may be too computationally intensive if executed by a general purpose processor. In some embodiments, motion estimation The metering engine 434 is typically used by the graphics processor component to assist in video decoding and processing functions that are sensitive or adaptive to the direction or magnitude of motion within the video material.

In some embodiments, image scaling and filtering module 436 performs image processing operations to improve the visual quality of the resulting images and video. In some embodiments, the scaling and filtering module 436 processes the image and video material during a sampling operation prior to providing the data to the execution unit array 414.

In some embodiments, GPE 410 includes data 444, which provides an additional mechanism for the graphics subsystem to access memory. In some embodiments, the data 444 facilitates memory storage including rendering target writes, constant buffer reads, scratch memory space read/write, and media surface access operations. take. In some embodiments, the data 444 includes a cache memory space to cache access to the memory. The cache memory can be a single data cache or divided into multiple caches (eg, render buffer cache, constant buffer cache, etc.) for accessing multiple subsystems of memory via data. In some embodiments, threads executing on execution units in execution unit array 414 communicate with the data stream by exchanging messages via data distribution interconnections coupled to each of the subsystems of GPE 410.

Execution unit

FIG. 5 is a block diagram of another embodiment of a graphics processor 500. An element of Figure 5 having the same reference number (or name) as the elements of any other figures herein may operate or perform work in any manner similar to that described elsewhere herein. Use, but not limited to.

In some embodiments, graphics processor 500 includes a ring interconnect 502, a pipeline front end 504, a media engine 537, and graphics cores 580A-580N. In some embodiments, the ring interconnect 502 couples the graphics processor to other processing units, including other graphics processors or one or more general purpose processor cores. In some embodiments, the graphics processor is one of a number of processors integrated within a multi-core processing system.

In some embodiments, graphics processor 500 receives a batch of commands via ring interconnect 502. The incoming command is interpreted by command stream 503 in pipeline front end 504. In some embodiments, graphics processor 500 includes scalable execution logic to perform 3D geometry processing and media processing via graphics cores 580A-580N. Command streamer 503 provides commands to geometry pipeline 536 for 3D geometry processing commands. For at least some of the media processing commands, the command streamer 503 provides the commands to the video front end 534, which is coupled to the media engine 537. In some embodiments, media engine 537 includes a video quality engine (VQE) 530 for video and post-image processing and a multi-format encoding/decoding (MFX) 533 engine to provide hardware accelerated encoding and decoding of media data. In some embodiments, geometry pipeline 536 and media engine 537 each generate an execution thread for thread execution resources provided by at least one drawing core 580A.

In some embodiments, graphics processor 500 includes scalable thread execution resources characterized by modular cores 580A-580N (sometimes referred to as core slices), each core having multiple sub-cores 550A-550N, 560A-560N ( Sometimes called a core sub-slice). In some embodiments, the graphics processor 500 There may be any number of graphics cores 580A through 580N. In some embodiments, the graphics processor 500 includes a graphics core 580A having at least a first sub-core 550A and a second sub-core 560A. In other embodiments, the graphics processor is a low power processor with a single sub-core (eg, 550A). In some embodiments, the graphics processor 500 includes a plurality of graphics cores 580A-580N, each including a collection of first sub-cores 550A-550N and a collection of second sub-cores 560A-560N. Each of the sub-cores in the set of first sub-cores 550A-550N includes at least a first set of execution units 552A-552N and media/texture samplers 554A-554N. Each of the sub-cores in the set of second sub-cores 560A-560N includes at least a second set of execution units 562A-562N and samplers 564A-564N. In some embodiments, each sub-core 550A-550N, 560A-560N shares a common set of resources 570A-570N. In some embodiments, the shared resources include shared cache memory and pixel operation logic. Other shared resources may also be included in various embodiments of the graphics processor.

6 illustrates thread execution logic 600 that includes an array of processing elements employed in some embodiments of GPE. The elements of Figure 6 having the same reference numbers (or names) as the elements of any other figures herein may operate or function in any manner similar to that described elsewhere herein, but are not limited thereto.

In some embodiments, thread execution logic 600 includes a pixel shader 602, a thread dispatcher 604, an instruction cache 606, an expandable execution unit array (including a plurality of execution units 608A-608N), a sampler 610, a data cache 612. And information 埠 614. In one embodiment, the included components are interconnected via interconnect structures that are linked to the various components. In some embodiments The thread execution logic 600 includes one or more of the memory via the instruction cache 606, the data buffer 614, the sampler 610, and the execution unit array 608A-608N to a memory such as system memory or cache memory. Multiple connections. In some embodiments, each execution unit (eg, 608A) is an individual vector processor capable of executing multiple simultaneous threads and processing multiple data elements in parallel for each thread. In some embodiments, execution unit arrays 608A-608N include any number of individual execution units.

In some embodiments, execution unit arrays 608A-608N are primarily used to execute "shader" programs. In some embodiments, the execution units in arrays 608A-608N perform a set of instructions that include native support for a number of standard 3D graphics shader instructions such that execution from graphics libraries (eg, Direct 3D and OpenGL) is performed with minimal conversion. Shader program. Execution units support vertex and geometry processing (eg, vertex programs, geometry programs, vertex shaders), pixel processing (eg, pixel shaders, fragment shaders), and general processing (eg, computation and media shaders).

Each of the execution unit arrays 608A-608N operates on an array of data elements. The number of data elements is the "execution size" or the number of channels used for the instruction. The execution channel is a logic unit for the execution of data element access, masking, and flow control within the instruction. For a particular graphics processor, the number of channels can be independent of the number of physical arithmetic logic units (ALUs) or floating point units (FPUs). In some embodiments, execution units 608A-608N support integer and floating point data types.

The execution unit instruction set includes a single instruction multiple data (SIMD) instruction. Various data components can be stored in the scratchpad as package data types and executed The unit will process the various components based on the size of the component's data. For example, when operating on a 256-bit wide vector, the 256-bit vector of the vector is stored in a scratchpad, and the execution unit uses the vector as four separate 64-bit packed data elements (four words ( QW) size data element), eight independent 32-bit package data elements (double word (DW) size data elements), sixteen independent 16-bit package data elements (word (W) size data elements), or Thirty-two independent 8-bit data elements (bytes (B) size data elements) are manipulated. However, different vector widths and scratchpad sizes are possible.

One or more internal instruction caches (e.g., 606) are included in thread execution logic 600 to cache thread instructions for execution units. In some embodiments, one or more data caches (eg, 612) are included to cache thread data during thread execution. In some embodiments, a sampler 610 is included to provide texture sampling for 3D operations and media sampling for media operations. In some embodiments, sampler 610 includes a dedicated texture or media sampling function to process texture or media material during the sampling process prior to providing the sampled material to the execution unit.

During execution, the graphics and media pipeline passes thread initialization requests to thread execution logic 600 via thread generation and dispatch logic. In some embodiments, thread execution logic 600 includes a local thread dispatcher 604 that arbitrates thread initialization requests from the drawing and media pipelines and instantiates the requested threads on one or more execution units 608A-608N. For example, a geometry pipeline (eg, 536 of Figure 5) dispatches vertex processing, tessellation, or geometry processing threads to thread execution logic. 600 (Figure 6). In some embodiments, thread dispatcher 604 can also process execution time thread generation requests from the execution shader program.

Once a set of geometric objects have been processed and rasterized into pixel data, pixel shader 602 is invoked to further calculate the output information and cause the results to be written to the output surface (eg, color buffer, depth changer, template) Buffers, etc.). In some embodiments, pixel shader 602 calculates values of various vertex attributes to be interpolated between rasterized objects. In some embodiments, pixel shader 602 then executes a pixel shader program that supplies an application interface (API). To execute the pixel shader program, pixel shader 602 dispatches the thread to the execution unit (eg, 608A) via thread dispatcher 604. In some embodiments, pixel shader 602 uses texture sampling logic in sampler 610 to access texture data stored in texture maps in memory. Arithmetic operations on texture data and input geometry calculate pixel color data for each geometry, or discard one or more pixels to avoid further processing.

In some embodiments, data file 614 provides thread execution logic 600 - a memory access mechanism that outputs processed data to memory for processing on the graphics processor output pipeline. In some embodiments, the data cartridge 614 includes or is coupled to one or more cache memories (eg, data cache 612) to cache data for memory access via the data cartridge.

FIG. 7 is a block diagram of a graphics processor instruction format 700 in accordance with some embodiments. In one or more embodiments, the graphics processor execution unit supports a set of instructions having instructions in multiple formats. The implementation box shows the components that are typically included in the execution unit instructions, while the dashed lines include optional or only Components included in a subset of instructions. In some embodiments, the described and illustrated instruction formats 700 are macro instructions because they are instructions that are supplied to the execution unit, as opposed to micro-operations that result when the instruction is processed, ie, the instruction is decoded.

In some embodiments, the graphics processor execution unit natively supports the instructions of the 128-bit format 710. The 64-bit compaction command format 730 can be used for some instructions based on selected instructions, instruction options, and operand numbers. The native 128-bit format 710 provides access to all instruction options, while some options and operations are limited in the 64-bit format 730. Native instructions available in 64-bit format 730 vary from embodiment to embodiment. In some embodiments, a set of index values in an index field 713 is used to partially compact the instructions. The execution unit hardware references a set of compact tables based on the index values and uses the compact table output to reconstruct the native instructions of the 128-bit format 710.

For each format, the instruction opcode 712 defines the operations to be performed by the execution unit. The execution unit executes the respective instructions in parallel across a plurality of data elements of the respective operands. For example, in response to the add instruction, the execution unit performs a simultaneous add operation across the various color channels representing the texture element or picture element. By default, the execution unit executes each instruction across all data channels of the operand. In some embodiments, the instruction control field 714 enables control of certain execution options, such as channel selection (eg, prediction) and data channel order, eg, data swizzle. For 128-bit instructions 710, the execution size (exec-size) field 716 limits the number of data channels that will be executed in parallel. In some embodiments, the execution size field 716 is not suitable for use in the 64-bit compaction command format 730.

Some execution unit instructions have up to three operands, including two source operands src0 722, src1 722, and a destination 718. In some embodiments, the execution unit supports dual destination instructions in which one of the destinations is implied. The data manipulation instruction may have a third source operand (eg, SRC2 724), wherein the instruction opcode 712 determines the number of source operands. The last source operand of the instruction may be an immediate (eg, hard coded) value passed with the instruction.

In some embodiments, the 128-bit instruction format 710 includes access/addressing mode information 726 specifying, for example, a direct register addressing mode or an indirect register addressing mode. When the direct register addressing mode is used, the register address of one or more operands is directly provided by the bits in instruction 710.

In some embodiments, the 128-bit instruction format 710 includes an access/addressing mode field 726 that specifies an addressing mode and/or an access mode for the instruction. In one embodiment, the access mode is used to define the data access alignment of the instructions. Some embodiments support access modes including a 16-byte aligned access mode and a 1-byte aligned access mode, wherein the byte alignment of the access mode determines the access alignment of the instruction operand. For example, when in the first mode, the instructions 710 can use byte-aligned addressing for source and destination operands, and in the second mode, the instructions 710 can use 16-byte aligned addressing for all Source and destination operands.

In one embodiment, the addressing mode portion of the access/addressing mode field 726 determines whether the instruction is to use direct or indirect addressing. When using the direct register addressing mode, the bits in instruction 710 provide one or more operations directly The scratchpad address of the operand. When the indirect scratchpad addressing mode is used, the scratchpad address of one or more operands can be calculated based on the addressed scratchpad value and the addressed immediate field in the instruction.

In some embodiments, the instructions are grouped to simplify the opcode decoding 740 based on the opcode 712 bit field. For 8-bit opcodes, bits 4, 5, and 6 allow the execution unit to determine the type of opcode. The precise opcode grouping shown is merely exemplary. In some embodiments, the mobile and logical opcode group 742 includes data movement and logic instructions (eg, move (mov), compare (cmp)). In some embodiments, the move and logical group 742 shares five most significant bits (MSBs), where the move (mov) instruction is in the form of 0000xxxxb and the logical instruction is in the form of 0001xxxxb. Flow control command group 744 (e.g., call, skip (jmp)) includes instructions in the form of 0010xxxxb (e.g., 0x20). Miscellaneous instruction group 746 includes a mix of instructions, including synchronization instructions (eg, wait, send) in the form of 0011xxxxb (eg, 0x30). The parallel math instruction group 748 includes arithmetic instructions (eg, add, multiply (mul)) of the sub-components in the form of 0100xxxxb (eg, 0x40). Parallel math group 748 performs arithmetic operations in parallel across data channels. The vector math group 750 includes an arithmetic instruction (eg, dp4) in the form of 0101xxxxb (eg, 0x50). Vector math groups perform arithmetic such as dot product calculations on vector operands.

Drawing pipeline

FIG. 8 is a block diagram of another embodiment of a graphics processor 800. Element of Figure 8 having the same reference number (or name) as the elements of any other figures herein The device may operate or function in any manner similar to that described elsewhere herein, but is not limited thereto.

In some embodiments, graphics processor 800 includes a graphics pipeline 820, a media pipeline 830, a display engine 840, thread execution logic 850, and a rendering output pipeline 870. In some embodiments, graphics processor 800 is a graphics processor within a multi-core processing system that includes one or more general purpose processing cores. The graphics processor is controlled by a scratchpad write to one or more control registers (not shown) or via a ring interconnect 802 to the graphics processor 800. In some embodiments, ring interconnect 802 couples graphics processor 800 to other processing elements, such as other graphics processors or general purpose processors. Commands from ring interconnect 802 are interpreted by command streamer 803, which provides instructions to individual elements of drawing pipeline 820 or media pipeline 830.

In some embodiments, command streamer 803 instructs the operation of vertex skimmer 805, which reads vertex material from memory and executes vertex processing commands provided by command streamer 803. In some embodiments, vertex skimmer 805 provides vertex material to vertex shader 807, which performs coordinate space conversion and illumination operations on the various vertices. In some embodiments, vertex skimmer 805 and vertex shader 807 execute vertex processing instructions by dispatching execution threads to execution units 852A, 852B via thread dispatcher 831.

In some embodiments, execution units 852A, 852B are arrays of vector processors having a set of instructions for performing drawing and media operations. In some embodiments, execution units 852A, 852B have specificities for each An array or an additional L1 cache 851 that is shared between the arrays. The cache can be configured as a data cache, an instruction cache, or a single cache, which is divided into data and instructions in different partitions.

In some embodiments, drawing pipeline 820 includes tessellation elements to perform hard-accelerated tessellation of 3D objects. In some embodiments, the stylized hull shader 811 configures tessellation operations. The programmable field shader 817 provides a backend evaluation of the tessellation output. The tessellator 813 operates in the direction of the hull shader 811 and contains dedicated logic to generate a set of subtle geometric objects based on a coarse geometric model provided as input to the drawing pipeline 820. In some embodiments, tessellation elements 811, 813, 817 can be bypassed if tessellation is not used.

In some embodiments, the complete geometry may be processed by geometry shader 819 via one or more threads assigned to execution units 852A, 852B, or may proceed directly to cutter 829. In some embodiments, the geometry shader operates on the entire geometric object, rather than operating on patches of vertices or vertices as in previous stages of the drawing pipeline. Geometry shader 819 receives input from vertex shader 807 if tessellation is disabled. In some embodiments, geometry shader 819 can be programmed through a geometry shader program to perform geometric tessellation if the tessellation unit is deactivated.

The trimmer 829 processes the vertex data prior to rasterization. The cutter 829 can be a fixed function cutter or a programmable cutter with a crop and geometry shader function. In some embodiments, the rasterization and depth test component 873 in the render output pipeline 870 dispatches a pixel shader to The geometric object is converted to its per pixel representation. In some embodiments, pixel shader logic is included in thread execution logic 850. In some embodiments, the application can bypass rasterization 873 and access the unrasterized vertex data via stream output unit 823.

The graphics processor 800 has an interconnect bus, interconnect structure, or some other interconnect mechanism that allows data and messages to be transferred between the main components of the processor. In some embodiments, execution units 852A, 852B and associated cache 851, texture and media sampler 854, and texture/sampler cache 858 are interconnected via data 856 to perform memory access, and processing Rendering output pipeline component communication. In some embodiments, sampler 854, cache 851, 858, and execution units 852A, 852B each have separate memory access paths.

In some embodiments, the render output pipeline 870 includes a rasterizer and depth test component 873 that converts the vertice-based object into an associated pixel-based representation. In some embodiments, the rasterization logic includes a windower/masker unit to perform fixed function triangles and line rasterization. The associated render cache 878 and depth cache 879 are also available in some embodiments. Pixel manipulation element 877 performs pixel-based operations on the material, however in some examples, pixel operations associated with 2D operations (eg, bit-block image transfer with color mixing) are performed by 2D engine 841, or by display The controller 843 replaces the display time with the superimposed display plane. In some embodiments, the shared L3 cache 875 can be used for all of the drawing elements, allowing for sharing of data without the use of main system memory.

In some embodiments, graphics processor media pipeline 830 includes media engine 837 and video front end 834. In some embodiments, video front end 834 receives a pipeline command from command streamer 803. In some embodiments, media pipeline 830 includes a separate command stream. In some embodiments, video front end 834 processes the commands prior to transmitting the media commands to media engine 837. In some embodiments, media engine 837 includes thread generation functionality to generate threads for dispatching to thread execution logic 850 via thread dispatcher 831.

In some embodiments, graphics processor 800 includes display engine 840. In some embodiments, display engine 840 is external to processor 800 and coupled to the graphics processor via ring interconnect 802, or some other interconnect bus or structure. In some embodiments, display engine 840 includes a 2D engine 841 and a display controller 843. In some embodiments, display engine 840 includes dedicated logic that is capable of operating independently of the 3D pipeline. In some embodiments, display controller 843 is coupled to a display device (not shown), such as a system-integrated display device in a laptop or an external device attached via a display device connector Display device.

In some embodiments, graphics pipeline 820 and media pipeline 830 can be configured to perform operations based on multiple graphics and media programming interfaces, and are not specific to any one application interface (API). In some embodiments, the driver software for the graphics processor translates API calls specific to a particular drawing or media library into commands that can be processed by the graphics processor. In some embodiments, providing open graphics from the Khronos Group Support for OpenGL and OpenCL, Open3D libraries from Microsoft, or support for both OpenGL and D3D. Support for the Open Source Computer Vision Library (OpenCV) is also available. Future APIs with compatible 3D pipelines can also be supported in the case of pipelines that map future APIs to the pipeline of the graphics processor.

Drawing pipeline programming

FIG. 9A is a block diagram showing a graphics processor finger command format 900 in accordance with some embodiments. FIG. 9B is a block diagram showing a drawing processor command sequence 910 in accordance with an embodiment. The solid lined boxes in Figure 9A illustrate the elements typically included in the drawing commands, while the dashed lines include elements that are optional or only included in a subset of the drawing commands. The exemplary graphics processor command format 900 of FIG. 9A includes a data field, a command opcode 904, and associated material 906 for the command to identify the target client 902 of the command. Sub-opcode 905 and command size 908 are also included in some commands.

In some embodiments, the client 902 specifies a client unit of the drawing device that processes the command material. In some embodiments, the graphics processor command parser verifies the client field of each command to adjust the further processing of the command and route the command material to the appropriate client unit. In some embodiments, the graphics processor client unit includes a memory interface unit, a rendering unit, a 2D unit, a 3D unit, and a media unit. Each client unit has a corresponding processing pipeline that processes the commands. Once the order is ordered by the client Upon receipt, the client unit reads the opcode 904 and, if present, reads the sub-opcode 905 to determine the operation to be performed. The client unit uses the information in the data field 906 to execute the command. For some commands, an explicit command size 908 is required to specify the size of the command. In some embodiments, the command parser automatically determines the size of at least some of the commands based on the command opcode. In some embodiments, the commands are aligned via a multiple of a double word.

The flowchart in FIG. 9B shows an exemplary drawing processor command sequence 910. In some embodiments, the software or firmware of the data processing system characterizing the embodiment of the graphics processor uses the version of the command sequence shown to set, execute, and terminate a set of drawing operations. The example command sequences are shown and described for purposes of example only, as embodiments are not limited to these particular commands or sequences of such commands. In addition, commands can be issued as command batches in a sequence of commands such that the graphics processor will process the sequence of commands at least partially simultaneously.

In some embodiments, the drawing processor command sequence 910 can begin with a pipeline clearing command 912 to cause any active drawing pipeline to complete the currently pending command for the pipeline. In some embodiments, 3D pipeline 922 and media pipeline 924 do not operate simultaneously. Execution pipeline clearing causes the active drawing pipeline to complete any pending commands. In response to the pipeline clearing, the command parser for the graphics processor will pause the command processing until the active graphics engine completes the pending operation and the associated read cache is invalid. Optionally, any material marked as "dirty" in the render cache can be flushed to memory. In some embodiments, the pipeline clearing command 912 can be used for pipeline synchronization or prior to placing the graphics processor in a low power state.

In some embodiments, the pipeline selection command 913 is used when the command sequence requires the graphics processor to explicitly switch between pipelines. In some embodiments, the pipeline select command 913 is only needed once within the execution context before issuing the pipeline command, unless the context is for issuing commands for both pipelines. In some embodiments, a pipeline clearing command 912 is required immediately prior to pipeline switching via pipeline select command 913.

In some embodiments, pipeline control command 914 configures the graphics pipeline for operation and is used to program 3D pipeline 922 and media pipeline 924. In some embodiments, the pipeline control command 914 configures the pipeline status for the active pipeline. In one embodiment, the pipeline control command 914 is used for pipeline synchronization and is used to clear data for one or more cache memories within the active pipeline before processing the command batch.

In some embodiments, the return buffer status command 916 is used to configure a set of return buffers for individual pipelines to write data. Some pipeline operations require the allocation, selection, or configuration of one or more return buffers that write intermediate data into the one or more return buffers during processing. In some embodiments, the graphics processor also uses one or more return buffers to store the output data and perform cross-thread communication. In some embodiments, return buffer status 916 includes selecting the size and number of return buffers to be used for a set of pipeline operations.

The remaining commands in the command sequence differ based on the pipeline in effect for the operation. Based on pipeline decision 920, the command sequence is for the slave 3D pipeline The 3D pipeline 922 design begins with state 930 or is designed for media pipeline 924 starting from media pipeline state 940.

The commands for the 3D pipeline state 930 include 3D state setting commands for vertex buffer states, vertex component states, constant color states, depth buffer states, and other state variables that will be configured prior to processing the 3D primitive commands. The values of these commands are determined, at least in part, by the particular 3D API in use. In some embodiments, the 3D pipeline state 930 command can also selectively disable or bypass certain pipeline components if those components will not be used.

In some embodiments, the 3D primitive 932 command is used to submit 3D primitives for processing by the 3D pipeline. Commands and associated parameters that are passed to the graphics processor via the 3D primitive 932 command are forwarded to the vertex extraction function in the drawing pipeline. The vertex extraction function uses the 3D primitive 932 command material to generate a vertex data structure. The vertex data structure is stored in one or more return buffers. In some embodiments, the 3D primitive 932 command is used to perform vertex operations on the 3D primitive via the vertex shader. To process the vertex shader, the 3D pipeline 922 dispatches the shader execution thread to the graphics processor execution unit.

In some embodiments, the 3D pipeline 922 is triggered by executing a 934 command or event. In some embodiments, the scratchpad write triggers command execution. In some embodiments, execution is triggered via a "go" or "kick" command in the command sequence. In one embodiment, command execution is triggered using a pipeline synchronization command to clear a sequence of commands from the drawing pipeline. The 3D pipeline will perform geometric processing for the 3D primitives. Once the operation is complete, the resulting geometry will be rasterized and the pixel engine will color the resulting pixels. Additional commands to control pixel shading and pixel back end operations may also be included for those operations.

In some embodiments, the graphics processor command sequence 910 follows the media pipeline 924 path when performing media operations. In general, the particular use and manner of programming for media pipeline 924 depends on the media or computing operations to be performed. Specific media decoding operations may be offloaded to the media pipeline during media decoding. In some embodiments, the media pipeline can also be bypassed, and media decoding can be performed in whole or in part using resources provided by one or more general processing cores. In one embodiment, the media pipeline also includes components for general purpose graphics processor unit (GPGPU) operations, wherein the graphics processor is configured to execute using a computation shader program that is not explicitly associated with rendering of the graphics primitives SIMD vector operation.

In some embodiments, media pipeline 924 is configured in a similar manner as 3D pipeline 922. A set of media pipeline status commands 940 are assigned or placed before the media object command 942 in the command sequence. In some embodiments, media pipeline status command 940 includes data to configure media pipeline elements that will be used to process media objects. This includes data for configuring video decoding and video encoding logic within the media pipeline, such as encoding or decoding formats. In some embodiments, the media pipeline status command 940 also supports the use of one or more indicators to point to an "indirect" status element that includes a batch of status settings.

In some embodiments, the media item command 942 supplies the metrics to the media item for processing by the media pipeline. Media objects include a memory buffer, It contains the video material to be processed. In some embodiments, all media pipeline states must be valid before the media object command 942 is issued. Once the pipeline status is configured and the media object command 942 is queued, the media pipeline 924 is triggered via an execution command 944 or an equivalent execution event (eg, a scratchpad write). The output from media pipeline 924 can then be post-processed by operations provided by 3D pipeline 922 or media pipeline 924. In some embodiments, GPGPU operations are configured and executed in a manner similar to media operations.

Drawing software architecture

FIG. 10 illustrates an exemplary drawing software structure for data processing system 1000 in accordance with some embodiments. In some embodiments, the software architecture includes a 3D graphics application 1010, an operating system 1020, and at least one processor 1030. In some embodiments, processor 1030 includes a graphics processor 1032 and one or more general purpose processor cores 1034. The drawing application 1010 and the operating system 1020 are each executed in a system memory 1050 of the data processing system.

In some embodiments, the 3D drawing application 1010 includes one or more shader programs that include shader instructions 1012. The shader language instructions can be high order shader languages such as High Order Shader Language (HLSL) or OpenGL Shader Language (GLSL). The application also includes executable instructions 1014 in a machine language format suitable for execution by general purpose processor core 1034. The application also includes a drawing object 1016 defined by vertex data.

In some embodiments, operating system 1020 is a Microsoft® Windows® operating system from Microsoft Corporation, a proprietary UNIX-like operating system, or an open source UNIX operating system that uses a variant of the Linux kernel. When using the Direct3D API, the operating system 1020 uses the front end shader compiler 1024 to compile any shader instructions 1012 in the form of HLSL into lower order color shader languages. The compilation can be a just-in-time (JIT) compilation or the application can perform a shader precompilation. In some embodiments, the high order shader is compiled into a low order shader during compilation of the 3D drawing application 1010.

In some embodiments, the user mode drawing driver 1026 includes a back end shader compiler 1027 for converting the shader instructions 1012 into a hardware specific representation. When the OpenGL API is used, the GLSL high-level language form of the color wheel instructions 1012 is passed to the user mode drawing driver 1026 for compilation. In some embodiments, the user mode drawing driver 1026 uses the operating system core mode function 1028 to communicate with the core mode drawing driver 1029. In some embodiments, core mode drawing driver 1029 communicates with graphics processor 1032 to dispatch commands and instructions.

IP core implementation

One or more aspects of at least one embodiment can be implemented by a representative code stored on a machine readable medium, the machine readable media representation and/or logic defined within the integrated circuit, such as a processor. For example, machine readable media can include instructions that represent various logic within the processor. When read by a machine, the instructions cause the machine to make logic to perform the techniques described herein Surgery. This representation, referred to as an "IP core," is a reusable logic unit for an integrated circuit that can be stored on a tangible, machine-readable medium as a hardware model that describes the structure of the integrated circuit. . The hardware model can be supplied to various customers or manufacturing facilities that load the hardware model onto the manufacturing machine that manufactures the integrated circuit. The integrated circuit can be fabricated such that the circuit performs the operations described in association with any of the embodiments described herein.

11 is a block diagram of an IP core development system 1100 that can be used to fabricate integrated circuits to perform operations in accordance with an embodiment. The IP core development system 1100 can be used to create a modular, reusable design that can be integrated into a larger design or used to construct an entire integrated circuit (eg, a SOC integrated circuit). The design facility 1130 can generate a software simulation 1110 of the IP core design in a high-level programming language (eg, C/C++). Software Simulation 1110 can be used to design, test, and verify the behavior of IP cores. A scratchpad transfer level (RTL) design can then be built or synthesized from the simulation model 1100. The RTL design 1115 is an abstraction of the behavior of the integrated circuit of the digital signal stream between the modeled hardware registers, including the associated logic performed using the modeled digital signals. In addition to the RTL design 1115, lower order designs at the logic level or the transistor level can be created, designed, or synthesized. Therefore, the specific details of the initial design and simulation can be different.

The RTL design 1115 or equivalent may be further synthesized by the design facility as a hardware model 1120, which may be a hardware description language (HDL) format or some other representation of the physical design material. The HDL can be further simulated or tested to verify the IP core design. Non-volatile memory 1140 (eg, hard drive, flash memory, or any non-volatile storage medium) can be used to store IP The core design is passed to a third party manufacturing facility 1165. Alternatively, the IP core design can be transmitted (eg, via the internet) via wired connection 1150 or wireless connection 1160. Manufacturing facility 1165 can then fabricate an integrated circuit that is at least partially designed in accordance with the IP core. The fabricated integrated circuit can be configured to perform operations in accordance with at least one embodiment described herein.

12 is a block diagram depicting an exemplary system single-chip integrated circuit 1200 that can be fabricated using one or more IP cores in accordance with an embodiment. The exemplary integrated circuit includes one or more application processors 1205 (eg, CPUs), at least one graphics processor 1210, and may additionally include an image processor 1215 and/or a video processor 1220, any of which may A modular IP core that is designed from the same or multiple different facilities. The integrated circuit includes peripheral or busbar logic including a USB controller 1225, a UART controller 1230, an SPI/SDIO controller 1235, and an I ² S/I ² C controller 1240. In addition, the integrated circuit can include a display device 1245 coupled to one or more of a high resolution multimedia interface (HDMI) controller 1250 and a mobile industry processor interface (MIPI) display interface 1255. The storage may be provided by a flash memory subsystem 1260 that includes a flash memory and a flash memory controller. The memory interface can be provided via memory controller 1265 for accessing SDRAM or SRAM memory devices. Some integrated circuits additionally include an embedded security engine 1270.

In addition, other logic and circuitry may be included in the processor of integrated circuit 1200, including additional graphics processors/cores, peripheral interface controllers, or general purpose processor cores. Device and method for pattern driving and adaptive virtual drawing processing unit

In current graphics processor unit (GPU) virtualization implementations, virtual machine monitors (VMMs), and in particular virtual GPU (vGPU) device models, capture and simulate guest access to privileged GPU resources for security and more Work, while accessing CPUs through performance-critical resources (for example, such as CPU access to graphics memory). GPU commands, once committed, are executed directly (in the GPU) without VMM intervention. Therefore, it is possible to achieve near-native performance.

In an architecture where the GPU performs workloads from multiple vGPU guests, switching from one of the vGPU guests to the next vGPU guest will result in a GPU context switch and the required hardware level ring engine context will be saved/restored. Unlike a very lightweight CPU with a context, the GPU context is very heavy. Therefore, GPU context switching takes significantly more time than CPU context switching. Analysis data shows that GPU context switching may take 100 to 300 microseconds (us), while CPU context switching may only take hundreds of nanoseconds (ns).

Therefore, the GPU scheduler policy will not trigger context switching as frequently as in the CPU. Usually, it happens every few milliseconds (ms). A typical GPU scheduler provides a quantum based policy to schedule vGPU instances periodically. It should be noted that the period here can be fixed, or weighted, or even different for each additional strategy. Each vGPU, once scheduled, is provided with a quota in which commands prepared by the CPU (CMD) are executed until the quota is consumed, and/or until the vGPU is locked (such as due to a flag, a wait event, etc.). Although this strategy is in the CPU The side performs well, but it poses an additional challenge to the GPU side because the execution of the vGPU instance depends on the available CMD prepared by the CPU. That is, a vGPU instance (VM) might have a quota to run GPU CMD, but there may not be a ready CMD at a given point in time (ie, the GPU must wait for the CMD from the CPU to be ready, which may take time) . Thus, the GPU can remain in an idle state until the next CMD is available, during which time the GPU loop is wasted. Figure 13 illustrates an exemplary scenario where VM1's GPU utilization is approximately 20%, with approximately 80% of GPU resources being wasted waiting for the next available command.

One option is to schedule the vGPU instance out immediately when it detects that no CMD is available. However, because the context switching cost in the GPU is so high compared to the CPU, it can result in very high switching costs for frequent GPU context switching. So for the GPU scheduler, it is a challenge to determine the set of conditions for vending (routing out) a vGPU instance when no CMD is available.

Embodiments of the invention described herein implement a pattern driven, adaptive scheme (PDSAS) for efficient yielding. As depicted in FIG. 14, in one embodiment, PDSAS logic 1470 is built into or on top of an existing GPU scheduler 1412, and indeed, can be implemented by any existing GPU scheduler.

A. Virtual Drawing Processing Overview

Additional details of the exemplary embodiment 1400 shown in FIG. 14 will now be provided, followed by (in section B) the operations performed by the PDSAS logic 1470. Detailed description. The illustrated embodiment includes a plurality of VMs, such as VM 1430 and VM 1440, managed by a hypervisor 1410 (sometimes referred to as a virtual machine monitor (VMM)) that can access all of the GPU features in GPU 1420. Array. Virtual GPU (vGPU) 1460A-B can access all of the functionality provided by GPU hardware 1420 based on GPU virtualization technology. In various embodiments, hypervisor 1410 can track and manage the resources and lifecycle of one or more vGPUs. Although only two vGPUs 1460A-B are shown in FIG. 14, hypervisor 1410 may include other vGPUs. In some embodiments, the vGPU can interact with a native GPU driver. The VM 1430 or VM 1440 can access the entire array of GPU features through the vGPU 1460A-B. The vGPU context can be switched per quota or per event. In some embodiments, context switching may occur per GPU rendering engine (eg, a 3D rendering engine or a blitter rendering engine). Periodic switching allows multiple VMs to share physical GPU 1420 in a manner that is transparent to the workload of VMs 1430, 1440.

Much like a single central processing unit (CPU) core, which can be allocated a limited amount of time by the VM 1430, the GPU 1420 can also be allocated a limited amount of time by the VM 1430. Another virtualization model is time sharing, where GPU 1420 or a portion thereof can be shared by multiple VMs, such as VM 1430 and VM 1440, in a multiplexed manner. Other GPU virtualization models may also be used in other embodiments. In various embodiments, the graphics memory associated with GPU 1420 can be partitioned and assigned to various vGPUs in hypervisor 1410.

In various embodiments, a graphics conversion table (GTT) may be used by the VM and/or GPU 1420 to map graphics processor memory to system memory. Or used to convert a GPU virtual address to a physical address. In some embodiments, hypervisor 1410 can manage the mapping memory map via a shadow GTT, and the mapping GTT can be maintained in a vGPU instance, such as vGPU 1460A-B. In various embodiments, each VM may have a corresponding mapping GTT to maintain a mapping between the mapped memory address and the physical memory address. In some embodiments, the mapping GTT can be shared and maintain a mapping of multiple VMs. In some embodiments, individual VMs (eg, VM 1430 and VM 1440) may include both per-process and global GTT.

Some embodiments may use system memory as the graphics memory. System memory can be mapped to multiple virtual address spaces by GPU page tables. Different embodiments can support global graphics memory space and per program graphics memory address space. The global graphics memory space can be a virtual address space, for example, 2 GB, mapped through a Global Graphics Transformation Table (GGTT). The lower portion of this address space may be referred to as an "aperture" and may be accessed by both GPU 1420 and a CPU (not shown). The upper portion of this address space is referred to as a high graphics memory space or a hidden graphics memory space, which can be accessed only by GPU 1420. In various embodiments, a mapped global graphics conversion table (SGGTT) can be used by VM 1430, VM 1440, hypervisor 1410, or GPU 1420 to convert a drawing memory address into individual system memory locations in accordance with a global memory address space. site.

The hypervisor 1410 can use the command parser 1418 to detect potential memory working sets of the GPU rendering engine for commands submitted by the VM 1430 or VM 1440. In various embodiments, VM 1430 can have individual Command buffers (not shown) to hold commands from different workloads (eg, 3D workloads and media workloads). Similarly, VM 1440 can have individual command buffers (not shown) to hold commands from these different workloads.

In various embodiments, the command parser 1418 can scan for commands from the VM and determine if the command includes a memory operand. If it contains a memory operand, the command parser 1418 can read the associated drawing memory space map from, for example, the GTT for the VM and then write it to the workload specific portion of the SGGTT. After the workload's command buffer has been scanned in its entirety, the SGGTT that holds the memory address space map associated with this workload can be generated or updated. Moreover, by scanning pending commands from VM 1430 or VM 1440, command parser 1418 can also increase the security of GPU operations, such as by mitigating malicious operations.

In some embodiments, an SGGTT can be generated to hold the conversion of all workloads from all VMs. In some embodiments, an SGGTT can be generated to hold, for example, a conversion of all workloads from only one VM. The workload specific SGGTT portion may be constructed by command parser 1418 as needed to preserve the conversion of a particular workload (eg, a 3D workload from VM 1430 or a media workload from VM 1440).

B. Pattern Driven, Adaptive Solutions (PDSAS) for efficient yielding

As described above, embodiments of the present invention implement a pattern for efficient yielding Drive, Adaptive Solutions (PDSAS). As shown in FIG. 14, in an embodiment, PDSAS logic 1470 is built into or on top of an existing GPU scheduler 1412, and in fact, can be implemented by any existing GPU scheduler.

One embodiment of the present invention monitors activities and produces an analytic pattern of GPU execution states during operation, including how long the GPU can remain in an idle state for a given VM. In one embodiment, this is done by judging the time difference from the time the last CMD was completed until the time the new CMD is available. These past execution patterns of the VM can then be used to predict the behavior of the VM and extend the GPU scheduler 1412 to improve the efficiency of whether to make better decisions on the GPU.

To perform these operations, one embodiment of PDSAS Logic 1470 adds a stamp to the execution of each VM GPU command to determine how to use the GPU resource's pattern. In particular, PDSAS Logic 1470 collects statistics on the duration of the point in time at which the GPU completes the execution of the previously submitted CMD (possibly batch format) (where there is initially no committable CMD) to the new CMD available.

In one embodiment, the analytics data includes GPU busy time and/or GPU idle time for each VM, as shown in FIG. In this example, each VM is offered a quota of 15ms. VM1 has a GPU resource utilization of approximately 20% within its quota (ie, there is a significant idle period), and VM2 has a GPU resource utilization of approximately 80% within its quota (with significantly less idle periods).

CPU schedulers and GPU schedulers may be unknown. VM one There are four scheduling possibilities: 1) Scheduled-in only by CPU scheduler, 2) Scheduled by GPU scheduler only, 3) Only by CPU and GPU scheduler Cheng, 4) is not scheduled by the GPU or CPU scheduler.

In one embodiment, PDSAS logic 1470 then performs the analysis of the determined utilization patterns to obtain a profile of possible latency (ie, GPU idle per VM). One embodiment applies a function P(t), where t is the wait duration and P(t) is the probability (eg, 0.01%). An example of such a probability distribution curve for P(t) is shown in Figure 15, which emphasizes the mean time (T _average ) and T _80% (i.e., the probability that the next CMD will arrive within a specified time period). 80%) The point on the curve. This example is derived from a 3Dmark workload.

Once the probability curve is determined, the PDSAS logic 1470 can calculate and predict the waiting cost ("W-Cost") at a certain probability. In one embodiment, PDSAS logic 1470 simply performs its evaluation using the average latency T _average : W-Cost = T _average , where T _average is calculated as:

In one embodiment, since the data is sampled in small time intervals such as 10 ns (0.01 ms), the above formula becomes a discrete function as follows:

N = T _average /0.01

Of course, other values can also be used for W-Cost. For example, in an embodiment, W-Cost is set to the maximum possible wait duration of the next command (T0), as indicated in FIG. In one embodiment, W-Cost is in Variables that are dynamically adjusted during runtime based on the specified policy (some examples of which are discussed below).

The sampling technique described above can last, for example, for a certain period of time of 10 minutes. In one embodiment, the patterns in the sampling period may be used equally and/or different weights may be used to distinguish between the most recent data and the old data (eg, more weight is applied to more current data) . For example, the latest 1-minute sampling data can be considered to have the best heuristics, and, therefore, can have an 80% weight, while the remaining 9 minutes of data can have a 20% weight. In this implementation, the final P(t) = Pa(t) * 0.8 + Pb(t) * 0.2, where P(t) is the distribution curve of possible latency. Of course, the underlying principles of the invention are not limited to any particular manner of determining the weight and/or the time period during which the weight is applied.

One embodiment of PDSAS Logic 1470 enhances GPU scheduler 1412 by predicting the cost and gain of vGPU instances. For example, PDSAS Logic 1470 can perform the above evaluation and then compare the gain and loss of the two methods. In one embodiment, the PDSAS logic 1470 yields the vGPU if the cost of waiting is greater than or equal to the cost of the handover. which is:

If W-Cost>=SW-Cost (switching cost) + Thres0, the vGPU is given out.

As mentioned above, W-Cost is the latency of the GPU idle if the GPU has not been released. SW-Cost is the switching cost, and Thres0 is the minimum gain limit for generating the GPU. As described above, in an embodiment, W-Cost is the average cost of waiting for the vGPU to obtain a new CMD that is operational, and SW-Cost is the cost of a vGPU context switch (eg, 0.3 ms), and Thres0 is a threshold that can be set to maintain policy resiliency and conservativeness.

In one embodiment, utilizing PDSAS logic 1470 that implements the above scheme, GPU utilization is increased, and thus overall system GPU throughput is increased. However, in some cases, unbalanced GPU resource allocation may be introduced between VMs. That is, a specific VM can be allocated to resources that are less than resources that should be allocated even in a long-term perspective. In one embodiment, by addressing the use of individual GPUs for individual VMs and extending the scheduler to increase the priority of the VM, the VM will then have more opportunities to be scheduled to come in. A VM, and/or if it is scheduled to come in and has a CMD to execute, will be given a longer time quota. Different scheduler policies can also be used, such as priority based schedulers, or loop schedulers, or any combination thereof. The above method can be used to make a VM with a lower resource allocation have a higher probability of being scheduled in the next cycle.

A method in accordance with an embodiment of the present invention is illustrated in Figures 17A-B. The method is subdivided into those operations performed by the vCPU (Fig. 17A), as well as those implemented by the GPU scheduler (Fig. 17B). At 1701, the CPU of the VM begins execution of the GPU workload, and, at 1702, the GPU workload generates a sequence of commands that are placed in the command buffer 1705. If during the sampling period (determined at 1703), the P(t) curve is updated and sampling begins at 1704.

Turning to Figure 17B, at 1706, the GPU scheduler selects the next vGPU for execution (e.g., for a given quota such as 15ms). If it is within the quota (determined at 1707), then at 1708, the vGPU is currently performing work negative. Loaded. When the vGPU execution is completed at 1709, a determination is made at 1710 as to whether the new command is available from the command buffer. If so, the process returns to 1707. If not, start sampling at 1711. If there are pending workloads in other VMs (as judged at 1712), the PDSAS logic is applied at 1713. If not, the process returns 1707 for the current vGPU instance. A determination is made at 1714 as to whether to give up another vGPU instance (eg, using the PDSAS techniques described herein). If so, the process returns to 1706. If not, processing returns to 1707 for the current vGPU instance.

Accordingly, some embodiments of the invention described herein define a strategy for computing WCost compared to SW-Cost and Thres0. In some architectures, the SW-cost has been determined to be approximately 0.1 to 0.3 ms, sampled at runtime. There are also several ways to choose Thres0. In one embodiment, Thres0=0. In another embodiment, Thres0 is a fixed value, such as 0.2ms as a conservative schedule (such as for a RT VM/vGPU), or -0.1ms as a positive factor to motivate the schedule. In yet another embodiment, the value is determined from the average GPU execution time of the next vGPU (ie, as idle may be determined for the VM, the busy state may also be determined for the VM). Here, for example, the next VM represents the next visitor scheduler to be scheduled to come in. For example, for the next vGPU with a running 3D workload, Thres0 can be set to 0.5ms if the average GPU busy time is 0.5ms per command buffer. The VM will want to switch to the next VM with a total handover cost = SW-Cost + 0.5ms = 0.8ms, which means it needs to wait an average of 0.8ms until the GPU ownership is switched again (ie, with Implemented in 2 VMs). However, it should be noted that the basic principles of the invention are not limited to any particular way of computing Thres0.

Three exemplary strategies will now be described (1) W-Cost=T _average , (2) W-Cost=T80%, and (3) W-Cost=(1-D)* T _average (where D is as follows) The balance factor described).

1. W-Cost=T _average

In this embodiment, the cost is not to let the GPU and the GPU idle waiting T _average time. If the switching cost, SW-cost=0.3, and Thres0 is the average execution time of a vGPU under 0.5ms, the total cost is 0.8ms. In this strategy, W-Cost can be relatively consistent as long as the vGPU maintains the same workload, as the workload may have a fixed pattern. Therefore, if T _average > 0.8ms, the current vGPU can give up the GPU when each CMD is completed.

As shown in the flow chart in Figures 17a-b, the advantage of this strategy is that we can maximize the use of GPU cycles, but it can affect vGPU1 (the concession). In other embodiments, the vGPU1 workload is high priority and even if we give the GPU out to vGPU2, it should not be affected, as if the physical GPU could be scheduled back to vGPU1 sooner or later (eg before the average wait time expires). This is very important for high priority or instant workloads (eg, live streaming media, video conferencing, etc.). The downside is that it always tries to keep the GPU for a VM or always choose to give up the GPU every cycle. VMs with small T _average and small Thred0 values (indicating that they have a busy workload) will then tend to consume most of the GPU resources. It requires the help of a scheduler to balance the GPU resources between the various VMs.

2. W-Cost=T _80%

As mentioned above, T80% means that 80% of the next command will arrive within this duration. The following formula can be used to determine T80%. Although 80% of the values are used in this exemplary embodiment, various other percentages can still be used while still complying with the basic principles of the invention (eg, 90%, 95%, 99%, Tmax, etc.).

In one embodiment, this implementation has the same effect as described in the W-Cost=T _average strategy described above, with the main difference being that if T _80% > T _average , W-Cost can be a larger value. FIG 18 illustrates a T _80% for one exemplary embodiment of the W-Cost =, as described above using the same workload VM1 and VM2. As shown in Figure 18, VM1 is more easily scheduled to VM2 due to the larger W-Cost value. As shown, the switching in 1801 is similar to the switching described above. However, the switching in 1802 is very different because the GPU alternately processes the CMDs from each of VM1 and VM2, giving a larger W-Cost value. In one embodiment, the idle time in each CMD buffer may not be large enough, and, therefore, the workload itself may be slightly affected/delayed. Therefore, this implementation is more suitable for non-instant workloads. However, as shown at 1802, this embodiment may result in increased overall GPU utilization.

Two other examples of W-Cost are W-Cost=Max and W-Cost= 0. In the first case, W-Cost=Max will immediately give up the GPU. In the second case, W-Cost=0, the GPU will not be given up during the VM quota.

Figure 19 depicts an exemplary probability curve indicating the probability at different time values and including the balance factor.

3. W-Cost=(1-D)* T _average

In this embodiment, D is a balance factor between (-1.0 and 1.0). A negative D value indicates that the PDSAS logic 1470 will be more inclined to give up the GPU execution time. A positive D value indicates that the PDSAS logic 1470 will be more prone to remain within its GPU quota (eg, 15ms). The D factor is added to affect the in-balance GPU resource allocation between multiple VMs. D will be adjusted at runtime based on the GPU utilization of each VM. It is known that without the help of a scheduler, Strategy #1 or Strategy #2 may cause some VMs to take up most of the GPU time.

Below are examples of 2 VMs. The coefficient D is calculated using D = 1 - VM% * TotalVM. In this example, VM% represents the percentage of allocation of GPU time for the current VM. For the case of 2 VM visitors, TotalVM=2. If VM% is 30% (which should be 50% for 2 VMs), D=0.4*T _average . W-cost=0.6*T _average . Using these techniques, VMs (which used to have 30% GPU time) are more likely to allocate more GPU resources due to the introduction of small W-costs.

It should be noted that the above technique for predicting the idleness of the next CMD can also be used to predict when an interrupt arrives. In this implementation, the IRQ handler You can wait for this time to combine the incoming interrupts and process them together. This basically reduces the total number of traps injected into the guest VM.

In the detailed description, reference is made to the accompanying drawings, in which FIG. Other embodiments may be utilized, and structural or logical changes may be made without departing from the scope of the disclosure. Therefore, the following detailed description is not to be considered as limiting,

Various operations may be described as a plurality of separate acts or operations in sequence, in a manner that is most helpful in understanding the claimed subject matter. However, the order of description should not be construed as implying that the operations must be sequential. In particular, these operations may not be performed in the order in which they are rendered. The described operations may be performed in a different order than the described embodiments. In additional embodiments, various additional operations may be performed and/or the operations described may be omitted.

For the purposes of this disclosure, the terms "A and/or B" mean (A), (B), or (A and B). For the purposes of this disclosure, the terms "A, B, and/or C" mean (A), (B), (C), (A and B), (A and C), (B and C), or (A, B, and C). In the context of the disclosure "a" or "a first element" or its equivalent, the disclosure includes one or more such elements, neither requiring nor excluding two or more such elements . Furthermore, ordinal indications (eg, first, second, or third) for identifying elements are used to distinguish multiple elements and do not indicate or imply the required number of such elements. Or a limited number, unless otherwise specified, they do not indicate a particular location or order of such elements.

Reference to an embodiment or an embodiment in the description means that a particular feature, structure, or characteristic described in connection with the embodiments is included in at least one embodiment of the invention. The description may use terms such as "in one embodiment", "in another embodiment", "in some embodiments", "in an embodiment", "in various embodiments", etc. One or more identical or different embodiments are referred to. In addition, the terms "including", "including", "having", etc., used in connection with the embodiments of the present disclosure are synonymous.

In the embodiments, the terms "engine" or "module" or "logic" may refer to one or all of the following: special application integrated circuits (ASICs), electronic circuits, processors (shared, dedicated, or group) And/or memory (shared, dedicated, or group) of one or more software or firmware programs, combinatorial logic, and/or other suitable components that provide the functionality. In an embodiment, the engine or module can be implemented in any combination of firmware, hardware, software, or firmware, hardware, and software. Embodiments of the invention may include various steps that have been described above. The steps can be embodied by machine executable instructions that cause a general purpose or special purpose processor to perform the steps. Alternatively, these steps may be performed by a particular hardware component that includes hardwired logic for performing the steps, or by any combination of programmed computer components and custom hardware components.

As described herein, instructions may refer to hardware such as special application integrated circuits (ASICs) that are configured to perform certain operations or have predetermined functions. A specific configuration, or a software instruction stored in a memory embodied in a non-transitory computer readable medium. Thus, the techniques shown in the figures can be implemented using code and data stored and executed on one or more electronic devices (e.g., terminal stations, network elements, etc.). Such an electronic device can read media storage and (internal and/or network and other electronic devices) communication code and data using a computer device, and the computer machine can read the media such as a non-transitory computer machine to be readable. Storage media (eg, diskette; CD-ROM; random access memory; read-only memory; flash memory device; phase change memory) and temporary computer devices can read communication media (eg, electricity, light, sound) Or other forms of propagating signals - such as carrier waves, infrared signals, digital signals, etc.).

Moreover, such electronic devices typically include a collection of one or more processors coupled to one or more other components, such as one or more storage devices (non-transitory machine readable storage media), user input / Output devices (eg, keyboard, touch screen, and/or display), and network connections. The coupling of the processor set to other components is typically through one or more bus bars and bridges (also known as bus bar controllers). The signals of the storage device and the load network traffic represent one or more machine readable storage media and machine readable communication media, respectively. Thus, a storage device of a given electronic device typically stores code and/or data for execution on the set of one or more processors of the electronic device. Of course, different combinations of software, firmware, and/or hardware can be used to implement one or more portions of embodiments of the present invention. Numerous specific details are set forth to provide a thorough understanding of the invention. However, it is obvious to those skilled in the art. It is to be understood that the invention may be practiced without a part of these specific details. In some instances, well-known structures and functions are not described in the details of the details. Therefore, the scope and spirit of the present invention should be judged based on the following claims.

1400‧‧‧Exemplary embodiment

1410‧‧‧Super Manager

1412‧‧‧GPU scheduler

1418‧‧‧Command Profiler

1420‧‧‧GPU

1430‧‧‧VM

1440‧‧‧VM

1460A, 1460B‧‧‧Virtual GPU (vGPU)

1470‧‧‧PDSAS logic

Claims (25)

An apparatus comprising: a graphics processing unit (GPU) for processing a drawing command and responsively rendering a plurality of image frames; a hypervisor for virtualizing the GPU to share the plurality of virtual machines (VMs) a GPU that subdivides the GPU for each VM into a plurality of quotas; and scheduling logic to monitor GPU utilization, including GPU busy times for each VM during its assigned quota and/or Or GPU idle time, and used to store usage data reflected by the GPU during each quota period; the scheduling logic is configured to predict a waiting cost within a given quota of the first VM based on the utilization data, and To further predict the cost of the second VM, if the waiting cost is greater than the cost of the second VM, the scheduling logic yields the GPU to the second VM. The device of claim 1, wherein the scheduling logic is used to give the GPU to the second VM if the cost of the waiting is greater than a sum of the cost of the second VM and a specified threshold. . The device of claim 2, wherein the threshold comprises a value of 0 or greater than zero. The apparatus of claim 1, wherein the GPU busy time and/or GPU idle time is monitored, the schedule logic stamping commands within a set of GPU commands that each VM will execute within each quota. Such as the device of claim 1 of the patent scope, wherein the scheduling logic It is configured to store the utilization data reflecting the utilization of the GPU as a probability curve that reflects the probability that a new command will be received within a given quota for different time periods. The apparatus of claim 5, wherein the at least one time period includes an average amount of time to expect a new command based on the probability curve. The apparatus of claim 5, wherein at least one time period is associated with a specified percentage that includes a likelihood that a command will be received within a specified time period. The apparatus of claim 1, wherein the scheduling logic is configured to determine whether the command is pending for the second VM, and the GPU is only allowed if the command is pending for the second VM Out to the second VM. The apparatus of claim 2, wherein the scheduling logic dynamically adjusts the specified threshold in response to a workload condition on each of the VMs. A method comprising: selecting a first virtual machine (VM) to execute a command on a graphics processor unit (GPU); monitoring GPU utilization data, including GPU busy of the first VM during one or more assigned quotas Time and/or GPU idle time; storing usage data reflecting the utilization of the GPU; predicting a cost of waiting within the first quota of the first VM and a cost of giving out to the second VM based on the utilization data; and if waiting The cost is greater than the cost of giving up the second VM, then The GPU is given out to the second VM. The method of claim 10, wherein if the waiting cost is greater than a sum of the cost of the second VM and a specified threshold, the GPU is given out to the second VM. The method of claim 11, wherein the threshold comprises a value of 0 or greater than zero. A method of claim 10, wherein monitoring the GPU busy time and/or GPU idle time, stamping a set of GPU commands to be executed by each VM within each quota. The method of claim 10, further comprising storing the utilization data reflecting the utilization of the GPU as a probability curve, the probability curve reflecting a probability of receiving a new command within a given quota of different time periods. The method of claim 14, wherein the at least one time period comprises an average amount of time to expect a new command based on the probability curve. The method of claim 14, wherein at least one time period is associated with a specified percentage that includes a likelihood that a command will be received within a specified time period. The method of claim 10, further comprising: determining whether the command is pending for the second VM, and only releasing the GPU to the second when the command is pending for the second VM VM. For example, the method of claim 11 of the patent scope further includes: dynamically adjusting the workload conditions on each of the VMs. The specified threshold. A system comprising: a memory for storing data and a code; a central processing unit (CPU) including an instruction cache for caching a portion of the code, and a data for quickly accessing a portion of the data The CPU further includes execution logic for executing at least some of the code in the code and responsively processing at least some of the data, at least a portion of the code including a drawing command; a graphics processing unit (GPU) For processing the drawing commands and responsively rendering a plurality of image frames; a hypervisor for virtualizing the GPU to share the GPU among a plurality of virtual machines (VMs) to be used by the hypervisor The GPU of each VM is subdivided into multiple quotas; and scheduling logic to monitor GPU utilization, including GPU busy time and/or GPU idle time of each VM during its assigned quota, and used to store The utilization data utilized by the GPU during the quota period; the scheduling logic is configured to predict the waiting cost within the given quota of the first VM based on the utilization data, and to further predict the yield to the first VM cost, if the cost is greater than the wait to make a cost of the second VM, the scheduling logic that allows the GPU to the second VM. The system of claim 19, wherein the scheduling logic is used to give the GPU to the second VM if the cost of the waiting is greater than a sum of the cost of the second VM and a specified threshold. . A system as claimed in claim 20, wherein the threshold comprises a value of 0 or greater than zero. A system of claim 19, wherein the GPU busy time and/or GPU idle time is monitored, the scheduling logic stamping a command within a set of GPU commands that each VM will execute within each quota. The system of claim 19, wherein the scheduling logic is configured to store the utilization data reflecting the utilization of the GPU as a probability curve, the probability curve reflecting a probability of receiving a new command within a given quota for different time periods. . A system as claimed in claim 23, wherein the at least one time period comprises an average amount of time to expect a new command based on the probability curve. A system of claim 23, wherein at least one time period is associated with a specified percentage that includes a likelihood that a command will be received within a specified time period.