Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F9/00—Arrangements for program control, e.g. control units
  • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
  • G06F9/44—Arrangements for executing specific programs
  • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
  • G06F9/45533—Hypervisors; Virtual machine monitors
• • G—PHYSICS
  • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
  • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
  • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
  • G09G5/36—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
  • G09G5/363—Graphics controllers

Definitions

• the present invention is related to the field of computer systems, and in one aspect to processing of specialized hardware resources such as computer graphics units.
• GPUs In the field of computer graphics, it is known to use specialized graphics processing units or GPUs to provide hardware-based acceleration of certain graphics operations.
• a contemporary GPU may perform operations such as texture mapping, high-speed polygon rendering, and shading in hardware circuitry having a highly parallel structure for high processing throughput.
• Processing performed by a GPU is generally for purposes of rendering graphical images of an application program on a graphical display. GPUs are especially useful for many graphics-intensive computer applications, including video games and high-end graphical composition and editing tools for example.
• a GPU is not divisible like memory or storage, and thus cannot be assigned for use by different VMs on that basis.
• One approach to GPU virtualization is used in virtual-computing systems based on the so-called XEN architecture, which features an open-source hypervisor and a control VM running the Linux operating system.
• Linux provides native support for a graphics application programming interface (API) known as OpenGL.
• Systems running the Windows® operating system generally utilize a different graphics API of a family referred to as DirectX.
• DirectX graphics application programming interface
• a specific DirectX API used for three-dimensional graphics is referred to as "Direct 3D" or "D3D”.
• Wine open-source conversion program
• the Wine program is deployed in the control VM, and the guest VMs are configured with special drivers that route D3D function calls generated by application programs to the control VM.
• Wine processes the D3D function calls using OpenGL operations.
• OpenGL can impose performance penalties as compared to systems using a native D3D driver for the GPU.
• First is the conversion process itself, which imparts extra delay and may reduce graphics throughput.
• functions provided by D3D but not by OpenGL must be executed in some alternative manner, such as some form of emulation for example, which can greatly reduce performance over GPU-assisted execution. It would be desirable for Linux-based virtual computing systems such as XEN systems to provide support for the Windows D3D API without unduly sacrificing graphics performance.
• a technique for virtualizing a specialized hardware resource such as graphics hardware is disclosed that overcomes limitations of prior virtualization techniques, such as discussed above, providing for flexibility in deployment without a need for
• a disclosed computing system includes a graphics processing unit and main processing circuitry that is operative to execute computer program instructions forming a hypervisor, a control virtual machine and a specialized rendering virtual machine used for graphics processing.
• the hypervisor provides for virtualization of hardware of the computing system as in conventional virtual-computing platforms, while the control virtual machine manages the rendering virtual machine and guest virtual machines of the computing system.
• the technique supports shared use of the graphics processing unit by application programs of the guest virtual machines that generate graphics information (generally including graphics commands and graphics data) according to a specified graphics API such as the D3D API.
• the rendering virtual machine includes a graphics driver that is native to the graphics processing unit, and it is assigned pass-through access to the graphics processing unit by the control virtual machine. That is, the rendering virtual machine is able to directly communicate with the GPU to control its operation, without virtualization by the hypervisor as is done for other hardware resources.
• the rendering virtual machine receives graphics information from an application program via an inter- virtual-machine communication channel, and uses the graphics driver to control operation of the graphics processing unit to perform graphics rendering operations based on the graphics information.
• the rendering VM is separate from the control VM, it can use an operating system that supports the native graphics driver (such as Windows) while the control VM may use a different operating system that may not be compatible with the driver but has other advantages such as open- source availability (such as Linux).
• the control VM is not directly involved in the graphics processing, so that performance- reducing conversion between different graphics APIs is not necessary.
• the rendering virtual machine may be realized as a relatively low- function machine - it may not require functions such as network access, a complicated file system, a user interface etc. Thus it may use a so-called "embedded" operating system which is less resource-intensive and less expensive than a full-function operating system such as normally used in general-purpose VMs.
• the disclosed technique has broader application to virtualization of other types of hardware resources.
• Figure 1 is a block diagram of a computer system
• Figures 2(a) through 2(c) are schematic depictions of graphical user interface display screens
• Figure 3 is block diagram of certain elements of the computer system; and Figure 4 is a flow diagram of certain operations of the computer system.
• Figure 1 shows a computer having a set of hardware resources (or hardware) 10 and software-implemented components including a hypervisor 12 and a collection of virtual machines (VMs) shown as a control VM 14, a rendering VM 16, and one or more guest VMs 18 (two guest VMs 18-1 and 18-2 shown).
• the hypervisor 12, the control VM 14 and the rendering VM 16 are privileged system components, meaning that they generally have direct access to some or all of the hardware 10, while the guest VMs 18 are relatively less privileged and rely on virtualization and other functionality of the privileged system components for their use of the hardware 10.
• the hardware 10 generally includes one or more processors, memory, and one or more high-speed data buses providing functional interconnections for data transfer among these components.
• the hardware 10 also generally includes I/O circuitry which may include, for example, network interface circuitry, storage device(s) such as flash memory or magnetic disk(s), and interface circuitry to user interface devices such as a pointing device, keyboard, and graphical display device.
• I/O circuitry may include, for example, network interface circuitry, storage device(s) such as flash memory or magnetic disk(s), and interface circuitry to user interface devices such as a pointing device, keyboard, and graphical display device.
• the hardware 10 includes video interface circuitry (VIDEO INTFC) 19 including a graphics processing unit (GPU) 20.
• the video interface circuitry 19 provides an interface to a graphical display device such as an LCD display (not shown).
• the GPU 10 includes circuitry specially tailored to performing graphics rendering operations. Such circuitry may also be referred to as a "graphics accelerator”.
• the GPU 20 includes a capability of three-dimensional (3-D) rendering.
• the control VM 14 is a special VM used for management and control of the computer, including the instantiation of the rendering VM 16 and guest VMs 18 as well as configuration and control of certain aspects of their operation. This control function is indicated in Figure 1 by control connections 22.
• the control VM 14 may be referred to alternatively as a "control domain", "domain 0" or “domO", the latter designations being applicable in a system in which successively instantiated VMs are given corresponding sequential numbers 0, 1, 2, ... .
• the control VM 14 also provides an interface to a human operator of the computer, such as by a "console” or similar user interface program as generally known in the art.
• the control VM 14 executes the Linux operating system. Certain specific functions of the control VM 14 are described below.
• the guest VMs 18 are typically full- function VMs that are preferably as close to fully virtualized as possible, meaning that they can execute software (especially an operating system) with little modification from a non-virtualized form.
• each guest VM 18 includes an operating system (O/S) 24 and one or more application programs (APPL) 26.
• the operating system 24 may be the Windows® operating system as used in personal computers. It should be noted that different guest VMs 18 may execute different types of operating systems 24.
• the operating system 24 is shown as including a graphics "pseudo driver" (PSEUDO DRV) 28 that is operatively coupled to the rendering VM 16 via an inter- virtual-machine (inter- VM) communication channel 30.
• the inter- VM communication channel 30 is realized using a first inter- VM (V TO V) channel interface (V-V) 31 in the guest VM 18 and a second inter- VM channel interface 33 in the rendering VM 16 (these are referred to as "client-type” and “server-type” respectively below).
• V TO V inter- VM
• V-V inter- VM channel interface
• the operating system typically includes a graphics driver operatively coupled directly to graphics processing circuitry such as a GPU to control its execution of graphics operations at a fairly low level (e.g., programming parameters and operation codes, monitoring status and completion of operations, etc).
• the pseudo driver 28 in contrast, functions primarily to merely transfer graphics commands, responses and data (collectively "graphics information") between the application program 26 and the rendering VM 16 via the inter- VM communication channel 30. Actual processing of graphics commands is performed by the rendering VM 16 and GPU 20 as described below.
• the rendering VM 16 is a special-purpose VM providing graphics processing capability to the guest VMs 18. It is assigned direct, "pass-through" access 32 to the GPU 20, meaning that a native driver (NTV DRV) 34 controls and communicates directly with the GPU 20 as in a conventional non-virtualized setting, with no virtualization being provided by the hypervisor 12.
• the rendering VM 16 employs an operating system 36 that may be relatively less functional than the operating system 26 of the guest VMs 18, because of the special purpose nature of the rendering VM 16.
• the operating system 36 may not support a conventional file system or rich user interface, and it may support only limited storage and provide little or no user extensibility. In one
• the operating system 36 may be a so-called “embedded” or “real-time” operating system, a type used for a variety of specialized computing devices including cell phones as generally known in the art.
• the operating system 36 may be a member of the Windows Embedded family of embedded operating systems.
• the operating system 36 is shown as including inter- VM channel interfaces 33 each operatively coupled to a respective inter- VM communication channel 30 and the native driver 34. In operation, each inter- VM channel interface 33 operates
• the hypervisor 12 and VMs 14 - 18 are shown as separate from the hardware 10, which is a typical and useful depiction for purposes of description. It should be understood that these software-implemented components effectively include the main processing circuitry during their execution.
• the hypervisor 12 for example, includes the main processing circuitry of the hardware 10 when executing instructions of a hypervisor program.
• a similar description can be applied to the VMs 14 - 18.
• the arrangement of Figure 1 may be particularly suited for use with application programs 26 having a graphics-intensive workload, such as video games for example.
• application programs 26 having a graphics-intensive workload such as video games for example.
• it may be a personal device such as a desktop, console or mobile platform.
• the arrangement may also find use in a server-type computing device, such as a server providing for remote display of rendered graphics for an application program executing on the server.
• VMs may use other operating systems, such as those of the Windows® family, that may use a different graphics API of a proprietary family known as DirectX, one version of which is called Direct 3D or "D3D" and used for 3-dimensional graphics rendering.
• DirectX a proprietary family known as DirectX
• D3D Direct 3D
• Wine a conversion program known as Wine to convert between the D3D and OpenGL APIs. Because not all graphics functions provided by D3D are available in OpenGL, graphics performance in such systems is typically lower than in non-virtualized systems using native D3D drivers for the graphics hardware.
• One of the advantages of the structure of Figure 1 is the ability to use the native driver 34 for the GPU 20 (e.g., a D3D driver) notwithstanding that the driver may be incompatible with the operating system (e.g., Linux) of the control VM 14.
• the native driver 34 is placed within the specialized rendering VM 16 which uses a compatible operating system (e.g., Windows), and the control VM 14 establishes the inter- VM communication channels 30 to enable graphics information (e.g., D3D function calls and data) to be transferred between the guest VMs 18 and the shared native driver 34.
• the native driver 34 is also provided with the pass-through access to the GPU 20 as described above.
• the use of a Linux-based control VM 14 does not affect graphics performance, because it is not directly involved in graphics processing as in prior systems using
• Figures 2(a) through 2(c) provide a simplified depiction of operation of the computer system from a user's perspective.
• each of these represents a screen of a graphical display device, such as an LCD display, during operation.
• Figure 2(a) shows a control screen 40 forming part of a user interface for a system control program executed by the control VM 14.
• the screen 40 is used by a user to select which of the guest VMs 18 is in "focus", i.e., has its graphics output appearing on the display device and accepts input from the user input device(s) (keyboard, mouse etc.).
• the control screen 40 may have an area 42 where control features such as icons, pulldown menus etc. may be located and/or activated from.
• a user selects the focus by activating a corresponding icon 44.
• activation of the icon 44-1 causes a screen 46-1 for the guest VM 18-1 to be displayed ( Figure 2(b)), while activation of the icon 44-2 causes a screen 46-2 for the guest VM 18-2 to be displayed ( Figure 2(c)).
• the screen 46-1 includes two windows 48, 50 laid on a desktop background for the guest VM 18-1, while the screen 46-2 shows one large window 52 for guest VM 18-2.
• a user command to change focus may be accomplished using just the keyboard, such as by a sequence of keys representing a system-level control command, or by use of the mouse by bringing it to an edge of the screen to cause a control screen or toolbar to appear.
• Figure 3 illustrates the set of components involved in graphics processing, arranged vertically to reflect functional location relative to the application programs 26 and GPU 20 of Figure 1.
• the pseudo- drivers 28 At an upper location facing the applications 26 are the pseudo- drivers 28.
• Each pseudo-driver 28 communicates with a client-type inter- VM interface 31, which in term communicates with a server-type inter- VM interface 33 via the respective inter- VM communication channel 30.
• Each server-type inter- VM interface 33 communicates with the native driver 34 that controls operation of the GPU 20.
• FIG 4 is a simplified flow diagram illustrating pertinent operation of the computer of Figure 1, specifically graphics-related operation of the rendering VM 16.
• the rendering VM 16 receives graphics information from an application program 26 of a guest VM 18 via an inter- VM communications channel 30.
• the graphics information is in accordance with a graphics API supported by the graphics processing unit 20, e.g., the above-mentioned D3D graphics API.
• the rendering VM 16 employs the server-type inter- VM interface 33 to the inter- VM communications channel 30.
• the graphics information is sent using transport messages of the inter- VM communications channel 30.
• the server-type inter- VM interface 33 passes the received graphics information to the native driver 34.
• the native driver 34 controls operation of the graphics processing unit 20 to perform graphics rendering operations using the received graphics information. This control will be in the form of programming parameters and operation codes, monitoring status and completion of operations, etc. as discussed above.
• the result of the rendering operations by the GPU 20 may represent a screen to be displayed on a local display device, such as the screens 42, 46 discussed above. In some cases, the result may be used in a different manner.
• a remote display device i.e., a local application 26 generates graphics information for a screen to be displayed on a display device attached to another physical computer.
• the result of the rendering operations may be returned to the requesting application 26, for example, to enable the application 26 to send the result to the other computer where the result will be displayed.
• Other back-end processing of the rendering result is possible.
• the disclosed technique may also be generalized to take advantage of the use of a specialized VM (analogous to the rendering VM 18) that is more compatible with the guest VMs 18 than is the control VM 14.
• guest VMs 18 employ an operating system of a first type (such as Windows), while the control VM 14 uses a second operating system of a different type (such as Linux), and that there is either no support or only limited support under the second operating system for a system function that is more fully supported by the first operating system.
• system function (or control software for the system function, such as a device driver) can be deployed on a specialized VM (akin to the rendering VM 16) which runs the first operating system, and an inter- VM communication scheme can be employed to enable the function to be accessed from the guest VMs 18.
• This arrangement enables a virtualizing computer which is based on the second operating system to support functions of the first operating system in a native fashion.

Landscapes

• Engineering & Computer Science (AREA)
• Software Systems (AREA)
• Theoretical Computer Science (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• General Engineering & Computer Science (AREA)
• Computer Graphics (AREA)
• Computer Hardware Design (AREA)
• Stored Programmes (AREA)
• Processing Or Creating Images (AREA)
• Image Generation (AREA)

Priority Applications (3)

Applications Claiming Priority (2)

Publications (1)

Family

ID=49117963

Family Applications (1)

Country Status (5)

Cited By (5)

Families Citing this family (40)

Citations (2)

Family Cites Families (5)

• 2012
  • 2012-08-23 US US13/592,891 patent/US9146762B2/en active Active
• 2013
  • 2013-08-22 WO PCT/US2013/056145 patent/WO2014031833A1/en not_active Ceased
  • 2013-08-22 EP EP13759069.1A patent/EP2888662B1/en active Active
  • 2013-08-22 IN IN1709DEN2015 patent/IN2015DN01709A/en unknown
  • 2013-08-22 CN CN201380054128.XA patent/CN104737129B/zh not_active Expired - Fee Related

Patent Citations (2)

Also Published As

Similar Documents

Legal Events