Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
  • G06F13/14—Handling requests for interconnection or transfer
  • G06F13/20—Handling requests for interconnection or transfer for access to input/output bus
  • G06F13/24—Handling requests for interconnection or transfer for access to input/output bus using interrupt
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F9/00—Arrangements for program control, e.g. control units
  • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
  • G06F9/44—Arrangements for executing specific programs
  • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
  • G06F9/45533—Hypervisors; Virtual machine monitors
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F9/00—Arrangements for program control, e.g. control units
  • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
  • G06F9/46—Multiprogramming arrangements
  • G06F9/48—Program initiating; Program switching, e.g. by interrupt
  • G06F9/4806—Task transfer initiation or dispatching
  • G06F9/4812—Task transfer initiation or dispatching by interrupt, e.g. masked

Definitions

• each VM as hosted on a computing device is for all intents and purposes a computing machine, although in virtual form, and thus represents itself as such both to the use application thereof and to the outside world.
• the VM and/or a use application thereof can and in fact do issue hardware requests for hardware resources of the VM, even though the VM might not in reality have such hardware resources.
• such hardware requests are intercepted or otherwise redirected toward the host, and such host services such hardware requests based on the hardware resources thereof, typically with the requesting VM and/or use application thereof being none the wiser.
• a host deploys each VM thereof in a separate partition, address space, processing area, and/or the like.
• Such host may include a virtualization layer with a virtual machine monitor ("VMM") or the like that acts as an overseer application or 'hypervisor', where the virtualization layer oversees and/or otherwise manages supervisory aspects of each VM of the host, and acts as a possible link between each VM and the outside world.
• the VMM may be a separate application running in its own address space or may be integrated more closely with the host operating system, either directly or as an operating system extension of some sort, such as a device driver.
• the VMM of the host may intercept or otherwise redirect hardware requests that originate from each VM of the host and/or a use application thereof, and may at least assist in servicing the requests, again with the requesting VM and/or use application thereof being none the wiser.
• Many computing systems comprise multiple processors.
• Processors in a multiprocessor virtual machine environment may operate in a guest mode or in a VMM mode.
• a processor uses virtual machine definitions to manage the virtual machine's guest operating system and applications, translating arguments and managing system resources without intervention from the VMM.
• the guest operating system or applications may need system resources that must be managed by the VMM.
• the VMM may be required for error handling, system faults, or interrupt handling. In these situations, the processor operates in a VMM mode.
• Modern processing systems include support for interrupts, which allow processors to be notified of external events.
• an interrupt causes a processor to stop what it's doing, record its current execution location so it can resume execution after servicing the interrupt, and then execute a specified interrupt service routine.
• An EOI command is typically delivered to an interrupt controller though an I/O port or a memory-mapped I/O access, such as a read from or write to a register.
• processing an EOI command may consume tens or hundreds of cycles.
• processing an EOI command may consume thousands of cycles.
• Some virtual machine monitors use interrupts as a basis for inter-partition messaging. If software running within one partition needs to communicate with software running within a second partition on the same physical machine, it is able to do so through the use of inter-partition messages. When a message is sent by one processor, the virtual machine monitor may send an interrupt to the processor that is the intended recipient of the message, causing the recipient processor's interrupt service routine to process the message and respond to its contents.
• a guest operating system may program certain interrupt sources as "automatic end-of-interrupt" ("auto- EOI").
• auto-EOI automatic end-of-interrupt
• a virtualized interrupt controller clears the bit in an interrupt service register corresponding to a delivered interrupt without waiting for an explicit end-of-interrupt (“EOI") command.
• EOI end-of-interrupt
• the auto-EOI interrupt may not block the delivery of other interrupts.
• Figure 6 is a flow chart illustrating a way of handling an interrupt request
• Figure 7 depicts a timeline for an example of interrupt priorities
• Figure 8 is a flow chart illustrating a way of handling an interrupt request using an automatic EOI in accordance with the teachings herein;
• Figure 9 is a flow chart illustrating a way of handling an inter-processor message according to the teachings herein.
• One or more programs that may implement or utilize the processes described in connection with the invention, e.g., through the use of an API, reusable controls, or the like. Such programs are preferably implemented in a high level procedural or object oriented programming language to communicate with a computer system. However, the program(s) can be implemented in assembly or machine language, if desired. In any case, the language may be a compiled or interpreted language, and combined with hardware implementations. [0024] Although exemplary embodiments may refer to utilizing aspects of the invention in the context of one or more stand-alone computer systems, the invention is not so limited, but rather may be implemented in connection with any computing environment, such as a network or distributed computing environment.
• Interrupts are used in modern computing systems for a variety of purposes including, by way of examples, to notify processors of external events and to facilitate communication between processors of a multiprocessor system.
• an interrupt interrupts normal processing and temporarily diverts flow of control to an interrupt service routine ("ISR").
• ISR interrupt service routine
• Various activities of a computing system can trigger interrupts. Some examples are pressing a key on a keyboard, receiving a network packet, and writing to or reading from a disk.
• Inter-processor interrupts are a type of interrupt by which one processor may interrupt another processor in multiprocessor environment. IPIs may be used as a basis for inter-processor messaging.
• EOI commands can be skipped in many cases, significantly reducing the virtualization overhead related to interrupt delivery.
• an end of message EOI need only be sent when a second message is already queued for slot containing a just-processed message.
• a physical interrupt can be selectively EOIed irrespective of whether it is the highest priority in-service interrupt.
• Virtualization provides a mechanism for increasing flexibility while enhancing security and reliability.
• Processors, memory, and I/O devices are examples of subsystems that can be virtualized.
• a virtual interface and virtual resources available through the virtual interface are mapped onto the interface and resources of a real system on which the virtualization is implemented.
• Virtualization can be applied not only to subsystems, but to an entire machine.
• a virtual machine's architecture is implemented in a layer of software on a real machine.
• computer systems generally comprise one or more layers of software running on a foundational layer of hardware. This layering is done for reasons of abstraction. By defining the interface for a given layer of software, that layer can be implemented differently by other layers above it.
• each layer only knows about (and only relies upon) the immediate layer beneath it. This allows a layer or a "stack" (multiple adjoining layers) to be replaced without negatively impacting the layers above said layer or stack.
• software applications upper layers
• lower layers typically rely on lower levels of the operating system (lower layers) to write files to some form of permanent storage, and these applications do not need to understand the difference between writing data to a floppy disk, a hard drive, or a network folder. If this lower layer is replaced with new operating system components for writing files, the operation of the upper layer software applications remains unaffected.
• VM virtual machine
• FIG. 1 is a diagram representing the logical layering of the hardware and software architecture for a virtualized environment in a computer system.
• a virtualization program 110 runs directly or indirectly on the physical hardware architecture 112.
• the virtualization program 110 may be (a) a virtual machine monitor that runs alongside a host operating system or (b) a host operating system with a hypervisor component, where the hypervisor component performs the virtualization.
• the term virtual machine monitor is used as a general term for any of the various types of virtualization programs.
• the virtualization program 110 virtualizes a guest hardware architecture 108 (shown as dashed lines to illustrate the fact that this component is a partition or a "virtual machine"), that is, hardware that does not actually exist but is instead virtualized by the virtualizing program 110.
• a guest operating system 106 executes on the guest hardware architecture 108, and a software application 104 can run on the guest operating system 106.
• the software application 104 can run in a computer system 102 even if the software application 104 is designed to run on an operating system that is generally incompatible with a host operating system and the hardware architecture 112.
• FIG. 2 illustrates a virtualized computing system comprising a host operating system (host OS) software layer 204 running directly above physical computer hardware 202, where the host OS 204 provides access to the resources of the physical computer hardware 202 by exposing interfaces to partitions A 208 and B 210 for the use by operating systems A and B, 212 and 214, respectively.
• the host OS 204 may go unnoticed by operating system layers 212 and 214 running above it.
• the host OS 204 may be a specially designed operating system with native virtualization capabilities or, alternately, it may be a standard operating system with an incorporated hypervisor component for performing the virtualization (not shown).
• partition A 208 which may be, for example, a virtualized Intel 386 processor
• partition B 210 which may be, for example, a virtualized version of one of the Motorola 680X0 family of processors.
• guest OSs guest operating systems
• a 212 and B 214 are guest operating systems
• partition A 208 and partition B 214 are virtualized computer hardware representations that exist only as software constructions. They are made possible due to the execution of specialized virtualization software(s) that not only presents partition A 208 and partition B 210 to Guest OS A 212 and Guest OS B 214, respectively, but which also performs all of the software steps necessary for Guest OS A 212 and Guest OS B 214 to indirectly interact with the real physical computer hardware 202.
• the physical computer hardware 202 may comprise a single central processing unit (CPU) 222, as in a uniprocessor environment, or multiple CPUs 222, 224, 226 as in a multiprocessor environment.
• FIG 3 illustrates an alternative virtualized computing system where the virtualization is performed by a VMM 304 running alongside the host operating system 306.
• the VMM 304 may be an application running above the host operating system 306 and interacting with the computer hardware 302 only through the host operating system 306.
• the VMM 304 may instead comprise a partially independent software system that on some levels interacts indirectly with the computer hardware 302 via the host operating system 306, but on other levels the VMM 304 interacts directly with the computer hardware 302 (similar to the way the host operating system interacts directly with the computer hardware).
• the VMM 304 may comprise a fully independent software system that on all levels interacts directly with the computer hardware 302 (similar to the way the host operating system interacts directly with the computer hardware) without utilizing the host operating system 306 (although still interacting with the host operating system 306 in order to coordinate use of the computer hardware 302 and avoid conflicts and the like).
• two partitions, A 308 and B 310 lie conceptually above the VMM 304.
• guest OSs guest operating systems
• Running on top of guest OS A 312 are two applications, application Al 316 and application A2 318, and running on top of guest OS B 314 is application Bl 320.
• the physical computer hardware 302 may comprise a single central processing unit (CPU) 322, as in a uniprocessor environment, or multiple CPUs 322, 324, 326 as in a multiprocessor environment.
• CPU central processing unit
• the physical computer hardware 402 may comprise a single central processing unit (CPU) 422, as in a uniprocessor environment, or multiple CPUs 422, 424, 426 as in a multiprocessor environment.
• CPU central processing unit
• interrupt controllers track requested and in-service interrupt requests. This is often done through the use of two bit vectors where each bit represents an individual interrupt source. One bit vector is called the interrupt request register 518, and a second is called the interrupt service register 520.
• an interrupt controller 516 receives a request for an interrupt, it sets a corresponding bit in the interrupt request register 518.
• the interrupt controller 516 delivers an interrupt to a processor 510, 512, or 514, it clears a corresponding bit in the interrupt request register 518 and sets a corresponding bit in the interrupt service register 520.
• the interrupt controller 516 receives an EOI, it knows that the corresponding interrupt is no longer being serviced, and so clears the corresponding bit in the interrupt service register 520.
• a device wishing to signal an interrupt drives a voltage on an interrupt request line to a predetermined level defined as “active” and holds it there until the interrupt has been serviced.
• an interrupt request is signaled by a level transition on an interrupt request line wherein a device wishing to signal an interrupt drives a pulse onto the interrupt request line and then returns the line to its quiescent state.
• an interrupt controller detects the IRQ 604, it determines whether the IRQ has a higher priority than any currently in-service interrupts 606, possibly by examining the interrupt service register 520 ( Figure 5). If a higher priority interrupt is in- service when the IRQ is detected, then the interrupt controller flags a corresponding bit in the interrupt request register 608 so as to record the pending request. If the requested interrupt has a higher priority than any in-service interrupt, then the interrupt controller flags a corresponding bit in the interrupt service register 610 and signals an appropriate processor to run a corresponding interrupt service routine 612.
• FIG. 7 describes an example that illustrates the general concept of interrupt priority and is not intended to be limiting. Suppose that at time ti an interrupt source of priority 10 requests an interrupt 702. The interrupt controller interrupts the processor which invokes an interrupt service routine 704 associated with the interrupt source.
• an interrupt source of priority 200 requests an interrupt 706 destined for the same processor.
• the interrupt controller interrupts the processor again, and the ISR 708 for the priority 200 interrupt begins executing while the ISR for the priority 10 interrupt is suspended 710.
• a third interrupt source at priority 50 requests an interrupt 712 at time t3, before the priority 200 ISR 708 completes.
• the interrupt controller will defer delivery 714 of this interrupt until the processor has completed execution of the priority 200 ISR at time t 4 .
• the priority 50 ISR will be invoked 716 after the priority 200 ISR 708 completes. When the priority 50 ISR 716 completes, at time t 5 , execution of the priority 10 ISR resumes 718 and is completed at time t 6 .
• processors and interrupt controllers may be virtualized. This is done through a combination of software (for example, a VMM) and virtualization assists provided by hardware.
• the EOI command is emulated by the VMM. This is done by intercepting accesses to an EOI port or register. The intercept invokes a software handler within the VMM that performs the same functions as a physical interrupt controller in response to the EOI.
• the combination of the intercept and the software handler can require thousands or tens of thousands of cycles. This adds significant overhead to the ISR when executing within a virtualized environment.
• a VMM accepts interrupts and redirects them to guest operating systems as virtual interrupts.
• Interrupts can be generated from a variety of sources, including, by way of examples and not limitation, physical hardware devices, a partition emulating a hardware device, a partition wishing to post a message or signal an event to another partition, or the VMM wishing to signal a partition.
• the VMM typically issues an EOI command to a physical interrupt controller after an interrupt has been accepted. For level-triggered interrupts it is generally not safe to issue an EOI command until the ISR in the guest operating system has run and issued an EOI command to the virtual interrupt controller.
• Certain physical interrupt controllers like the APIC allow only the highest- priority in-service interrupt to be EOIed.
• the VMM may need to selectively EOI an interrupt which is not the highest-priority in-service interrupt.
• the VMM accepts the first interrupt and redirects it to the guest operating system. Before the ISR in the guest has issued an EOI command, the second interrupt arrives. Subsequently the guest operating system issues an EOI command for the first interrupt. In this situation the VMM cannot EOI the first interrupt since a higher priority interrupt is already in-service and issuing an EOI command to the physical interrupt controller will EOI the higher priority interrupt.
• a guest operating system is allowed to program some interrupt sources as "auto-EOI.”
• auto-EOI When an interrupt source is marked as auto-EOI, the traditional interrupt prioritization behavior is modified.
• An auto- EOI interrupt does not block the delivery of other interrupts.
• an auto-EOI interrupt behaves similarly to a lowest-priority interrupt in that any other interrupt, including other auto-EOI interrupts, are allowed to interrupt the execution of its associated ISR.
• the bit associated with the auto-EOI interrupt in the interrupt service register is immediately cleared. Effectively, the virtualized interrupt controller automatically generates an EOI at the time the auto-EOI interrupt is delivered. With an auto-EOI interrupt, it is desirable that the interrupt source moderate itself by not requesting subsequent interrupts until it knows the previous interrupt was handled. Otherwise, each succeeding interrupt would interrupt the previous ISR, potentially overflowing the processor's stack.
• the auto-EOI property is specified in a virtual register associated with a synthetic interrupt source (SINT). The format of the virtual register is as follows:
• the Auto-EOI flag indicates that an implicit EOI should be performed by the VMM when an interrupt is delivered to the virtual processor.
• the VMM will automatically clear the corresponding flag in the in-service register of the virtual interrupt controller. If the guest enables this behavior, then it must not perform an explicit EOI in its interrupt service routine.
• Figure 8 depicts the life cycle of an auto-EOI interrupt request according to the disclosure herein.
• a device or software begins the process by asserting an IRQ 802.
• an interrupt controller detects the IRQ 804
• a virtualized interrupt controller allows a VMM to selectively EOI physical interrupts when the physical interrupt controller does not support such functionality. This may be accomplished by maintaining a list of pending EOIs, i.e., interrupts that need to be EOIed at a later time. For example, when a guest operating system issues an EOI command, the VMM may check whether the interrupt being EOIed is indeed the highest priority in-service interrupt in the physical interrupt controller. If not, the VMM simply adds the interrupt to the list of pending EOIs. If, on the other hand, the interrupt being EOIed is the highest priority in-service interrupt, the VMM not only EOIs the current interrupt, but also EOIs other interrupts on the list of pending EOIs.
• pending EOIs i.e., interrupts that need to be EOIed at a later time. For example, when a guest operating system issues an EOI command, the VMM may check whether the interrupt being EOI
• VMMs use interrupts as the basis for inter-partition messaging.
• a partition is an isolation boundary enforced by a VMM and is the "container" for a virtual machine. If software running within one partition needs to communicate with software running within a second partition on the same machine, it is able to do so through the use of inter-partition messages.
• These messages typically contain small payloads. For example, in the case of one known hypervisor, they message payload may comprise up to 240 bytes plus a 16-byte header.
• the hypervisor may deliver an interrupt to that virtual processor. This causes a corresponding ISR to be invoked.
• the ISR is responsible for reading the message and reacting to its contents. As described above, the ISR generally must "EOI" the interrupt after the interrupt has been serviced. In this case, the EOI would be sent after the message has been read. Inter-partition messaging must be as fast as possible.
• the traditional EOI mechanism adds an undesirable overhead, using possibly tens of thousands of cycles to notify the virtualized interrupt controller that the message has been read and that subsequent messages and lower priority pending interrupts can be delivered. [0060] In accordance with the disclosure herein, the overhead of the traditional
• EOI mechanisms for signaling that an inter-partition message has been processed can be avoided.
• a message slot is provided for each SINT and the layout of a message is defined by the data structures described in Table 1 below. typedef struct ⁇
• FIG. 9 is a flow chart depicting an embodiment of inter-processor message handling in accordance with the disclosure herein.
• a sending processor posts an inter-processor message 902 corresponding to a specified SINT which has been designated as auto-EOI.
• the VMM appends the message to a message queue 904 and determines whether the message slot corresponding to the specified SINT is empty 906. If a previous message is still present in the message slot, the VMM sets the Message Pending bit in the header of the message in the slot 908. If the message slot is empty, the VMM copies the message to the message slot 910 and sends an interrupt associated with the specified SINT to the receiving processor 912.
• a guest OS running on the receiving processor When a guest OS running on the receiving processor receives an interrupt associated with a SINT, its ISR reads the message from the corresponding message slot and performs an action based on the message type and payload 914. When processing of the message is completed, the ISR clears the message type 916. For example, in accordance with the data structures defined in Table 1, the ISR may clear the message type by writing a specified value to HV MESSAGE TYPE. The ISR then examines the Message Pending bit of the just-processed message 918. If the Message Pending bit is not set, meaning that no further message is queued for the message slot, no further action is required by the ISR 920. This should be the case most often. In particular, there is no need for the ISR to send an EOI command, thus avoiding considerable computational overhead.
• EOM end-of-message

Landscapes

• Engineering & Computer Science (AREA)
• Theoretical Computer Science (AREA)
• Software Systems (AREA)
• Physics & Mathematics (AREA)
• General Engineering & Computer Science (AREA)
• General Physics & Mathematics (AREA)
• Bus Control (AREA)
• Debugging And Monitoring (AREA)
• Multi Processors (AREA)

Priority Applications (4)

Applications Claiming Priority (2)

Publications (1)

Family

ID=39492588

Family Applications (1)

Country Status (6)

Cited By (2)

Families Citing this family (30)

Citations (4)

Family Cites Families (26)

• 2006
  • 2006-12-06 US US11/635,455 patent/US7533207B2/en active Active
• 2007
  • 2007-11-13 JP JP2009540365A patent/JP4882005B2/ja not_active Expired - Fee Related
  • 2007-11-13 WO PCT/US2007/084522 patent/WO2008070410A1/en not_active Ceased
  • 2007-11-13 EP EP07864330.1A patent/EP2122474B1/en active Active
  • 2007-11-13 CN CN2007800452399A patent/CN101553792B/zh not_active Expired - Fee Related
  • 2007-11-13 KR KR1020097013858A patent/KR101514088B1/ko not_active Expired - Fee Related

Patent Citations (4)

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events