DE112016004347T5 - Real-time local and global data center network optimizations based on platform telemetry data - Google Patents

Abstract

In one embodiment, a system includes platform logic that includes a plurality of processor cores and resource allocation logic. The resource allocation logic may receive a processing request and direct the processing request to a processor core of the plurality of processor cores, wherein the processor core is selected based at least in part on telemetry data associated with the platform logic, the telemetry data indicating a topology of at least a portion of the platform logic.

Description

REFER TO RELATED APPLICATIONS

This application claims priority over the non-provisional (replacement) filed on 26 September 2015 U.S. Patent Application No. 14 / 866,869 entitled "REAL-TIME LOCAL AND GLOBAL DATACENT NETWORK OPTIMIZATION BASED ON PLATFORM TELEMETRY DATA", which is incorporated herein by reference in its entirety.

TERRITORY

The present disclosure relates generally to the field of computer development, and more particularly to real-time local and global data center network optimizations based on platform telemetry data.

BACKGROUND

A data center may include one or more platforms, each including at least one processor and associated memory modules. Each data center platform may enable the performance of any suitable number of processes associated with different applications running on the platform. These processes can be performed by the processors and other associated logic of the platforms.

list of figures

• 1 ein Blockdiagramm von Komponenten eines Datenzentrums nach einigen Ausführungsformen;
• 2 ein Blockdiagramm einer Datenzentrumsverwaltungsplattform nach einigen Ausführungsformen;
• 3 ein Blockdiagramm einer Datenanalysemaschine nach einigen Ausführungsformen;
• 4 ein Blockdiagramm einer zentralen Verarbeitungseinheit nach einigen Ausführungsformen;
• 5 ein Verfahren zum Zuweisen von Plattformressourcen basierend auf Telemetriedaten nach einigen Ausführungsformen.

Show it:

• 1 a block diagram of components of a data center according to some embodiments;
• 2 a block diagram of a data center management platform according to some embodiments;
• 3 a block diagram of a data analysis engine according to some embodiments;
• 4 a block diagram of a central processing unit according to some embodiments;
• 5 a method of allocating platform resources based on telemetry data, in accordance with some embodiments.

Like reference numerals and designations in the various drawings refer to like components.

DETAILED DESCRIPTION

1 shows a block diagram of components of a data center 100 according to some embodiments. In the illustrated embodiment, the data center includes 100 a variety of platforms 102 , the data analysis engine 104 and the data center management platform 106 passing through the net 108 coupled together. A platform 102 can the platform logic 110 with one or more central processing units (CPUs) 112, storing 114 (which may include any number of different modules), chipsets 116 , Communication interfaces 118 and any other suitable hardware and / or software for running a hypervisor 120 or any other operating system that is capable of running processes on the platform 102 associated with running applications. In some embodiments, a platform 102 as a host platform for one or more guest systems 122 that call these applications act.

In certain embodiments, different units of the data center may be used 100 such as the data center management platform 106 , the hypervisor 120 or one or more I / O device drivers 124 (eg, an Ethernet device driver) capable of being on the platform 102A Running processes resources (such as resources, the cores of the CPUs 112 are assigned) of the platform logic 110 assign. In some systems, algorithms used by conventional I / O device drivers may attempt to optimize platform utilization by assigning queues or flows associated with guest machines to CPU cores, based on which core of a connection is allocated, or may be a simple one Use round-robin algorithm to identify target CPU cores. However, such systems are limited because these optimizations are only local platform optimizations based on local considerations.

In various embodiments of the present disclosure, devices (eg, the data center management platform 106 , the hypervisor 120 or the I / O device driver 124) of the data center 100 Processes by guest systems 122 according to telemetry data, one or more platforms 102 associated resources of the platform logic 110 to assign. Accordingly, optimizations made by these units may be based on a global view of the state of the platform 102 and / or the entire data center 100 as shown by the telemetry data. As an example, the I / O device driver 124 may be based on platform-external behavior at runtime, platform performance features over and within sockets to accommodate the CPUs 112 and external events over one or more virtual or traditional networks of the data center 100 , Routing workloads to resources (such as CPU cores). As another example, the hypervisor 120 or another operating system based on the state information determine which core an I / O device driver 124 is to be attached to.

Any suitable telemetry data regarding the platform (s) 102 can through any suitable entity, such as by the hypervisor 120 or the other operating system, one or more I / O device drivers 124, one or more platform telemetry devices, such as the chipsets 116 associated administrative machines 126 , a unit outside the data center 100 or a combination thereof, collected or otherwise provided. As non-limiting examples, the platform telemetry data may be collected using run-time sampling, runtime microbenches, or pre-benchmarking. In some embodiments, this information may be available to the I / O device driver 124, the hypervisor 120 or other operating system, a management machine 126 , the data center management platform 106 and / or the data analysis engine 104 to be provided. Each of these units can process the data and make optimization decisions based on the data. These decisions may then be made by an I / O device driver 124, the hypervisor 120 or another operating system.

As non-limiting examples, the telemetry data may be congestion congestion; Memory access pattern; the utilization and health of the cores of the CPUs 112 ; the current caching and / or storage bandwidth levels per thread, application, virtual machine or other boundary; topology-based information, such as which CPU cores have higher access to I / O bandwidth and / or memory bandwidth and lower latencies due to their placement on the CPU die; and other information.

As described above, various embodiments of the present disclosure may use logic to optimize the performance of the guest systems 122 within a single platform or across multiple platforms 102 include. Such embodiments may allow for optimization in deployments, including Network Function Virtualization (NFV) and Software Defined Networking (SDN). For example, the ability to migrate virtual network functions (VNFs) from one platform to another, and other emerging NFV and SDN use cases, provide an opportunity for local optimization based on the provision of a global data center virtual networking 100 perform. In addition, global consideration may allow for local optimizations that provide better security, quality-of-service (QoS), service level differentiation, traffic monitoring, service function chain (SFC) traffic engineering through SDN and NFV provisioning ,

Based on one or more of the platforms 102 associated telemetry data may include the performance of one or more I / O device drivers 124, platforms 102 and / or the entire data center 100 be improved over different vectors; These include gross throughput, latency, performance determinism, non-uniform memory access (NUMA) locality, scalability, power and other suitable features. Such optimizations can also improve security for flow-based QoS on the platforms 102 and be used via SFCs.

Every platform 102 can the platform logic 110 include. The platform logic 110 includes, among other logic, the functionality of the platform 102 allows one or more CPUs 112 , the memory 114 , one or more chipsets 116 and the communication interface 118 , Although three platforms are shown, the data center 100 include any suitable number of platforms. In various embodiments, a platform 102 be arranged on a circuit board, which in a housing, a rack or other suitable structure, which several through the network 108 (which may include, for example, a rack or backplane switch) having interconnected platforms is installed.

The CPUs 112 Each may comprise any suitable number of processor cores. The cores can pass through one or more on the CPU 112 and / or the chipset 116 located controllers with each other, with the memory 114 , with at least one chipset 116 and / or with the communication interface 118 be coupled. In certain embodiments, a CPU 112 executed inside a socket that is permanent or detachable with the platform 102 is coupled. The CPU 112 is in connection with below 4 described in more detail. Although four CPUs are shown, one platform can 102 include any suitable number of CPUs.

The memory 114 may include any form of volatile or non-volatile memory including, but not limited to, magnetic media (e.g., one or more tape drives), optical media, Random Access Memory (RAM), read-only Memory (read only memory, ROM), flash memory, removable media, or any other suitable local or remote memory component (s). The memory 114 is for short, medium and / or long term storage by the platform 102 usable. The memory 114 can be any appropriate, from the platform logic 110 store used data or information, including software embedded in a computer readable medium and / or embedded or otherwise stored coded logic (eg, firmware). The memory 114 can store data by cores of CPUs 112 be used. In some embodiments, the memory may be 114 Also include the storage of commands by the cores of the CPUs 112 or other processing elements (eg, on the chipsets 116 resident logic) can be executed to the administrative engine 126 or other components of the platform logic 110 to provide assigned functionality. Additionally or alternatively, the chipsets 116 each include memory that is any of the here in relation to the memory 114 may have described features. The memory 114 Also, the results and / or intermediate results of the different, by the CPUs 112 or processing elements on the chipsets 116 saved calculations and determinations. In various embodiments, the memory may be 114 include one or more system memory modules that are controlled by memory controllers (external or to the CPUs 112 can be integrated) with the CPUs are coupled. In various embodiments, one or more particular modules of the memory may be used 114 a particular CPU 112 or other processing device or multiple CPUs 112 or other processing devices.

A platform 102 can also have one or more chipsets 116 include any suitable logic to support the operation of the CPUs 112 include. In various embodiments, the chipset may 116 on the same package as a CPU 112 or on one or more other packages. Each chipset can be any suitable number of CPUs 112 support. A chipset 116 may also include one or more controllers for interfacing other components of the platform logic 110 (eg the communication interface 118 or the memory 114 ) with one or more CPUs. Additionally or alternatively, the CPUs 112 include integrated controls. For example, the communication interface 118 via integrated I / O controls residing on each CPU directly to the CPUs 112 be coupled.

In the depicted embodiment, each chipset includes 116 also an administrative machine 126 , The administrative machine 126 may be any suitable logic to support the operation of the chipset 116 include. In a particular embodiment, the management engine 126 (which may also be referred to as an innovation machine) capable of real-time telemetry data from the chipset 116 , CPU (s) 112 and / or memory 114 that through the chipset 116 , other components of the platform logic 110 and / or various connections between components of the platform logic 110 managed to collect. Any suitable telemetry data may be collected, such as the current CPU cache usage, the current memory bandwidth usage, and the current I / O bandwidth usage by each guest 122 or component thereof (eg, thread, application, etc.) or any I / O device (eg, Ethernet device or hard disk drive). Additional telemetry data may include a size of available space or bandwidth, a size of available CPU cache space, or bandwidth or available I / O bandwidth for each I / O device. In addition, temperatures, currents and / or voltages from different points of the platform 102 , such as at one or more locations of each core, one or more locations of each CPU 112 , one or more positions of the chipsets 116 or at other positions of the platform 102 , are collected (eg the air inlet and outlet temperatures can be measured).

In various embodiments, the management engine operates 126 as an asynchronous out-of-band data processing means capable of interfacing with the various elements of the platform logic 110 to make telemetry data without or with minimal disturbance of the running processes on the CPUs 112 to collect. For example, the management engine 126 a dedicated processing element (eg, a processor, a controller, or other logic) on the chipset 116 include the functionality of the administrative machine 126 (for example, by executing software instructions), and thus processing cycles of the CPUs 112 for the guest systems 122 preserve associated operations. In addition, the dedicated logic for the administrative machine 126 asynchronous to the CPUs 112 operate and capture at least some of the telemetry data without increasing the load on the CPUs.

The administrative machine 126 may also filter or otherwise process data that it collects. For example, the management engine 126 in some embodiments, calculate metrics of a service level agreement (SLA) based on their collected telemetry data and SLA violations by reporting those violations to the data center management platform 106 or local processing or enforcing enforcement actions.

In various embodiments, the management engine reports 126 the data that it collects and / or the results of its data filtering or other processing to other elements in the data center, such as a multiple I / O device driver 124, the hypervisor 120 or another operating system, the data analysis engine 104 and / or the data center management platform 106 , In some embodiments, the telemetry data is regularly reported to one or more of these devices. In certain embodiments, a critical event, such as a congested core or too high a temperature, may be reported prior to the normal telemetry data reporting interval (eg, a notification may be sent immediately upon discovery).

In various embodiments, the management engine may 126 include programmable code that is configurable to set which CPU (s) 112 the chipset 116 managed and / or which telemetry data is collected.

The chipsets 116 Each also includes a communication interface 128 , The communication interface 128 is to communicate signaling and / or data between the chipset 116 and one or more I / O devices, one or more networks 108 and / or one or more with the network 108 coupled devices (e.g., the data center management platform 106 or the data analysis engine 104 ) usable. For example, the communication interface 128 for sending and receiving network traffic, such as data packets. In a particular embodiment, the communication interface comprises 128 one or more Network Interface Controllers (NICs), also known as network interface cards or network adapters. A NIC may include electronic circuitry to use any suitable physical layer and data link layer standard, such as Ethernet (eg, as per the IEEE standard 802.3 defined), Fiber Channel, InfiniBand, Wi-Fi, or any other appropriate standard. A NIC may include one or more physical ports that may be coupled to a cable (eg, an Ethernet cable). A NIC can communicate between any appropriate element of the chipset 116 (eg the administration machine 126 or the switch 130 ) and another with the net 108 allow coupled device. In some embodiments, the network may 108 include a switch with bridging and / or routing capabilities that external to the platform 102 is located and operable to various, via the data center 100 (eg, on different platforms) to link distributed NICs together. In various embodiments, one NIC may be integrated into the chipset (ie, reside on the same integrated circuit or circuit board as the rest of the chipset logic) or on another integrated circuit or circuit board that is electromechanically coupled to the chipset.

In certain embodiments, the communication interface 128 the communication of data associated with management and monitoring functions performed by the management engine 126 be performed (for example, between the administrative engine 126 and the data center management platform 106 and / or the data analysis engine 104 ). In various embodiments, the management engine may 126 Elements (eg one or more NICs) of the communication interface 128 to report the telemetry data (for example, to the data center management platform 106 or the data analysis engine 104 ) to the use of NICs of the communication interface 118 for the guest systems 122 reserve allocated operations. In some embodiments, the communication interface 128 also allow integrated or off-platform 1/0 devices (eg, drives, other NICs, etc.) into the platform to communicate with the CPU cores.

The switch 130 can interfere with different (for example, NICs provided) ports of the communication interface 128 pair and data between these ports and different components of the chipset 116 (eg one or more with the CPUs 112 coupled Peripheral Component Interconnect Express (PCIe) lanes. The switch 130 can be a physical or virtual (ie software) switch.

The platform logic 110 can be an additional communication interface 118 include. Similar to the communication interface 128 is the communication interface 118 for communicating signaling and / or data between the platform logic 110 and one or more networks 108 and one or more with the network 108 coupled devices usable. For example, the communication interface 118 for sending and receiving network traffic, such as data packets. In a particular embodiment, the communication interface comprises 118 one or more physical NICs. These NICs can communicate between any appropriate element of the platform logic 110 (eg CPUs 112 ) and another with the net 108 coupled device (eg, elements of other platforms or through one or more networks to the network 108 coupled remote nodes). In certain embodiments, the communication interface 118 allow non-platform devices (eg, drives, other NICs, etc.) to communicate with the CPU cores. In various embodiments, NICs may be the communication interface 118 through I / O controllers (external or to the CPUs 112 can be integrated) with the CPUs.

The platform logic 110 can receive and execute any suitable types of processing requests. A processing request may be any request to use one or more resources of the platform logic 110 , such as one or more cores or associated logic. For example, a processing request may be a processor core interrupt; a request to instantiate a software component, such as an I / O device driver 124 or the virtual machine 132 ; a request to process one from a virtual machine 132 or from outside the platform 102 located device (such as one with the network 108 coupled network nodes) received network packets; a request to execute a workload (such as a process or thread) that is a virtual machine 132 , one on the platform 102 running application, the hypervisor 120 or another on the platform 102 associated with the current operating system; or another suitable request.

In various embodiments, processing requests may be to the guest systems 122 be assigned. A guest can be a single virtual machine (such as the virtual machine 132a or 132b ) or several virtual machines operating together (for example, a virtual network function (VNF)). 134 or a service function chain (SFC) 136 ). As depicted, various embodiments may have a variety of types of on the same platform 102 existing guest systems 122 include.

A virtual machine 132 can emulate a computer system with its own dedicated hardware. A virtual machine 132 can be a guest OS over the hypervisor 120 operate. The components of the platform logic 110 (eg the CPUs 112 , the memory 114 , the chipset 116 and the communication interface 118 ) can be virtualized so that it appears to the guest operating system that the virtual machine 132 has its own dedicated components.

A virtual machine 132 may include a virtualized NIC (vNIC) that is used by the virtual machine as its network interface. A vNIC can be assigned a media access control (MAC) address, which allows multiple virtual machines 132 in a network are individually addressable.

In some embodiments, a virtual machine 132b be paravirtualized. For example, the virtual machine 132b advanced drivers (for example, drivers that provide higher performance or higher bandwidth interfaces to underlying resources or capabilities provided by the hypervisor 120 be provided). For example, an extended driver may provide a faster interface to the underlying virtual switch 138 for a higher network performance over standard drivers.

The VNF 134 can comprise a software implementation of a functional building block with defined interfaces and behaviors that can be used in a virtualized infrastructure. In certain embodiments, a VNF 134 one or more virtual machines 132 which together provide specific functionalities (e.g., wide area network (WAN) optimization, virtual private network (VPN) completion, firewall operations, load balancing operations, security functions, etc.). One on the platform logic 110 ongoing VNF 134 can provide the same functionality as traditional dedicated hardware implemented network components. For example, a VNF 134 Components for performing any suitable NFV workloads, such as virtualized evolved packet core (vEPC) components, mobility management entities, 3rd generation partnership project (3GPP) control, and data tier components, etc., are included.

The SFC 136 is a group of VNFs 134 organized as a chain to perform a series of operations, such as network packet processing operations. Service chaining may provide the ability to maintain an ordered list of network services (eg, firewalls, Load distributor), which are linked together to create a service chain.

A hypervisor 120 (Also known as a virtual machine monitor) may be logic for generating and operating the guest systems 122 include. The hypervisor 120 may include guest operating systems that are executed by virtual machines with a virtual operating platform (ie, the virtual machines appear to be running on separate physical nodes although they are in fact consolidated on a single hardware platform) and the execution of the guest operating systems by the platform logic 110 manage. Services of the hypervisor 120 may be provided by virtualization in software or by hardware-based resources that require minimal software intervention, or both. Many instances of a variety of guest operating systems can be served by the hypervisor 120 to get managed. Every platform 102 may be a separate instantiation of a hypervisor 120 exhibit.

The hypervisor 120 can be a native or bare-metal hypervisor directly on the platform's logic 110 runs to control the platform logic and manage the guest operating systems. Alternatively, the hypervisor 120 a hosted hypervisor that runs on a host operating system and abstracts the guest operating systems from the host operating system. Various embodiments may include one or more non-virtualized platforms 102 include; In this case, any suitable features or functions of the hypervisor described herein may be used 120 apply to an operating system of the non-virtualized platform.

The hypervisor 120 may collect any suitable telemetry data (such as those described above) and the telemetry data to any suitable entity, such as one or more administrative engines 126 , the data center management platform 106 or the data analysis engine 104 , Report. In one embodiment, the hypervisor reports 120 the telemetry data via an in-band solution (ie using the same, by the guest systems 122 used communication interface) through the communication interface 118 to the data center management platform 106 and / or to the data analysis engine 104 , The hypervisor 120 may also include one or more I / O device drivers 124 (either directly or through an intermediate interface) with the management engine 126 and allow communication between the two units.

The hypervisor 120 can be a virtual switch 138 involve the virtual machines of the guest systems 122 can provide virtual switching and / or routing functions. The virtual switch 138 can include a logical switching fabric that contains the vNICs of the virtual machines 132 coupled together, creating a virtual network that allows virtual machines to communicate with each other. The virtual switch 138 can also communicate via physical NICs of the communication interface 118 with one or more networks (eg network 108 ) to communicate between the virtual machines 132 and one or more outside the platform 102 located network nodes (for example, a virtual machine running on another platform 102 running, or a node that is using the internet or another network with the platform 102 coupled). The virtual switch 138 may comprise a software element based on components of the platform logic 110 is performed. In various embodiments, the hypervisor may 120 communicate with any appropriate entity (e.g., an SDN controller) that can cause the hypervisor 120 the parameters of the virtual switch 138 in response to changing conditions in the platform 102 (eg the addition or removal of virtual machines 132 or the identification of optimizations that can be made to improve the performance of the platform).

The hypervisor 120 can also use the resource allocation logic 144 which may include logic for determining the allocation of platform resources based on the telemetry data. The resource allocation logic 144 Also can logic to communicate with different units of the platform 102 to implement such optimizations, such as components of the platform logic 102 and resource allocation logic 142 one or more I / O device drivers 124. For example, the resource allocation logic 144 specify which resources of the platform logic 110 be used to the virtual switch 138 or implement one or more I / O device drivers 124. As another example, the resource allocation logic could 144 the I / O to virtual machines 132 that have more resources to handle the load than other virtual machines. In certain embodiments, the resource allocation logic 144 the movement of processes associated with applications and / or virtual machines via cores of the platform logic 110 conduct.

The hypervisor 120 may include any suitable number of I / O device drivers 124. The I / O device driver 124 includes one or more software components that allow the hypervisor 120 communicates with a physical I / O device. In different Embodiments may use the underlying physical I / O device with any of the CPUs 112 be coupled and data to the CPUs 112 send and data from the CPUs 112 receive. The underlying I / O device can use any suitable communication protocol, such as PCI, PCIe, Universal Serial Bus (USB), Serial Attached SCSI (SAS), Serial ATA (SATA), InfiniBand, Fiber Channel, an IEEE 802.3 protocol, an IEEE 802.11 protocol or other current or future signaling protocol.

The underlying I / O device may include one or more ports that are operable to connect to cores of the CPUs 112 to communicate. In one example, the underlying I / O device is a physical NIC or a physical switch. For example, in one embodiment, the underlying I / O device of the I / O device driver 124 is a NIC of the communication interface 118 with several connections (eg Ethernet connections).

In other embodiments, underlying I / O devices may include any suitable device capable of providing data to the CPUs 112 to transmit and receive, such as audio / video (A / V) device control (eg, a graphics accelerator or audio controller); a data storage device controller, such as a flash memory device, a magnetic disk or optical disk controller; a wireless transceiver; a network processor or controller for another input device, such as a monitor, a printer, a mouse, a keyboard, or a scanner; or another suitable device.

In various embodiments, when a processing request is received, the I / O device driver 124 or the underlying I / O device may issue an interrupt (such as a message-signaled interrupt) to any of the cores of the platform logic 110 send. For example, the I / O device driver 124 may send an interrupt to a kernel selected to perform an operation (eg, for a virtual machine 132 or a process of an application). Before the interrupt is delivered to the core, the core input data (eg, network packets) may be at the underlying I / O device and / or one of the CPUs 112 the core associated I / O block are cached. In some embodiments, the I / O device driver 124 may configure the underlying I / O device with instructions related to where interrupts are to be sent.

In various embodiments, the I / O device driver 124 includes monitoring logic 140 , which may be capable of collecting real-time and / or benchmark telemetry data associated with the I / O device driver 124. In some embodiments, the hypervisor may 120 and / or the I / O device driver 124 perform a series of benchmark tests to obtain various telemetry data. The benchmark tests may take place at any appropriate time, for example when the platform 102 is turned on when the I / O device driver 124 is initialized or in the platform 102 a configuration change occurs. In various embodiments, the benchmark tests may be performed by the hypervisor 120 or the I / O device driver 124 (in some embodiments in response to a request from the management engine 126 to run benchmark tests). In some embodiments, certain benchmark tests may be performed during a period in which measured telemetry indicates that the benchmark tests would not interfere with normal processes performed by the CPU cores.

In some embodiments, the hypervisor may 120 and / or the I / O device driver 124 go through the cores during a benchmark test and collect data associated with each core. The benchmark tests may include latencies, bandwidths, determinisms (ie, the extent to which the amount of time required for a particular task is predictable), or jitter associated with each CPU core (eg, with respect to core-to-system memory communication , the core and other cores, the core and I / O devices such as NICs, the core and one or more caches, etc.) or other suitable information. As an example, small data transfers (such as a single cache line) may be through various components of the platform (eg, cores, I / O devices, various CPU sockets, memory 114 etc.) and the latencies of each transmission can be measured.

In some embodiments, the benchmark tests may determine the various topological data of the platform 102 allow. For example, the obtained data may be the positioning of cores with respect to one another (eg, the amount of time required to communicate from one core to another may indicate how close the cores are to each other), the positioning of cores with respect to one or more I's / O devices (eg, cores that respond more quickly to requests from an I / O device than are closer to the I / O device) can be used to position cores with respect to a memory module of the present invention memory 114 whose cores display certain shared caches (and their positioning relative to the shared caches), the positioning of cores with respect to particular connections (such as a QuickPath interconnect), or other appropriate information. The topology data may also include topology information that includes measurements across multiple CPUs (eg, how close a particular core of the CPU is 112a a specific core of the CPU 112b is). The topology data may allow the dynamic discovery of irregularities in shared platform resources (eg, in some embodiments, certain cores may have a higher available memory bandwidth than others, and other cores may have a higher I / O bandwidth than others).

In certain embodiments, telemetry data may indicate the topology or performance of the components of the platform 102 from an external source. For example, these data may be based on an identification of various components of the platform logic 110 (such as a serial number or stockkeeping unit (SKU) of a component) from a database storing information about the configuration of the components. In various embodiments, such information could be in the hypervisor 120 or I / O device drivers 124 or the hypervisor 120 or I / O device drivers 124 are otherwise accessible.

Telemetry data can also be used during normal operation of the platform 102 collected when the I / O device driver 124 passively monitors the traffic within the underlying I / O device. During such monitoring, the I / O device driver 124 may collect any suitable information, such as those described above in connection with the benchmarks. In various embodiments, the data collected at runtime may indicate real-time conditions based on the traffic flow, such as the load of each core, cache, and memory module. In at least some embodiments, the bandwidths and latencies measured at run time may reflect a bandwidth amount or average latencies available based on the current traffic flow in comparison with a maximum bandwidth or latency that might be measured during the benchmark test.

In some embodiments, the monitoring logic may be 140 Filter or otherwise process the data that it collects. In various embodiments, the monitoring logic reports 140 the data it collects and / or the results of its data filtering or other processing to other elements in the Data Center 100 , such as the administrative engine 126 , the hypervisor 120 or the other operating system, the data analysis engine 104 and / or the data center management platform 106 , In some embodiments, the telemetry data is regularly reported to one or more of these devices. In various embodiments, the monitoring logic may 140 include programmable code that is configurable to set which telemetry data is collected by the monitoring logic.

The collection of in-band telemetry data may require that workloads for the telemetry collection be disrupted. In addition, the collection of telemetry data may be restricted if certain cores are dedicated to VNF line rate packet processing and should not be disturbed. In addition, the collection of in-band telemetry data consumes computing cores, cache space, memory bandwidth, and other resources, which can affect existing workloads by reducing their bandwidth, increasing jitter, or increasing total latency.

In various embodiments, the I / O device driver 124 and / or the hypervisor 120 Collect and / or report telemetry data in an out-of-band manner. Such embodiments may interface with the hypervisor 120 and the administrative engine 126 include. The administrative machine 126 can receive the collected telemetry data through the interface and the data to the data analysis engine 104 and / or the data center management platform 106 send. Such embodiments may allow out-of-band access to the I / O device without disturbing the in-band accesses from the core (s) operating the I / O device driver 124. Such embodiments can significantly reduce overheads and interrupt rates that occur in response to reconfiguration. In another embodiment, the hypervisor 120 Telemetry data through a communication interface (eg the interface 118 ), also through the guest systems 122 However, the data may conveniently be forwarded to the data center management platform through network interfaces that are unused or underutilized 106 and / or the data analysis engine 104 Report the impact on the performance of the platform 102 to minimize.

The I / O device driver 124 may also use the resource allocation logic 142 include, the May include logic for determining the allocation of platform resources based on the telemetry data. The resource allocation logic 142 may also include logic for implementing such optimizations that includes communicating with the underlying I / O device to direct the I / O device to perform the optimizations.

Logic on the platform 102 may include the ability to determine the allocation of platform resources based on the telemetry data (including real-time measurements and / or benchmark data). For example, the logic may determine to which core an interrupt is to be forwarded, which cores to avoid, where to position a process associated with a particular virtual machine or group of virtual machines, which allocate resources to an I / O device driver 124 (eg which kernel should be used to operate the I / O device driver 124 and / or which caches or memory modules should be used to store data used by the I / O device driver 124), how to reconfigure the platform is to optimize resource usage or make other appropriate resource allocation decisions. In some cases, optimization can involve a different platform. For example, a process, virtual machine, or other workload could be located on or migrated to another platform with an assigned core, cache, and / or storage of that platform.

Any suitable logic may make one or more of these optimization decisions. For example, the resource allocation logic 142 I / O device driver 124, the data analysis engine 104 , the data center management platform 106 , the resource allocation logic 144 of the hypervisor 120 or any other operating system to be able to make such decisions (either alone or in combination with other elements of the platform 102 ). In a particular embodiment, the data center management platform 106 (using in-band or out-of-band communication) with the hypervisor 120 to specify the optimizations that should be used to comply with the policies stored on the data center management platform.

In various embodiments, the optimizations may be made in real time based on the traffic volume and workload on the cores, as measured by the collected telemetry data. Global (eg, platform-wide) or local (eg, with respect to a particular virtual machine, a guest, a guest running application, etc.) metrics that can be optimized by the logic of the platform can minimize this performance latency of a processing request, minimizing the impact of neighboring cores on the delivery of interrupts, minimizing interrupt latency, and maximizing overall throughput and performance. In some cases, a number of local optimizations (such as selecting specific cores for specific workloads) can be made to achieve a global optimization point (such as maximizing performance or minimizing the latency of a particular application).

As mentioned above, one of the optimizations may involve the placement of an I / O device driver 124 and associated resources. For example, powerful I / O device drivers can be run on cores, the CPU's I / O blocks 112 are closest. Resources associated with the I / O device driver 124 may also be located at optimal locations. For example, queues utilized by the I / O device driver 124 (eg, for storing instructions used to drive the driver or as a buffer for driver-directed inbound or outbound packets) or hardware DMA descriptors may be dedicated Cache or memory module (eg, one with high power and / or near the core operating the I / O device driver 124).

Certain workloads of the guest systems 122 (including core interrupts) may be assigned to a particular core based on the telemetry data, such as core bandwidth or core-to-cache, core-to-I / O or core-to-memory bandwidth. For example, high-priority workloads or high-priority connections may be routed to high-performance cores, while low-priority workloads may be routed to relatively low-performance cores. Workloads can also be assigned to caches or stores based on any suitable telemetry data. For example, a particular cache may include one or more partitions to store data associated with higher priority workloads. As another example, local memory to a particular CPU may be used for processes running on that CPU.

In some embodiments, when workloads are distributed among the cores, the hypervisor may 120 direct more workloads to the more powerful cores than to the lower performing cores. In some cases For example, cores showing problems such as overheating or heavy loads may be assigned fewer tasks than other cores, or they may be bypassed altogether (at least temporarily). Workloads, the applications, services, containers, and / or virtual machines 132 can be distributed to cores based on network load and traffic patterns rather than just CPU and memory usage metrics.

If some processes use the I / O device at a relatively high rate, these processes may be affinity-focused on a core that is close to the I / O device (ie, near an I / O block of a CPU 112 ) is located. Similarly, if a communication flow is latency sensitive, it may be placed on a high performance I / O device (eg, NIC) and a high performance core near the I / O device.

In various embodiments, the optimizations may include moving core control structures through the I / O device driver 124. For example, a shared ring used to communicate software running on a first core could be moved to a more optimal core along with its interface driver.

The optimizations can also change the structure of the guest systems 122 who can use communications between multiple virtual machines (e.g., VNF 134 or SFC 136 ), consider. For example, the workloads associated with a first virtual machine of an SFC may be located on a core that is adjacent to a core that executes workloads associated with a second virtual machine of the SFC so that communications from the first virtual machine can be quickly sent to the second virtual machine ,

In some embodiments, the optimizations may enable dynamic scalability, which may be particularly important, as they are based on maintaining high QoS with respect to service chaining and other guest systems 122 refers. In a particular embodiment, the I / O device driver 124 may collect telemetry data indicating the traffic flow. If the traffic flow rate on the platform 102 is low or moderate, the I / O device driver may send interrupts to cores closest to the I / O device (eg, NIC) to increase latency / bandwidth locality for better performance and lower latency maximize. For example, the I / O device driver 124 may load-distribute the interrupts to a selected number of high-performance cores. In some embodiments, if the flow rate can be effected without a reduction in performance by a single high-performance core, all interrupts can be sent to that core. When the interrupts are load-balanced across a large number of cores, a simple round-robin technique or methodology that directs a higher percentage of requests to one or more higher-performance cores can be used for load balancing.

If the flow rate is high (eg, above a certain threshold), then the interrupts can be split across all cores to maximize throughput across the platform. In some embodiments, the number of cores that the I / O device driver 124 sends interrupts may increase in proportion to the traffic flow rate to ensure high performance of existing flows.

This allows workloads, the applications, services, containers or virtual machines 132 are assigned to cores based on network load and traffic patterns rather than just CPU and memory usage metrics.

Further, in some embodiments, optimizations (such as the selection of cores) may be based on one or more packet features detected by the I / O device driver 124. As an example, small packets of the order of 64 bytes may be keep-alive of phone applications on handheld devices in a LTE mobile network and require a fast response, and would thus be connected to a low-latency core (eg, a core near the I / O). Device) are sent. As another example, latency sensitive transmission control protocol (TCP) packets may also be sent to a low latency core.

In various embodiments, the placement of a workload may also be based on compatibility with platform logic components 110 based. For example, workload requirements may be against an SKU, a cache type, a supported bus, a QPI version, or any other suitable feature of the CPU 112 be checked to determine if the workload can be operated by a core of the CPU. Should any CPUs 112 Being incompatible with the workload may be the best remaining core in the platform logic 110 selected for the workload.

In various embodiments, the data center management platform 106 Telemetry data from multiple platforms 102 receive and manage optimizations over the same. The data center management platform 106 can with the hypervisors 120 (eg in an out-of-band way) or other operating systems of the different platforms 102 communicate to implement optimizations directed by the data center management platform. In this way, the data center management platform can control workload placement and overall data center performance.

In some embodiments, the logic of the platform 102 include a closed loop that would compare performance before performing optimization and after performing optimization to see how optimization has impacted performance. The closed loop may be operable to optimize placement by the hypervisor such that processes utilizing the I / O device driver 124 are closer to the I / O device driver 124 and to dynamically evaluate the improvements. The logic that implements the closed loop may reside in any appropriate unit of data center 100 such as a hypervisor 120 or other operating system, the I / O device driver 124 or a combination thereof.

The elements of the platform logic 110 may be coupled together in any suitable manner. For example, a bus may couple any of the components together. A bus may be any known connection, such as a multi-drop bus, a mesh connection, a ring connection, a point-to-point connection, a serial connection, a parallel bus, a coherent (e.g. Cache coherent) bus, a layered protocol architecture, a differential bus, or a Gunning Transceiver Logic (GTL) bus.

Elements of the data system 100 may be in any suitable manner, such as by one or more networks 108 be coupled with each other. A net 108 may be any suitable network or combination of one or more networks that can operate on one or more appropriate network protocols. A network may represent a series of nodes, points, and connected communication paths for receiving and transmitting information packets propagating through a communication system. For example, a network may include one or more firewalls, routers, switches, security applications, antivirus servers, or other useful network devices. A network provides communicative interfaces between sources and / or hosts and may be any local area network (LAN), wireless local area network (WLAN), metropolitan area network (MAN), intranet, extranet , Internet, wide area network (WAN), virtual private network (VPN), cellular network, or any other suitable architecture or system that enables communication in a network environment. A network may include any number of hardware or software elements that are coupled (and in communication) through a communication medium. In various embodiments, the guest systems may 122 through the net 108 communicate with nodes that are outside the data center 100 are located.

2 shows a block diagram of a data center management platform 106 according to some embodiments. The data center management platform 106 includes at least one CPU besides any other suitable hardware 202 , the memory 204 and the communication interface 206 having any suitable features as described above with reference to the CPUs 112 , the memory 114 and the communication interface 118 described, may include the operations of the data center management platform 106 to enable. In various embodiments, the data center management platform may 106 from the platforms 102 of the data center 100 different (for example, it may be in another physical module or on a different board). The data center management platform 106 can through the communication interface 206 with each of the platforms 102 and / or the data analysis engine 104 communicate and telemetry data from the platforms 102 and run the platforms to perform optimizations as described herein. In one embodiment, the communication interface uses 206 an out-of-band solution for communication with the management machines 126 and an in-band solution to connect directly to the platforms 102 running hypervisors 120 or operating systems.

The policy database 208 The Customer Service Level Agreement (SLA) includes logic for mapping one to one or more platforms 102 running application to an SLA, so the data center management platform 106 assess whether performance objectives related to the application are met. SLAs may be based on any suitable measures, such as metrics, virtual machines or VNF operations (e.g., a virtual machine provisioning latency and reliability, a virtual machine clock failure, a virtual machine dead on arrival, etc.) or virtual network operations (eg packet delays, delay fluctuations, power failures, Connection status, policy integrity, etc.).

The security monitoring and policy orchestrator 210 can provide security monitoring and management logic within the data center 100 include. For example, the security guard and policy orchestrator 210 include intrusion detection and mitigation, denial and risk mitigation, anti-virus and other security features. The security monitoring and policy orchestrator 210 can be a global view of the data center 100 and virtual machine deployments 132 maintained within the data center from a security standpoint and connections between different network segments within the data center 100 (eg communication / bridging across different VLANs).

The traffic monitoring and engineering module 212 includes logic for monitoring the traffic on the platforms 102 of the data center and to guide optimizations for that traffic. The module 212 can with the data analysis engine 104 , the administrative machines 126 from different platforms and / or the hypervisors 120 or other operating systems from different platforms to receive telemetry data, determine appropriate optimizations, and direct execution of the optimizations. In some embodiments, the data center management platform 106 the hypervisor 120 or the other operating system to perform in-band benchmark tests to capture additional telemetry data. In various embodiments, the data center management platform may include data aggregation logic and analytic algorithms, and / or the functionality of the data analysis engine 104 use to determine which optimizations to perform.

As an example of an optimization that could be made, the module may 212 a specific platform 102 and / or certain elements of the platform logic 110 for a workload or the instantiation of a virtual machine 132 , a VNF 134 or an SFC 136 select based on the performance optimization for this element or for the data center as a whole. As another example, the module 212 the migration of a workload or virtual machine 132 , VNF 134 or SFC 136 from a platform (e.g., 132a) to another platform (e.g., 132b) to improve performance for the particular migrated element or for the data center as a whole. In some embodiments, the module may 212 the hypervisors 120 different platforms 102 Guidelines for through the hypervisors 120 provide optimizations (including optimizations made by the I / O device driver 124). In one application, the module can 212 recognize that an SFC 136 poor performance (eg, a predetermined QoS is not met), and try the performance of the SFC 136 improve by having a hypervisor 120 This leads to one of the SFC 136 associated workload to move from a lower performing core to a more powerful core. In another application, the module 212 recognize that connections between two virtual machines 132 are weak and one or both of the virtual machines 132 move to other cores to increase the connection performance.

3 shows a block diagram of a data analysis machine 104 according to some embodiments. The data analysis machine 104 includes at least one CPU besides any other suitable hardware 302 , the memory 304 and the communication interface 306 having any suitable features as described above with reference to the CPUs 112 , the memory 114 and the communication interface 118 described, may involve the operations of the data analysis machine 104 to enable. In various embodiments, the data analysis engine may be 104 from the platforms 102 of the data center 100 and / or the data center management platform (e.g., it may reside in a different physical module or on a different board). The data analysis machine 104 can through the communication interface 306 with each of the platforms 102 and / or the data center management platform 106 communicate and telemetry data from the platforms 102 and / or the data center management platform 106 receive and process the telemetry data. In one embodiment, the communication interface uses 306 an out-of-band solution for communication with the management machines 126 and an in-band solution to connect directly to the platforms 102 running hypervisors 120 or operating systems.

The data analysis module 308 may include any suitable logic to store, process, and / or learn the collected telemetry data for future optimization decisions by the platforms 102 or the data center management platform 106 to steer. The data analysis module 308 can aggregate telemetry data, filter telemetry data, average telemetry data, or otherwise process the telemetry data.

4 shows a block diagram of a central processing unit (CPU) 112 , with various other components of the platform 102 coupled, according to some embodiments. Although the CPU 112 shows a particular configuration, the cores and other components of the CPU can 112 be arranged in any suitable manner. The CPU 112 may include any processor or processing device, such as a microprocessor, an embedded processor, a digital signal processor (DSP), a network processor, an application processor, a coprocessor, a system on a chip , SOC) or another device for executing code. The CPU 112 In the depicted embodiment, four processing elements (the cores 402 in the illustrated embodiment), which may include asymmetric processing elements or symmetric processing elements. However, the CPU can 112 any number of processing elements that may be symmetric or asymmetric.

In one embodiment, a processing element refers to hardware or logic to support a software thread. Examples of hardware processing elements include: a thread unit, a thread slot, a thread, a processing unit, a context, a context unit, a logic processor, a hardware thread, a kernel, and / or any other element disclosed in the It is able to hold a state for a processor, such as an execution state or an architectural state. In other words, in one embodiment, a processing element refers to any hardware that is capable of being independently assigned to a code, such as a software thread, an operating system, an application, or other code. A physical processor (or processor socket) typically refers to an integrated circuit that optionally includes any number of other processing elements, such as cores or hardware threads.

A core may refer to logic that resides on an integrated circuit and is capable of maintaining an independent architectural state, with each independently maintained architectural state associated with at least some dedicated execution resources. A hardware thread may refer to any logic that resides on an integrated circuit and is capable of maintaining an independent architectural state, with independently maintained architectural states sharing access to execution resources. As can be seen, the boundary between the nomenclature of a hardware thread and a kernel overlaps when certain resources are shared and others are dedicated to an architectural state. Nevertheless, an operating system often views a core and a hardware thread as a single logical processor, with the operating system being able to schedule operations on each logical processor individually.

The physical CPU 112 , as in 4 shown, contains four cores - the cores 402A . 402B . 402C , and 402D Although a CPU may include any suitable number of cores. Here are the cores 402 be regarded as symmetric cores. In another embodiment, the cores may include one or more out-of-order processor cores or one or more in-order processor cores. However, the cores can 402 individually from any kernel type, such as a native kernel, a software managed kernel, a kernel adapted to execute a native instruction set architecture (ISA), a kernel adapted to execute a translated ISA , a co-developed nucleus or another known nucleus. In a heterogeneous core environment (ie, asymmetric cores), one form of translation, such as binary translation, can be used to schedule or execute code on one or both cores.

A core 402 may include a decode module coupled to a fetch unit to decode fetched items. In one embodiment, fetch logic includes individual sequencers, the thread slots of the cores 402 assigned. Usually, it is a core 402 a first ISA assigned to the core 402 executable commands defined / specified. Often, machine code instructions that are part of the first ISA include a portion of the instruction (referred to as an opcode) that references / specifies a instruction or operation to be performed. The decode logic may include circuitry that recognizes these instructions at their opcodes and forwards the decoded instructions in the pipeline for processing as defined by the first ISA. For example, in one embodiment, decoders may include logic that is designed or adapted to recognize particular instructions, such as transaction instructions. Due to the recognition by the decoder undertakes the architecture of the core 402 certain predefined actions to perform tasks associated with each command. It is important to note that any of the tasks, blocks, acts and methods described herein may be performed in response to a single or multiple instructions, some of which may be new or old instructions. Decoder of the cores 402 recognize the same ISA (or a subset thereof) in one embodiment. Alternatively, in one heterogeneous core environment a decoder of one or more cores (eg the nucleus 402B ) recognize a second ISA (either a subset of the first ISA or its own ISA).

In various embodiments, the cores 402 Also include one or more arithmetic logic units (ALUs), floating point units (FPUs), caches, instruction pipelines, interrupt handling hardware, registers, or other suitable hardware for handling the cores 402 to enable.

The bus 408 can be any suitable, with the cpu 112 represent coupled connection. In one example, the bus 408 the CPU 112 with another platform logic CPU 102 couple (eg via QPI). The I / O blocks 404 provide interface logic for coupling the I / O devices 410 and 412 to cores of the CPU 112 In various embodiments, an I / O block 404 may include I / O control that resides on the same package as the cores 402 integrated or may simply include interface logic for coupling to off-chip I / O control. As an example, the I / O blocks 404 may include PCIe interface logic. Likewise, the memory controller 406 Interface logic for coupling the memory 114 with cores of the CPU 112 In various embodiments, the memory controller 406 on the same package as the cores 402 integrated. In alternative embodiments, a memory controller may be located off-chip.

In the depicted embodiment, the cores are 402 via the I / O blocks 404A and 404B, the memory controller 406 and the bus 408 with each other and with other components of the platform 102 coupled. 4 shows nonuniformities that may exist in different topologies and through benchmark tests performed (eg by the hypervisor 120 or the I / O device driver 124) can be detected and taken into account when optimizing within the data center 100 be made.

As various examples, in the illustrated embodiment, the core 402A a relatively high bandwidth and lower latency to the bus 408 coupled devices (eg other CPUs 112 ) and the NICs 410 but a relatively low bandwidth and higher latency to memory 114 or to the core 402D exhibit. The core 402B can have relatively high bandwidths and low latency to both the NICs 410 as well as the PCIe solid state drive (SSD) 412 and moderate bandwidths and latencies to the bus 408 coupled devices and to the core 402D exhibit. The core 402C would have relatively high bandwidths and low latency to memory 114 and to the core 402D , The core 402D Finally, it would have a relatively high bandwidth and low latency to the core 402C , but relatively low bandwidths and high latency to the NICs 410 , to the core 402A and to the bus 408 coupled devices.

Optimizations within the platform 102 can be based on the unevenness present within the CPUs of the platform. For example, if a virtual machine 132 often with another virtual machine 132 (for example, if the first virtual machine could be a block in an SFC and the second virtual machine could be the next block), then the first virtual machine might be on the core 402C can be arranged, and the second virtual machine could be on the core 402D kernel thread conflicts may make it undesirable to place both virtual machines on the same core). Another example may be to provide NIC interrupts (through an I / O device driver 124) to the core 402A or 402B for high bandwidth and low latency. Yet other examples may include placing one frequently on the memory 114 accessing virtual machine on the core 402C or one common to the SSD 412 accessing process on the core 402B include.

5 FIG. 12 illustrates a method for allocating platform resources based on telemetry data, according to some embodiments. In step 502 becomes a platform 102 switched on. In step 504 Benchmark-based telemetry data is collected. The collection of this data may be done by any suitable entity, such as the data center management platform 106 , one or more administrative machines 126 , the hypervisor 120 or another operating system, or one or more I / O device drivers 124, are initiated and / or collected. Any suitable telemetry data can be collected. In a particular embodiment, the collected telemetry data may be a topology of at least part of the platform logic of the platform 102 Show. For example, for each core of the platform logic, the topology data may include an indication of bandwidth or latency from the core to each of the one or more memory modules of the platform logic. As another example, for each core of the platform logic, the telemetry data may include an indication of bandwidth or latency from the core to any other core of the platform logic. As another example, for each core of the platform logic, the telemetry data may be an indication of bandwidth or latency from the core to each of the one or more I / O devices. Include platform logic devices. In step 506 For example, the telemetry data may be transferred from the unit that collected it to any other suitable entity, such as the data center management platform 106 , the administrative machine 126 , the hypervisor 120 or the other operating system, an I / O device driver 124, or the data analysis engine 104 , be reported.

In step 508 can gather additional telemetry data and then in step 510 be reported. During these steps, any suitable telemetry data, such as any of those described above in connection with step 504 or telemetry data described in connection with the previous figures. Once again, the telemetry data from the unit that collected them can be sent to any other suitable entity, such as the data center management platform 106 , one or more administrative machines 126 , the hypervisor 120 , one or more I / O device drivers 124 or the data analysis engine 104 , be reported.

Parallel to step 508 and / or step 510 can in step 512 Processing requests are received and in step 514 Platform resources based on the telemetry data. A processing request may include any suitable request involving the use of the platform logic, such as a request to provide a guest, a request to provide an I / O device driver 124, a request to perform a workload (e.g., a process or threads) associated with a guest system and / or application, a network packet addressed to a guest, or other suitable processing request. The processing request may be at any suitable entity, such as the hypervisor 120 , at the I / O device driver 124 or at the data center management platform 106 to be received. The receiving unit can be used alone or in combination with other units of the data center 100 Assign platform resources to the request based on available telemetry data. The platform resources may be allocated in a manner that optimizes the performance of the processing request, the performance of the entire platform, and / or the performance of the entire data center. The steps 508 - 514 For example, any appropriate number of times may be performed that allow resource allocation decisions to be made based on the current state of the platform.

Some of the in 5 If appropriate, the steps shown may be repeated, combined, modified or omitted, and additional steps may be added to the flowchart. In addition, steps may be performed in any suitable order without departing from the scope of certain embodiments.

"Logic" (as used in platform logic, for example 110 , Monitoring logic 140 , Resource allocation logic 142 , Resource allocation logic 144 or in other references to logic in this application) may refer to hardware, firmware, software, and / or respective combinations for performing one or more functions. In various embodiments, logic may include a microprocessor or other processing element operable to execute software instructions, discrete logic such as an application specific integrated circuit (ASIC), a programmed logic device such as an environment programmable gate Field programmable gate array (FPGA), memory device containing instructions, combinations of logic devices (such as would be found on a printed circuit board), or other suitable hardware and / or software. Logic may include one or more gates or other circuit components. In some embodiments, logic may also be implemented entirely as software.

A design can go through different stages, from creation to simulation and manufacturing. Data that represents a design can represent the design in several ways. First, as is helpful in simulations, the hardware can be represented by a hardware description language (HDL) or other functional description language. In addition, at various stages of the design process, a circuit level model with logic and / or transistor gates may be created. Further, at a certain level, most designs achieve a level of data representing the physical placement of various devices in the hardware model. In the case of using conventional semiconductor manufacturing techniques, the data representing the hardware model may be the data that specifies the presence or absence of various characteristics on different mask layers for masks used to fabricate the integrated circuit. In some implementations, such data may be stored in a database file format, such as Graphic Data System II (GDS II), Open Artwork System Interchange Standard (OASIS), or a similar format.

In some implementations, software-based hardware models and HDL and other functional descriptor language objects may include Register Transfer Language (RTL) files among other examples. Such objects may be mechanically parsable such that a design tool accepts the HDL object (or model), parses the HDL object for attributes of the described hardware, and determines a physical circuit and / or on-chip layout from the object can. The output of the design tool can be used to fabricate the physical device. For example, a design tool may determine configurations of various hardware and / or firmware elements from the HDL object, such as bus widths, registers (including sizes and types), memory blocks, physical link paths, fabric topologies among other attributes that would be implemented to realize the system modeled in the HDL object. Design tools may include tools for determining the topology and fabric configurations of single-chip systems (SoC) and other hardware devices. In some cases, the HDL object may be used as a basis for developing models and design files that may be used by manufacturing equipment to fabricate the described hardware. In fact, an HDL object itself may be provided as an input to manufacturing software to provide the described hardware.

In any design representation, the data may be stored in any form of machine-readable medium. A memory or a magnetic or optical storage medium, such as a disk, may be the machine-readable medium for storing information transmitted over an optical or electrical wave that has been modulated or otherwise generated to transmit such information. When an electrical carrier wave indicating or carrying the code or design is transmitted to the extent that copying, buffering or retransmission of the electrical signal is performed, a new copy is made. Thus, a communications provider or a network provider may store an article, such as carrier-encoded information, which embodies techniques of embodiments of the present disclosure, at least temporarily on a tangible, machine-readable medium.

A module as used herein refers to any combination of hardware, software and / or firmware. As an example, a module includes hardware, such as a microcontroller associated with a non-transitory medium, for storing code adapted to be executed by the microcontroller. Thus, in one embodiment, the reference to a module refers to the hardware that is specifically configured to recognize and / or execute the code to be held on a non-transitory medium. Further, in another embodiment, the use of a module refers to the code-containing non-transitory medium that is specifically adapted to be executed by the microcontroller to perform predetermined operations. And as may be deduced, in yet another embodiment, the term "module" (in this example) may refer to the combination of the microcontroller and the non-transitory medium. Module boundaries, shown as separate, usually vary often and overlap each other, if necessary. For example, a first and a second module may share hardware, software, firmware, or a combination thereof, while retaining some independent hardware, software, or firmware as appropriate. In one embodiment, the use of the term "logic" includes hardware, such as transistors, registers, or other hardware, such as programmable logic devices.

The use of the phrase 'to (m)' or 'configured to' in one embodiment refers to the arranging, assembling, finishing, offering to sell, import and / or develop a device, hardware, logic or an item to perform a designated or specific task. In this example, a device or element thereof that is not operating is still configured to perform a designated task when it is designed, coupled, and / or connected to perform the designated task. As a purely illustrative example, a logic gate may provide a 0 or a 1 during operation. On the other hand, a logic gate that is 'configured' to provide an enable signal to a clock does not include every potential logic gate that can provide a 1 or a 0. Instead, the logic gate is one that is coupled in a certain way so that during operation the 1 or 0 output is to activate the clock. It should again be noted that the term 'configured to' does not require operation but instead refers to the latent state of a device, hardware, and / or element, where the device, hardware, and / or element is in latent state Condition are designed to perform a specific task when the device, the hardware and / or the element work.

Further, the use of the phrases capable of 'and / or' operable, to 'in one embodiment refers to a device, a logic, a Hardware and / or an item designed to allow use of the device, logic, hardware and / or element in a specified manner. It should be noted, as noted above, that the use of 'to (m)', 'capable,' or 'operable, to' in one embodiment refers to the latent state of a device, logic, hardware, and / or of an element, wherein the device, the logic, the hardware and / or the element do not work but are designed to allow the use of a device in a specified manner.

A value as used herein includes any known representation of a number, state, logic state, or binary logic state. Often, the use of logic levels, logic values or logical values is also referred to as 1s and 0s, which is simply binary logic states. For example, a 1 refers to a high logic level and 0 refers to a low logic level. In one embodiment, a memory cell, such as a transistor or a flash cell, may be capable of holding a single logical value or multiple logical values.

However, other representations of values have also been used in computer systems. For example, the decimal number ten may also be represented as a binary value of 418A0 and a hexadecimal A character. Therefore, a value includes any representation of information that is capable of being held in a computer system.

In addition, states can be represented by values or parts of values. As an example, a first value, such as a logical one, may represent a default state, while a second value, such as a logical zero, may represent a non-default state. Moreover, in one embodiment, the terms 'reset' and 'set' refer to a default value or state, or an updated value or state. For example, a default value may include a high logical value, i. H. Reset, while an updated value may include a low logical value, i. H. ,Set'. It should be noted that any combination of values may be used to represent any number of states.

The embodiments of methods, hardware, software, firmware or code set forth above may be implemented via instructions or code stored on a machine-accessible, machine-readable, computer-accessible or computer-readable medium and executable by a processing element. A non-transitory machine-accessible / readable medium includes any mechanism that provides information (i.e., stores and / or transmits) information in a form that is readable by a machine such as a computer or electronic system. For example, a non-transitory machine-accessible medium includes Random Access Memory (RAM) such as static RAM (SRAM) or dynamic RAM (DRAM); Read-only memory (ROM); a magnetic or optical storage medium; Flash memory devices; electrical storage devices; optical storage devices; acoustic storage devices, another form of storage device for holding information received from transitory (transmitted) signals (e.g., carrier waves, infrared signals, digital signals, etc.) different from non-transitory media capable of receiving information therefrom.

Instructions used to program logic to execute embodiments of the disclosure may be stored within memory in the system, such as DRAM, cache, flash memory, or other memory. Furthermore, the commands may be distributed over a network or by other computer-readable media. Thus, a machine readable medium may include any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer), but is not limited to floppy disks, optical disks, compact disc read only memories (Compact Disc read-only memory, CD-ROMs) and magneto-optical disks, read-only memory (ROMs), random access memory (RAM), erasable programmable read-only memory (Erasable Programmable Read -Only Memory, EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), magnetic or optical cards, flash memory, or a tangible machine-readable memory used in the transmission of information over the Internet via electrical, optical, acoustic or other forms of propagation signals (eg carrier waves, infrared signals, digital signals, etc.). Accordingly, the computer-readable medium includes any type of tangible machine-readable medium capable of storing or transmitting electronic instructions or information in a form readable by a machine (e.g., a computer).

In one embodiment, a system includes platform logic that includes a plurality of processor cores, and Resource allocation logic. The resource allocation logic is to receive a processing request and direct the processing request to a processor core of the plurality of processor cores, wherein the processor core is selected based at least in part on telemetry data associated with the platform logic, the telemetry data indicating a topology of at least a portion of the platform logic.

The system may further include an I / O device driver that includes at least a portion of the resource allocation logic, and routing the processing request to the processor core may include sending an interrupt to the processor core through the I / O device driver. The I / O device driver may perform benchmark tests on the platform logic to collect the telemetry data.

The telemetry data indicating the topology of at least part of the platform logic may include an indication of bandwidth or latency from the processor core to a memory of the platform logic. The telemetry data indicating the topology of at least a portion of the platform logic may include an indication of bandwidth or latency from the processor core to another processor core of the plurality of processor cores. The telemetry data indicating the topology of at least a portion of the platform logic may include an indication of bandwidth or latency from the processor core to an I / O device of the platform logic. The telemetry data may also indicate current bandwidths used by the processor cores.

The resource allocation logic may select the processor core based on the processor core with a highest expected performance for the processing request of the plurality of processors when a total utilized bandwidth of the plurality of processors falls below a certain threshold. The resource allocation logic may select the processor core based on optimizing the overall bandwidth of the plurality of processors when a total utilized bandwidth of the plurality of processors exceeds a certain threshold.

The processing request may include a request to instantiate an I / O device driver. The processing request may include a request to instantiate a virtual machine. The processing request may be associated with a service function chain running on the platform logic or a virtual network function.

The system may further include a management engine for collecting additional telemetry data associated with the platform logic in an out-of-band manner with respect to the plurality of processor cores. The management engine may send the additional telemetry data to a data center management platform that receives telemetry data from a plurality of individual platforms, each platform including at least one processor core. The management engine may perform benchmark tests on the platform logic to collect the telemetry data indicating the topology of at least part of the platform logic.

A hypervisor or operating system executed by the platform logic may perform benchmark tests on the platform logic to collect the telemetry data indicating the topology of at least part of the platform logic. The system may further comprise a hypervisor comprising the resource allocation logic, wherein the hypervisor's resource allocation logic is to select the processor core based on performance requirements associated with one or more of the processing request.

In one embodiment, a method may include receiving a processing request at an input / output device driver; by the input / output device driver, selecting a processor core from a plurality of processor cores based at least partially on the processor cores associated telemetry data; and by the input / output device driver, sending an interrupt signal associated with the processing request to the selected processor core.

The telemetry data may indicate a topology of at least a portion of the processor cores. The telemetry data may include bandwidth or latency from the processor core to memory coupled to at least some of the processor cores; a bandwidth or latency from the processor core to another processor core of the plurality of processor cores; and / or indicate a bandwidth or latency from the processor core to an input / output device.

The method may further include performing benchmark tests on the processor cores by the input / output device driver to collect the telemetry data. The method may further include selecting the processing core based on the processor core having a highest expected performance for the processing request A plurality of processors include when a total utilized bandwidth of the plurality of processors falls below a certain threshold. The method may further comprise selecting the processor core based on optimizing the overall bandwidth of the plurality of processors when a total utilized bandwidth of the plurality of processors exceeds a certain threshold.

The processing request may include a request to instantiate an I / O device driver or a request to instantiate a virtual machine.

The method may further comprise collecting additional telemetry data by an administrative engine in an out-of-band manner with respect to the plurality of processor cores.

A system may comprise a plurality of platforms, wherein one platform comprises a plurality of processor cores; and a data center management platform coupled through a network with the plurality of platforms. The data center management platform may receive telemetry data from the plurality of platforms, the telemetry data indicating performance cores associated with the processor cores of each of the platforms; receive an indication of at least one workload to be executed by a platform from the plurality of platforms; select a platform from the plurality of platforms based on the telemetry data received from the platforms; and direct the placement of the workload to the selected platform.

The workload may include a service function chain or a virtual network function, and directing the workload placement to the selected platform may include migrating the service function chain or virtual network function from a first platform of the plurality of platforms to the selected platform.

The telemetry data may also indicate a topology of processor cores of a platform. The data center management platform may collect and transmit at least a portion of the telemetry data from one or more of a management engine of a platform, the management engine collecting and transmitting the telemetry data in an out-of-band manner with respect to the processor cores of the platform; a hypervisor running on the platform; and an input / output device driver of the platform.

In one embodiment, at least one machine-readable storage medium may have instructions stored thereon. The instructions, when executed by a machine, may cause the machine to receive a processing request at an input / output device driver; selecting, by the input / output device driver, a processor core from a plurality of processor cores based at least in part on telemetry data associated with the processor cores; and send, by the input / output device driver, an interrupt signal associated with the processing request to the selected processor core.

The telemetry data may include an indication of a topology of at least a portion of the processor cores. The telemetry data may include an indication of bandwidth or latency from the processor core to another processor core of the plurality of processor cores. The telemetry data may include an indication of bandwidth or latency from the processor core to an I / O device. The instructions, when executed, may further cause the machine to perform benchmark tests on the plurality of processor cores to collect the telemetry data.

In one embodiment, a system includes platform logic comprising a plurality of processor cores; Means for receiving a processing request and means for directing the processing request to a processor core of the plurality of processor cores, wherein the processor core is selected based at least in part on telemetry data associated with the platform logic, the telemetry data indicating a topology of at least a portion of the platform logic.

The directing the processing request to the processor core may include sending an interrupt to the processor core. The telemetry data may include an indication of a topology of at least a portion of the platform logic. The telemetry data may include an indication of bandwidth or latency from the processor core to another processor core of the plurality of processor cores. The telemetry data may include an indication of bandwidth or latency from the processor core to an I / O device. The system may further comprise means for performing benchmark tests on the platform logic for collecting the telemetry data.

The reference to "1 embodiment" or "an embodiment" in this specification means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present disclosure. Thus, the occurrence of the phrases "in 1 embodiment" or "in one embodiment" at various points in this description does not necessarily always refer to the same embodiment. Furthermore, the particular properties, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

In the foregoing description, a detailed description has been given with reference to specific exemplary embodiments. It is understood, however, that various modifications and changes may be made thereto without departing from the broader spirit and scope of the disclosure as set forth in the appended claims. Accordingly, the description and drawings are to be considered in an illustrative rather than a limiting sense. Further, the above use of embodiment and other exemplary language does not necessarily refer to the same embodiment or the same example, but may refer to different and different embodiments as well as possibly to the same embodiment.

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• US 14866869 [0001]

Claims (31)

System comprising: Platform logic that includes a variety of processor cores; and Resource Assignment Logic for: Receive a processing request and Routing the processing request to a processor core of the plurality of processor cores, wherein the processor core is selected based at least in part on telemetry data associated with the platform logic, the telemetry data indicating a topology of at least a portion of the platform logic. System after Claim 1 and further comprising an I / O device driver that includes at least a portion of the resource allocation logic, and wherein routing the processing request to the processor core includes sending an interrupt to the processor core through the I / O device driver. System after Claim 1 , further comprising an I / O device driver including at least a portion of the resource allocation logic, wherein the I / O device driver is to perform benchmark tests on the platform logic to collect the telemetry data. System after Claim 1 wherein the telemetry data indicating the topology of at least a portion of the platform logic includes an indication of bandwidth or latency from the processor core to a memory of the platform logic. System after Claim 1 wherein the telemetry data indicating the topology of at least a portion of the platform logic includes an indication of bandwidth or latency from the processor core to another processor core of the plurality of processor cores. System after Claim 1 wherein the telemetry data indicating the topology of at least a portion of the platform logic includes an indication of bandwidth or latency from the processor core to an I / O device of the platform logic. System after Claim 1 The telemetry data further indicates current bandwidths used by the processor cores. System after Claim 7 wherein the resource allocation logic is to select the processor core based on the processor core with a highest expected performance for the processing request of the plurality of processors when a total utilized bandwidth of the plurality of processors falls below a certain threshold. System after Claim 7 wherein the resource allocation logic is to select the processor core based on optimizing the overall bandwidth of the plurality of processors when a total utilized bandwidth of the plurality of processors exceeds a certain threshold. System after Claim 1 wherein the processing request comprises a request to instantiate an I / O device driver. System after Claim 1 wherein the processing request comprises a request to instantiate a virtual machine. System after Claim 1 wherein the processing request is associated with a service function chain or a virtual network function running on the platform logic. System after Claim 1 and further comprising a management engine for collecting additional telemetry data associated with the platform logic in an out-of-band manner with respect to the plurality of processor cores. System after Claim 13 wherein the management engine is further to transmit the additional telemetry data to a data center management platform that receives telemetry data from a plurality of individual platforms, each platform including at least one processor core. System after Claim 13 wherein the management engine is further to perform benchmark tests on the platform logic to collect the telemetry data indicating the topology of at least part of the platform logic. System after Claim 1 wherein a hypervisor or operating system executed by the platform logic is to perform benchmark tests on the platform logic to collect the telemetry data indicating the topology of at least part of the platform logic. System after Claim 1 and further comprising a hypervisor comprising the resource allocation logic, wherein the hypervisor's resource allocation logic is to further select the processor core based on one or more performance requirements associated with the processing request. A method, comprising: receiving a processing request at an input / output device driver; by the input / output device driver, at least partially, selecting a processor core from a plurality of processor cores telemetry data associated with the processor cores, the telemetry data indicating a topology of at least a portion of the processor cores; and by the input / output device driver, sending an interrupt signal associated with the processing request to the selected processor core. System comprising: a plurality of platforms, wherein a platform comprises a plurality of processor cores; and a data center management platform coupled to the plurality of platforms through a network, the data center management platform serving to: Receive telemetry data from the plurality of platforms, the telemetry data indicating performance cores associated with the processor cores of each of the platforms; receive an indication of at least one workload to be executed by a platform from the plurality of platforms; select a platform from the plurality of platforms based on the telemetry data received from the platforms; and direct the placement of the workload to the selected platform. System after Claim 19 wherein the workload comprises a service function chain or virtual network function, and directing the workload placement to the selected platform includes migrating the service function chain or virtual network function from a first platform of the plurality of platforms to the selected platform. At least one machine-readable storage medium with instructions stored thereon, the instructions, when executed by a machine, to cause the machine to: receiving, at an input / output device driver, a processing request; selecting, by the input / output device driver, a processor core from a plurality of processor cores based at least in part on telemetry data associated with the processor cores, and send, by the input / output device driver, an interrupt signal associated with the processing request to the selected processor core. Medium after Claim 21 wherein the telemetry data comprises an indication of a topology of at least a portion of the processor cores. Medium after Claim 21 wherein the telemetry data includes an indication of bandwidth or latency from the processor core to another processor core of the plurality of processor cores. Medium after Claim 21 wherein the telemetry data comprises an indication of bandwidth or latency from the processor core to an I / O device. Medium after Claim 21 The instructions, when executed, are further intended to cause the machine to perform benchmark tests on the plurality of processor cores to collect the telemetry data. System comprising: Platform logic that includes a variety of processor cores; Means for receiving a processing request and Means for routing the processing request to a processor core of the plurality of processor cores, wherein the processor core is selected based at least in part on telemetry data associated with the platform logic, the telemetry data indicating a topology of at least a portion of the platform logic. System after Claim 26 wherein directing the processing request to the processor core includes sending an interrupt to the processor core. System after Claim 26 wherein the telemetry data comprises an indication of a topology of at least a portion of the platform logic. System after Claim 26 wherein the telemetry data includes an indication of bandwidth or latency from the processor core to another processor core of the plurality of processor cores. System after Claim 26 wherein the telemetry data comprises an indication of bandwidth or latency from the processor core to an I / O device. System after Claim 26 and further comprising means for performing benchmark tests on the platform logic to collect the telemetry data.

Images