DE102019120689A1 - TECHNIQUES FOR THE EFFICIENT OPERATION OF A PROCESSING SYSTEM BASED ON THE ENERGY CHARACTERISTICS OF INSTRUCTIONS AND MACHINE LEARNING - Google Patents

Abstract

An integrated circuit, such as a graphics processing unit (GPU), comprises a dynamic power controller for setting the operating voltage and / or frequency. The controller can receive the current power used by the integrated circuit and a predicted power based on instructions pending in a plurality of processors. The controller determines settings that must be made to the operating voltage and / or frequency to minimize the difference between the current power and the predicted power. A reinforcing in-system learning mechanism is included to adjust the parameters of the controller yourself.

Description

TERRITORY

This technology relates to the power management of an integrated circuit and, more particularly, to a dynamic voltage and / or frequency setting for power management in an integrated circuit. Even more particularly, this technology relates to managing performance based on sensing upcoming processor instructions and predicting / refining performance control system parameters with an empowering in-system learning mechanism. Use cases include the performance management of massive parallel processors of a GPU.

BACKGROUND

Graphics processors (GPUs) have become ubiquitous. No longer just for graphics, GPUs are now used for a wide range of applications that benefit from intensive computing operations that include artificial intelligence, real-time pattern recognition and autonomous vehicle control, and countless other applications.

Many GPUs are massively parallel - which means that they contain many computing elements (e.g. programmable streaming multi-processors (SMs)) that work in parallel. This massive parallel architecture allows developers to break down a complex calculation into parallel pieces, which because they are performed in parallel, are completed much faster. Although overly fast, such an arrangement of parallel computing elements can consume a lot of power and generate a lot of heat. Therefore, power management has become an important aspect of the design and operation of the GPU (and other) complex integrated circuits.

Just as a dancer uses more energy in fast dancing than in slow dancing, a digital circuit usually consumes more power when it works faster. In most integrated circuits, the operating speed is controlled by a clock circuit. The clock circuit sends out a "beat" (clock signal) to various circuits on the chip. The circuits synchronize their operation with the "beat" of the metronome-like clock signal. The faster the beat, the faster the circuits work and the more power they consume. However, the faster the beat, the more “movements” the circuits can perform in a given time.

The power supply voltage also has an effect on speed and power consumption. Transistors (e.g. MOSFETs) are used to build the digital logic circuits that are the building blocks of modern high-speed processors. The transistors operate (switch) faster when the power supply voltage is increased. One way to think about it is this: Switching a MOSFET transistor involves charging and discharging its capacitance. A rising DC voltage that supplies power to the MOSFET enables the capacitance of the MOSFET to be charged faster, thus increasing the switching speed of the transistor. This is similar to using higher water pressure to fill a bucket faster. The fastest speed at which the MOSFET can be clocked can thus depend on the power supply voltage.

However, operating circuits at higher voltages causes them to consume more power while increasing their potential operating speed. Generally speaking, the more power the circuit consumes, the more heat it generates. While fans or heat sinks are typically used to remove excessive heat, undue heating wastes power that could otherwise be used in the calculation. For mobile applications such as autonomous vehicles, portable devices, and the like, wasted power unnecessarily reduces battery life. Even for stationary applications such as servers and desktop computers, wasting power is bad for the environment and increases operating costs.

While the simplest solution is to slow down the circuit's operating speed to save power, this will adversely affect the number of operations per second the circuit can perform, and thus the speed at which it can perform complex calculations. In many demanding applications, including, but not limited to, autonomous vehicles and user interactive servers and other computing devices, high speed performance is more important for safety-critical and other calculations that need to be performed in real time or near real time.

There is therefore a compromise between operating speed and power consumption. In order to increase the number of operations per second, one can change the rate (frequency) of the clock signal (and if to support the higher clock rate, also the power supply voltage) at the expense of additional power consumption increase. To save power, one can reduce the clock signal rate (and, if desired, the power supply voltage) at the expense of slower processing.

To achieve this compromise, some GPUs and other processors provide dynamic control over the clock rate (and in some cases, the power supply voltage to support the higher clock rate) depending on the computational load that enables the circuits to "dance slowly" when processing requirements are lower and allows them to "dance fast" when more computing speed is required. Such dynamic control can reduce overall power consumption and heat dissipation while maximizing speed performance.

While much work has been done in the past, there is a need for further improved solutions that provide a customizable but tunable system that can handle a sudden load step and load release without affecting overall performance.

list of figures

• 1 und 2 veranschaulichen schematisch ein Beispielsystem einschließlich einer proaktiven Leistungsverwaltungssteuerung gemäß bestimmter Beispielausführungsformen.
• 3 veranschaulicht ein Ablaufdiagramm einer Leistungssteuertechnik unter Verwendung aktueller und prädiktiver GPU-Leistung gemäß bestimmter Beispielausführungsformen.
• 4 und 5 veranschaulichen schematisch eine Schaltung, die eine proaktive Leistungsverwaltungstechnik mit einem Lernmechanismus gemäß bestimmter Beispielausführungsformen implementiert.
• 6 ist ein schematisches Diagramm eines skalierten, bestärkenden Lernens basierten Proportional-Integral-Derivative-Systems.
• 7 ist ein schematisches Diagramm eines skalierten, bestärkenden Lernens basierten Proportional-Integral-Derivative-Systems.
• 8 ist ein schematisches Diagramm eines skalierten, bestärkenden Lernens basierten Proportional-Integral-Derivative-Systems, das zwei PID-Controller und zwei Lernsysteme umfasst.
• 9 veranschaulicht ein bestärkendes Lernen-basiertes Rückkopplungssteuerverfahren gemäß bestimmter Beispielausführungsformen.

The following detailed description of exemplary, non-limiting, illustrative embodiments is to be read in conjunction with the drawings, in which:

• 1 and 2 schematically illustrate an example system including proactive performance management control in accordance with certain example embodiments.
• 3 illustrates a flow diagram of a power control technique using current and predictive GPU performance according to certain example embodiments.
• 4 and 5 schematically illustrate a circuit that implements a proactive power management technique with a learning mechanism according to certain example embodiments.
• 6 Figure 11 is a schematic diagram of a scaled, reinforcing learning based proportional-integral-derivative system.
• 7 Figure 11 is a schematic diagram of a scaled, reinforcing learning based proportional-integral-derivative system.
• 8th FIG. 11 is a schematic diagram of a scaled, reinforcing learning based proportional integral derivative system that includes two PID controllers and two learning systems.
• 9 illustrates an encouraging learning-based feedback control method according to certain example embodiments.

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

Many GPUs engage dynamic voltage frequency scaling (DVFS) during runtime, which uses a load-based adjustment of the voltage and frequency of the GPU to get the best performance under a power envelope. In this context, load is defined as the energy intensity of the work performed on the GPU. Math-dominating workloads are a huge burden because the math operations that run on streaming multiprocessors (SMs) within the GPU can consume high energy in one clock cycle. In contrast, if the work is memory-dominant and does not use as many math operations in the SMs, the work is considered moderate or low load (in such cases, the SMs can consume much of their time to go to the storage system wait to retrieve data from memory).

By nature, load-based DVFS is reactive in nature. However, this load-based DVFS can lead to disadvantageous behavior, where an operating point at high voltage / high frequency could suddenly experience a load step on the GPU. This occurs in principle because the work started on the GPU can quickly change from a math-dominating regime to a memory-dominating regime and vice versa within short time scales where the DVFS cannot work. A load step is when the energy consumed in the GPU increases because high energy operations can occur at a given voltage and frequency. For the GPU, this is when it executes high performance math instructions. This condition can cause peak power to be exceeded on short time scales, which in turn can stress the electrical / power delivery system for the GPU and lead to voltage drops or reliability issues.

High energy instructions (such as matrix math instructions (MMA) or tensor core instructions that enable the acceleration of large matrix operations in the heart of artificial intelligence) significantly increase the energy consumed in a clock cycle and can have an impact on power delivery networks to avoid reliability or timing issues to cause. This can lead to GPU margins must entertain who eat their way into performance. To address these issues, we need an adaptive but tunable system to ensure that we can modulate energy consumption to mitigate these adverse effects without compromising overall performance. Basically, the current procedures for handling such effects do not achieve the optimal working point management, as described below.

1. 1) Präventiv - Deckeln der betriebliche Höchstfrequenz, um derartige Ausschläge zu verhindern oder zu begrenzen; und
2. 2) Reaktiv - Durch Einsetzen von Sensoren, welche die erhöhte Leistung/Strom abfühlen und dann mittels Verringerns der Betriebsspannung und Frequenz durch adaptive Spannung-Frequenz-Skalierung tätig werden.

Current practices to prevent performance spikes in short time scales include:

1. 1) Preventive - capping the maximum operational frequency to prevent or limit such rashes; and
2. 2) Reactive - By using sensors that sense the increased power / current and then operate by reducing the operating voltage and frequency through adaptive voltage-frequency scaling.

These practices can be effective, but have their reservations / limitations.

The first approach limits the maximum frequency at which the GPU can operate reliably and limits performance. This approach ensures, by design, that peak power is limited on short time scales, but the capped maximum operating frequency is likely to limit performance on longer time scales, which can be important for measuring the GPU's energy efficiency.

The second approach is a reactive approach and depends on the time to sense and activate that latencies can have to be effective, but the timescales of effectiveness typically do not extend below 100 psec.

The reason for this is because there is a cost to measure the change in the load in a reactive manner, since this change in the load manifests itself as a change in the power / current decrease associated with the sensing latency. When a system designer tries to go to ever lower timescales to manage peak performance, he runs up against some limits of external components (e.g. channel multiplexing) that can be the bottleneck in speeding up sensing.

An alternative approach would be to provide an on-the-spot notice of the change in workload (load step / release) so that this proactive approach can guide performance management to discontinue delivery to prevent reliability issues. However, detecting a sudden and large change in the workload is certainly difficult in the front end stages of the GPU pipeline, especially in computing workloads where the program semantics are not visible at the front end of the GPU pipeline. However, stages later in the GPU pipeline (e.g. in the streaming multiprocessor (SM) of NVIDIA GPUs) there is an ideal location for the acquisition since the semantics are visible because the instructions in the program can be decoded to reflect such changes in the load.

The load change management is not only limited to the load step (i.e. an increase in current / power), but there is also a need to manage sudden decreases in power (load release). A sudden load release can cause the power supply to overshoot due to the energy stored in the reactive components, causing an inductive kickback, which can also cause the power regulation system to trip.

Detecting a sudden and large change in workload is particularly difficult in the front end stages of the GPU pipeline, but the middle stages of the GPU (such as SMs in NVIDIA GPUs), however, have instructions in the programs that are likely to result in load step / release behavior can understand before executing the instructions.

Instruction predecoding to estimate future performance needs

Sample embodiments provide a proactive sampling technique that can be used to identify a potential or likely load step event (such as an SM) using instruction predecoding. Instruction predecoding by an instruction look-ahead predecoder that works on a queue or buffer of instructions fetched before they are executed enables the instruction to be marked / flagged before the instruction is sent for execution. Such instruction look-ahead predecoders can be used in addition to the typical instruction decoder to decode the instruction for execution. Instruction preview predecoding enables notification of an upcoming load step / enable event before the event actually occurs a few clock cycles later. This advance notification of the event to the power management unit enables Make changes to prevent peak performance scenarios.

A power management unit tracks the operating voltage and frequency and can use this notification to take action to move the voltage and / or frequency towards the operating regime to stop high performance events. Moving either voltage and / or frequency to reduce performance in this way can reduce electrical reliability problems in shorter time scales than other approaches, but without compromising performance.

From an analysis of the architecture, example embodiments can clearly identify the instructions that, when executed, can result in high power consumption in a clock cycle, which can result in higher performance. The instruction predecoding can classify these instructions, for example, if they pass through the front end of the SM pipeline and report these events to a GPU level power management unit (PMU). Once high energy instructions are flagged (e.g. in the SM), the information is consumed to trigger changes in hardware and / or software to manage the pace or release of peak performance.

At the GPU level, the PMU aggregates these notifications, for example, from all SMs in the GPU and can then calculate the potential for the size of the sudden increase or decrease in work based on the temporal and spatial properties of these event occurrences. This enables the PMU to make a proportional change depending on the full GPU workload, as there could be a variation in the workload running on each SM. The PMU makes changes to the appropriate time scale to implement only a clock frequency or a voltage / frequency change to limit the peak power in anticipation of the high-performance event, thereby avoiding overshoot and improving system-level operation.

In one non-limiting example embodiment, each SM includes a predecoder that outputs a digital code that is encoded to identify the power of instructions to be executed in the future. The digital codes of the various SMs are accumulated, filtered, and smoothed through digital processing to feed a control system. Since it would be restrictive to derive the parameters for the control system with a monitored offline training system, sample embodiments continuously refine the control system parameters with a learning mechanism in the system. Example embodiments enable improved performance within a performance envelope (Perf / W) with robust operation at the system level in short time scales; for example in server-based systems that desirably have the highest efficiency (performance with a performance envelope) with high reliability of system operation.

activation

In non-limiting example embodiments, sensing high energy instruction changes from the SM is used to activate changes in hardware or software to manage a step or release of peak performance. At the GPU level, the PMU aggregates these notifications from all SMs in the GPU and can then calculate the potential for increasing or decreasing the sudden work based on the temporal and spatial characteristics of these event occurrences.

One approach according to example embodiments is to use a “PID” (proportional-integral-derivative) proportional-integral-derivative (PID) controller that enables fixed or self-defined parameters for the controllers. The PID controller output can only change a frequency, only change a voltage, or perform a voltage / frequency change based on the time scale at which the power is to be managed.

In a non-limiting example embodiment, the high energy instruction notifications from the SMs are aggregated in an IIR (infinite impulse response) filter, the output of which provides an effective change in the computing power of the GPU. The computing energy together with the set voltage and frequency give a near future performance estimate. Example embodiments take the difference from this future performance estimate versus the current measured performance. This difference, also known as error or delta, is fed into a PID controller. Sample embodiments provide an adaptive PID controller where the PID control parameters are not static but are updated based on a reinforcement learning (RL) approach. The machine learning approach can have a different time scale than the operation of the PID, which, for example, enables the PID to work much faster than the adaptability of the learning approach.

Because workloads can vary widely and in short time scales, some use Non-limiting example embodiments, a reinforcement learning (RL) approach using an actor-critical or Q-machine learning (ML) to update the PID parameters. The empowering learning can observe the effect of certain operating parameters of the PID on the step and release of the peak performance and can adjust these operating parameters in order to optimize the performance management performance.

In non-limiting example embodiments, the PID controller moves the frequency and / or voltage to reduce (or at least minimize) the error to 0 and, in parallel, inputs (errors, measured performance and predicted performance) into a processor or controller of the PMU fed. The PMU processor / controller runs a self-tuning learning algorithm that can periodically update the controller parameters based, for example, on the properties of the error.

The process of operating hardware (HW) and tuning software (SW) enables a higher quality control system that predicts power coverage to implement only a frequency or a voltage / frequency change to increase the peak power in anticipation of the high performance event limit to improve system level operation and reliability when working at maximum efficiency (Perf @ W). As reinforcing machine learning gains more experience with performance management results on the particular GPU it is working on, it will tune performance more effectively to avoid over-performance (step and release) scenarios that the PID controller would otherwise have based on the expected computational load would order. In some embodiments, reinforcing machine learning is able to command SMs to change their instruction execution (e.g., by delaying the execution of high performance applications or otherwise the load balancing performance required for instruction execution between different time intervals) by system overruns and over-releases to avoid.

Consideration at a high level of performance management control

1 and 2 schematically illustrate an example system 100 that includes a proactive, predictive, for example, machine learning-based, performance management control in accordance with certain example embodiments.

System 100 can have one or more GPUs 102 (0), ... 102 (N) that communicate with one another via one or more PCI buses or other digital communication bus complex. GPUs 102 can with assigned memories 104 (0) 104 (N) connected. In some embodiments, the system 100 include one or more CPUs (not shown) or other control processors associated with the one or more GPUs 102 communicate over the digital communication bus complex, and the control processors and GPUs can access their respective associated memories and / or a merged memory.

The GPUs 102 can be any type of GPU, such as an architecture with a large number of parallel processors 116 , For example, any GPU 102 a processing complex of a large number of streaming multiprocessors (SMs) 116 (0), ... 116 (M), which are configured for highly parallel operation.

1 illustrates the system 100 that shows how the GPU 102 Performance through a power distribution network 110 is fed. A performance management unit ("PMU" or "performance management") 140 monitors the operating voltage and / or the current to the GPU through the power distribution network 110 supplied power and / or the operating frequency of the GPU 102 , The performance management unit 140 also receives signals 142 of one or more parallel processors 116 , indicate the performance required to complete instructions that have not yet been executed. For example, the processors 116 in connection with the service management 140 use instruction look-up predecode to determine what power the processors will need to execute pending or future instructions before the instructions are dispatched for execution by the processors (e.g., by a scheduler or similar structure). Based on the current operating voltage and / or frequency of the GPU 102 and the signals from the processors 116 predicts or estimates the performance management unit 140 the amount of power that processors will need in the near future and controls the voltage applied to the GPU and / or the operating frequency of the GPU clock signal so that the GPU 102 appropriate levels of power are supplied at the time when the instructions are from the processors 116 be carried out.

As an example, if there is a math dominant workload that consumes high energy in one clock cycle, it will be executed by the processors 116 to be planned, the performance management 140 signaling from the processors 116 received, indicating that they are about to perform a math-dominant workload. In response, service management predicts 140 that the processors 116 higher performance will take a few clock cycles in the future, that is, a few cycles later than the cycles currently being executed by the processors. Based on the predicted performance and current from the GPU 102 drawn performance can the performance management 140 change the operating frequency and / or voltage that the GPU 102 is provided so that the GPU operates at optimal performance (not too high and not too low) to execute the instructions quickly and efficiently. When a memory-dominant workload that consumes lower energy in one clock cycle is about to be executed by the processors 116 to be planned is the performance management 140 Receive signals indicating that instructions that require lower performance are executed a few clock cycles later. Based on the predicted performance and that of the GPU 102 drawn current performance can the performance management 140 reduce the operating frequency and / or voltage that the GPU 102 is provided so that the GPU can continue to run the memory-dominant workload efficiently without using more power than necessary.

In some exemplary embodiments, if the instruction predecode, performance management may 140 alarms that many high performance applications, such as tensor or other intensive computing instructions, are included from a number of different processors 116 to be run and performance management 140 predicts that such an implementation is likely to cause unacceptably high peak power consumption, e.g. due to overshoot, the power manager may reduce the clock frequency, the power supply voltage, or both the voltage and the frequency to limit the power in anticipation of the high power event. Limiting the power by lowering the frequency and / or voltage in anticipation of the high performance event can avoid overcurrent situations that can lead to circuit damage, power supply fluctuations, or excessive heating.

As in 1 shown, monitors performance management 140 a signal 114 the current and / or voltage sampling information that a signal conditioner and multiplexer (MUX) 112 is received, the one or more power rails of the power distribution network 110 scans. The signal 114 indicates the current performance of the GPU 102 provided. In some embodiments, the GPU 102 about a signal 115 select certain power rails for monitoring. Service management 140 also receives signals 142 of one or more parallel processors 116 , The GPU 102 controls the power it receives by controlling the power distribution network 110 about a signal 125 based on a difference between that by the signal 115 shown current performance and based on signals 142 certain predictive performance.

As explained in more detail below, performance management can 140 Self-learning and self-tuning include parameters that are used to predict and adjust the operating voltage and / or frequency settings. In some embodiments, self-learning may be partially through a warp from one of the parallel processors 116 performed using one or more deep neural networks created by a hardware-based deep learning accelerator (DLA) 141 be accelerated as part of the GPU 102 is included.

In some embodiments, performance management 140 on the same substrate or die as the parallel processors 116 be arranged. In some embodiments, one or more components of performance management 140 be provided on a substrate or die that is different from the substrate or die on which the parallel processors 116 are arranged.

According to example embodiments, the system can 100 be a board that includes one or more GPUs, one or more control processors, such as CPUs, and associated memory and / or memory management circuitry. In some example embodiments, the GPU 102 be a Systemon Chip (SoC) that includes a GPU. In still other embodiments, the system can 100 instead of the GPU 102 an integrated circuit with a different type of processor, the frequency-voltage control system and an ADC on the same or different substrate. Embodiments of this application are not limited to supplying power to the GPU and monitoring the power used by the GPU in the manner illustrated. Other embodiments known to those skilled in the art can be used to deliver power to the GPU and / or to monitor power used by the GPU.

More detailed schematic block diagram

2 illustrates the system 100 in more detail that the power distribution network 110 and sample components of performance management 140 show that on the GPU 102 to be provided. The GPU 102 includes a Machine learning based system (e.g. including one or more neural networks) 150 configured to monitor the power used by the GPU to predict the power used by the GPU 102 will be needed if it executes certain instructions to be executed soon, and the voltage and / or frequency power settings of a power control system 122 to control.

As explained above, the GPU 102 Power consumed varies based on processing workloads that are processed by the processors 116 sent (to be sent). For example, if all (or most) of a large number of parallel processors 116 (e.g. SMs) of the GPUs 102 are busy processing a stream of arithmetic instructions at high supply voltages and clock speeds, the GPU 102 a high amount of current from the board and / or through the voltage regulator 105 pull. In contrast, when a stream of memory-dominant instructions are processed, the GPU can 102 draw significantly lower amounts of power when working at the same clock rates, for example because many of their parallel processors 116 Much of the time may be idle due to memory access latencies. The processors can do the same thing as food servers waiting for the rush at a football field snack bar 116 don't have much to do other than wait while the GPU 102 to the store 170 accesses. In the example embodiments, the memory dominant instructions can be processed at lower power supply voltages and clock speeds, thereby reducing power consumption and heat generation. On the other hand, if the GPU 102 is ordered (e.g., by a CPU, not shown) to perform complex calculations on the data retrieved from the memory, the streaming multiprocessors 116 the GPU (similar to the soccer field hotdog stand workers when halftime begins and hungry fans only a few minutes before the start of the second half to get a hotdog or pretzel and return to their seats) as fast as they can, to perform their tasks.

In the disclosed embodiments, the GPU includes 102 dynamic voltage-frequency scaling (DVFS) to determine the voltage supplied to the GPU and / or the frequency of the GPU's internal core clocks based on that from the GPU 102 drawn current power and the predicted power that will be determined based on instructions in the near future from the parallel processors 116 are to be carried out. In rough analogy, sample embodiments keep an eye on the soccer field and the crowd and try to predict when the halftime rush will begin, how long the snack stands will be, and what customers are likely to order when they reach the head of the queue. This prediction enables snack stand workers to prepare a sufficient number of hot dogs and pretzels in time without having to prepare too many.

power supply

2 shows one or more power supply units (PSUs) in more detail 108 , performance different parts of the system 100 feeds directly and also via power rails or connections from a PCI bus 106 to be provided. For example, GPU (s) 102 be supplied primarily or exclusively by power that a voltage regulator 105 from the PSU 108 directly and / or via the power bus of the PCI bus 106 receives. The voltage regulator 105 levels the power supply voltage levels from the PCI bus 106 and / or the PSU 108 down to (programmable) voltage level (s) required to power the GPUs 102 to operate, and supplies the downgraded voltage (s) to the GPUs 102 via GPU power rails 127 , In some embodiments, the PSU 108 alternatively or in addition to the GPUs 102 Performance directly over performance tracks 111 respectively.

In some embodiments, the voltage regulator (VR) performs 105 a DC-to-DC conversion from a high voltage to a lower voltage. This conversion is used in some example embodiments because of processing chips such as the GPU 102 , are designed to operate at lower voltages (e.g. 1VDC-2VDC) than the standard 12VDC or 5VDC power supply voltages required for the PCI bus or PCIe bus 106 and / or the power supply unit 108 are typical that distribute power on the onboard power distribution network. The voltage regulator 105 is also designed to regulate the DC output voltage that it supplies to the GPU. The exit 127 of the voltage regulator supplies current to the GPU power rails 127 for distribution to the different parts of the GPU 102 and their associated supporting components and circuits.

As described above, there are multiple input power rails 127 the GPU 102 Performance too. Thus, although not specifically illustrated, the voltage regulator 105 include a separate instance for each rail. That is, in some embodiments, the voltage regulator 105 can include multiple voltage regulators, each of which can be controlled separately and / or independently, around a respective reference voltage etc. for a respective he multiple power rails on the GPU 102 adjust.

performance monitor

The GPU 102 can use an analog-to-digital converter (ADC) 118 include that it uses to produce a "power monitor in" analog signal 114 to be monitored (e.g. digitally sampled). The analog signal 114 can represent, for example, the current current and / or voltage levels that are present on the onboard input power rails 111 and GPU power rails 127 to be provided.

The GPU 102 uses an interface 115 to an analog multiplexer (MUX) 112 control to select a particular one of the multiple power rails to be scanned for a particular time division multiplex (TDM) time slot. Thus, in some example embodiments, the GPU 102 the MUX 112 control to select a "round robin" sequence of different analog signal levels that allow the GPU to periodically monitor different power supply signal levels that include power levels that are currently the GPU 102 be fed yourself. In other embodiments, the GPU controls 102 the MUX 112 to select inputs to monitor on an as-needed basis.

The signal conditioning and MUX circuitry 112 thus selects between the several input power rails 127 from which the GPU 102 supply and multiplexes the selected rail on the ADC 118 via the MUX-ADC interface 114 to signal with their selection to supply the GPU at a certain time, which is executed by the GPU itself in non-restrictive example embodiments. The circuit arrangement 112 can also perform signal processing on the signal supplied to the GPU. For example, the circuit arrangement 112 use a voltage divider or other circuitry to scale voltage levels and / or convert current levels to voltage levels representative of the current levels. The "power monitor in" signal 114 in some embodiments ( 1 ) an analog voltage whose level represents or indicates the amount of current that the GPU 102 draws; and (2) in response to a different selection by the MUX 114 , include an analog voltage (which can be scaled down) that represents the voltage level applied to the GPU. The GPU can use these two values 102 determine the amount of power (in watts) that she is currently consuming.

The external signal processing and the MUX circuit arrangement 112 made possible by means of multiplexing between the several input power rails 127 Monitor multiple input levels using a single “performance monitor in” analog input to the GPU 102 and the ADC 118 thus reducing the GPU's I / O pin count 102 , The MUX-ADC interface 114 can refer to the special features of MUX 112 and ADC 118 be highly sensitive. the interface 114 is configured with the MUX and ADC, which match each other in terms of impedance, signal range, etc. However, other arrangements are possible. For example, the GPU 102 with multiple ADCs 118 be equipped to enable it to process multiple performance monitoring inputs simultaneously.

In an example embodiment, the analog-to-digital converter (ADC) 118 configured to monitor the power supply voltage and current levels that the GPU 102 are currently provided. As in 2 shown, receives the analog off-chip multiplexer (MUX) in one embodiment 112 as inputs levels of power supply voltages and currents that are used to power the GPU 102 To perform. Signal conditioning such as voltage divider / scaler circuits and the like in the MUX 112 , provides signals representative of voltage levels and current levels that the PSU 108 either directly or via the voltage regulator 105 supplies. The GPU 102 controls the MUX 112 about the signal 115 to select certain level signals to feed the GPU a “monitor in” input. The U.S. Patent Application 16 / 046,916 (filed July 31, 2018, entitled "Voltage / Frequency Scaling for Overcurrent Protection with On-Chip ADC") discloses additional details on using an on-chip ADC to monitor GPU performance and the power delivered to the GPU to control.

In another embodiment, the ADC 118 Monitor "power in" by the voltage regulator 105 is provided to determine current performance by the GPU 102 is used.

Processor pre-decoding and aggregation

The processors 116 are configured to generate signals that, based on the type of instruction, indicate a level of performance required by instructions to be executed soon. The processors 116 generate such signals for instructions to be executed soon before executing these instructions. Each of the processors 116 may decode (e.g., the op code of) a respective pending instruction for a given cycle (e.g., partially) to determine whether that instruction is a low power instruction or is a high performance instruction. In one embodiment, any processor 116 generate a binary signal or flag to indicate whether a high or low level of performance will be used by the respective processor (e.g., setting a flag when a processor unit will execute instructions that require a high level of performance). In other embodiments, the signal may indicate expected or future power consumption during a given cycle through a range (e.g., a value between 0 and n, where n is an integer). The processors 116 can continuously parse instructions received for near future execution, and provide performance estimates for executing pending instructions a few cycles before they are executed.

In an example embodiment, a filter / scaler receives 420 an aggregate of the performance estimation signals from multiple processors 116 , filters the aggregate to remove certain types of noise and scales the result. The filter / scaler 420 combines or compares the scaled result with the performance level currently used by the GPU, such as by the ADC 118 sensed / detected. The result of the combination or comparison forms an error or delta signal, which is used to change the control signals sent to the frequency-voltage controller 122 Taxes.

Service management 140 thus receives signals from the ADC 118 which are those of the GPU 102 indicates current power used, and signals indicating estimated future power needs from the processor 116 specify and determines whether to change the operating voltage and / or frequency. More specifically, performance management 140 include circuitry, such as the filter / scaler 420 to the signals from the processors 116 to accumulate, a performance requirement for the GPU 102 for a given future cycle, estimate an error between that for the GPU 102 estimated performance and that of the GPU 102 Determine current power used and output control signals to change the operating frequency and / or voltage in time to meet the changed power requirement. Service management 140 can keep the current settings if, for example, the error is below a certain threshold, indicating that the power requirement does not change much.

Frequency and / or voltage controller and learning system As will be explained in more detail below, the frequency / voltage control system 122 a proportional integral derivative (PID) controller configured to receive a signal indicating the error and signals to move the frequency and / or voltage up or down to reduce the error to zero , A learning system 150 , which may be implemented in software and / or hardware, may be configured to operate the frequency / voltage control system 122 monitor and update parameters (e.g., constants or coefficients used by the PID controller) that are used to determine the output signals to determine the performance of the frequency / voltage control system 122 to improve.

For example, the learning system 150 the control parameters of the frequency / voltage control system 122 modify so that the error is reduced faster and / or reduce overshoots when the frequency and / or voltage is moved. The learning system 150 can get the error from the GPU 102 current power used and / or that used by the processors 116 receive predicted power and execute a self-tuning learning algorithm derived from the frequency / voltage control system 122 modified parameters used. The learning system 150 can operate the frequency / voltage control system 122 Monitor over a predetermined number of cycles and periodically update parameters by the frequency voltage controller 122 to be used.

The learning system 150 can be on a different (e.g. slower) time scale compared to the frequency-voltage controller 122 work and in some cases may include software running on one or more of the processors 116 runs like through an accelerator for deep learning 141 accelerated. In such embodiments, the learning system 150 partially used to predict the performance that a processor 116 will need to run the learning system itself.

The output control signals of the frequency / voltage control system 122 can have at least one voltage regulator 105 control (e.g. via GPIO pins on the GPU), the performance of the PCI bus or similar bus 106 and / or the power supply unit (PSU) 108 receives and performance of the GPU 102 over power rails 127 supplies. The signal conditioner and multiplexer 112 receives GPU power rails 127 and also onboard input power rails 111 , The output control signals of the frequency / voltage control system 122 can also generate internal or external control signals that control the clock rate (s) of clock signals that a clock generator 107 generated to trigger different GPU circuits synchronously ("clocking").

As an example, if the activity levels in the processors 116 are low, the frequency / Voltage control system 122 work to decrease one or more supply voltages to reduce the amount of power supplied to the GPU. If the activity levels are high, the frequency / voltage control system can 122 Increase one or more supply voltages to provide more power to the GPU to meet the requirements of the increased processing workload.

Manage performance

3 FIG. 14 is a flow diagram of an example performance control technique using current and predictive GPU performance in accordance with certain example embodiments. In some sample embodiments, the process 300 (ie, operations 302 - 312 ) in hardware circuitry components on the GPU 102 carried out. In some other sample embodiments, in addition to on the hardware of the GPU 102 performed operations, the process 300 Operations include that of other hardware components external to the GPU 102 be performed. In yet other example embodiments, the process can 300 , in addition to operations performed by hardware, include operations performed in software running on the GPU 102 and / or other parts of the system 100 running. In general, however, in order to provide adequate speed performance, the 3 operations shown in hardware with the exception of deep learning 312 that can be done in software with hardware acceleration.

The process 300 may be continuous during the operation of a computer system in which a system such as the system 100 and / or a GPU, such as the GPU 102 , is used.

The operation 302 monitors the current performance used by the GPU. In embodiments, monitoring input power uses an ADC 118 that of the GPU 102 input power samples. Monitoring can be done by the ADC 118 comprise, the input current and / or voltage samples into the voltage regulator and converts the sampling results into a digital output of any desired resolution.

During surgery 304 can each of the parallel processors 116 received instructions process to predictive power signals 142 for pending instructions (to be issued soon). The pending instructions from the various parallel processors 116 can be aggregated and used to predict GPU performance for an upcoming processing cycle while the pending instructions are executing (see block 304 ).

During surgery 306 the difference (error) between the predicted GPU performance and the current performance is determined. Before determining the error, the predicted performance can be scaled so that it is on the same scale as the determined current performance.

During surgery 308 determines the GPU 102 Frequency and / or voltage settings to reduce the error. The determination can be made using an adaptive proportional-integral-derivative (three-term) controller (“PID controller”) 440 (see below) to determine the settings.

During surgery 310 can the GPU 102 based on that during surgery 308 determination performed, change the clock frequency (ie, program a new one) and / or change a power voltage to increase or decrease the power drawn by the GPU. If the GPU 102 For example, if a clock frequency setting alone can effect the required change in power, then it can set the frequency (or rate) for one or more clocks. A voltage can be adjusted by offsetting the voltage sent to the voltage regulator to either bring the current power back below the limit or to increase it.

In some embodiments, parameters are the operation 306 updated using an adaptive PID controller (which may be implemented in hardware as is well known to those skilled in the art) based on a machine learning approach (block 312 ). Because the workloads in the parallel processors 116 Diverse and may vary on short time scales, a reinforcement learning approach using actor-critical learning can be used to measure the PID parameters on a different time scale than the process of 3 to update. For example, the process of 3 from block 302 - 310 quickly between individual processing cycles of processors 116 based on hardware-maintained voltage and / or frequency parameters, whereas the deep learning block 312 can be performed over a number of such individual processing cycles and generates outputs that are used to update the hardware-maintained voltage and / or frequency parameters more occasionally.

That way the process can 300 the power input into the GPU 102 according to the workloads in the parallel processors proactively control to optimize processing performance and power consumption. Learning 312 is added to provide a second level of dynamic control to update the parameters used to adjust the performance setting for the GPU 102 to determine.

Empowering machine learning

4 and 5 schematically illustrate a circuit 400 using a proactive performance management technique with a learning mechanism 150 implemented according to certain example embodiments. The circuit 400 includes an ADC 118 , Filter and scaling circuits 420 , a comparator or totalizer 422 , a state vector register 430 , a PID controller 440 , a frequency / voltage controller 450 and a learning system 460 ,

The circuit 400 can be made on the same substrate as the processing system (e.g. GPU 102 ) includes. Including the circuit 400 on the substrate of GPU 102 provides direct, fast feedback and responsiveness. Unlike reactive systems that respond to changes in current performance measurements, systems of example embodiments allow adjustments to be made proactively by the processing system because the performance control system experiences a slowdown or increase in processing between times when instructions for execution are presented and of the time when they are actually executed can predict.

The ADC 118 can be configured to monitor the power supply voltage and current levels generated by a processing system (e.g., a GPU 102 ) that use multiple processors 116 (eg programmable streaming multiprocessors). The ADC 118 is configured to receive input signals representing voltages and / or currents supplied to the processing system and to produce a digital output signal representing the power currently used by the processing system.

The filter and scaler circuits 420 can use prediction signals from each of the processors 116 received, indicating how much performance instructions to perform in the near future will or will likely require. For example, the processors 116 report whether pending instructions will result in high energy consumption or low energy consumption in one clock cycle. The signal from each processor 116 may be a high performance / low performance binary flag signal, or it may provide a range of different values that, at any desired level of resolution, indicate how much power may be required in the near future to execute instructions in the instruction pipeline are, but have not yet been executed, but will soon be executed (or may, if speculative execution is allowed).

How 5 shows the processors 116 one predecoder each 117 configured to signal the likely power of instructions to be executed in the future. The filter circuit 420a can be configured to provide an aggregate power signal from all processors 116 to recieve. Such an aggregated signal can be generated by an aggregator 422 are generated that, in one example embodiment, include combinatorial logic that tunes an aggregated signal or otherwise for presentation to the filter 420a calculated quickly. The signals represent how much power pending instructions a predetermined number of clock cycles later through the bank of processors 116 are likely to require execution when executed. In a non-limiting example embodiment, these signals do not predict how much power will actually be required, but only that the instructions are likely to consume more or less power based on their complexity or other instructional properties. For example, in one example embodiment, a processor 116 , which parses a memory load instruction, signal that the upcoming instruction is likely to require low (re) performance. The same processor 116 Parsing an arithmetic instruction may signal that the upcoming instruction is likely to require high (higher) performance.

In many embodiments, the processor 116 do not "know" how much power they will actually consume by executing a given instruction. This is like a long-distance runner who can have a rough idea of how much energy in calories he is likely to consume when running a half marathon, but will generally not be able to predict energy consumption down to calories. There are too many variables (e.g. environmental conditions such as temperature, actual speed as opposed to a planned speed, exact body mass etc.) that influence the actual energy consumption. Furthermore, the exact performance requirements of any given processor 116 characteristics of that particular processor due to Manufacturing and process variations that can lead to tiny structural differences from GPU to GPU or even from one processor to another on the same GPU. The exact power usage will generally also depend on the particular sequence of software instructions that the processors 116 To run.

In the example shown, the filter and scaler circuits include 420 one or more filters 420a (e.g. an IIR (infinite impulse response) filter) to reduce the noise in the aggregated signal from the processors 116 Will be received. The impulse response from such a filter is "infinite" because there is feedback in the filter; if a pulse (a single "1" sample followed by many "0" samples) is entered, theoretically an infinite number of non-zero values will result. However, other implementations could use an FIR filter, a combination of an FIR filter and an IIR filter, or any other suitable filter design.

The output of the filter 420a is a near-future performance estimate. This near future performance estimate becomes a scaler 420b provided that scales the output so that it matches the output of the ADC 118 can be combined (the scaler can be controlled in part by the size of the ADC output).

The circuit 400 can also be a circuit arrangement (for example a summing node 422 ) include a difference or error between the measurement of the ADC 118 to determine the current performance and the performance of the filter / scaler 420 estimates to be used in a predetermined number of cycles.

The state vector register 430 receives / holds the error value to provide a value X (t) by the PID controller 440 can be used to determine frequency and / or voltage settings. In some exemplary embodiments, X (t) takes into account error values for multiple clock cycles (eg errors for the current cycle, one or more past cycles, and / or one or more future cycles). The output X (t) of the state vector circuit 430 can include several components. In some example embodiments, the output X (t) can provide the error as a function (e.g., a multiplier function, scaling function, or unit function). In some embodiments, the state vector circuit 430 take an error that is provided in continuous form and a discrete value for the PID controller 440 provide. In an example embodiment, the output X (t) of the state vector circuit 430 can be represented by e (t), Δe (t), Δ2e (t) ... The entire vector X (t) can be the learning system 460 and a subset of the state vectors are provided, such as the component e (t) to the PID controller 440 can be provided.

The PID controller 440 receives a representation of the error between the current power and the predicted power and calculates an output W (t) depending on specific parameters by the PID controller 440 are stored in a "tuning constant" register 440a. As is well known, the PID controller combines 440 proportional control with additional integral and derivative settings that help the controller to automatically compensate for changes in the system. As is well known, the PID controller enforces 440 a feedback to match a setpoint. Proportional tuning involves correcting a target proportional to the difference; it approaches zero errors but never reaches them. The integral tuning accumulates the error result from the "P" action in order to increase the correction factor. As this sometimes leads to an overshoot, derivative tuning is used to minimize this overshoot by slowing the applied correction factor as you approach the goal. In the example embodiment, the PID levels of the PID controller 440 parameterized with increment values or tuning constants for the proportional, integral and derivative terms. These parameters are in the read / write register 440a saved in the sample embodiment and can be from the learning system 150 can be changed dynamically as described below.

The output W (t) of the PID controller 440 specifies changes to the operating voltage and / or frequency so that the error between the current power and the predicted power is reduced. The output of the PID controller 440 will stop setting the frequency and voltage when the error goes to zero. The output W (t) of the PID controller 440 can choose between (a) a frequency only setting, (b) a voltage only setting, and (c) a voltage and frequency change based on the time scale at which power should be managed.

The output W (t) of the PID controller 440 becomes the frequency / voltage controller 122 for setting the operating frequency and / or voltage of the processing system (eg GPU 102 ) provided. Operations of the frequency / voltage controller 450 can be done in hardware and / or software.

Based on the output W (t) the frequency / voltage controller provides 122 the operating frequency and / or voltage to indicate an increase or decrease in that of the GPU 102 to allow drawn power. For example, if it is determined that frequency adjustment alone can affect the required change in performance to reduce the error, then the frequency / voltage controller can 450 trigger a frequency change to the frequency (or rate) for one or more clocks 127 adjust. The PID controller 440 can determine whether only the operating frequency or only the voltage or both the voltage and the frequency is to be changed. In some embodiments, the level of error can determine which operating settings should be changed. For example, if the error is above a predetermined threshold, the PID controller 440 Call for changes in the operating voltage. The operating voltage and frequency can be changed to limit the peak power in anticipation of a high performance event, thereby improving system level operation and reliability when operating at maximum efficiency.

In some embodiments, the frequency / voltage controller 122 adjust the voltage by offsetting the voltage sent to the voltage regulator to either bring the current power back below the limit or increase it.

As mentioned above, the PID controller enables 440 fixed or self-tuned parameters for the controller. Because workloads can be diverse and vary in short time scales, empowering learning can be done using Actor-Critic or Q-Learning, for example 150 used to be in the register 440a saved parameters of the PID controller 440 to update. The learning system 150 can also take into account the variability of the processing system (e.g. silicon variability) that would otherwise not be taken into account by presetting the parameters of the PID controller 440 and by preventing the PID controller 440 expected performance requirements overcompensated.

The learning system 150 can continuously get the error X (t) from the state vector 430 and the current measured power from the ADC 118 receive. The learning system 150 may also have predicted performance from the processors in some embodiments 116 and / or aggregator 422 receive. In the example embodiment, the learning system is correct 150 the parameters 440a of the PID controller 440 from. The learning system 150 outputs K (t), which forms a PID control vector that is updated to give a dynamic property based on correction effectiveness.

The learning system 460 includes one or more deep neural networks that periodically update the controller parameters based on the effectiveness of correcting the error. If the PID 440 for example at the time of manufacture with tuning constants 440a which have been empirically based on simulations are not necessarily initialized with the real hardware properties of a particular GPU 102 to match. The learning system 150 can, however, implement reinforcing learning to the actual correlation between the value X (t) of the state vector 430 that predicts the current decrease for a particular set of processor instructions and the actual current / power decrease of the GPU 102 to learn when the GPU executes these instructions. The learning system 150 can use statistical and probability analysis to provide a more optimal mapping for the PID controller 440 to act between X (t) and W (t) for a variety of different X (t) s, despite the potential complexity of the output signal W (t) and without being programmed with any preconceived model of the system or mapping his.

In example embodiments, machine learning 150 through a hardware accelerator for deep learning ("DLA") 141 can be accelerated and can carry out a deep reinforcing learning algorithm, such as actor-critical reinforcing learning or Q-learning. See e.g. Konda et al., "Actor-Critic like learning algorithms for Markov decision processes", SIAM Journal on Control and Optimization, 38 (1): 94-123 (1999) ; Sutton et al., Reinforcement Learning: An Introduction (MIT Press Cambridge, MA, 1995) ; and Grondman et al, "A Survey of Actor-Critic Reinforcement Learning: Standard and Natural Guideline Gradients" IEEE TRANSACTIOS ON SYSTEMS, MAN AND CYBERNETICS-PART C: APPLICATIONS AND REVIEWS (2012).

• Bestärkendes Lernen ist ein Framework, bei dem ein Agent (oder Controller) sein Verhalten durch Wechselwirken mit seiner Umgebung optimiert. Nach Ergreifen einer Aktion in einem Zustand, empfängt der Agent eine skalare Belohnung von der Umgebung, die dem Agent eine Angabe der Qualität dieser Aktion gibt. Die Funktion, welche die Aktion angibt, die in einen bestimmten Zustand vorzunehmen ist, wird die Richtlinie genannt. Das Hauptziel des Agents ist es, eine Richtlinie zu finden, welche die totale akkumulierte Belohnung, auch die Rendite genannt, maximiert. Durch Folgen einer gegebenen Richtlinie und Verarbeitung der Belohnungen, kann der Agent Schätzungen der Rendite erstellen. Die Funktion, die diese geschätzte Rendite darstellt, ist als die Wertfunktion bekannt. Die Verwendung dieser Wertfunktion ermöglicht dem Agent, indirekten Gebrauch von vergangenen Erfahrungen zu machen, um über zukünftige Aktionen Entscheidungen zu treffen, um einen bestimmten Zustand aufzunehmen oder nicht Actor-Critic-Verfahren kombinieren die Vorteile von Nur-Actor- und Nur-Critic-Verfahren. Während der parametrisierte Actor den Vorteil des Berechnens kontinuierlicher Aktionen ohne den Bedarf an Optimierungsprozeduren auf einer Wertfunktion bringt, ist der Verdienst des Critic, dass er den Actor mit Kenntnis niedriger Varianz der Performance beliefert. Genauer gesagt, ermöglicht die Schätzung des Critic der erwarteten Rendite dem Actor, mit Gradienten zu aktualisieren, die eine niedrigere Varianz aufweisen, was den Lernprozess beschleunigt. Die niedrigere Varianz wird für eine größere Ausrichtung am Start des Lernens eingetauscht, wenn die Schätzungen des Critic bei weitem nicht genau genug sind. Actor-Critic-Verfahren weisen gewöhnlicherweise im Gegensatz zu Nur-Critic-Verfahren gute Konvergenzeigenschaften auf.

As described by Grondmon et al. summarized:

• Empowering learning is a framework in which an agent (or controller) optimizes its behavior by interacting with its environment. After taking an action in a state, the agent receives a scalar reward from the environment that gives the agent an indication of the quality of that action. The function that specifies the action to be taken in a particular state is called the policy. The main goal of the agent is to find a policy that maximizes the total accumulated reward, also called the return. By following one Given the policy and processing of the rewards, the agent can make estimates of the return. The function that represents this estimated return is known as the value function. The use of this value function enables the agent to make indirect use of past experiences to make decisions about future actions, to record a certain state or not Actor-Critic methods combine the advantages of Actor-only and Critic-only methods , While the parameterized actor has the advantage of calculating continuous actions without the need for optimization procedures on a value function, the Critic's merit is that it provides the actor with knowledge of low variance in performance. More specifically, the Critic's estimate of the expected return enables the actor to update with gradients that have lower variance, which speeds up the learning process. The lower variance is exchanged for greater focus at the start of learning when the Critic's estimates are far from accurate. Actor-critical processes usually have good convergence properties in contrast to critical-only processes.

Deep neural networks or deep neural networks (DNNs) based on processors such as the PPU 200 were developed for various applications, from self-driving cars to faster drug development, from automatic image headings in online image databases to intelligent real-time voice translation in video chat applications. Deep learning is a technique that models the neural learning process of the human brain, learns continuously, becomes increasingly intelligent, and delivers faster and more accurate results over time. A child is first taught by an adult to correctly identify and classify different forms in order to finally be able to identify forms without coaching. Likewise, a deep learning or neuronal learning system must be trained in object recognition and classification so that it can identify basic objects, hidden objects, etc. more intelligently and efficiently and at the same time assign context to the objects. In general, empowering learning is somewhere between supervised and unsupervised learning. Whereas with supervised learning you have a target label for each training example and with unsupervised learning you have no labels at all, with reinforcing learning you have to have sparse and time-delayed labels - the rewards. Based on these rewards only, the agent must learn to behave in the area. See https://neuro.cs.ut.ee/demystifying-tiefereinforcement-Learn/

At the simplest level, neurons in the human brain look at various inputs that are received, each of these inputs is assigned levels of meaning, and an output is passed to other neurons to respond to. An artificial neuron or perceptron is the most basic model of a neural network. In one example, a perceptron can receive one or more inputs that represent various features of an object that the perceptron is trained to recognize and classify, and each of these features is assigned a specific weight based on the importance of that feature for defining the shape based on an object.

A DNN (Deep Neural Network) model contains several layers of many connected perceptrons (e.g. nodes) that can be trained with enormous amounts of input data in order to solve complex problems quickly and with high accuracy. In one example, a first layer of the DNN model breaks down a supplied image of a car into different sections and searches for basic patterns such as lines and angles. The second layer assembles the lines to look for higher level patterns such as wheels, windshields and mirrors. The next layer identifies the type of vehicle and the last few layers create a label for the input image, which identifies the model of a particular automobile brand.

Once the DNN is trained, the DNN can be used and used to identify and classify objects or patterns in a process called inference. Examples of inferences (the process by which a DNN extracts useful information from a given receipt) include identifying handwritten numbers on checks deposited in ATMs, identifying pictures of friends in photos, providing movie recommendations to over fifty Millions of users, identifying and classifying different types of cars, pedestrians and road hazards in driverless cars, or translating human speech in real time.

During training, the data flows through the DNN in a forward propagation phase until a prediction is generated that indicates a label corresponding to the input. If the neural network does not correctly label the input, then errors will occur between the correct label and the predicted one Label analyzed and weights adjusted for each trait during a backward propagation phase until the DNN correctly identified the inputs and other inputs in a training record. Training complex neural networks requires enormous amounts of parallel computing performance, which includes floating point multiplications and additions, by the PPU 200 get supported. Inferencing is less computationally intensive than training because it is a latency-sensitive process in which a trained neural network is applied to new inputs that it has never seen before to classify images, translate speech, and generally new information derive.

Neural networks rely heavily on matrix math operations, and complex multi-layer networks require enormous amounts of floating point performance and bandwidth for both efficiency and speed. With thousands of cores optimized for matrix math operations and ten to hundreds of TFLOPS in performance, the PPU is 200 a computing platform capable of delivering the performance required for artificial intelligence and machine learning applications based on deep neural networks.

und so weiter.Example of a non-limiting reinforcing learning-based proportional-integral-derivative controller 6 shows an example of a more detailed architecture for a reinforcing learning-based PID controller 440 , In this example implementation, the input signal y_ref (t) represents the reference power that we want to manage, and y (t) is an output signal that represents the sum of the current sensor power + the estimated future increase in power observed due to the high energy instructions becomes. An adder 422 ' calculates the difference between y_ref (t) and y (t) by an error signal e (t) for application to the state vector 430 to create. The state vector x (t) is thus a vector derived from e (t), where $$\begin{array}{l}:\text{x}\left(\text{t}\right)=\left[\text{x}1\left(\text{t}\right).\text{x}2\left(\text{t}\right).\text{x}3\left(\text{t}\right)...\text{xn}\left(\text{t}\right)\right]=[\text{e}\left(\text{t}\right).\text{delta}\left(\text{e}\left(\text{t}\right)\right).\\ \text{delta}2\left(\text{e}\left(\text{t}\right)\right)....]:\text{e}\left(\text{t}\right)=\text{y}\_\text{ref}\left(\text{t}\right)-\text{y}\left(\text{t}\right).\text{delta}\left(\text{e}\left(\text{t}\right)\right)=\\ \text{e}\left(\text{t}\right)-\text{e}\left(\text{t}-1\right).\text{delta}2\left(\text{e}\left(\text{t}\right)\right)=\text{e}\left(\text{t}\right)-2*\text{e}\left(\text{t}-1\right)+\text{e}\left(\text{t}-2\right)\end{array}$$

and so on.

The PID controller 440 will use an element of this state vector, typically e (t), for example, that goes to the block of incremental PID controllers 440 goes. The output of the PID controller 440 is Δu (t), which is at the input of the process 472 is applied to the frequency voltage controller 122 and associated power adjustment mechanism. A Z-1 (memory) operator 470 provides hysteresis so that the output Δu (t) of the PID controller 440 is used to adjust the current u (t) signal that is applied to the process 472 to control. The Z-1 operator 470 stores u (t) and supplies it as u (t-1) for input into an adder 474 , which adds Δu (t) to u (t-1) to produce u (t).

The learning block 150 includes an actor 476 , a critical 480 and a self-adapting module ("SAM") 478 , In general, the system 150 a neural network-based model that uses the updated coefficients K (t) for the PID controller 440 will generate. An entry in the Critic 480 is r (t), also known as a reward as an entity in reinforcing learning systems. In this particular case, r (t) is a scalar value derived from e (t) and e (t-1). The objective of learning is to achieve maximum maximum r (t).

In this particular arrangement, the actor leads 476 its output K '(t) the self-adapting module ("SAM") 478 to, which generates K (t) and this signal as a coefficient input to the PID controller 440 supplies. The coefficients form the vector of PID parameters in this example.

Meanwhile, the Critic creates 480 an output V (t) based on inputs x (t) and r (t) and applies the output V (t) to the SAM 478 which adapts to the edition of the Critic and transforms K '(t) into K (t).

Two control loop architecture

The PID controller 440 and the learning system 150 thus provide two control loops, the first of which is the PID controller 440 and the second comprises the learning system running on top of (and on a different time scale) the first loop. The two control loops provide for self-learning and self-setting of parameters by the PID controller 440 used to control the power control parameters for the GPU 102 based on future power usage estimates made by the processors 116 are provided as a result of the processors parsing their instruction formats to be executed soon.

In non-limiting example embodiments, the loop of the PID controller runs 440 continuously and regardless of the loop of the learning system 150 , but can be updated as needed, as determined by the learning system loop. The learning system loop monitors the operation of the loop of the PID controller 440 to see when a difference (e.g. a reduction in error) due to the currently set Weighting coefficient of the PID controller 440 is made. If no difference is made or the reduction in error can be improved, the learning system loop can 150 modify one or more coefficients by the PID controller 440 be used. The loop of the learning system 150 can update the coefficients of the PID controller during a period of time when the PID controller 440 the frequency and / or voltage settings are not updated.

As mentioned above, in some exemplary embodiments, the learning system 150 receive additional information from other components of the processing system. For example, the learning system 460 Information from the processors 116 received via the pending instructions scheduled to run. The learning system 460 can use this additional information to help him find the coefficients of the PID controller 440 to determine and update.

Example of a Non-Restrictive, Scaled, Learning-Based Proportional-Integral-Derivative-System to control the system based on separate metrics. In the context of predictive performance management, the scaled, reinforcing learning-based PID system can be used to manage performance (separate metric) and performance (separate metric), so that the highest potential operating point is reached. 7 Figure 11 is a schematic diagram of a scaled, reinforcing learning-based proportional-integral-derivative system. In 7 become one or more PID controllers 440 used a process 472 to control and one or more learning systems 150 update coefficients for the one or more PID controllers 440 ,

As in 7 shown the process 472 based on total PID controller outputs 440 to be controlled. Each PID controller can manage a separate metric that is one or more common parameters of the process 472 manage.

In the in 7 The example shown is each of the PID controllers 440 is with a respective learning system 150 coupled, which determines the coefficients for the coupled PID controller. The learning systems 155 determine the coefficients for the PID controller 440 based on the one or more editions of the process 472 , In other examples, the learning systems determine 150 the coefficients based on one or more error signals that are in the PID controller 440 can be entered. In some examples, a single learning system (e.g. RL agent 1 ) Coefficients for two or more PID controllers (e.g. PID controller 1 and PID controller 2 ) determine.

8th shows an example architecture for a scaled, reinforcing learning-based PID system, which comprises two PID controllers and two learning systems. In this example implementation, a first PID controller 440A and a second PID controller 440B used a process 472 to control, which can represent the frequency-voltage controller and associated power adjustment mechanism. Input signals θ _ref and x _ref can represent separate metrics that we want to manage, and θ (t _k ) and x (t _k ) can represent current metric values and the estimated future increase in metric values. As an example, x (t _k ) may represent the sum of the current sensor power + the estimated future increase in power that will be observed due to the high energy instructions, and θ (t _k ) may be the sum of the current performance + the estimated future increase in the Represent performance that will be observed based on the high energy instructions. In some example implementations, the first metric can tension the process 472 represent and the second metric is a frequency of the process 472 represent.

The first PID controller 440A receives a first metric error signal e _p (t _k ), which represents a difference between θ _ref and θ (t _k ). The second PID controller 440B receives a second error signal e _c (t _k ), which represents a difference between x _ref and x (t _k ). The first PID controller 440A and the second PID controller 440B output values u _p (t _k ) and u _c (t _k ), which are determined based on the received errors and the coefficients K (t) set in the controllers. As in 8th shown, the output values u _p (t _k ) and u _c (t _k ) can be summed and the process 472 to be provided.

The output values u _p (t _k ) and u _c (t _k ) are summed if separate metrics have one or more common parameters of the process 472 to adjust. As an example, if the operating clocks are set based on performance and performance (since clocks can affect both), the outputs are used to drive / set common parameters that control the process.

In other examples, the output values u _p (t _k ) and u _c (t _k ) can be the process 472 be provided directly to determine how the output values u _p (t _k ) and u _c (t _k ) will drive / set parameters that affect the process 472 Taxes.

As in 8th shown, the first PID controller 440A and the second PID controller 440B each be coupled to a respective learning system for updating coefficients of the controller. A first learning system 150A generates one or more update coefficients K _PP , K _IP and / or K _DP for the first PID controller 440A , A second learning block 150B generates one or more update coefficients K _PC , K _IC and / or K _DC for the second PID controller 440B , The first learning block 150A and the second learning block 150B receive one or more components of a state vector (n ₁ (t _k ), n ₂ (t _k ), n ₃ (t _k ), n ₄ (t _k )) representing the discrete values of the outputs of the process 472 represents. The state vector values n ₁ (t _k ), n ₂ (t _k ), n ₃ (t _k ) and n ₄ (t _k ) are determined by the outputs θ (t _k ), θ (t _k ), x (t _k ) and x (t _k ) of the process 472 derived.

In the in 8th shown example receives the first learning block 150A State vector values n ₁ (t _k ) and n ₂ (t _k ) and the second learning block 150B receives state vector values n ₁ (t _k ), n ₂ (t _k ), n ₃ (t _k ) and n ₄ (t _k ). While 8th shows that the learning blocks receive different sets of values, the first learning block 150A and the second learning block 150B receive the same set of values or different sets of values.

In other examples, the learning systems 150A and or 150B use an element (e) of a state vector of the error signal e _p (t _k ) and / or error signal e _c (t _k ) to update the coefficients of the first and / or second PID controller.

Example of a non-limiting reinforcing learning-based feedback loop control method

9 FIG. 12 is a flow diagram of an encouraging learning-based feedback control method in accordance with certain example embodiments.

First of all, the operation is initialized 510 one or more parameters. These parameters can include, for example, the parameters set by the PID controller 440 used to control the power control parameters for the GPU 102 to influence.

The operation 520 feels an error, for example, by generating an error signal e (t) for application to the state vector 430 as provided by calculating the difference between y_ref (t) and y (t).

The operation 530 uses reinforcement learning to determine whether the sensed error is a system error that requires correction. Empowering learning is able to correlate immediate actions with the delayed returns it generates. Similar to humans, encouraging learning algorithms sometimes have to wait a little to see the result of their decisions. You can thus work in a delayed return environment where it can be difficult to understand which action leads to which result over many time steps. Thus, reinforcing learning algorithms can be expected to perform better in ambiguous environments, such as performance control with unknown factors (e.g., the degree of parallelism of certain instructions / threads / warps from certain software applications or functions thereof, the number of GPU transistors that be activated and deactivated based on such a parallel operation, etc.) while choosing between any number of possible actions. In example operation of a non-limiting embodiment, the critique of a neural network has been trained by an empowering machine learning system on previous performance adjustment operations in response to previous sensed errors and evaluated an accumulated reward, i.e., a value assessment of how advantageous it is to be in a particular state , If the neural network determines that the rated reward is high based on the sensed error, the state vector is not changed and the "senmse error" operation 520 is repeated continuously. However, if the neural network detects that the reward is low based on the sensed error, additional processing is called to change the state vector (discussed below).

To change the state vector when the rated reward is low, the non-limiting example embodiment performs an operation 540 by, that is, a state vector-based parameter calculation based on the neural network (in other embodiments, the state vector-based calculation could be carried out in other ways, such as incrementing or decrementing by a certain amount, etc.). This calculation can, for example, calculate coefficients K '(t) for the PID controller 440 include. In a non-limiting example embodiment, the state vector-based parameter calculation can be performed by the actor 476 based on the state vector x (t) derived from the error signal e (t).

In a non-limiting example embodiment, the operation performs 550 stochastic perturbations to transform calculated coefficients K '(t) into K (t). Transforming coefficients K '(t) into K (t) can be done by the self-adapting module ("SAM") 478 based on the output V (t) from the Critic 480 be performed. During surgery 560 the new coefficients K (t) derived through the neural network are applied to the PID controller 440 applied.

Applying new coefficients K (t) to the PID controller 440 causes the operating point for the GPU power settings to move (block 570 ). This operating point movement can be the setting of the output frequency of the frequency-voltage controller 122 and / or adjusting the output voltage using the power adjustment mechanism. The result of the process by the frequency voltage controller 122 and the associated power adjustment mechanism is performed, sets the GPU's new frequency and / or voltage operating point. The GPU then processes instructions using the new operating point (block 580 ). In the meantime, the neural network is learning 1 of the effect of this operation and uses this additional learning on the next iteration of the process of 9 ,

Overcurrent prevention

In some exemplary embodiments, the frequency / voltage controller 450 delay, exclude or prevent execution of the pending instructions in one or more processors (eg SMs). For example, the frequency / voltage controller 450 and / or machine learning 150 the output from the PID controller 440 receive and determine when the execution of the instructions would result in frequency and / or voltage settings that exceed predetermined limits (eg safe limits). Based on this determination, the frequency / voltage controller can 450 Output signals to delay or stop processing pending instructions for one or more cycles. The signals to delay or stop processing the instructions can be output to one or more of the processors via conventional privilege mechanisms known to those skilled in the art. The frequency / voltage controller 450 can determine in how many processors 116 the processing of high-energy instructions must be delayed in order to avoid exceeding predetermined safety limits. For example, if a determination is made that the execution of the pending instructions in the processors 116 The frequency / voltage controller can cause the power settings to exceed predetermined limits and thus cause potential overheating, voltage drops or other undesirable effects 450 transmit a delay signal to one or more processors so that all high power consumption instructions are not executed during a given clock cycle. Such load balancing can offset the high performance computation over time with minimal impact on overall speed performance.

Exemplary embodiments of this disclosure are not limited to the use of a PID controller and actor-critical or Q-learning described in 4 and 5 is illustrated. Other feedback controllers and learning systems can be used with the exemplary embodiments disclosed in this application. For example, a PID or other neural network controller (if fast enough) can be used to determine changes that should be applied to the operating frequency and / or voltage of the GPU based on the error between current and predicted performance.

As described above, conventional GPUs have been limited to using reactive mechanisms to manage the performance of the GPU. Such reactive mechanisms, however, leave the internal components of the GPU vulnerable to electrical reliability problems (e.g. internal component damage and / or that the GPU is switched off and falls off the communication buses). Embodiments of this disclosure provide proactive sensing of instructions to predict a probable load step / release event, which enables the upcoming load step / release event to be known before the event actually occurs a few clock cycles later. Predicting the load step / enable event allows the GPU's power settings to be adjusted (e.g., by changing the voltage and / or frequency) before the event happens to prevent peak performance scenarios.

Moving the voltage and / or frequency in this manner can reduce electrical reliability problems in shorter time scales than other approaches, but without sacrificing performance. It also sees the learning system that is parallel to the PID controller 440 operates to dynamically adjust parameters that are used to determine the changes that should be made to the operating voltage and / or frequency to anticipate and prevent such undesired peak power scenarios.

GPUs, which include proactive performance management systems according to embodiments, provide increased reliability on shorter timescales than other approaches, however without affecting performance, and are therefore applicable in computing environments that require high levels of reliability. GPUs and SoCs that include one or more GPUs according to embodiments may be implemented in data centers, server environments, game computers, laptop computers, and other computing environments. Embodiments can also be used in environments such as autonomous vehicles, robots, and the like. For example, embodiments may improve the operational reliability of autonomous vehicle applications by ensuring that the GPU does not even drop off the communication bus due to overcurrent situations. Embodiments can also improve the power consumption of autonomous (or other) vehicles by implementing a faster power management response.

However, embodiments are not limited to GPUs and SoC including GPUs. Other embodiments may provide on-chip ADC on CPU chips or other types of processors.

Embodiments of the article and functional operations described herein can be implemented in one of the following: digital electronic circuits, specifically embodied computer software or firmware, computer hardware including the structures disclosed in this application and their structural equivalents and combinations thereof. Such embodiments may be implemented as one or more modules of computer program instructions encoded on a specific non-transitory computer storage medium to execute or control data processing devices (ie, one or more computer programs). The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access storage device or a combination of one or more thereof.

Each of the above items is incorporated by reference for all purposes, as if they were expressly listed.

While the invention has been described in connection with what is currently considered the most practical and preferred embodiments, it is to be understood that the invention is not to be limited to the disclosed embodiments, but on the contrary is intended to cover various modifications and equivalent arrangements which are included within the spirit and scope of the appended claims.

QUOTES INCLUDE IN THE DESCRIPTION

This list of documents listed by the applicant has been generated automatically and is only included for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent literature cited

• US 16046916 [0054]

Non-patent literature cited

• Konda et al., "Actor-Critic like learning algorithms for Markov decision processes", SIAM Journal on Control and Optimization, 38 (1): 94-123 (1999) [0091]
• Sutton et al., Reinforcement Learning: An Introduction (MIT Press Cambridge, MA, 1995) [0091]

Claims (25)

GPU, comprising: Parallel processors, each comprising an instruction decoder that decodes instructions prior to execution and signals expected performance of instruction execution; a power controller that adjusts power that was at least partially delivered to the processors in response to the signaled expected power usage; and a neural network that adjusts the setting of the power that the controller performs, the neural network performing reinforcement learning to mitigate peak performance scenarios. GPU according to Claim 1 , wherein the instruction decoder includes an instruction look-ahead predecoder that predecodes instructions to be executed in the future and estimates how much power will be used to execute the instructions, and the parallel processors each include an additional instruction decoder that executes the instructions decoded. GPU according to Claim 1 or 2 wherein the power controller is configured to reduce power to avoid peak power overshoot. A GPU according to any preceding claim, wherein the neural network controls the parallel processors to modify their instruction execution to reduce peak performance requirements. Processing system comprising: multiple processors; and power management circuitry configured to: Compare performance used by the multiple processors and predicted performance estimated to be used by the multiple processors in the future; generate one or more control signals to change the performance of the plurality of processors in response to the comparison; and perform reinforcing learning to adjust the generation of the one or more control signals. Processing system according to Claim 5 where reinforcing learning uses actor-critical machine learning or Q-learning machine learning. Processing system according to Claim 5 or 6 wherein the power management circuitry includes a proportional integral derivative (PID) controller configured to receive a difference result of the comparison and which generate one or more signals to change the operating frequency and / or voltage of the processing system. Processing system according to Claim 7 , the reinforcing learning modifying operating coefficients of the PID controller. Processing system according to Claim 7 or 8th , wherein the PID controller generates the one or more signals every clock cycle and the reinforcing learning modifies operating coefficients of the PID controller at intervals that exceed a clock cycle. Processing system according to one of the Claims 5 to 9 wherein: each processor is configured to pre-decode instructions received from the respective processor and generate a signal based on the pre-decoded instructions that estimates the energy consumption level of pending instructions; and the power management circuitry is further configured to aggregate the signals that indicate the level of power consumption of each processor and predict performance based on the aggregated signals. Processing system according to one of the Claims 5 to 10 wherein the power management circuitry is configured to adjust the operating frequency and voltage of the processing system based on the one or more generated control signals. Processing system according to one of the Claims 5 to 11 wherein the power management circuitry is configured to control one or more of the processors based on the one or more generated control signals to delay execution of one or more pending instructions. Processing system according to one of the Claims 5 to 12 wherein the power management circuitry includes an analog-to-digital converter (ADC) configured to receive input signals representing voltages and / or currents supplied to the processing system and to generate a digital output signal based on the input signals representing the performance used by the processing system. Processing system according to one of the Claims 5 to 13 , the multiple processors and the power management circuitry is arranged on a same substrate. A method of dynamically controlling voltage and frequency settings for multiple processors configured to receive and execute instructions, the method comprising: Measuring the power currently used by the processors; Predicting processor performance by examining instructions to be executed by the processors; Determining an error between the current performance and the predicted performance; Generating one or more signals to change the operating frequency and / or voltage to reduce the error; and Perform reinforcing learning to adjust the generation of the one or more signals. Procedure according to Claim 15 wherein generating the one or more signals comprises using a proportional integral derivative (PID) controller to generate the one or more signals based on the error to change the operating frequency and / or voltage that is either is fed to several of the processors. Procedure according to Claim 16 wherein the PID controller generates the one or more signals at intervals that correspond to the clock cycles, and the reinforcing learning modifies the PID controller coefficients at intervals that exceed the intervals that correspond to the clock cycles. Procedure according to Claim 16 or 17 , whereby the reinforcing learning modifies the coefficients of the PID controller. Method according to one of the Claims 15 to eighteen , further comprising: generating a signal for each processor that represents the power consumption level of pending instructions executed on a clock cycle; and determine the predicted performance by aggregating the signals representing the energy consumption level of pending instructions for the corresponding clock cycle. Controller comprising: a performance monitor configured to monitor a voltage and / or a current applied to multiple processors; circuitry configured to receive signals from the plurality of processors representing the estimated power consumption to execute instructions in the future and to predict performance based on aggregation of the received signals to be performed by the processors in the future to execute the instructions Instructions will be used; and a self-learning and self-adjusting controller configured to control the power to be delivered to the processors in response to the monitored voltage and / or current and the predicted power. Controller according to Claim 20 , wherein the self-learning and self-tuning controller comprises: a proportional-integral-derivative (PID) controller configured to determine power to be delivered to the processors based on coefficients, and at least one neural network that is configured to perform reinforcing machine learning to modify the coefficients. Controller according to Claim 20 or 21 , wherein the self-learning and self-adjusting controller is arranged on the same substrate as the processors. Controller according to one of the Claims 20 to 22 , wherein the self-learning and self-adjusting controller is further configured to send a signal to one or more of the processors based on the particular power to be delivered to the processors to delay execution of at least one instruction. Controller according to one of the Claims 20 to 23 , wherein the self-learning and self-tuning controller comprises: a proportional-integral-derivative (PID) controller configured to determine performance to be delivered to the processors based on coefficients, and at least one neural network that is configured to perform actor-critical reinforcement machine learning to modify the coefficients based on a difference between the power applied to the multiple processors and the power consumption estimated to execute instructions in the future. Controller according to one of the Claims 20 to 24 , wherein the self-learning and self-tuning controller comprises: a plurality of proportional-integral-derivative (PID) controllers, each PID controller generating control signals based on a different metric that affects the operation of the multiple processors; and multiple neural networks, each neural network configured to perform reinforcing machine learning to modify coefficients of a respective PID controller, wherein the power supplied to the processors is controlled based on a sum of the outputs of the PID controllers.

Images