DE102020127707A1 - HIGH PERFORMANCE SYNCHRONIZATION MECHANISMS FOR COORDINATING OPERATIONS ON A COMPUTER SYSTEM - Google Patents

Abstract

In order to synchronize operations of a computer system, a new type of synchronization barrier is disclosed. In one embodiment, the disclosed synchronization barrier for certain synchronization mechanisms, such as "Arrive" and "Wait", can be shared to allow greater flexibility and efficiency in coordinating synchronization. In another embodiment, the disclosed synchronization barrier enables hardware components, such as certain copy or direct memory access (DMA) machines, to be synchronized with software-based threads.

Description

BACKGROUND & SUMMARY

Massively parallel high-performance systems with multithreading multi-core processors - systems that contain many processing cores working in parallel - process data much faster than was possible in the past. These processing systems break down complex calculations into smaller tasks that are carried out simultaneously by parallel computing cores. This “divide and conquer” approach enables complex calculations to be performed in a fraction of the time it would take if only one or a few processors worked on the same calculations in succession. But such parallel processing also creates the need for communication and coordination between parallel execution threads or blocks.

One possibility to coordinate their states for different execution processes is the synchronization by means of a barrier. When synchronizing using a barrier, each process in a collection of parallel processes usually waits at a barrier until all other processes in the collection have caught up. No process can cross the barrier until all processes have reached the barrier.

The 1A-1H together form a flipchart animation that illustrates an example of such synchronization by means of a barrier. 1A provides threads or other execution processes T1 , T2 , ... TK Three threads are shown, but there could be hundreds or thousands of threads. The threads begin at a common point (eg an initialization point, a previous barrier, etc. - here referred to as the “start line” for illustration purposes).

One thread TK is a "rabbit" - it runs faster than other threads and moves faster towards a barrier (here graphically represented by a level crossing and the designation "arriving waiting point", the meaning of which is explained below).

Another thread T1 is a "turtle" - it runs slower than other threads and moves more slowly towards the barrier.

The 1B , 1C , 1D show the different threads approaching the barrier at different speeds. 1D shows the "rabbit" thread TK , the one before the "turtle" thread T1 arrives at the barrier. Since the "rabbit" thread TK cannot go beyond the barrier until the "turtle" thread T1 also arrives at the barrier, the "rabbit" thread would wait at the barrier (according to the state of the art - but see below for the "shared" arrival / waiting functionality). This could potentially involve many cycles of delays - if for example the "turtle" thread T1 is slow, because it is waiting for the service from the main memory, the "rabbit" thread TK may wait a long time ( 1E) before the "turtle" thread catches up and reaches the barrier.

As soon as the last trailing "turtle" thread T1 reached the barrier ( 1F) , the barrier is lifted (in 1G represented by the opening of the level crossing barrier) and all threads can cross the barrier.

An example of a useful application that benefits from synchronization using a barrier is "asynchronous computation". With asynchronous computation, the GPU usage is increased by the fact that the tasks are not scheduled in strict order, but rather without observing the order, so that "later" (according to the order) calculations at the same time as "earlier" (according to the order ) Calculations can be performed. For example, when rendering graphics, an asynchronous computation can allow the shader to run concurrently with other work, rather than having a shader run sequentially with other workloads. While the GPU API can be designed to assume that most or all of the calls are independent, it also gives the developer control over how the tasks or tasks are scheduled and can implement barriers to ensure correctness , e.g. B. when one operation depends on the outcome of another. See, for example, U.S. Patent No. 9,117,284 and 10,217,183 .

Hardware-based synchronization mechanisms are included in GPUs to support such types of synchronization functions with a barrier. See e.g. Xiao et al., "Inter-Block GPU Communication via Fast Barrier Synchronization", 2010 IEEE International Symposium on Parallel & Distributed Processing (IPDPS) (April 19-23, 2010). Computational GPUs with such hardware-based synchronization capabilities have traditionally been programmed in a mass-synchronous style - broad parallel tasks with synchronization by means of a barrier within and a split / merge between them. See, for example, U.S. Patent Application Publication No. 2015020558.

In modern GPU architectures, many threads of execution run concurrently, and many warps, each spanning many threads, also run concurrently. For example, when threads need to perform more complex communications or collective operations in a warp, the developer can use NVIDIA's CUDA "_syncwarp" primitive to synchronize threads. The _syncwarp primitive initializes hardware mechanisms that cause an executing thread to wait until all threads specified in a mask have called the primitive with the same mask before resuming execution. For more details see e.g. US Patent No. 8,381,203 ; 9,158,595 ; 9,442,755 ; 9,448,803 ; 10,002,031 ; and 10,013,290 ; and see also https://devblogs.nvidia.com/usingcuda-warp-level-primitives/; and https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#memory-fence-functions.

Although hardware-implemented barriers have proven useful, it is sometimes helpful for a program to use more than one barrier at the same time. For example, a program can potentially use a first sync barrier to block a first set of threads and a second, different sync barrier to block another set of threads (or sometimes the same sync barrier is reused to block the same set of threads over and over again block as they traverse their execution paths). In the past, in order to perform operations with multiple barriers, a software developer typically had to tell the compiler how many barriers were needed in advance. In systems where synchronization barriers were hardware implemented, a limited number of synchronization barriers were available. Some programs needed or could have used more synchronization barriers than were supported in the hardware by the hardware platform.

1. 1. Programme, die mehr als eine physikalische Hardware-Barriere benötigten, hatten Schwierigkeiten bei der Zuweisung dieser Barrieren.
2. 2. Barrieren mit der klassischen „arrive-and-wait“-Schnittstelle verbergen die Synchronisierungslatenz nicht gut (in Bezug auf die Flipchart-Animation in 1A-1H muss der „Hase“-Thread möglicherweise eine Weile warten und nichts tun, bis die „Schildkröten“-Threads aufholen).
3. 3. Kopier-Maschinen (wie DMA-Einheiten (Direct Memory Access)) können in der Regel nicht direkt an der hardwarebasierten Synchronisierung teilnehmen, da sie keine Software-Threads sind.

Due to additional uses of and the demand for synchronization barriers, there is a need to improve the mapping of synchronization barriers. In particular, certain previous implementations and approaches of hardware accelerated barriers have significant shortcomings:

1. 1. Programs that required more than one physical hardware barrier had difficulty assigning these barriers.
2. 2. Barriers with the classic “arrive and wait” interface do not hide the synchronization latency well (with regard to the flipchart animation in 1A-1H the "rabbit" thread may have to wait a while and do nothing for the "turtle" threads to catch up).
3. 3. Copy machines (such as DMA units (Direct Memory Access)) can usually not participate directly in the hardware-based synchronization because they are not software threads.

It has long been possible to implement synchronization barriers in software, but software-implemented barriers do not necessarily provide the same level of performance as hardware-implemented barriers. For example, in the past some developers used hardware to implement as many barriers as the platform hardware supported, and when more (or other types of) barriers were needed, they implemented additional barriers in software. Developers who implemented synchronization barriers in software often suffered performance degradation. In particular, over-assigning barriers could result in fewer execution paths and correspondingly lower performance. It has not always been an easy compromise for developers to determine if using more barriers and fewer tasks could improve performance.

There is a need to improve the assignment of hardware-accelerated and / or -assisted synchronization barriers in such a way that the software assignment can be made flexible without, however, adversely affecting performance.

Figure list

• The 1A-1H together form a temporal sequence of images that comprise a flipchart animation that illustrates the use of a synchronization barrier (to view the animation, start with 1A an electronic copy of this patent and presses the page down key repeatedly);
• 2 Figure 13 shows an example of a non-limiting instruction stream;
• The 2A & 2 B Figure 4 is block diagrams of an example of a non-limiting system using the present method of synchronization using a barrier;
• 3 Figure 3 is a block diagram of an example of a non-limiting memory-based synchronization barrier primitive;
• 4th FIG. 3 is a block diagram of an exemplary, non-limiting combination of hardware and software functions used to manage the barrier of 3 can be used;
• The 5A-5F illustrate exemplary flow diagrams of operations associated with barrier functions;
• The 6th & 7th show illustrative examples of two-threaded barriers;
• 8th Figure 12 shows an illustrative example of three stage pipeline streaming code; and
• 9 shows an example of a non-limiting loop code structure.

DETAILED DESCRIPTION OF EXEMPLARY NON-RESTRICTING EMBODIMENTS

1. 1. Sie ist im Speicher implementiert und allokiert daher wie der Speicher.
2. 2. Die „Arrive“- (Ankommen-) und die „Wait“- (Warten-) Operation werden aufgeteilt, damit nicht betroffene Tasks bzw. Arbeiten dazwischen ausgeführt werden können.
3. 3. Asynchrone Kopierhardware von demselben Streaming-Multiprozessor (SM) kann sich als virtueller oder „moralischer“ Thread beteiligen.

A new type of barrier is introduced that solves the problems described above:

1. 1. It is implemented in memory and is therefore allocated like memory.
2. 2. The "Arrive" and "Wait" operations are split up so that unaffected tasks or work in between can be carried out.
3. 3. Asynchronous copy hardware from the same streaming multiprocessor (SM) can participate as a virtual or "moral" thread.

The implementation of barriers in memory is quite feasible and is generally implemented by software. Shared software barriers are less common, but they also exist in practice. Hardware acceleration is provided for this idiom (s), and hardware copiers are integrated with the hardware acceleration as if the hardware copiers were "moral" threads.

The programming effort required to create a barrier with richer functionality and good performance is greatly improved. It also enables more asynchronous copy operations to be introduced into the streaming multiprocessors (SMs) of a GPU by providing an innovative way of synchronizing with the copy operations that improves SM performance by offloading work on the core threads .

The present example of a non-limiting method thus shifts additional functionality into barriers, which in turn can increase the spread of synchronization barriers. In particular, much more code will have to (and can) potentially use multiple sync barriers than in the past.

Synchronization function with split arrival / waiting barrier

As explained above, synchronization by means of a barrier is often or typically defined as a construct in which a certain amount of threads or other processes are blocked at the synchronization point and when all threads or other processes in a certain amount have arrived at the synchronization point , all threads or other processes are then released. Please refer 1A-1H .

In particular, in many earlier implementations, threads such as the "rabbit" thread in 1A ff. that arrive at the synchronization point early, simple and did no useful work until they were released by the arrival of later ending threads. A typical scenario was that whenever more threads arrived at the sync point, there was soon just one lagging thread waiting for all the other threads to be waiting for before they were all released and allowed to move on to the next stage of processing.

Assume that a programmer implements a barrier in software. Assume that the programmer / developer wrote code to implement the barrier. An economic process would be “arrive and wait”. This is what an "Open MP" barrier implemented and how previous CUDA barriers were implemented. In an exemplary implementation, each thread arriving at the synchronization point would say "I have arrived". The program would then count the number of threads that arrived. If not all threads have arrived, the program would block. Such a system would simply wait and continue polling the count until the count reaches the correct value indicating that all threads have arrived. The polling loop is wasteful because the threads are not doing useful work while they wait, the processor spends a lot of time polling the number, and the system is consuming resources that could otherwise be used for useful work.

Some newer software-based approaches have decoupled the two events of arriving at the synchronization point and blocking at the synchronization point. Exemplary, non-limiting embodiments similarly provide decoupling of how a thread or other process arrives and waits at a synchronization point. In exemplary, non-limiting embodiments, this method is used so that threads arrive first at the synchronization point where their arrival is considered. However, in non-limiting implementations, the threads do not need to block when they arrive at the sync point. Rather, they can do other work that is not related to the synchronization point (i.e. work that does not need to be synchronized through that particular synchronization barrier, but is more asynchronous with respect to that particular synchronization barrier). Once they complete that other work and need to get back to the work that requires syncing through the barrier, they can be blocked if necessary. However, if the other work is significant enough, by the time the thread completes the other work, all the other threads have already arrived at the sync point and there is no blocking at all. In such situations, exemplary non-limiting implementations simply determine that all threads have arrived at the synchronization point and are atomically blocked and released without actually delaying or stopping further processing of a thread (with the exception of threads running out of other work, which they could do while waiting for the release at the sync point).

Exemplary, non-restrictive embodiments thus subdivide the “arrive and wait” or arriving and waiting function into two different atomic functions: (1) arriving or arrive and (2) waiting or wait. The "arrive" part of the function is all of the bookkeeping and other management that typically needs to be done prior to implementing the barrier, but it does not cause any thread to actually be blocked. Because the threads are not blocked, they are allowed to do work unrelated to the barrier.

For example, a barrier is often used to implement phases of computation over a data structure. So the synchronization barrier is used to prevent threads from using the data structure until all threads have finished updating the data structure. In non-limiting embodiments, threads that have already arrived at the sync point may be allowed to perform other useful work that does not involve that data structure while waiting for other threads to arrive at the sync point.

Example - red and green data structures

As an example in 2 It is assumed that there are two data structures: Even if patent drawings are not in color, for the sake of simplicity reference is made to two different data structures, which the system updates as the “red” data structure and the “green” data structure. 2 is actually more general than updating two different data structures (e.g., updating a single data structure and then doing "other work" unrelated to that data structure while waiting for updates to that data structure to complete), but it is useful to explain how the process shown could be used to update two different (eg a “red” and a “green”) data structures.

A "red" synchronization barrier is created (2100 ') to provide a barrier to the "red" data structure. Once threads have finished updating the "red" data structure (2702) and arrive at the "red" sync point (2200 ') after completing their respective operations on the "red" data structure, they can begin other work, such as e.g. work in relation to the "green" data structure ( 2704 ), and update that data structure while waiting for the "red" sync barrier - thus protecting the "red" data structure work to be completed. An additional, "green" synchronization barrier could be used in a similar manner if desired to protect the "green" data structure.

Once the threads have finished working with the “green” data structure, they can work on the “red” data structure again - but before taking any further steps with the “red” data structure, they must ensure that the previous processing phase has been completed . At this point, if previous processing, as managed by the "red" synchronization primitive, has not yet completed, the threads may have to wait ( 2300 ' ) until the processing phase is complete. Since the atomic operations "arrive" ( 2200 ' ) and "wait" ( 2300 ' ) but are separated in time by any length of time, for example thousands of cycles, can do a lot of useful work ( 2704 ) of all threads that have arrived at the "red" synchronization point ( 2200 ' ), but are not blocked, but instead are free to do any useful work other than work on the “red” data structure, to be done collectively (2704).

It may turn out that the synchronization primitive never actually blocks a thread. If all the threads are designed to run when they arrive ( 2200 ' ) start working on the "green" data structure at the "red" synchronization point ( 2704 ), and if the time the threads spend working on the "green" data structure exceeds the time it takes for the last lagging thread to arrive at the synchronization point after working on the "red" data structure, then none will the threads are blocked. Rather, when the last lagging thread arrives at the synchronization point, the synchronization primitive will change state to the next processing phase, and when the threads check the state of the synchronization point, they will find that the processing phase has changed and they are no longer blocked. Accordingly, no thread blocks and no cycles are wasted.

Another way of describing this scenario is: The synchronization barrier requires all threads or other processes to arrive at the synchronization point at the same time and does not allow any thread or process to continue until they have all arrived. For example, rather than an "arrive and wait" scenario, non-limiting embodiments transform the "arrive" event into an execution window between the "arrive" ( 2200 ' ) and the "wait" ( 2300 ' ). No thread is allowed to use the "wait" point ( 2300 ' ) happen until all threads at least the "arrive" point ( 2200 ' ) achieved. However, this does not prevent the "arrived" threads from performing other tasks ( 2704 ) that are not protected by the synchronization barrier.

As one non-limiting example, when all threads and other processors have arrived at the synchronization point, the synchronization barriers are reset to start the next phase of processing. Thus, in accordance with exemplary, non-limiting implementations, a synchronization barrier is a reusable object that can be used to manage multiple synchronization points for the same set of threads. Once a first phase is complete, the next phase begins, which can be managed by the same synchronization barrier, and then the next but one phase, which can also be managed by the same synchronization barrier, begins, and so on.

1. 1. load-/store- bzw. Lade-/Speicher-Vorgänge in einem VOR_ANKOMMEN_ABSCHNITT eines Therads sind garantiert sichtbar für load-/store-Vorgänge in einem NACH_WARTEN_ABSCHNITT anderer mitwirkender Threads;
2. 2. load-/store-Vorgänge in einem NACH_WARTEN_ABSCHNITT eines Threads sind garantiert nicht sichtbar für load-/store-Vorgänge in einem VOR_ANKOMMEN_ABSCHNITT anderer mitwirkender Threads; und
3. 3. load-/store-Vorgänge in einem MITTEL_ABSCHNITT eines Threads weisen keine Sichtbarkeitsgarantiereihenfolge bezüglich anderer Threads auf.

More specifically, in one example of a non-limiting embodiment, each thread involved in the arriving wait barrier calls two functions in turn, first the ARRIVE function and then the WAIT function. The model of the arriving-wait-barrier divides a contribution to a barrier in a program into three sections: an ARRIVAL_SECTION, a MIDDLE_SECTION and a WAIT_SECTION with the following axioms: BEFORE_ ARRIVAL_SECTION ARRIVE [arrive-wait-barrier-address] MIDDLE_SECTION WAIT [Arriving-Waiting-Barrier-Address] AFTER_WATCH_SECTION where the following applies to exemplary embodiments:

1. 1. load / store or load / store processes in an ARRIVAL_SECTION of a Therad are guaranteed to be visible for load / store processes in a WAIT_SECTION of other participating threads;
2. 2. load / store operations in a WAIT_PART of a thread are guaranteed not to be visible to load / store operations in an ARRIVAL_SECTION of other contributing threads; and
3. 3. load / store operations on a MID_SECTION of a thread do not have a visibility guarantee order with respect to other threads.

In non-limiting embodiments, for example, arrive-wait barriers enable overlapping barriers to be pending resolution.

Such an implementation can be carried out for a software-implemented barrier, a hardware-implemented barrier or a hybrid hardware / software-implemented barrier. For example, the previous CUDA hardware-based_synch primitive could be modified with circuit changes to implement the arrive-and-wait strategy as two separate atomic functions (arrive and wait) as described above. However, in the example of a non-limiting technology, additional benefits are obtained by implementing the synchronization barrier as a memory-based, hardware-accelerated barrier.

Storage-based synchronization barriers

For the purpose of terminological clarification, the term “barrier” can have different meanings depending on the level of abstraction. At a lower level of abstraction, systems usually have physical memory that implements the memory. A device that implements load and store is used for reading and writing to this memory.

At a next level of abstraction, if the memory is used for data communication between processes, a mechanism can be provided that ensures that all relevant data has been written to the physical memory before a flag is set that indicates to another process that the Data is available for communication with the other process. Without some sort of barrier, another process could be trying to read the data before it was written or while it was being written and the message could be incomplete or incorrect. Such barriers that protect against this are usually referred to as “storage barriers”.

A memory barrier in the example context above is not a synchronization primitive, but rather a non-side effect instruction that is used to make operations in a program order visible on machines like many modern GPUs that rearrange memory transactions. Many modern GPUs have such memory barriers, e.g. an instruction called "memory fence", such as the CUDA command "_thread-fence_block ()". These storage barriers are on an abstraction level that is below the typical synchronization barriers.

It is possible to use memory barriers to implement synchronization primitives. Locks and mutexes (objects for mutual exclusion) are further examples of basic synchronization elements. A mutex grants thread after thread exclusive access to a critical resource. A barrier does something similar, but with certain differences.

In general, a sync barrier, like the one in 1A and the following is shown, various calculation phases in which many threads working in one phase arrive at a barrier, these threads being blocked until all other relevant threads do their work in this one Phase completed. As soon as all threads have arrived at the barrier, the barrier is lifted and a new processing phase begins. Such synchronization barriers are typically defined as a synchronization "primitive" and in some contexts can be viewed as an object. Thus, while synchronization primitives can be implemented in some contexts using “memory barriers”, the discussion in the present description is intended to separate the concept of a “memory barrier” from a “synchronization barrier”.

Exemplary non-limiting embodiments use a memory-based synchronization barrier, that is, they implement synchronization barriers using memory (and, in some cases, associated memory barriers). Implementing a synchronization barrier as storage virtualizes the synchronization barrier in the same way that storage is virtualized. Furthermore, there is no practical limit to the number of barrier objects that can be instantiated, at least in the event that each virtual storage space can be used to support and store a synchronization barrier.

For example, assume that a synchronization barrier object 64 Bytes of memory required. It follows that a memory-based synchronization barrier scheme allows the designer to have as many synchronization barriers as available memory can accommodate additional 64-byte memory elements. In modern GPU architectures with unified memory, the global memory can be excessively large, which means that a very large number of synchronization barriers can be accommodated. This is an improvement over hardware-based synchronization barriers, where typically only a limited number of barriers can be used, depending on the particular hardware implementation, chip design, available chip size, etc.

Instantiating synchronization barrier objects in memory greatly eases the performance tradeoffs discussed above, since implementing objects in memory is a simple problem that most developers have mastered. Since a developer can instantiate so many barrier objects (although not really unlimited, the number is practically unlimited as the size of the main or global memory increases), there is no need to compromise between the number of synchronization barriers and the task Capacity to enter.

Because synchronization barriers are stored, i.e. implemented in memory in exemplary embodiments, they also benefit from sharing the memory and the memory hierarchy. Previous hardware synchronization barrier circuits were often implemented directly in the processor. Therefore, such hardware implemented barriers generally did not apply between different processors. In other words, each processor could have its own hardware-based barriers that it could use to manage tasks or multi-threaded tasks running on that processor, but those hardware barriers were not in coordinating activities outside of that processor useful, for example in a system with multiple processors that may have been involved in implementing the same processing phase in parallel. Such coordination typically required the use of shared global (CPU) memory, which could be slow and have other performance issues.

In contrast, for example, non-limiting embodiments that implement a synchronization barrier using memory instructions allow functionality to be supported outside the confines of a processor, GPU, or SOC (System-on-a-Chip). In particular, synchronization barriers can now be implemented at any level of the storage hierarchy, including, for example, levels shared across multiple cores, multiple streaming multiprocessors, multiple GPUs, multiple chips, multiple SOCs, or multiple systems, and in some cases, storage based on such hierarchies -Caches are cached.

Example of a non-limiting memory-based system implementation

For example, in an exemplary system, referring to 2 with one or more CPU (s) 101 , one or more GPU (s) 110 with one or more local graphics memories 114 , one or more main memories 115 and one or more advertisements 112 the synchronization barrier in the local graphics memory 114 stored and used by all streaming multiprocessors (SMs) 204 (please refer 2 B) inside the GPU 110 be shared, or they could be in main memory 115 saved and used by the CPU 101 and the GPU 110 shared. More precisely, the local graphics memory 114 be organized in different hierarchies, such as level 3 cache, level 2 cache, Level 1 cache, shared memory, etc., all held by a memory management unit (MMU) 212 managed as it is in 2 B is shown. Because the storage hierarchy is used to determine which storage locations are shared by which resources, the storage of a synchronization barrier at different levels of the storage hierarchy controls which compute resources can share the synchronization barrier (and also which compute resources cannot access the barrier) .

Such memory-implemented synchronization barriers can therefore be used to move between threads running on a common core 206 run, between different warps running on different cores, different processes running on the same or different GPUs 200 running, synchronizing the same or different SOCs, etc. Thus, a certain barrier is no longer limited to the synchronization of threads that are processed in parallel on a certain processor, but can also be used to transfer many more threads or other designs over any number of different cores, GPUs, processors, processor architectures, chips and Synchronize across systems.

This capability is made possible by implementing the memory-based synchronization barrier at an appropriate level of the memory hierarchy so that it can be accessed and shared by multiple processes and / or processors while being protected, for example, by commonly used memory barriers. For scalability, as the storage hierarchy expands, more threads can leverage these synchronization barriers, while smaller hierarchies may support more limited uses of the barrier.

Using synchronization barriers to synchronize hardware

An additional limitation of most previous hardware synchronization barrier implementations was that they could block software execution, but not necessarily hardware processes. One process typically performed by previous GPU hardware is copy operations. During a processor, memory commands such as load and store, which are issued by the load / store unit 208 can be used to copy from one storage location to another, so-called "copy machines" have long been used to speed up the copying of data to and from the storage. 210 as in 2 B are shown, or "Direct Memory Access" (DMA) controllers (eg hardware-accelerated or hardware-based copy operators) are used. In many systems, software can send commands to a dedicated copy / DMA machine 210 send, which carries out the data transfer. Such a copy machine can store data for a thread from a large number of contiguous locations in memory 214 , which each correspond to an offset from a base address of the thread, copy and store the data contiguously in the system memory. The copier 210 can also copy data for a thread from memory locations in system memory and the data in GPU memory 214 to save. See, for example, U.S. Patent Publication No. 20140310484.

Often such copying processes have to be completed before a next processing phase can begin, since z. B. the transition to the next processing phase may depend on the completion of an update of a data structure in memory. However, because previous implementations of synchronization barriers were based on blocking threads rather than hardware, an additional mechanism beyond the hardware-based synchronization primitives was required to ensure that the correct data was present after all of the threads monitored by the synchronization primitive were completed before the next phase of processing could begin. In other words, in earlier traditional approaches, such additional hardware-based machines (e.g. copier machine 210 or any other hardware) generally does not participate in the same synchronization barrier process as executing threads. While one previous solution was to wrap a software operator around the hardware-based DMA / copy operator so that the software operator did not finish until the hardware operator had finished, this approach placed additional constraints on software design that were not always desirable or efficient.

In contrast to such prior approaches, an exemplary non-limiting feature is that direct memory access (DMA) copy operations performed by a copy machine 210 (or similar or other operations performed by hardware such as a calculating machine) are integrated into the software-implemented but hardware-accelerated synchronization primitive so that the same synchronization primitive can be used to create a Providing barriers for software processes, hardware-based processes and hybrids of hardware- and software-based processes.

An example of a non-limiting feature of the implementations presented here is therefore the synchronization of transactions of copy machines 210 with the technology of synchronization barrier. Such an integration can for example be carried out with purely hardware-based barrier implementations, but the example of a non-limiting memory-based synchronization barrier method described here offers additional advantages in terms of performance and flexibility compared to a purely hardware implementation.

In one non-limiting embodiment described herein, hardware operations behave such as copy machine operations once initiated 210 , from the standpoint of the synchronization barrier, as if they were full-fledged threads, that is, as if they were a stream of execution of software instructions written by a programmer or compiled by a compiler. The implementation is elegant in that it is easy to describe: the hardware operation behaves as if it were "morally" a thread. In some examples of non-limiting embodiments, there can be many fine-grained hardware operations, such as copy operations, that are performed concurrently and concurrently, and they can all be synchronized with a common synchronization barrier (s).

With massively parallel modern GPUs, complex calculations can most often be performed collectively. Thus, the calculations can be carried out collectively using a large number of threads, which in turn jointly an even larger number of hardware-based operations such as DMA operations by one or any number of copying machines 210 can start. For example, assume that 100 Threads are running concurrently and that each of these 100 Threads a DMA operation through one or more associated copy machines 210 initiated. Using exemplary, non-limiting features of the method presented here, the same synchronization barrier can synchronize the 100 DMA operations and the 100 threads (ie, from the standpoint of synchronization by the synchronization primitive, the DMA operations "look" like threads), thereby creating a synchronization for a total of 200 processes ( 100 Software threads and 100 hardware-based DMA operations). Such a functionality is provided, for example, by hardware acceleration circuits, the interfaces between the MMU 212 and the copier (s) 210 ready so that the copier (s) 210 Can / can change values of the memory-based synchronization basic element (e.g. increment and reset counter values). The present method can be expanded so that any number of fine-grained DMA operations can be synchronized on the same barrier.

With a massively parallel architecture that can support many threads, it might be inefficient to program each individual thread to wait for each hardware-based operation to complete. The current example of a non-limiting method instead provides a synchronization primitive that allows the large number of threads (and, in some embodiments, copy operations) to wait together for one or more hardware-based operations (e.g., copy) to complete the next processing phase depends.

In this case, the barrier primitive is a different type of mechanism than a semaphore or flag (which were sometimes used in previous approaches to synchronizing with hardware-based processes) because the new synchronization primitive provides collective synchronization. It is different from a thread setting a flag or semaphore for another thread. Instead, it allows N threads to be blocked pending completion of potentially M hardware-based copies, where N and M are any non-negative integers. Such collective functionality need not necessarily be limited to software-based or memory-based barrier methods, but can be implemented in software, hardware, or both.

Hardware accelerated synchronization barrier

For example, to enable the above functionality and achieve higher performance, non-limiting embodiments provide hardware accelerated implementation of memory-based barriers. The implementation is called by software commands, but integrates additional hardware functionality, such as hardware-based copying, into the same synchronization barrier mechanism that is used to block software processes such as threads.

While it would be possible, for example according to non-limiting implementations, to implement such a function entirely in software, hardware acceleration is used to at least implement the function of resetting a barrier more efficiently, and it is also used to interface with To create hardware-based processes such as DMA copy machines and thus to enable hardware functions to reset a barrier. In some embodiments, a dedicated hardware-based accelerator could be used to cache the synchronization barrier.

In previous software-implemented versions, the last thread to arrive recognized that it was the last thread to arrive and modified a counter accordingly, adding the compliment to the current value of the counter to reset the counter to a starting value. As an example, refer to the Java “Phaser”, which is usually implemented in Java virtual machines. DMA machines are not written in software in some example implementations. Because the DMA machines may be responsible for resetting the barrier in some cases, and because they are not software, such resetting is desirable in these hardware implementations. For this reason, exemplary non-limiting implementations provide a hardware accelerated reset operation. However, other exemplary, non-limiting implementations of the method presented here could be applied to phasers, latches, or other synchronization primitives that are not barriers. Such a technique could also be applied for use with semaphores.

Software implementation

1. 1. Die erwartete Anzahl von Ankünften für jedes Mal, wenn die Barriere benutzt wird (Feld 1908).
2. 2. Die verbleibende Anzahl von Ankünften, die erforderlich sind, um die Barriere aufzuheben (Zählerfeld 1904).
3. 3. Die Barrierephase (zur Wiederverwendung der Barriere) (Feld 1902).

In non-limiting embodiments, each arrive wait barrier state is an implemented (eg, 64-bit) data structure stored in memory 1900 . Like it in 3 contains this data structure 1900 The following:

1. 1. The expected number of arrivals for each time the barrier is used (field 1908 ).
2. 2. The remaining number of arrivals required to lift the barrier (counter field 1904 ).
3. 3. The barrier phase (to reuse the barrier) (field 1902 ).

In accordance with exemplary, non-limiting implementations, arrive-wait barriers allow individual threads to work together so that the count of the fields 1904 , 1908 can be viewed as a thread counter.

Like it in 3 is the first field 1902 a phase counter which shows the current processing phase of the arriving-waiting barrier. When the barrier is reset, the phase counter 1902 can be increased to the next phase number. In an exemplary implementation, the phase counter 1902 be a one-bit status flag (ie a one-bit counter) that is toggled (incremented) each time the barrier is reset. Such a state flag can be used to indicate to threads that the state of the barrier has changed for the next processing phase - which means that the threads are not (or no longer) blocked by the barrier. In other, non-limiting embodiments, a greater resolving power of the phase indicator could be desired, for example in order to differentiate between more than just the current phase and a subsequent phase. Such other implementations could, for example, increment a multi-bit counter each time the barrier is reset to keep track of which of N processing phases is currently in effect, where N is an integer.

As in 3 is further shown, an additional field comprises an arrival counter 1904 , which indicates how many threads / processes have arrived (or in one implementation, the remaining arrivals required for the barrier to clear). In an exemplary implementation, each time a thread or other process arrives at the barrier, the arrival counter is checked 1904 incremented. When the arrival counter 1904 is incremented to a predetermined known value, it indicates that all threads and processes have arrived and the barrier can change state and move on to the next stage of processing.

When the last lagging thread (or hardware process) arrives and the barrier is saturated / reset, the arrival counter can 1904 can be reset to an initial value, for example in the third field 1908 in the in 3 shown non-limiting embodiment is included - namely for an expected number of arrivals to be used each time the arrive-wait barrier is used.

As discussed below, exemplary non-limiting embodiments allow the software to read the values of the arrival counter 1904 and the field "Expected number of threads" 1908 changes dynamically after the barrier is created.

In the exemplary, non-limiting embodiments, the in 3 data structure shown 1900 stored somewhere in memory. As previously discussed, the storage type / hierarchical level of storage is used to store the data structure 1900 is used, chosen to provide the desired amount of inter-thread sharing. When a thread implements a synchronization barrier of the exemplary non-limiting embodiment herein, the call to the synchronization primitive in the thread directly or indirectly assigns a memory address for the stored instance of the data structure 1900 representing the synchronization primitive. This differs from previous hardware-based synchronization primitives, where calling the synchronization primitive would likely represent a reference number or ID. A memory address (which, depending on the system, can be a physical or virtual memory address) specifies where in memory the instantiation of the synchronization primitive referenced by the thread 1900 is to be found.

Example; Hardware accelerated implementation

4th Figure 4 is a block diagram of an exemplary, non-limiting, hardware accelerated, memory-based implementation of the synchronization primitive 1900 out 3 . As previously described, the counters are 1902 for the current phase, the arrival counter 1904 and the memory field 1908 stored in memory for the expected number and by the load / store unit 208 accessible, which in exemplary embodiments is capable of atomic operations (those in a processor instruction register 1952 are stored and by a conventional instruction decoder 1954 decoded) like (arithmetic functions like atomicAdd ()), atomicSub (), atomicExch (), atomicMin (), atomicMax (), atomiclnc (), atomicDec (), atomicCAS ()); bitwise functions like atomicAnd (), atomicOr (), atomicXor ()); and perform other functions. Such atomic operations allow a streaming multiprocessor to determine the value of the synchronization primitive 1900 To change "on the spot". An MMU 212 is modified to have hardware circuitry that makes it the DMA control / copy machine 210 allows the arrival counter 1904 Modify "in place" in a similar fashion and the synchronization primitive 1900 reset when an operation on the copier causes the arrival counter decoder 1956 (which acts as a comparator that compares the count of the counter with a predetermined value and resets the counter and the phase indicator depending on the results of the comparison) determines that no further threads or copy machine operations are expected before the synchronization barrier can be reset . In such a case the decoder initiates 1956 a hardware reset of the instance of the synchronization primitive 1900 to see the phase indicator 1902 "To flip" and the expected number 1908 again in the arrival counter 1904 to load. In an exemplary embodiment, the LSU performs 208 - and / or MMU 212 circuit a kind of atomic operation with direct memory access to perform suitable atomic operations on the instance of the synchronization primitive stored in the memory 1900 initiate / perform to make these changes.

Therefore, for example, in non-limiting embodiments, the processor architecture is modified to provide additional circuitry that integrates hardware-based processes such as DMA into the implementation of the synchronization barrier so that the hardware is able to reset the memory that is blocking the synchronization barrier when it has completed its DMA task and realizes that it is the last lagging process required to proceed to the next stage of processing. In exemplary non-limiting implementations, the hardware changes need not worry about separating arriving from waiting, as described above, since hardware DMA control generally returns and is ready for a next task or task as soon as it is ready she has completed the previous one. The count given by the completion counter 1904 However, in exemplary non-limiting implementations, for example, it includes both the number of threads needed to complete and the number of DMA hardware operations needed to complete - ie, the counters 1904 , 1908 do not distinguish between a thread count and a DMA / copy machine count, but instead have one each only value that summarizes the number of threads and the number of copy operations that must be completed for the barrier to reset.

Example of a non-limiting implementation

Exemplary, non-limiting embodiments herein implement changes to the Instruction Set Architecture (ISA) to include instructions for accessing a new synchronization primitive that references memory-based storage of a synchronization barrier in memory. In addition, in exemplary non-limiting implementations, a thread has two different primitive calls instead of a primitive call: a primitive “arrive” call to the barrier and a primitive “wait” call to the barrier. Between these two calls, the thread can, as explained in advance and in 2 have instructions that are unrelated to the barrier and that can be carried out without violating the barrier.

During the initialization, the system first sets up the instance of the synchronization barrier in the memory and stores the corresponding data there that the system must hold to implement the barrier (e.g. arrival counter, phase counter). Typically, the software development kit (SDK) provided by the system developer may have a library that contains these various function calls to initiate a synchronization barrier. Similarly, the ISA of the processing system is modified to include new instructions for the sync barrier arrive and the sync barrier wait.

• • eine _create-Funktion 2100 (siehe 5A) wird verwendet, um eine Ankommen-Warten-Barriere im Speicher einzurichten.
• • eine _arrive-Funktion 2200 (siehe 5B) wird von einem Thread verwendet, um seine Ankunft an einer Ankommen-Warten-Barriere anzuzeigen. Die Barrierephase wird von dieser Funktion zur Verwendung mit der _wait-Funktion zurückgegeben.
• • eine _wait-Funktion 2300 (siehe 5C) wird verwendet, um auf die Freigabe einer Ankommen-Warten-Barriere für die vorgesehene Phase zu warten.
• • a _dropthread-Funktion 2400 (siehe 5D) entfernt permanent einen Thread von einer Ankommen-Warten-Barriere. Dies ist nützlich, wenn ein Thread endet.
• • eine _addonce-Funktion 2500 (siehe 5E) fügt dem Zählerstand einer Ankommen-Warten-Barriere einen bestimmten Wert hinzu.
• • eine _droponce-Funktion 2600 (siehe 5F) reduziert den Zählerstand einer Ankommen-Warten-Barriere um einen bestimmten Wert.

In an exemplary, non-limiting embodiment, the following software functions may be used to create a memory-stored instance of an arrive-wait barrier primitive 1900 manage:

• • a _create function 2100 (please refer 5A) is used to set up an arriving-wait barrier in memory.
• • an _arrive function 2200 (please refer 5B) is used by a thread to indicate its arrival at an arrive-wait barrier. The barrier phase is returned by this function for use with the _wait function.
• • a _wait function 2300 (please refer 5C ) is used to wait for an arrival-wait barrier to be released for the intended phase.
• • a _dropthread function 2400 (please refer 5D ) permanently removes a thread from an arrival-wait barrier. This is useful when a thread ends.
• • an _addonce function 2500 (please refer 5E) adds a certain value to the count of an arrival-wait-barrier.
• • a _droponce function 2600 (please refer 5F) reduces the count of an arriving-waiting-barrier by a certain value.

In addition, some non-limiting embodiments have an additional ARRIVES.LDGSTSBAR.64 instruction that signals that all DMA transfers from this thread have been completed and updates the arriving count in the arriving wait barrier accordingly.

_create

5A shows an example of the _create function 2100 who created a new barrier. In this example, the calling code is responsible for ensuring that _create is only called in one thread. The _create function 2100 receives as parameters "Ptr" (the direct or indirect memory location at which the data structure of the barrier is to be stored in the memory) and "ExpectedCount" (the expected total number of accumulation of threads and DMA operations that are synchronized by this particular barrier) . In the example shown, the counters are initialized in the data structure ( 2102 ). The barrier counter is set to "ExpectedCount" (increment upon arrival) ( 2108 ), and the now initialized barrier is stored in the specified location in memory ( 2110 ). The function then executes a return ( 2112 ). As previously described, in one exemplary embodiment the barrier may be stored in shared memory, but in other embodiments it may be stored in any desired memory in the memory hierarchy that can be negotiated with the scope of the synchronization.

• _create(BarPtr, NumThreads);
• Eingabe: BarPtr, NumThreads;
• Initialisieren der Barriere für die angegebene Anzahl von Threads.
• Kern-Grundelement: Cooperative Thread Array- (CTA-) breite geteilte Barriere (zugewiesen durch CTA-breite Synchronisierung).
• BarPtr = Zeiger auf zugewiesene Barriere, die in einem gemeinsam genutzten oder jedem anderen Speicherort gespeichert ist, an dem der Zustand der opaken Barriere gespeichert ist.
• NumThreads: die Anzahl der an dieser Barriere beteiligten Threads, die ankommen müssen, bevor das Warten aufgehoben wird.
• _arrive

A more specific example is:

• _create (BarPtr, NumThreads);
• Input: BarPtr, NumThreads;
• Initialize the barrier for the specified number of threads.
• Core Primitive: Cooperative Thread Array (CTA) wide shared barrier (assigned by CTA wide sync).
• BarPtr = pointer to assigned barrier, stored in a shared or any other location that stores the state of the opaque barrier.
• NumThreads: the number of threads involved in this barrier that must arrive before the wait is released.
• _arrive

5B shows an example of the _arrive function 2200 , which indicates that a thread has arrived at a barrier and returns the barrier phase that is required for the _wait () function 2300 should be used. In the example shown, the _arrive function reads 2200 the phase of the barrier at the memory location [Ptr] and stores the phase locally (eg in a register to which the thread has access). It should be noted that in the example of a non-limiting embodiment, from a software perspective, the _arrive function 2200 does not change the state of the barrier, but merely reads the current phase value from the barrier. In the exemplary implementation, the call to the _arrive function 2200 however, the effect (based on the hardware acceleration associated with the implementation of barriers) of reducing the number of threads indicated by the barrier's thread counter, thereby “registering” with the barrier that the thread has arrived at the synchronization point and so the barrier no longer waits for that particular thread to reach its defined synchronization point.

In the exemplary, non-limiting embodiment, the call to the _arrive function 2200 placed anywhere in a thread, and it is the position of the call to the _arrive function 2200 that defines the synchronization point within the thread. The developer and / or an optimizing compiler ensure that the number of threads that call a _arrive function 2200 (+ DMA or other suitable hardware calls), matches the expected number of arrivals programmed in the barrier.

• _arrive(BarPhase, BarPtr);
• Eingabe: BarPtr;
• Ausgabe: BarPhase

A more specific, non-limiting example:

• _arrive (BarPhase, BarPtr);
• Input: BarPtr;
• Output: BarPhase

Indicates that a thread has arrived; returns the barrier phase to be used for the wait statement.

In an exemplary embodiment, a wait instruction may be issued by the copy machine 210 can be initiated independently of any software thread of execution. This can be done by having the hardware in the MMU 212 or LSU 209 generates a fused atomic load / store instruction (LDGSTS) to the shared memory, which essentially performs a direct memory access (“DMA”) by the hardware machine to the instance of the basic element stored in the shared memory.

_wait (BarPtr, BarPhase)

As in 5C is shown, the _wait function is waiting 2300 that the arriving-waiting barrier for the intended phase is lifted. The _wait function 2300 again reads the phase of the barrier at the memory location [Ptr] and compares it ( 2304 ) with the one previously used by the _arrive function 2200 read phase. 5C thus defines an infinite loop that waits until the phase of the barrier changes before allowing the thread to continue. A thread that does the _wait function 2300 is blocked at the barrier until the barrier phase changes.

_wait (BarPtr, BarPhase) must consume the BarPhase returned by a previous call to _arrive. In one non-limiting exemplary embodiment where _arrive causes side effects, _wait must also cause those side effects. But it is much more typical that the opposite will happen - namely _wait causes side effects and _arrive must also cause these side effects.

Since in the example shown the _wait function 2300 uses a value obtained from an _arrive function 2200 returned should be _wait 2300 are only called after _arrive 2200 was called. The two functions could be called immediately one after the other, or any number of commands or other functions unrelated to the barrier could be called between calls to the _arrive function 2200 and calling the _wait function 2300 to be placed. The developer (and / or an optimizing compiler) could do useful work between a _arrive function call 2200 and a wait function call 2300 so that processor cycles are not unnecessarily wasted. When the thread executes the _wait function 2300 calls after the state of the barrier phase has already changed, the thread is not blocked by the barrier, but instead executes the next instruction after the function call with only a short (e.g. one or two cycle) delay that occurs when performing the operations 2302 , 2304 the 5C arises.

• _wait(BarPtr, BarPhase)
• Eingabe: BarPhase;

A more specific example:

• _wait (BarPtr, BarPhase)
• Input: BarPhase;

• Jedes Wait(BarPtr) weist ein entsprechendes Arrive(BarPtr) auf, und die BarPhase von Arrive(BarPtr) wird als Eingabe für _wait(BarPtr) bereitgestellt; ein Aufruf von _wait(BarPtr) kann nicht auf ein _wait(BarPtr) ohne ein dazwischenliegendes _arrive(BarPtr) folgen.

Wait for all expected arrivals for the barrier for the specified phase of the barrier to arrive.
for a thread for a barrier BarPtr:

• Each Wait (BarPtr) has a corresponding Arrive (BarPtr), and the BarPhase of Arrive (BarPtr) is provided as input to _wait (BarPtr); a call to _wait (BarPtr) cannot follow a _wait (BarPtr) without an intervening _arrive (BarPtr).

A call to _arrive (BarPtr) should not follow a _arrive (BarPtr) without an intervening _wait (BarPtr).
_dropthread

The _dropthread function 2400 in 5D is used in exemplary embodiments to permanently remove a thread from an arrive-wait barrier. This function is useful when a thread needs to be terminated. The effect of this function is to decrease (decrement) the value for the expected number of arrivals at the barrier so that the barrier is no longer waiting for that particular thread (block 2402 ). In the example shown, the operation increments the value stored in the barrier, thereby reducing the number of counts required to reach a “waiting for no more threads” count.

• _dropThread (BarPtr)
• Eingabe: BarPtr;
• Entfernt einen Thread von der Barriere. Nützlich, wenn sich ein Thread beenden möchte.
• _addonce (BarPtr, Count)

A more specific example:

• _dropThread (BarPtr)
• Input: BarPtr;
• Removes a thread from the barrier. Useful when a thread wants to end itself.
• _addonce (BarPtr, Count)

The _addonce function 2500 in 5E adds a certain value to the counter reading of the arriving-waiting-barrier, and the _droponce function 2600 in 5F reduces the count of the arriving-waiting-barrier by a certain value. All overflows can be handled explicitly by the software.

• _add(BarPtr, AddCnt)
• Eingabe: BarPtr, AddCnt;

A more specific example:

• _add (BarPtr, AddCnt)
• Input: BarPtr, AddCnt;

AddCnt adds additional expected arrivals for this barrier. It is only added for a single use of the barrier.

For all threads involved in the barrier:

The sum of all AddCnt corresponds to the number of _arrive () calls.

The thread that provides the _addCnt can be different from the thread that executes the _arrive ().

A thread should not execute _add (BarPtr) between _arrive (BarPtr) and _wait (BarPtr).

• • Spezifizierung der erwarteten Ankunftsanzahl bei der Erstellung der Barriere.
• • Spezifizierung der erwarteten Ankunftsanzahl jedes Mal, wenn die Barriere von dem Arrive verwendet wird (wie bei der bestehenden SM CTA-Barriere müssen alle Threads die gleiche erwartete Ankunftsanzahl angeben)
• • Spezifizierung der erwarteten Ankunftsanzahl jedes Mal, wenn die Barriere von Wait verwendet wird (wie bei der bestehenden SM CTA-Barriere müssen alle Threads die gleiche erwartete Ankunftsanzahl angeben)
• • hybrid: Spezifizierung der erwarteten Ankunftsanzahl bei der Erstellung der Barriere, aber zusätzliche erwartete Ankünfte können durch das Arrive hinzugefügt werden.

Other ISA approaches are possible. The main approaches differ in the way in which the expected barrier arrival count is reported. Some options include:

• • Specification of the expected number of arrivals when creating the barrier.
• • Specification of the expected number of arrivals each time the barrier is used by the arrive (as with the existing SM CTA barrier, all threads must specify the same expected number of arrivals)
• • Specification of the expected number of arrivals each time the barrier of Wait is used (as with the existing SM CTA barrier, all threads must specify the same expected number of arrivals)
• • hybrid: Specification of the expected number of arrivals when creating the barrier, but additional expected arrivals can be added by the Arrive.

• Barriere-Zustand
  • * Phase: Phase der Barriere.
  • * Zählerstand: Zählerstand der Barriere.
  • * ThreadCnt: Anzahl der an dieser Barriere beteiligten Threads

 _create(BarPtr,InitCnt)
      barrier[BarPtr].initCnt = initCnt;
      barrier[BarPtr].cnt = initCnt;
      barrier[BarPtr].phase = 0;
      _add(BarPtr, addCnt)
      // Erhöhen der Ankunftsanzahl

       barrier[BarPtr].cnt += addCnt;
       Arrive_function(BarPtr) // dies ist nicht Teil der API in einer Ausführungsform

      // Dekrementieren der Ankunftsanzahl

       barrier[BarPtr].cnt--;

      // Prüfen, ob die Ankunft die Barriere aufhebt

       if (barrier[BarPtr].cnt == 0) { 

             // Aktualisieren der Phase und Rücksetzen des Zählerstands
             if (barrier[BarPtr].phase == 0) {barrier[BarPtr].phase = 1;} else {bar-
 rier[BarPtr].phase = 0;}
             barrier[BarPtr].cnt = barrier[BarPtr].init;
             // alle auf die Barriere wartenden Warps freigeben
             }unstall(BarPtr);

 }
             Arrive(BarPtr, addCnt, BarPhase)
      // Zurückgeben der Phase (optional)
      BarPhase = barrier[BarPtr].phase;

       // Arrive-Funktion ausführen

      Arrive_function(BarPtr);
      LDGSTS_Arrive(BarPtr)
      // Arrive-Funktion ausführen
      Arrive_function(BarPtr);
      BarWait(BarPtr,BarPhase)

       // wenn sich die Barriere in der gleichen Phase befindet, ist sie noch nicht
         freigegeben worden, dann anhalten, andernfalls weitermachen.
      if (barrier[BarPtr].phase == BarPhase) { stall(BarPtr); }
      DropThread(BarPtr)
      // Thread entfernt sich selbst von der Barriere

       barrier[BarPtr].initCnt--;

      // Arrive-Funktion ausführen
      Arrive(BarPtr);

Exemplary non-limiting micro-architecture (used to implement the block diagram of the 4th can be used):

• Barrier state
  • * Phase: phase of the barrier.
  • * Meter reading: Meter reading of the barrier.
  • * ThreadCnt: Number of threads involved in this barrier

 _create (BarPtr, InitCnt)
      barrier [BarPtr] .initCnt = initCnt;
      barrier [BarPtr] .cnt = initCnt;
      barrier [BarPtr] .phase = 0;
      _add (BarPtr, addCnt)
      // Increase the number of arrivals

       barrier [BarPtr] .cnt + = addCnt;
       Arrive_function (BarPtr) // this is not part of the API in any embodiment

      // Decrement the number of arrivals

       barrier [BarPtr] .cnt--;

      // Check that the arrival lifts the barrier

       if (barrier [BarPtr] .cnt == 0) { 

             // Update the phase and reset the counter reading
             if (barrier [BarPtr] .phase == 0) {barrier [BarPtr] .phase = 1;} else {bar-
 rier [BarPtr] .phase = 0;}
             barrier [BarPtr] .cnt = barrier [BarPtr] .init;
             // release all warps waiting for the barrier
             } unstall (BarPtr);

 }
             Arrive (BarPtr, addCnt, BarPhase)
      // return the phase (optional)
      BarPhase = barrier [BarPtr] .phase;

       // Execute the Arrive function

      Arrive_function (BarPtr);
      LDGSTS_Arrive (BarPtr)
      // Execute the Arrive function
      Arrive_function (BarPtr);
      BarWait (BarPtr, BarPhase)

       // if the barrier is in the same phase, it is not yet
         released, then pause, otherwise continue.
      if (barrier [BarPtr] .phase == BarPhase) {stall (BarPtr); }
      DropThread (BarPtr)
      // thread removes itself from the barrier

       barrier [BarPtr] .initCnt--;

      // Execute the Arrive function
      Arrive (BarPtr);

Cooperative data movement without thread sync. DMA task based shared barrier

The programming model can be a model similar to that of a multi-threaded barrier, except that the barrier is split.

• • <PRE>
• • BARRIER
• • <POST>

An existing multi-thread barrier can be described as follows:

• • <PRE>
• • BARRIER
• • <POST>

• • Die <PRE>.load/stores eines Threads sind garantiert sichtbar für die <POST>-load/stores anderer teilnehmender Threads.
• • Die <POST>-load/stores eines Threads sind garantiert nicht sichtbar für die <PRE>-load/stores anderer teilnehmender Threads.

The visibility rules:

• • The <PRE> .load / stores of a thread are guaranteed to be visible to the <POST> -load / stores of other participating threads.
• • The <POST> -load / stores of a thread are guaranteed not to be visible to the <PRE> -load / stores of other participating threads.

• • <PRE>
• • ARRIVE
• • <MIDDLE>
• • WAIT
• • <POST>

The shared multi-thread barrier can be described as follows:

• • <PRE>
• • ARRIVE
• • <MIDDLE>
• • WAIT
• • <POST>

• • Die <PRE>-load/stores eines Threads sind garantiert sichtbar für die <POST>-load/stores anderer teilnehmender Threads.
• • Die <POST>-load/stores eines Threads sind garantiert nicht sichtbar für die <PRE>-load/stores anderer teilnehmender Threads.
• • Die <MIDDLE>-load/stores eines Threads haben keine Sichtbarkeitsgarantie-Reihenfolge für andere Threads (zumindest keine Sichtbarkeitsgarantie, die von dieser speziellen Barriere bereitgestellt wird).

The visibility rules (the first two are the same as above):

• • The <PRE> -load / stores of a thread are guaranteed to be visible to the <POST> -load / stores of other participating threads.
• • The <POST> -load / stores of a thread are guaranteed not to be visible to the <PRE> -load / stores of other participating threads.
• • The <MIDDLE> -load / stores of a thread have no visibility guarantee order for other threads (at least no visibility guarantee provided by this particular barrier).

The LDGSTS “DMA” instruction is logically treated like an independent thread that has been split off or “forked” from the calling thread and executes an LDG / STS / ARRIVE, after which it ends or “this”.

Split barrier visibility issues

A split barrier means that multiple split barriers can overlap. All of the following overlaps are permitted and functionally correct without deadlock (see also 6th ). Pipeline-like Thread 0 Thread 1 Arrive (BarA) Arrive (BarA) Arrive (BarB) Arrive (BarB) Wait (BarA) Wait (BarA) Wait (BarB) Wait (BarB) nested Thread 0 Thread 1 Arrive (BarA) Arrive (BarA) Arrive (BarB) Arrive (BarB) Wait (BarB) Wait (BarB) Wait (BarA) Wait (BarA) different order per thread Thread 0 Thread 1 Arrive (BarA) Arrive (BarB) Arrive (BarB) Arrive (BarA) Wait (BarA) Wait (BarB) Wait (BarB) Wait (BarA) different barrier and order per thread Thread 0 Thread 1 Thread 2 Arrive (BarA) Arrive (BarC) Arrive (BarB) Arrive (BarB) Arrive (BarA) Arrive (BarC) Wait (BarA) Wait (BarA) Wait (BarB) Wait (BarB) Wait (BarC) Wait (BarC)

In some exemplary, non-limiting embodiments, the following overlaps should not be allowed as they would create a deadlock. → deadlock Thread 0 Thread 1 Arrive (BarA) Arrive (BarB) Wait (BarA) Wait (BarB) Arrive (BarB) Arrive (BarA) Wait (BarB) Wait (BarA)

Example of a state

6th represents an example of the state. 6th first shows the change from phase 0 to phase 1 (if all arrivals take place, the barrier phase changes and the count is reset). The operation of thread 1 "Wait (BarPhase_T1, BarPtr)" ensures that all loads / stores for phase 0 are visible to all threads involved for thread 1 before continuing. Similar wait operations in phase 1 and phase 2 provide similar functionality for these phases. Every time the barrier is deployed, it is a new phase, so in exemplary embodiments the internal phase indicator only needs to be a single bit to indicate which phase a particular Wait () must wait for.

7th shows another state example, which has load / store operations carried out by hardware operators such as the copy machine 210 are executed. The "Add (BarPtr, 2)" commands executed by Thread0 and Thread1 are used to add these two copy machine operations to the barrier.

In this example, the "Wait (BarPhase_T1, BarPtr)" operation ensures that all load / store operations in phase0 are visible for all threads involved for thread1 (marked by arrives) and that all LDGSTS results are visible in the shared memory ( marked by LDGSTS-Arrive). As soon as all arrives have occurred (see lines with arrows indicating phase = 0 and then phase = 1), the barrier phase changes and the count is reset.

8th shows an example of a three-stage pipeline streaming code. The 8th shows the priming of the loop, then iterations of the loop. 9 shows an example of a loop code structure with the following details: --- URF / RF values --- URF1 / URF2: Point to the barrier to be used RF1 / RFT: Point to the SMEM memory to be used iteration UR1 / R1 UR2 / R2 0 BarA / SBUfA BarC / SBufC 1 BarB / SBufB Ba rA / SB ufB 2 BarC / SBufC BarB / SBufB ... to repeat ...

All documents cited here are incorporated by reference as if they were specifically listed.

Although the invention has been described in connection with the embodiments presently considered to be most practical and preferred, it is to be understood that the invention is not intended to be limited to the disclosed embodiments, but on the contrary is intended to include various modifications and equivalent arrangements included in the spirit and scope of the appended claims are included.

QUOTES INCLUDED IN THE DESCRIPTION

This list of the documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent literature cited

• US 9117284 [0008]
• US 10217183 [0008]
• US 8381203 [0010]
• US 9158595 [0010]
• US 9442755 [0010]
• US 9448803 [0010]
• US 10002031 [0010]
• US 10013290 [0010]

Non-patent literature cited

• Xiao et al., "Inter-Block GPU Communication via Fast Barrier Synchronization", 2010 [0009]

Claims (24)

Comprehensive synchronization barrier: a data structure stored in memory, the data structure including a counter; wherein the counter is modified by an operation performed by either an execution thread or a hardware operator. Synchronization barrier after Claim 1 wherein the data structure, which is stored in a memory, comprises a phase flag, an arrival counter and a further value which is used to reinitialize the arrival counter when the barrier is reset. Synchronization barrier after Claim 1 or 2 wherein the hardware operator comprises hardware that performs copying. Synchronization barrier according to one of the preceding claims, wherein the operation comprises an ARRIVE which differs from a WAIT and / or a WAIT which differs from an ARRIVE. A synchronization barrier according to any one of the preceding claims, wherein the data structure is structured to be reset in response to a fused atomic load / store operation which can be initiated either by a hardware machine or a software thread. Comprehensive computing system: a synchronization barrier primitive stored in memory, the primitive including a counter and a phase indicator; and a memory access circuit which is dependent on either (a) execution of an instruction by a software thread or (b) when the counter indicates that all threads in a collection of threads and at least one copy operation have reached a synchronization point and all operations have completed in the cluster, reset the counter and change the phase indicator. System according to Claim 6 wherein the counter counts a sum of the number of copy operation completions and the number of execution thread arrival calls. System according to Claim 6 or 7th , the instruction consisting of an ARRIVE operation that does not include a WAIT operation or a WAIT operation that does not include an ARRIVE operation. System according to one of the Claims 6 to 8th wherein the primitive stored in memory further comprises a predetermined value, and wherein hardware resets the counter by loading the predetermined value when the counter indicates that all threads of a collection of threads and copy operations have reached a sync point and all copy operations in the cluster are complete. System according to Claim 9 , the system being structured to allow a thread to dynamically change the predetermined value. System according to one of the Claims 6 to 10 wherein the synchronization barrier primitive is stored in shared memory of a GPU. System according to one of the Claims 6 to 11 wherein the synchronization barrier primitive is stored in a memory hierarchy that determines threaded access to the primitive. System according to one of the Claims 6 to 12th which also has a comparator which compares the count of the counter with a predetermined value and resets the basic element depending on the results of the comparison. System according to one of the Claims 6 to 13th wherein the primitive phase indicator is configured to be read first by an ARRIVE instruction and then by a WAIT instruction so that a thread can determine whether the primitive phase indicator has changed a phase state. GPU instruction set architecture comprising: an ARRIVE operation which reads at least a phase indicator portion of a synchronization barrier primitive stored in a memory and causes a counter to increment; and a WAIT operation which reads at least the phase indicator portion of the primitive stored in a memory and the phase indicator portion read by the ARRIVE operation with the phase indicator portion of the primitive which is read by the WAIT Operation is read, compares to decide whether the phase state of the barrier has changed. GPU instruction set architecture according to Claim 15 which furthermore has an ADD operation which adds to a field which is stored with the synchronization barrier primitive the field which is used to reinitialize the primitive on a reset to a next phase state. GPU instruction set architecture according to Claim 15 or 16 which also includes a CREATE statement that initializes the synchronization barrier primitive and stores it in memory. GPU instruction set architecture according to one of the Claims 15 to 17th which further includes a merged load / store instruction that enables a hardware-based machine to reset the synchronization barrier primitive when the hardware-based machine has completed an assigned task. Comprehensive synchronization procedure: Storing in memory a synchronization barrier indication comprising a phase indicator and a counter; Executing an arrive statement from at least one thread, causing the counter to count and allowing the thread to read the phase indicator; Completing a task with a hardware controller causing the counter to count; Resetting the counter when the count of the counter indicates that a set of threads has executed arrive instructions and the hardware controller has completed tasks; and Execution of a wait instruction from the thread, whereby the thread is again allowed to read the phase indicator, whereby the thread is blocked depending on whether the phase indicator has changed values. Synchronization procedure according to Claim 19 which comprises opening an execution window from when a thread executes the arrive statement to when the thread executes the wait statement, the thread performing work within the execution window that is asynchronous with respect to the synchronization barrier. Comprehensive synchronization barrier: a counter which provides a synchronization barrier count; and circuitry operably connected to the counter that modifies the synchronization barrier count in response to completion of operations performed by execution threads and hardware operators. Synchronization barrier after Claim 21 , the counter being in a memory. Synchronization barrier after Claim 21 , the counter being stored in a shared memory of a GPU. Sync barrier after one of the Claims 21 to 23 wherein the circuit is configured to reset the synchronization barrier counter reading as a function of a merged atomic load / store process which can be initiated by a copy hardware and / or a software thread execution.

Images