DE102017213160B4 - Compilation for node device GPU-based parallel processing - Google Patents

Abstract

A computer-implemented method for performing analysis routine tasks, at least partially in parallel, by means of a plurality of node devices (1500), comprising:receiving an analysis routine that contains executable instructions for one or more tasks in the form of task routines for execution by at least one central processing unit (Central Processing Unit, CPU) of a node device to be performed on at least one data set,analyzing a current status of resources of at least one node device of the plurality of node devices to determine availability of at least one graphics processing unit (GPU) of the at least one node device determine,said resources comprising at least one CPU and at least one GPU, and. if the determining indicates that at least one GPU of the at least one node device is available for assignment of a first task of the analysis routine in the form of a first task routine,analyzing the first task routine to compile into a GPU task routine for execution by the at least one GPU such that the at least one GPU can perform multiple instances of the first task of the analysis routine, at least in part, in parallel, the analyzing comprising whether the first task routine includes an instruction that prevents compilation to produce a GPU task routine, and whether inputs and outputs the first task routine have dependencies that prevent compilation to create a GPU task routine, if the analyzing indicates that the first task routine can be compiled into a GPU task routine for execution by the at least one GPU,using a conversion rule to generate at least one instruction convert the first task routine into at least one corresponding GPU task routine instruction;compiling the at least one corresponding GPU task routine instruction for execution by the at least one GPU; assigning a dataset partition of the dataset to the at least one node device to enable access to the dataset partition by the at least one GPU, andassigning execution of the GPU task routine to the at least one node device to enable execution of the multiple instances of the first task with the dataset partition enable the at least one GPU using the GPU task routine.

Description

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of priority under 35 USC §119(e) from US provisional patent application serial no. 62/289,484 , filed February 1, 2016, US provisional patent application serial no. 62/379,512 , filed August 25, 2016, and US provisional patent application serial no. 62/394,411 , filed September 14, 2016. This application is a continuation of and claims the benefit of priority under 35 USC §120. from US patent application serial no. 15/422,154 , filed concurrently herewith on February 1, 2017.

BACKGROUND

It has become increasingly common to use grids of numerous node devices to perform analysis of large data sets (e.g., commonly referred to as "big data") in a distributed manner in which analysis routine tasks are performed at least partially in parallel across selected ones of the node devices be performed. In the operation of such grids, there are often competing goals in allocating the tasks of analysis routines to the node devices, including the goal of making full and uninterrupted use of the processing resources of each node device, and the goal of using the processing and memory resources of each node device, as well as the network bandwidth resources of the to use the grid more effectively. Adding to the complexity of addressing such competing targets is the common practice of sharing grid resources by causing the tasks of multiple analysis routines to be performed concurrently by the grid's node devices. A classic approach to task assignment is to simply assign the next task to be performed to the node device that is closest with available processing resources. Unfortunately, this classic approach means a decision to allow significant inefficiencies in the use of storage and network bandwidth.

Suchard, MA et al.: Understanding GPU Programming for Statistical Computation: studies in Massively Parallel Massive Mixtures.PMCID: PMC2945379 concerns parallelization, with CPUs and GPUs being treated differently. However, routines for GPUs are programmed separately and independently.

Hong, C. et al.: MapCG: Writing Parallel Program Portable between CPU and GPU. In: Proceedings of the PACT' 10, September 11 - 15, Vienna, Austria, 2010. pp. 217 - 226 proposes a programming framework in which programs can be written regardless of whether they are later executed on a CPU or a GPU . There is no check whether a CPU task routine can be converted into a GPU task routine.

U.S. 2011/252411 A1 discloses a network of computing nodes that provide different resources such as GPUs and/or CPUs for processing programs. It does not check whether it can be compiled into GPU code, but uses other criteria such as input size/type information, size thresholds, or the degree of parallelism. Furthermore, no analysis of the availability is described.

SUMMARY

The subject of the invention is defined by claim 1. Advantageous configurations can be found in the dependent claims.

character list

• 1A und 1B veranschaulichen jeweils ein Ausführungsbeispiel eines verteilten Verarbeitungssystems.
• 2 veranschaulicht ein Ausführungsbeispiel eines Seitenaustauschs.
• 3A und 3B veranschaulichen jeweils ein Ausführungsbeispiel der Zuweisung der Durchführung von Aufgaben einer Analyseroutine an Knotenvorrichtungen eines verteilten Verarbeitungssystems.
• 4A, 4B und 4C veranschaulichen zusammen ein Ausführungsbeispiel einer Zuweisung von Datensatzpartitionen und Aufgaben.
• 5A und 5B veranschaulichen zusammen ein Beispiel einer Durchführung einer zugewiesenen Aufgabe durch ein Ausführungsbeispiel einer Knotenvorrichtung.
• 6 veranschaulicht ein weiteres Ausführungsbeispiel einer Zuweisung von Datensatzpartitionen und Aufgaben.
• 7A, 7B und 7C veranschaulichen zusammen ein Ausführungsbeispiel einer Verzögerung der Zuweisung einer Datensatzpartition und einer Aufgabe.
• 8A, 8B und 8C veranschaulichen zusammen ein weiteres Ausführungsbeispiel einer Verzögerung der Zuweisung einer Datensatzpartition und einer Aufgabe.
• 9 veranschaulicht noch ein Ausführungsbeispiel einer Analyse von Taskroutinen und Zuweisung von Aufgaben durch eine Koordinierungsvorrichtung.
• 10 veranschaulicht noch ein Ausführungsbeispiel einer Analyse und Kompilierung einer Taskroutine durch eine Knotenvorrichtung.
• 11 veranschaulicht noch ein Ausführungsbeispiel von anfänglichen und nachfolgenden Zuweisungen von Aufgaben durch eine Koordinierungsvorrichtung.
• 12 veranschaulicht noch ein Ausführungsbeispiel einer Ableitung einer Zeitperiode einer Verzögerung zur Verwendung bei einer nachfolgenden Zuweisung einer Aufgabe.
• 13 veranschaulicht noch ein Ausführungsbeispiel einer verzögerten Zuweisung einer Aufgabe durch eine Koordinierungsvorrichtung.

The present disclosure is described in connection with the accompanying figures:

• 1A and 1B each illustrate an embodiment of a distributed processing system.
• 2 illustrates an embodiment of a page swap.
• 3A and 3B 12 each illustrate an embodiment of assigning node devices of a distributed processing system to perform tasks of an analysis routine.
• 4A , 4B and 4C 10 together illustrate one embodiment of an assignment of record partitions and tasks.
• 5A and 5B 10 together illustrate an example of a performance of an assigned task by an embodiment of a node device.
• 6 Figure 12 illustrates another embodiment of record partition and task assignment.
• 7A , 7B and 7C 10 together illustrate one embodiment of deferring the allocation of a record partition and a task.
• 8A , 8B and 8C 10 together illustrate another embodiment of deferring the allocation of a record partition and a task.
• 9 FIG. 12 illustrates another embodiment of analysis of task routines and assignment of tasks by a coordinator.
• 10 Figure 13 illustrates another embodiment of analysis and compilation of a task routine by a node device.
• 11 FIG. 11 illustrates another embodiment of initial and subsequent assignments of tasks by a coordinator.
• 12 FIG. 12 illustrates another embodiment of a derivation of a time period of delay for use in a subsequent assignment of a task.
• 13 FIG. 11 illustrates another embodiment of a delayed assignment of a task by a coordinator.

DETAILED DESCRIPTION

Various embodiments described herein are generally directed to techniques to improve the efficient use of processing, memory, and network bandwidth resources within a lattice of nodal devices to enable individual analysis routines to be completed in less time. Various embodiments are also generally directed to techniques for improving ease of use of multithreading support provided by at least some processors of a grid. Each node device of a lattice of node devices may include both volatile memory, which provides faster access to data and routines, and non-volatile memory, which provides higher storage capacity but with slower access than volatile memory. Additionally, each node of the lattice may include one or more central processing units (CPUs), or may include a combination of one or more CPUs and one or more graphics processing units (GPUs), any of which may be better suited to perform various types of analysis routine tasks. The initial assignments of data set partitions and analysis routine task(s) to node devices may be based on the availability of resources within the node devices and/or on an automated analysis of the task routines within the analysis routine to identify task routines that are designed to run on GPUs instead of CPUs can be converted and compiled. Such analysis can be performed either in connection with deriving the initial assignments or as part of compiling each of the task routines for execution by CPUs and/or GPUs. Subsequently, during execution of task routines of the analysis routine, when a node device becomes available so that it could be assigned a next task to perform on a particular dataset partition, such assignment to the one node device may be delayed by a predetermined period of time to another node device that is still performing a previous task on that particular dataset partition an opportunity to become available and assigned to perform that next task on that particular dataset partition. Such a delay in allocation may allow the temporary storage of the particular dataset partition within the volatile memory of the other node device to be utilized, thereby enabling the next task to be performed more quickly with the particular dataset partition. Such a delay in allocation may also avoid encountering a potentially greater delay associated with the transfer of the particular data set partition to the one node device.

A coordinator of a lattice of nodal devices may recurrently receive node data from each of the nodes of the nodal device lattice, updated periodically Displaying the level of availability of various processing, storage and/or network access resources provided in each. The coordinator may also receive an analysis routine comprising executable instructions for multiple task routines for multiple tasks to be performed on at least one dataset and specifying an order in which the tasks are to be performed. The coordinating device may further receive metadata indicative of various structural features of at least the one dataset. From the node data, the metadata and/or the analysis routine, the coordinator may derive initial assignments of dataset partitions of the at least one dataset to selected ones of the node devices and initial assignments of the task(s) to be performed by the selected node devices. The coordinating device can then transmit indications of the initial assignments to the selected node devices. As part of such initial assignments, multiple dataset partitions from at least the one dataset may be distributed among the selected node devices to allow the selected node devices to perform the same tasks, at least in part, in parallel with their respective dataset partitions. The coordinating device may additionally transmit to either a single storage device or a grid of storage devices storing the dataset indications of the distribution of dataset partitions to be made to the selected node devices as part of the initial assignments to allow the Start performing the initial task(s). In some embodiments, each of the selected node devices can retrieve one or more of the data set partitions from the one or more storage devices. In other embodiments, one or more of the data set partitions may be transferred to each of the selected node devices through the one or more of the storage devices.

With respect to the processing resources through which the coordinating device may receive node data recurrently, each of the node devices may integrate one or more GPUs in addition to or instead of integrating one or more CPUs. The one or more CPUs may use an internal processing architecture found to be well suited for serializing task routines that involve various input/output operations and/or branch operations that necessitate the execution of various sets of instructions on the results of various determinations . The one or more CPUs may each include one or more processing cores, each capable of supporting a relatively limited degree of parallel execution of instructions on a relatively limited number of threads of execution. In contrast, the one or more GPUs may employ an internal processing architecture that has been found to be well suited for the parallel processing of task routines that involve a relatively limited variety of computations and/or bitwise operations. In some embodiments, the one or more GPUs may be able to support the parallel processing of a relatively large set of instances of a task across a relatively large number of threads of execution when there are no dependencies between the instances of the task (sometimes referred to as referred to as “highly parallel”). Indeed, for a relatively limited variety of tasks, a single GPU within a single node device may be able to perform the same work as the CPUs of numerous separate node devices, but faster and less expensively. Thus, it may be advantageous to have at least a subset of node devices that integrate one or more GPUs capable of performing such a limited variety of tasks with such an increase in speed, and it may be advantageous to be able to be to automatically identify tasks in analysis routines that are of such a limited variety.

In embodiments of a node device lattice in which some but not all of the node devices integrate such GPUs, if an analysis routine comprises task routines for one or more tasks, . amenable to being executed by GPUs as highly parallel tasks faster than CPUs running . Arranging device for determining whether to prioritize the assignment of node devices that integrate GPUs or node devices that do not integrate GPUs to perform the tasks of the analysis routine. The coordinator may analyze the task routines of an analysis routine to identify tasks that are implemented with instructions and that operate on data in a manner that avoids dependencies that cause those tasks to be amenable to execution as highly parallel tasks via a large number of threads to be compiled by one or more GPUs. If no such task routine is found by such analysis, then the entirety of the analysis routine can be compiled for execution by CPUs only. Also, priority can be given to the assignment of analysis routine tasks to be performed by node devices that do not Integrating GPUs and/or integrating one or more CPUs and one or more GPUs. Such prioritization can be effected to leave as many of the nodes that integrate one or more GPUs available as possible to have tasks assigned to another analysis routine in which the instructions and/or interaction with data in one or more Task routines are amenable to being compiled for execution as highly parallel tasks by GPUs.

It should be noted, however, that although the results of analyzing the task routines of an analysis routine may have some impact on which node devices are selected for assignment of tasks, in some embodiments, indications of what processing resources are available among the available node devices may have some impact on whether the task routines are analyzed and/or compiled for execution by GPUs. In particular, in embodiments where none of the node devices that integrate GPUs are currently available to be assigned to any task (e.g., all node devices that integrate GPUs are assigned to perform tasks of a different and entirely unrelated analysis routine) , the coordinator may refrain from performing any analysis of the task routines to determine whether any of the task routines are amenable to being compiled for execution by a GPU since no GPUs are currently available to do so.

When analyzing the instructions within task routines, such analysis cannot at least prevent comparisons of instructions for each task routine with a list of instructions that are each known to prevent their corresponding tasks from being performed as highly parallel tasks by a GPU , and/or with a list of instructions each known to result in a task incapable of being performed as a highly parallel task by a GPU. Additionally, if a task routine does not include instructions that render its corresponding task incapable of being performed as a highly parallel task by a GPU, the manner in which the instructions are used within that task routine can be analyzed to determine whether the manner in which any instructions are used renders the corresponding task incapable of being performed as a highly parallel task by a GPU. Illustratively, if such instructions are used to perform operations on data in a way that would create dependencies between instances of a task routine such that those instances could not truly be performed in parallel, then the task routine's task may not be amenable to being a highly parallel task to be performed. If it is determined that a task routine's instructions and the manner in which those instructions are used do not prevent the corresponding task from being performed as a highly parallel task by a GPU, then compiling the task routine's instructions for such execution by one or more GPUs may be the default approach. If the node device lattice includes node devices that integrate different GPUs that do not share an instruction set, compilation of the task routine for such execution by one or more GPUs may result in multiple compilations of the task routine in support of each of the different GPUs.

In some embodiments, compiling a task routine to perform a task determined to be amenable to being performed as a highly parallel task by a GPU may involve converting instructions of the task routine that were not generated to a causing such highly parallel performance of the task by a GPU result in instructions generated to cause such performance of the task. A compiler, to perform such a conversion, may use a set of compilation rules, each associated with one or more particular instructions that may be present among the instructions of the task routine, and causing the one or more particular instructions to be converted into one or several other instructions that cause highly parallel execution by a GPU. Among such compilation rules may be rules that each cause a particular type of loop of instructions to be converted into another type of loop that is more amenable to highly parallel execution by a GPU (e.g., a loop with a reduced number of iterations ), and/or which cause the replacement of a particular type of loop of instructions with one or more other instructions for execution by a GPU in a highly parallel manner that does not entail the use of a loop. There may also be a set of compilation rules, each associated with a particular type of data structure that may be instantiated by the task routine's instructions or otherwise used, and that cause the data structure to be inverted into another data structure which is more amenable to use in highly parallel execution by a GPU.

By using such transformations of instructions of one or more task routines, the personnel writing the instructions of the task routines of an analysis routine can be given the opportunity to take advantage of the highly parallel processing capabilities of the one or more GPUs integrated in at least some of the node devices. without the need to write task routine instructions specifically for highly parallel execution by GPUs. In other words, these personnel may be spared the need to acquire the skills to architect and write the instructions that implement the tasks of an analysis routine in a manner designed for highly parallel execution by GPUs. Additionally, when the node device lattice includes node devices that integrate different GPUs that do not share an instruction set, such personnel are further spared the need to architect and write different versions of the task routines' instructions to accommodate the different idiosyncrasies of highly parallel execution by each of the to address different GPUs.

With respect to the memory resources through which the coordinating device may receive node data on a recurring basis, each of the node devices may incorporate memory capabilities implemented as a combination of volatile and non-volatile memory. The volatile memory may be implemented with one or more memory components that employ a memory technology that allows relatively fast access to data and/or routines, but is incapable of reading data and/or routines stored therein without a continuous supply of electrical power to keep. Such technologies include, but are not limited to, any of a variety of types of random access memory (RAM). The non-volatile memory may be implemented with one or more memory components that employ a memory technology capable of retaining data and/or routines stored therein, regardless of whether electrical power continues to be provided but not in the Capable of providing access that is as fast as that provided by various volatile memory technologies on which the volatile memory may be based. Such non-volatile memory technologies include, but are not limited to, the use of any of a variety of ferromagnetic and/or optical storage media.

Due to the faster access provided by volatile memory compared to non-volatile memory, instructions being executed by the one or more CPUs and/or the one or more GPUs integrated into each node device may be able to , stored within volatile memory where they can be read, written, and/or modified more quickly. However, due to the often lower cost and/or higher storage densities of the non-volatile memory components compared to the volatile memory components, the non-volatile memory can be implemented with a higher storage capacity than the volatile memory within each of the node devices. Although there may be data sets that are small enough and/or distributed across a large enough number of node devices to cause each data set partition of the data set distributed to a node device to be small enough to be entirely within volatile Being memory storable is intended that the data set partitions of the majority of data sets are each rather too large to do so.

As a result, within each node device, portions of routines being executed and/or data accessed by the one or more CPUs and/or the one or more GPUs may be swapped from non-volatile memory to volatile memory . As may be known to those skilled in the art, any of a variety of algorithms may be used to select portions of routines and/or data to be swapped into volatile memory and/or to select pages to be within volatile memory to be held while others are swapped back to non-volatile memory, including but not limited to any of a variety of demand-based and/or predictive algorithms. In one or more embodiments, one or more of the node devices may execute an operating system (OS) that includes a paging component that performs such swapping of uniformly sized areas of routines and/or data. Depending on various factors, such as the types of operations being performed, the frequency of accessing different areas of routines and/or data, and/or the number of routines being executed in parallel, a page of routine and/or data may be different retained within volatile memory for a longer or shorter period of time before being swapped back to non-volatile memory to free up space within volatile memory for another page of a routine and/or by release data. Thus, storage of portions of routines and/or data within volatile memory within each of the node devices may be time limited.

In node devices that integrate both one or more CPUs and one or more GPUs, there may be volatile memory for the one or more CPUs and another volatile memory for the one or more GPUs. However, there may be a single non-volatile memory, and portions of routines and/or data may be exchanged between the single non-volatile memory and either of the two volatile memories. In some embodiments, the operation of the one or more GPUs may be at least partially controlled by the one or more CPUs such that the one or more GPUs may not operate fully autonomously from the one or more CPUs. In such embodiments, the volatile memory associated with the one or more GPUs may also be accessible to the one or more CPUs, and a memory page management routine executed by the one or more CPUs to facilitate swapping of areas of Performing routines and/or data for the one or more CPUs may also perform such swapping of portions of routines and/or data for the one or more GPUs. As may be known to those skilled in the art, such swapping by the one or more CPUs on behalf of the one or more GPUs may occur due to the need for one or more driver routines to be executed by the one or more CPUs in order to access enable the non-volatile storage and/or utilize a file system used in storing data and/or routines as files with the non-volatile storage. Thus, regardless of the precise manner in which each of the selected node devices is provided with a record partition, such a received record partition may initially be stored entirely within non-volatile memory within each node device. After such receipt and storage, pages of the received dataset partition may then be swapped into volatile memory of the one or more CPUs and/or the one or more GPUs as needed to support performance of one or more analysis routine tasks with the dataset partition.

With respect to the network access resources over which the coordinator may receive node data recurrently, each of the node devices may integrate a network interface to a network used by the node device mesh to enable the node devices to communicate with each other, init, the coordinator, and/or one or more storage devices (e.g. a storage device grid). The task routines executed by the CPU(s) and/or the GPU(s) to perform analysis routine tasks may be distributed to node devices across the network by the coordinating device. Also, the data set partitions on which the tasks are performed may be transferred from the one or more storage devices to the node devices over the network, and data set partitions derived within the node may be transferred back to the one or more storage devices. As may be known to those skilled in the art, in some embodiments, the one or more GPUs of a node device may not be able to directly operate the network interface of the node device to effectuate an exchange of routines and/or data associated with a performance a task associated with the one or more GPUs. Instead, in a manner similar to swapping pages associated with the one or more GPUs between volatile and non-volatile memory, the one or more CPUs of the node device may thus operate the network interface on behalf of the one or more GPUs. As noted above, such action on behalf of the one or more GPUs by the one or more CPUs may be dictated by the need for one or more driver routines to be executed by the one or more CPUs to allow access to the network interface.

Just as accessing routines and/or data stored within non-volatile memory of a node device may be significantly slower than accessing routines and/or data stored within volatile memory, accessing routines and/or data stored within another device may be significantly slower over a network. Additionally, in some embodiments, accessing routines and/or data stored either within non-volatile memory or within another device over the network may be even slower for the one or more GPUs due to their dependency on the one or more multiple CPUs of the node device to take action to enable such access on behalf of the one or more GPUs. Thus, it may be deemed desirable, whenever possible, to maximize accesses made to routines and/or data while still being stored within volatile memory associated with the CPU(s) and/or GPU(s) making these accesses and to minimize accesses to routines and/or data while stored in non-volatile memory and/or within other devices such that access must be over the network. This may result in some node devices of the node device lattice becoming idle for varying periods of time to await the availability of certain node devices for use in performing specific tasks with specific data set partitions, rather than immediately assigning tasks to each node that is available for use in performing a task regardless of which node devices already have particular data set partitions within their memories. In other words, the assignment of a next task may be delayed for a period of time to allow a particular node device storing a particular data set partition to become available again to be assigned a next task that uses the particular data set partition instead of immediately assigning the next task to another node device to which the particular data set partition would need to be transferred. Additionally, if there are multiple tasks to be performed on a particular partition that can be performed using a GPU, delaying the assignment of the next of those multiple tasks may allow for the opportunity to assign that next of those tasks to a node device that integrates a GPU and that already stores the respective dataset partition, avoid a situation where immediate assignment of the next task to the next available node device may result in assignment to a node device that does not integrate a GPU, thus resulting in slower performance the next of these tasks.

Thus, for example, if a first node device that was engaged in performing a first task of analysis on a first dataset partition of a dataset completes its performance of the first task on the first dataset partition and thereby becomes available so that it could be assigned to a to perform the second task of the analysis with a second dataset partition of the dataset, the coordinating device may refrain from assigning the first node device to perform the second task with the second dataset partition for a predetermined period of time. The predetermined time period may be selected to provide an opportunity for a second node device that is still engaged in performing the first task with the second dataset partition to complete its performance of the first task with the second dataset partition such that the second node device in is able to perform the second task with the second data set partition. However, if the second node device becomes unavailable to be assigned to perform the second task with the second dataset partition, then another node device in which the second dataset partition is not already stored may be assigned to do so (e.g., the first knot device). Alternatively, in other embodiments, if there is yet another node device in which the second dataset partition was also stored in preparation for using that other node device as a backup in response to failure of a node device performing a task with the second dataset partition, and if the second node device becomes unavailable to be assigned to perform the second task with the second dataset partition and if such another node device is currently available to be assigned, such other node devices are so assigned.

In this way, use can be made of the fact that the second data set partition is already stored within the volatile and/or non-volatile memories of the second node device, so that the second data set partition does not have to be exchanged between devices in order to perform the second task with the enable second data set partition by another node device; In other words, had the first node device been assigned to perform the second task with the second data set partition, then the second data set partition would have had to be transferred to the first node device either from the second node device or from the one or more storage devices. In addition to consuming the available bandwidth of the network and network interface of at least the first node device, the first node device's performance of the second task with the second dataset partition would necessarily be delayed until at least sufficient of the second dataset partition was received by the first node device to allow execution to begin.

As another example, each of a first node device engaged in performing a first task of an analysis routine to generate a first dataset partition of a dataset and a second node device engaged in performing the first task to generate a second dataset partition of the record to generate both their implementations of the first complete task. However, while the first node device may be available to be assigned another task, the second node device (as a result of freeing node devices among multiple unrelated analyses) may be busy performing a task of an unrelated analysis routine such that the second node device may not yet be available to be assigned to perform a second task of the analysis routine on the second dataset partition that the second node device created itself. As previously mentioned, the coordinating device may refrain from assigning the first node device to perform the second task with the second dataset partition for a predetermined period of time. The predetermined time period may be selected to provide an opportunity for the second node device to complete its performance of the unrelated analysis routine's task so that the second node device becomes available to be assigned to perform the second task with the second data set partition. As already mentioned, in this way use can be made of the fact that the second data set partition is already stored within the volatile and/or non-volatile memories of the second node device, so that the second data set partition does not have to be exchanged between devices in order to carry out the to enable a second task with the second data set partition by another node device.

With general reference to the notations and nomenclature used herein, portions of the following detailed description may be presented in terms of program procedures executed by a processor of one or more networked machines. These procedural descriptions and representations are used by those skilled in the art to most effectively convey the substance of their work to others skilled in the art. A procedure is conceived here, and generally, as a self-consistent sequence of operations leading to a desired result. These operations are those that require physical manipulations of physical quantities. Usually, though not necessarily, these quantities take the form of electrical, magnetic, or optical communications capable of being stored, transmitted, combined, compared, and otherwise manipulated. It proves convenient at times, principally for reasons of common usage, to refer to what is being communicated as bits, values, elements, symbols, characters, terms, numbers, or the like. However, it should be noted that all of these and similar terms are associated with the corresponding physical quantities and are merely convenience labels applied to those quantities.

Furthermore, these manipulations are often referred to in terms, such as adding or comparing, that are commonly associated with mental operations performed by a human operator. However, no such skill of a human operator is necessary, or in most cases desirable, in any of the operations described herein which form part of one or more embodiments. Instead, these operations are machine operations. Useful machines for performing operations of various embodiments include machines that are selectively activated or configured by a routine stored in that written according to the teachings herein, and/or include devices specifically designed for the required purpose are constructed. Various embodiments also relate to devices or systems for performing these operations. These devices may be specially designed for the required purpose or may include a general purpose computer. The required structure for a variety of these machines will appear from the description given.

Reference is now made to the drawings, in which like reference numbers are used to refer to like elements throughout. In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding thereof. However, it may be apparent that the novel embodiments may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to facilitate a description thereof. The intent is to cover all modifications, equivalents, and alternatives within the scope of the claims.

The systems illustrated in some of the figures may be provided in various configurations. In some embodiments, the systems may be configured as a distributed system, where one or more components of the system are distributed across one or more networks in a cloud computing system and/or a fog computing system.

1A 11 illustrates a block diagram of one embodiment of a distributed processing system 1000 including one or more storage devices 1100 comprising a storage device lattice 1001, a coordinating device 1300, multiple node devices 1500 that can form a node device grid 1005, and/or a viewing device 1800 integrated. 1B 12 illustrates a block diagram of an alternative embodiment of the distributed processing system 1000 in which an alternative embodiment of the node devices 1500 integrates features of the one or more storage devices 1100 and/or performs functions thereof. In both of these embodiments of the distributed processing system 1000, and as will be explained in more detail, the node devices 1500 may operate together as the grid 1005 under the control of the coordinating device 1300, with each of the plurality of the node devices 1500 performing the same task at least partially in parallel with another of multiple dataset partitions 1131 of a dataset 1130 distributed among the multiple node devices 1500.

As illustrated, these devices 1100, 1300, 1500, and 1800 may exchange communications therebetween in connection with assigning and performing tasks of an analysis routine 1210 on one or more data sets 1130. Such communications may include the exchange of node states 1535, data set partitions 1131, and/or metadata 1135 of a data set 1130, analysis routine 1210 and/or task routines 1211 thereof, CPU task routines 1571, GPU task routines 1671, and/or result data 1830. However, one or more of devices 1100, 1300, 1500, and/or 1800 may also exchange other data over network 1999 that is in no way related to any assignment or performance of tasks of any analysis routine. In various embodiments, network 1999 may be a single network that may extend within a single building or other relatively limited area, a combination of connected networks that may extend over a significant distance, and/or may include the Internet. Thus, the network 1999 may be based on any variety (or combination) of communication technologies through which communications may be effected, including, without limitation, wired technologies using electrically and/or optically conductive cabling and wireless technologies using infrared , radio frequency (RF) or other forms of wireless transmission.

With reference to 1A For example, in various embodiments, each of the one or more storage devices 1100 may integrate one or more processors 1150, memory 1160, and a network interface 1190 to couple each of the one or more storage devices 1100 to the network 1999. Storage 1160 may include control routine 1140, one or more analysis routines 1210, each of which may integrate one or more task routines 1211, one or more datasets 1330, each of which may integrate metadata 1135, and/or one or more dataset partitions 1131 of the one or more Store records 1130. The control routine 1140 may incorporate a sequence of instructions operable on the processor 1150 of each of the one or more storage devices 1100 to implement logic to perform various functions. The processor 1150 of each of the storage devices 1100 may operate the network interface 1190 to exchange the analysis routine 1210 and/or one or more of the task routines 1211 thereof with the coordination device 1300 . Alternatively or additionally, the processor 1150 of each of the storage devices may operate the network interface 1190 to coordinate and effectuate an exchange of one or more data set partitions 1131 with one or more of the node devices 1500 via the network 1999 with the coordinating device 1300. In embodiments in which multiple of the storage devices 1100 operate together as the storage device lattice 1001, the sequence of instructions of the control routine 1140 on the processor 1150 of each of those storage devices 1100 may be operable to perform various functions, at least in part, in parallel with the processors 1150 of others memory devices 1100 to perform.

In some embodiments, the processors 1150 of the storage devices 1100 may cooperate to perform a collection function, in which each of the processors 1150 operates a corresponding one of the network interfaces 1190 to receive data items of one or more of the data sets 1130 over the network 1999, and may transmit the data items over assemble a time period into the one or more data sets 1130 . In such embodiments, data elements of a data set 1130 may be received over the network 1999 and/or otherwise from one or more other devices (not shown). By way of example, a variety of remotely located sensor devices (e.g. geological sensors distributed over a particular geological region or particle detection sensors located at different parts of a particle accelerator) may generate numerous data items which are then transmitted over the network 1999 to the storage devices 1100 where the various data elements are then assembled to form a data set 1130 to train. In other embodiments, storage devices 1100 may receive one or more of data sets 1130 from a variety of other devices (not shown), such as a grid of other node devices. Illustratively, such other devices may perform one or more processing operations that generate a data set 1130 (e.g., using Bayesian analysis to derive a prediction of the behavior of humans in a simulation of evacuating a burning building or to predict the behavior of structural components, of a bridge in response to various wind currents) and may then transmit a data set 1130 as an output to the storage device grid 1001.

Each of the one or more data sets 1130 may include any of a variety of types of data associated with any of a variety of objects. Illustratively, each of the datasets 1130 may include scientific observational data related to geological and/or meteorological events or from sensors used in laboratory experiments in fields such as chemistry or physics. In another example, the dataset may include indications of activities performed by a random sample of individuals from a population of humans in a selected country or community or population of endangered species being studied in the wild. As illustrated, each of the datasets 1130 may incorporate metadata 1135 that provide indications of structural features, including, but not limited to, aspects of the manner in which data items are organized and/or made accessible within each dataset 1130.

The tasks that the task routines 1211 of the analysis routine 1210 may cause one or more processors to perform may include any of a variety of data analysis tasks, data transformation tasks, and/or data normalization tasks. The data analysis tasks may include, but are not limited to, search and/or statistical analysis involving the derivation of approximations, numerical characterizations, models, evaluations of hypotheses, and/or predictions (e.g., a prediction by Bayesian analysis of crowd actions, trying to escape from a burning building, or the behavior of bridge components in response to a wind force). The data transformation tasks may include, but are not limited to, sorting, row and/or column based mathematical operations, row and/or column based filtering using one or more data items of a row or column, and/or reordering of data items within a data object. The data normalization tasks may include normalizing times of day, dates, monetary values (e.g. normalizing to a single currency unit), character spacing, use of delimiters (e.g. normalizing use of periods and commas in numeric values), use of formatting codes, Use of big or little endian encoding, use or lack of use of character bits, sets of bits used to represent integers and/or floating point values (e.g., bytes, words, double words or quad words), etc . lock in.

In various embodiments, the coordinating device 1300 may integrate one or more of a processor 1350, a memory 1360, an input device 1320, a display 1380, and a network interface 1390 to couple the coordinating device 1300 to the network 1999. The memory 1360 can be a control routine 1340, the metadata 1135 of a data record 1130, the analysiseroutine 1210, knot status 1535, allocation data 1330, tasks delay data 1335, a compilation routine 1440, compiling rules 1434, the CPU-Taskroutinen 1571, the GPU-Taskroutinen 1671 and// Save result data 1830. Control routine 1340 and compiler routine 1440 (if present within memory 1360) may each incorporate a sequence of instructions operable on processor 1350 to implement logic to perform various functions.

In various embodiments, each of the node devices 1500 may integrate one or more of a CPU 1550, a memory 1560, and a network interface 1590 to couple each of the node devices 1500 to the network 1999. Memory 1560 may store control routine 1540, one or more record partitions 1131, instance of node status 1535, compile routine 1440, compile rules 1434, one or more task routines 1211, and/or one or more CPU task routines 1571. The control routine 1540 and the compilation routine 1440 (if present within the memory 1560) may integrate a sequence of instructions operable on the CPU 1550 of each of the node devices 1500 to implement logic to perform various functions. In embodiments where multiple of the node devices 1500 operate together as the node device grid 1005, the sequence of instructions of the control rou Tine 1540 on the CPU 1550 of each of those node devices 1500 may be operable to perform various functions, at least in part, in parallel with the CPUs 1550 of other ones of the node devices 1500.

At least a subset of the node devices 1500 may additionally integrate a graphics controller 1600, which may integrate one or more of a GPU 1650 and memory 1660. Memory 1660 may store control routine 1640, one or more dataset partitions 1131, and/or one or more GPU task routines 1671. The control routine 1640 may integrate a sequence of instructions operable on the GPU 1650 of each of the node devices 1600 that integrates the graphics controller 1600 to implement logic to perform various functions. In embodiments where multiple of the node devices 1600 operate together as the node device grid 1005, the sequence of instructions of the control routine 1640 on the GPU 1650 of the graphics controller 1600 of each of those node devices 1500 may be operable to perform various functions, at least in part, in parallel with the GPUs 1650 of the graphics controller 1600 from other ones of the node devices 1500 to perform.

As in 2 As illustrated, memory 1560 may be divided into volatile memory 1561 and non-volatile memory 1563, and memory 1660 may include volatile memory 1661 but may not include non-volatile memory. Volatile memory 1561 and 1661 may be implemented with one or more volatile memory components 1562 and 1662, respectively. Volatile storage components 1562 and 1662 may each employ any of a variety of storage technologies that allow relatively fast access to data and/or routines stored therein (e.g., the illustrated example data set partition 1131), but that are not capable of retain data and/or routines stored therein without a continuous supply of electrical power. Such technologies include, but are not limited to, any of a variety of types of random access memory (RAM). The non-volatile memory 1563 may be implemented with one or more non-volatile memory components 1564 . The one or more non-volatile memory components 1564 may each employ a memory technology capable of storing data and/or routines stored therein, regardless of whether electrical power continues to be provided but is unable to access provide data and/or routines as fast as that provided by various volatile memory technologies on which volatile memory 1561 and/or 1661 may be based. Such technologies include, but are not limited to, any of a variety of technologies that employ ferromagnetic and/or optical storage media.

Due to differences in their respective technologies, the 1563 non-volatile memory may have a significantly larger storage capacity than either the 1561 or 1661 volatile memory. Thus, pages of data and/or routines stored within the 1563 non-volatile memory may be in each of the volatile memories 1561 and 1661 can be swapped in and out as a mechanism to enable the CPU 1550 and the GPU 1650 to take advantage of the faster access to the respective volatile memories 1561 and 1661 while overcoming the more limited memory capacities of each become. In particular, and as also illustrated, execution of a page component 1541 of control routine 1540 may cause CPU 1550 to effect such page swapping to support both its own operation and the operation of GPU 1650. The need for the CPU 1550 to perform a page swap on behalf of the GPU 1650 may result from the use of the one or more driver routines (not shown) executed by the CPU 1550 to allow the CPU 1550 to to access the one or more non-volatile memory components 1564.

Referring back to 1A In various embodiments, viewing device 1800 integrates one or more of a processor 1850, memory 1860, input device 1820, display 1880, and network interface 1890 to couple viewing device 1800 to network 1999. Memory 1860 may store one or more of control routine 1840, analysis routine 1210, and result data 1830. Control routine 1840 may incorporate a sequence of instructions operable on processor 1850 to implement logic to perform various functions. Processor 1850 may be caused by its execution of control routine 1840 to operate network interface 1890 to receive result data 1830 from one of node devices 1500 or from coordinating device 1300 over network 1999 upon completion of execution of analysis routine 1210. In some embodiments, the processor 1850 can also be caused to perform a visualization based on the result data 1830 generate to present on the display 1880 a representation of the results of the performance by a plurality of the node devices 1500 of the tasks of the analysis routine 1210's task routines 1211 .

Alternatively or additionally, processor 1850 may be caused by its execution of control routine 1840 to operate display 1880 and/or input device 1820 to provide a user interface through which an operator of viewing device 1800 may provide input thereto. This input may include a command to execute the analysis routine 1210 across a plurality of the node devices 1500 of the node device lattice 1005 to perform an analysis on at least one of the data sets 1130 stored by the one or more storage devices 1100 . In response to receiving the input command, the processor 1850 may be caused to operate the network interface 1890 to transmit the command and/or the analysis routine 1210 to the coordinator 1300 over the network 1999.

With reference to 1B The alternative embodiment of the distributed processing system 1000 illustrated therein differs from the embodiment 1A , by not including the one or more storage devices 1100. Instead, the node devices 1500 of the alternate embodiment may 1B directly perform the function of storing the one or more data sets 1130, eliminating the need for the one or more storage devices 1100 of the embodiment of the distributed processing system 1000 1A is excluded.

Referring to both embodiments of the two 1A and 1B It should be noted that in some embodiments, the functions performed by the coordination device 1300 may be performed by one of the node devices 1500 instead of the coordination device 1300 (e.g., embodiments that do not include the coordination device 1300). In such embodiments, such one of the nodal devices 1500 may additionally receive the metadata 1135 of one of the datasets 1130 from one of the storage devices 1100 (or from one of the other nodal devices 1500 in embodiments where the nodal devices 1500 perform the storage function of the one or more storage devices 1100). Moreover, such one of the node devices 1500 can additionally receive the node statuses 1535 from other ones of the node devices 1500 . Furthermore, such one of the node devices 1300 can additionally transmit the task routines 1211, the CPU task routines 1571 and/or the GPU task routines 1671 to other of the node devices 1500.

3A FIG. 1 illustrates an embodiment of the assignment of tasks and compilation of task routines performed by the scheduler 1300 in one of the embodiments of the distributed processing system 1000 of FIG 1A or 1B can be carried out. 3B 15 illustrates an alternative embodiment of the assignment of tasks that may be performed by the coordinating device, while the compilation of task routines may be performed by the node devices 1500 selected to be assigned the tasks in one of the embodiments of the distributed processing system 1000 one of 1A or 1B to perform.

With reference to 3A When executing the control routine 1340, the processor 1350 of the coordinating device 1300 can be caused to receive metadata 1135 indicative of structural features of one of the data sets 1130 and/or the analysis routine 1210 from the one or more storage devices 1100 and/or the viewing device 1800. The processor 1350 can also be caused to receive from the node devices 1500 the node statuses 1535 that are indicative of the processing, storage and/or network access resources that are integrated into each of the node devices 1500 and the extent to which each of these resources is currently available. The processor 1350 may use the metadata 1135, the analysis routine 1210 and/or the node states 1535 to derive initial assignments of at least one initial task of the analysis routine 1210 to selected ones of the node devices 1500 and an initial distribution of the data set partitions 1131 to the selected ones of the node devices 1500.

When executing the compilation routine 1440, the processor 1350 can be caused to analyze the executable instructions within each of the task routines 1211 of the analysis routine 1210 to identify those of the task routines 1211 that can be compiled for highly parallel execution by the GPUs 1650 that in at least a subset of the node devices 1500 may be integrated. Those of the Task Routines 1211 that can be so compiled for the GPUs 1650 can are compiled by the processor 1350 into corresponding GPU task routines 1671 that can be executed by the GPUs 1650. However, those of the task routines 1211 that cannot be so compiled for the GPUs 1650 can be compiled by the processor 1350 into corresponding CPU task routines 1571 that can be executed by the CPUs 1550 of the node devices 1500. The processor 1350 may then be caused to distribute the one or more CPU task routines 1571 and/or the one or more GPU task routines 1671 for the at least one initial task to the selected ones of the node devices 1500 as part of the assignment of the at least one initial task .

In executing the control routine 1540, the CPU 1550 may receive and store from each of the selected ones of the node devices 1500 the one or more CPU task routines 1571 and/or the one or more GPU task routines 1671 for the at least one initial task, as well as at least one of the initial distributed Record Partitions 1131. When a CPU Task Routine 1571 is received, the CPU 1550 may be caused to execute the CPU Task Routine 1571 to perform a task on the at least one Record Partition 1131 received. When a GPU task routine 1671 is received, the CPU 1550 can be caused to forward the GPU task routine 1671 to the GPU 1650 within the graphics controller 1600 of the node device 1500 together with the at least one received data set partition 1131, the GPU 1650 being executed by the control routine 1640 can be caused to execute the GPU task routine 1671 to perform a task with the at least one received dataset partition 1131.

Regardless of whether the CPU 1550 or a GPU 1650 of each node device 1500 performs a task with a corresponding data set partition 1131, the CPU 1550 and/or the GPU 1650 can be caused to recurrently update a corresponding one of the node statuses 1535 with indications of which task (n) is/are currently being performed and/or the extent to which various resources are currently available as a result. The CPU 1550 of each node device 1500 can be caused by further execution of the control routine 1540 to transmit the corresponding node status 1535 to the coordination device 1300 recurrently.

In another execution of the control routine 1340, the processor 1350 of the coordinating device 1300 may use the indications in the recurrently received node status 1535 of the tasks currently being performed and/or the current levels of availability of various resources within each node device 1500 to determine the time required is to complete various tasks. Processor 1350 may store indications of such lengths of time required for each task as part of task delay data 1335 . When one or more of the nodal devices 1500 complete previously assigned tasks, the processor 1350 may be caused to utilize such stored indications of time durations to determine a predetermined period of time by which to delay the assignment of one or more next tasks to one or more of the nodal devices 1500 shall be.

Such delays in the assignment of the next tasks may allow the assignment of these next tasks to those of the node devices 1500 in a manner that utilizes certain data set partitions 1131 already stored within the memories 1560 and/or 1660 of one or more of the node devices 1500. Further, this may take advantage of the temporary storage of record partitions 1131 within volatile memory 1561 and/or 1661 and may minimize the exchange of record partitions 1131 through the network 1999, which may consume longer periods of time than the lengths of the delays in the assignment of the next tasks. Furthermore, in embodiments where certain data set partitions 1131 were originally allocated at node devices that integrate one or more GPUs 1650, where both the previously allocated tasks and the next tasks are to be performed using the one or more GPUs 1650, such delays may occur the assignment of next tasks minimize instances where the next tasks are caused to be processed in other node devices that do not integrate GPU 1650 such that one or more CPUs 1550 are caused to perform the next tasks.

With reference to 3B For example, the assignment of tasks to selected ones of the node devices 1500 is essentially the same as the assignment of tasks embodiment 3A similarly, except that the compilation of the task routines 1211 of the analysis routine 1210 may occur within the selected node devices 1500 rather than within the coordinating device 1300. In particular, in deriving initial assignments from at least one initial on Given the analysis routine 1210 to the selected node devices 1500, the processor 1350 of the coordinator 1300 distributes the task routine(s) 1211 of the at least one initial task to the selected node devices 1500 instead of CPU 1571 or GPU 1671 task routines. The CPU 1550 of each of the selected nodes 1500 may execute the compilation routine 1440 to compile the distributed task routine(s) 1211 to generate the corresponding CPU task routine(s) 1571 and/or GPU task routine(s). ) 1671 within the selected node devices 1500 rather than the processor 1350 of the coordinating device 1300 doing so.

4A , 4B and 4C 15 together illustrate one embodiment of an assignment of tasks to be performed as part of the execution of an analysis routine to selected ones of the node devices 1500 of the node device trellis 1005. 4A and 4B 12 illustrate the collection and use of data associated with an analysis routine 1210, the metadata 1135 of a dataset 1130, and/or node status 1535 provided by the node devices 1500 to derive the assignments. 4C Figure 12 illustrates the distribution of task routines 1211, 1571 and/or 1671 to the selected node devices 1500.

With reference to 4A As illustrated, the control routine 1340 executed by the processor 1350 of the coordinator 1300 may include a monitor component 1345 to cause the processor 1350 to recurrently operate the network interface 1390 to receive the node statuses 1535 recurrently from each of node devices 1500 of node device lattice 1005 may be transmitted over network 1999. In some embodiments, each of the node devices 1500 may recurrently transmit a node status 1535 to the coordinating device 1300 at a regular time interval and independently of other ongoing activities as a form of "heartbeat signal" indicating continued functionality, as well as information about currently available resources and/or or current activities. In such embodiments, a failure to receive the node status 1535 by the coordinating device 1300 from a node device 1500, when expected, can be taken as an indication of a malfunction by the node device 1500, such that the resources of the node device can be considered unavailable and any task that is currently assigned can be assigned to another node device 1500.

As also illustrated, the control routine 1340 may also include an allocation component 1341 to cause the processor 1350 to allocate data set partitions 1131 of a data set 1130 along with analysis routine 1210 tasks performed on the allocated data set partitions 1131 to selected ones of the node devices 1500 of the node device lattice 1005 assigns. In preparation for such assignments, the processor 1350 can be caused to operate the network interface 1390 to retrieve the metadata 1135 of the data set 1130 and the analysis routine 1210 from one or more other devices via the network 1999 for use by the assignment component 1341. In embodiments where the distributed processing system 1000 includes one or more distinct storage devices 1100 (such as the example distributed processing system 1000 from 1A ), the metadata 1135 and/or the analysis routine 1210 may be provided to the coordination device 1300 via the network 1999 from the one or more storage devices 1100 . However, in embodiments where the distributed processing system 1000 does not include such disparate storage devices and the node devices 1500 provide distributed storage of data sets 1130 (such as the example distributed processing system 1000 of FIG 1B ), the metadata 1135 and/or the analysis routine 1210 may be provided to the coordinating device 1300 via the network 1999 from one or more of the node devices 1500.

In still other embodiments, viewer 1800 may provide analysis routine 1210 to coordinator 1300 . In such embodiments, viewer 1800 may provide a user interface through which viewer 1800 may be controlled to transmit a command to coordinator 1300 over network 1999 to cause analysis routine 1210 to be executed in a distributed manner to perform an analysis with data record 1130. Part of transmitting the command to the coordinating device 1300 may include transmitting the analysis routine 1210 from the viewing device 1800 to the coordinating device 1300 .

As further illustrated, in embodiments where the scheduler 1300 is to compile the task routines 1211 of the analysis routine 1210, the compilation routine 1440 may be performed by the processor 1350 and may include an analysis component 1441 to identify those of the task routines 1211 that are amenable to conversion and compilation for highly parallel execution by one or more GPUs 1650. For this purpose, the analysis component 1441 can also access the received analysis routine 1210, as can the assignment component 1341 of the control routine 1340.

With reference to 4B , as illustrated, node states 1535 may indicate what processing, memory, and/or network access resources are integrated into each node device 1500. As previously noted, node statuses 1535 may be periodically updated to also reflect the current level of current availability (e.g., current percentage of use) of each such resource.

Illustratively, the node statuses 1535 may provide indications of quantities, types, versions, and/or other internal architectural details of the processors that may be integrated into each of the node devices 1500. Thus, for each node device 1500, the node statuses 1535 may indicate the amounts of CPUs 1550 and/or GPUs 1650 present, as well as type information for each processor including, but not limited to, instruction set compatibility, revision level, cache size(s), amount of processing cores . and/or amount of threads that can execute per processing core. As will be explained in more detail, the provision of such information can help support a heterogeneous set of node devices 1500 within the node device grid 1005 that utilize a variety of different processors among them. Along with information about processing resources embedded in each of the node devices 1500, the node states 1535 may be periodically updated to provide indications of the current usage levels of different processors (e.g., different CPUs 1550 and/or GPUs 1650 within a single node device 1500 ), current usage levels of threads, and/or current usage levels of other particular processing features (e.g., usage levels of any specialized processing cores, logic units, extensions to instruction sets, different levels of cache, etc.).

Also by way of example, node statuses 1535 may provide indications of storage capacities of volatile memory 1561 and/or 1661 and/or non-volatile memory 1563 that may be incorporated into each of node devices 1500 . Thus, for each node device 1500, node states 1535 may indicate amounts and/or various parameters of memory components used to implement each of memories 1561, 1661, and/or 1563, including, but not limited to, types of interfaces used, page and/or Line sizes, access speeds and/or latencies and/or storage technologies used. Along with information regarding the memory resources embedded in each of the node devices 1500, the node statuses 1535 may be periodically updated to provide indications of the current usage levels of various volatile and/or non-volatile memories.

For further example, the node statuses 1535 may provide indications of network access capabilities and/or bandwidths of the network interface(s) 1590 that may be integrated into each of the node devices 1500. Thus, for each node device 1500, node statuses 1535 may indicate supported wired and/or wireless network interfaces, supported protocols, input and/or output buffer sizes, and so forth. Along with information regarding network access resources built into each of the node devices 1500, the node statuses 1535 may be periodically updated to provide indications of the current usage levels of bandwidth and/or buffers for each interface.

As also illustrated, metadata 1135 may provide indications of various structural features by which data items within data set 1130 may be organized and/or accessed. For example, metadata 1135 may indicate the overall size, type of data structure of dataset 1130 (e.g., binary tree, multidimensional array, linked list, etc.), data type(s) of data items (e.g., floating point, integer, text character, etc.). ), aspects of the indexing scheme used to access data items (e.g. number of dimensions, labels used, etc.), and/or other structural aspects. Also, for example, metadata 1135 may include indications of various constraints on the manner in which data set 1130 may be partitioned into data set partitions 1131, such as any partitioning scheme in which the data elements of data set 1130 are already organized and/or or a minimum size of the smallest atomic unit of data in which the data set 1130 that still allows independent processing of the resulting data set partitions 1131 (e.g., size of a row or draw in a two-dimensional array, etc.). As is known to those skilled in the art, such limitations on the manner in which data set 1130 may be partitioned may impose an upper limit on the set of data set partitions 1131 into which data set 1130 may be partitioned, effectively imposing an upper limit may be imposed for the set of node devices 1500 over which the resulting data set partitions 1131 may be distributed.

Analysis routine 1210 may be broken down into any number of task routines 1211, each including instructions specifying aspects of a respective task to be performed as a result of execution of that task routine 1211 as part of analysis routine 1210 execution. The instructions within each of the various task routines 1211 may indicate required processing resources (e.g., whether support is required for a particular extension to an instruction set) and/or storage capacity required to support data structures instantiated during execution , provide. Analysis routine 1210 may also provide indications of an order in which task routines 1211 should be executed. Alternatively or additionally, analysis routine 1210 may include definitions of inputs required to perform each task and/or definitions of outputs produced by each task, providing indications of an order in which task routines 1211 are performed may include indications of dependencies between task routines 1211, such as indications of where an output of one task routine 1211 is required as an input to another task routine 1211. The provision of indications of required inputs and/or outputs to be generated may be part of an implementation of a many task computing (MTC) architecture, in which the tasks of the analysis routine 1210 are each expressed independently, at least by their Inputs and outputs to make at least a subset of Analysis Routine 1210 tasks amenable to highly parallel execution.

The manner in which definitions of inputs and/or outputs, indications of the order of task execution, and/or indications of dependencies are provided may vary depending on attributes of the programming language(s) in which the analysis routine 1210 is generated. By way of example, in some embodiments, declaration headers for those of task routines 1211 that are written as callable functions in a manner that follows the conventions of a chosen programming language may be sufficient to provide such definitions. However, in other embodiments, a different file or other data structure may be created either within analysis routine 1210 or concomitantly with analysis routine 1210 to provide such definitions and/or displays. In particular, by way of example, an array may be generated in which each entry specifies required inputs, outputs to be generated, and/or a position within an order of execution. Regardless of the precise manner in which definitions of inputs and/or outputs and/or indications of dependencies and/or ordering in a file or other data structure are presented, in some embodiments such definitions and/or indications may represent a representation form a Directed Acyclic Graph (DAG) of the Analysis Routine 1210 tasks.

When executing the assignment component 1341, the processor 1350 can be caused to use the node statuses 1535, the metadata 1135 and/or parts of the analysis routine 1210 to make initial assignments of at least one initial task of the analysis routine 1210 to selected ones of the node devices 1500 and an initial distribution of at least one data set partition 1131 to each of the selected ones of the node devices 1500. In deriving such initial assignments, processor 1350 may be caused to initially derive the manner in which data set 1130 should be partitioned into data set partitions 1131 based at least in part on any specified constraints thereon that may be present within metadata 1135. and/or on the number of node devices 1500 currently indicated in node statuses 1535 as having sufficient storage resources. The processor 1350 may alternatively or additionally be caused to analyze the executable instructions within one or more of the task routines 1211 of the analysis routine 1210 to identify one or more particular processing resources required (e.g., floating point math, single instruction multiple, Data (SIMD) instruction support, etc.), and may select one of the nodes 1500 to which to assign an initial task or tasks based on indications in the node statuses 1535 of those of the node devices 1500 that currently have such processing resources as available (en). Regardless of the exact logic involved in deriving the initial assignments of tasks and/or data set partitions 1131 to the selected ones of the node devices 1500, the processor 1350 may store indications of such initial assignments as part of the assignment data 1330.

In embodiments where the scheduler 1300 is to compile the task routines 1211 of the analysis routine 1210, the scheduler 1300 may store the compilation rules 1434 for use in the execution of the compilation routine 1440 by the processor 1350. Compilation Rules 1434 may specify various aspects of compiling task routines 1211 of analysis routines 1210 to produce CPU task routines 1571 for execution by one or more CPUs 1550 and/or GPU task routines 1671 for execution by one or more GPUs 1650 . Under what is specified by the Compilation Rules 1434, aspects of the transformation of Task Routines 1211 instructions that were not originally created to take advantage of the highly parallel execution capabilities of the GPUs 1650 (by providing a relatively large set of threads of execution ), in statements generated to do so. Thus, Compilation Rules 1434 may specify aspects of the conversion and compilation of Task Routine 1211 instructions originally generated for execution by CPUs 1550 into instructions generated for execution by GPUs 1650.

Each of the one or more CPUs 1550 of each of the node devices 1500 may employ an internal processing architecture found well suited for the sequential processing of task routines 1211 that involve various input/output operations and/or branch operations that require the execution of various sets of Condition instructions within task routines 1211 on the results of various determinations. Each of the CPUs 1550 may each include one or more processing cores 1555 (referring to FIG 2 ), each capable of supporting a relatively limited degree of parallel execution of instructions on a relatively limited number of threads of execution. In contrast, the one or more GPUs 1650, which may be present in at least a subset of node devices 1500, may use an internal processing architecture deemed well suited for highly parallel processing of task routines 1211 that execute a relatively limited set of instructions for mathematical and/or bitwise operations that can be executed independently, so that there are no dependencies between numerous instances of a Task Routine 1211 executing in parallel. Indeed, in some embodiments, each of the GPUs 1650 may be capable of supporting parallel processing across hundreds, thousands, or even greater numbers of threads of execution.

If a Task Routine 1211's instructions are amenable to being converted for such parallel execution across such a large number of threads, the task performed by such a Task Routine 1211 may be executable in significantly less time. For example, a task of one of task routines 1211 that may take several days to perform using CPUs 1550 of node devices 1500 may instead take hours of a single day to perform using GPUs 1650. For tasks that can be implemented using the more limited instruction set of the GPUs 1650, a single one of the GPUs 1650 within a single one of the node devices 1500 can perform the equivalent work of numerous CPUs 1550 across numerous of the nodes 1500 in less time and at significantly less cost. Such an improvement in performance speed becomes even greater when multiple of the GPUs 1650 within multiple of the node devices 1500 operate in parallel to perform a task as a highly parallel task across the multiple threads supported by each, thereby providing an even higher parallel form of performing this task is provided.

However, there may be those of the task routines 1211 of the analysis routine 1210 that include particular instructions and/or perform particular operations that render the instructions of those task routines 1211 non-convertible and non-compilable for such highly parallel execution by the GPUs 1650. As previously mentioned, the GPUs 1650 can support a relatively limited instruction set. By way of example, many . Types of input/output operations are bound to be performed in a single-threaded fashion since various protocol handshakes and/or other features dictate sequential execution of steps. As a result, the Analysis Routine 1210 is intended to contain both Task Routines 1211, in which the instructions for conversion and compilation are accessible for highly parallel execution by the GPUs 1650, and Task Routines 1211, in which the instructions are not such as to be executed by which CPUs 1550 need to be compiled.

The Compilation Rules 1434 may include a list of instructions that, if present in a Task Routine 1211, at least do not prevent the conversion and compilation of the Task Routine 1211 instructions to create a corresponding GPU Task Routine 1671 in which the instructions perform cause the Task Routine 1211 task to be a highly parallel task using multiple threads of at least one GPU 1650 in execution. Alternatively or additionally, the compilation rules 1434 may include a list of instructions that, if present within a task routine 1211, prevent such conversion of the task routine 1211 to support such highly parallel execution by the GPUs 1650. Additionally, Compilation Rules 1434 may specify certain circumstances in which certain instructions that would not otherwise prevent such conversion may be used in a Task Routine 1211 (e.g., as part of an input/output operation or some other operation) that such a conversion is prevented.

For those of the task routines 1211 of the analysis routine 1210 that have instructions that may support conversion into other instructions that are compilable to perform the task(s) as highly parallel task(s) on the GPUs 1650, the compilation rules 1434 may be different Specify rules for performing such a conversion. Illustratively, Compilation Rules 1434 may include one or more rules for converting index values used in loops to be instantiated in a manner amenable to highly parallel execution that avoids index value collisions and dependencies between instances of a task that run in parallel can be avoided. Alternatively or additionally, there may be one or more rules for converting instructions implemented within a loop to cause an operation to be performed multiple times, sequentially, into instructions that cause multiple performances of that operation to be performed in parallel across multiple threads occur across at least one of the GPUs 1650. In some such casts involving a loop where the number of times the operation is performed may be large enough that they cannot all occur in parallel, the loop may not be eliminated and may instead be cast to use fewer iterations if a subset of performances of the operation occurs in parallel during each iteration. By way of example, Compilation Rules 1434 may be one or more rules for converting a data structure instantiated within a Task Routine 1211 whose data members are accessed sequentially into another data structure whose data members are accessed in parallel across multiple threads by at least one of the GPUs 1650 will include.

In some embodiments, the conversion may result in task routine 1211 instructions originally generated in a particular programming language being converted to different instructions generated in the same programming language. However, in some such embodiments, the instructions defined for use in each language may be supplemented by one or more additional instructions associated with aspects of the internal architecture specific to the GPU 1650 (e.g., .specific types of operands, specific types for returned values, organization of bits and/or bytes of variables for a register implementation, etc.). In other embodiments, the conversion may result in task routine 1211 instructions originally generated in one programming language being converted to different instructions generated in a different programming language. In some such embodiments, the other programming language may have been created by a vendor of the GPUs 1650 to provide a programming language configured to take full advantage of the features of the GPUs 1650.

The following is an example of a conversion of portions of a task routine 1211 that may be performed by the processor 1350 of the coordination device 1300 and/or by one or more CPUs 1550 of the node devices 1500. Shown first are portions of an example task routine 1211 generated in FCMP, a programming language offered in various products by SAS Institute Inc., Cary, NC, USA. In particular, for example, the declarations of the subroutine "income_statement_kernel" and the data structure "d_matrix_is" have been converted between the FCMP and C programming languages.

Corresponding portions of a corresponding GPU task routine generated in the conversion process performed by the processor 1350 or by one or more of the CPUs 1550 are shown next. In particular, the above parts of the example task routine 1211 generated in FCMP have been transformed to use a programming language extension that is part of the Compute Unified Device Architecture (CUDA) programming model to invoke functions of GPUs (e.g. the GPUs 1650) developed by by Nvidia Corporation, Santa Clara, CA, USA, a supplier of GPUs that can be used as the GPUs 1650.

As those skilled in the art can readily appreciate, it is often the case that programming code originally generated for execution using a first processing architecture is likely to be executed more efficiently by a first architecture processor than programming code originally generated for execution under generated using a second processing architecture and then converted for execution using the first processing architecture. Nevertheless, tests have confirmed that by converting the task routines 1211 by the compiler routine 1440, significant increases in the speed of execution of some tasks of an analysis routine 1210 can be realized, from generation for execution by CPUs 1550 to generation for highly parallel execution by the GPUs 1650.

Below is a table of execution time comparisons from testing an example stochastic calculation. Such a calculation may be a task implemented as one of the task routines 1211 within an example analysis routine 1210 . As illustrated, when the stochastic computation was implemented in SAS Macro Code (another programming language offered in various products by SAS Institute Inc.) for execution by a CPU (e.g., one of the CPUs 1550 of one of the nodes 1500) is written, the resulting amount of time required for execution is measured at approximately 93,600 seconds. If the same stochastic computation was instead written in FCMP for execution by multiple CPUs (e.g., multiple of the CPUs 1550 within one or more of the nodes 1500), the measured execution time was 763 seconds. However, when the same FCMP stochastic computation code was then converted to CUDA for highly parallel execution by one of the GPUs offered by Nvidia Corporation (e.g. one of the 1650 GPUs one of the 1500 nodes) resulted in a measured execution time of 73 seconds. Although 73 seconds is slower than the measured execution time of 11 seconds achieved when the same stochastic computation for highly parallel execution is written directly to CUDA by one of the GPUs offered by Nvidia Corporation, the measured execution time of 73 seconds is the achieved by using FCMP to CUDA conversion is still a very significant improvement over the measured execution time of 763 seconds achieved by running FCMP code through multiple CPUs. Technology used Stochastic calculation - time in seconds CPU SAS macro code 93,600 Code written in FCMP 763 GPU Code written in FCMP, and then converted to CUDA 73 Code written directly in CUDA 11

Thus, as can be seen from these measured execution times, such use of code conversion to allow compilation for such highly parallel execution allows personnel who do not have the skills or training to perform the task routines 1211 of the analysis routine 1210 write natively in CUDA while still taking advantage of highly parallel execution of the code they can write.

In embodiments where the coordinator 1300 is to compile the task routines 1211 of the analysis routine 1210, the processor 1350 may execute the analysis component 1441 as part of the derivation of the initial record partition assignments 1131 and the initial task(s). In particular, the processor 1350 can be caused by the analysis component 1441 to analyze each task routine 1211 of the analysis routine 1210 to identify those of the task routines 1211 where the instructions are amenable to conversion and compilation into corresponding GPU task routines 1671 in order to obtain a high-level allow the GPUs 1650 to perform their respective tasks in parallel. In this manner, the processor 1350 may be caused to use the indications within the compilation rules 1434 as to which instructions prevent such conversions and/or which instructions do not prevent such conversions. The processor 1350 can then be caused to use the results of such analysis of the task routines 1211 in the selection of node devices 1500 as part of the derivation of the initial assignments. In particular, if the analysis of the task routines 1211 results in a determination by the processor 1350 that none of the task routines 1211 can be compiled into corresponding GPU task routines 1671, the processor 1350 may limit the selection of the node devices 1500 to those that have one or more rere of the CPUs 1550, since no GPU task routines 1671 are generated from the task routines 1211. However, if analysis of task routines 1211 results in a determination that some of task routines 1211 can be compiled into corresponding GPU task routines 1671 while others cannot, then processor 1350 can limit the selection of node devices 1500 to those that both integrate one or more CPUs 1550 and one or more GPUs 1650 and therefore support the execution of both CPU 1571 task routines and GPU 1671 task routines spawned from different ones of the 1211 task routines. Furthermore, if the analysis of the task routines 1211 leads to a determination that all of the task routines 1211 can be compiled into the corresponding GPU task routines 1671, the processor 1350 can limit the selection of the node devices 1500 to those that integrate one or more of the GPUs 1650 .

With reference to 4C For example, in embodiments where the coordinator 1300 does not compile the task routines 1211 of the analysis routine 1210, the processor 1350 can be caused by the assignment component 1341 of the control routine 1340 to operate the network interface 1390 to map the task routine(s) 1211 of the assigned initial tasks to the to distribute selected node devices 1500. In such embodiments, each of the selected node devices 1500 may independently map the task routine(s) 1211 distributed to each of the selected nodes 1500 into the corresponding CPU task routine(s) 1571 and/or GPU task routine(s). ) 1671 in preparation for performing the initial task(s) assigned to each of the selected nodes 1500.

Alternatively, in embodiments where the coordinator 1300 is to compile the task routines 1211 of the analysis routine 1210 such that the coordinator 1300 stores the compiler routine 1440 for execution by the processor 1350, the compiler routine 1440 may include a compiler component 1444 for performing the compilation of the task routines 1211. In particular, the processor 1350 may be caused by the compiling component 1444 to compile at least the one or more task routines 1211 associated with the one or more initial tasks assigned to the selected ones of the node devices 1500 . For a task routine 1211 that has been determined by the processor 1350 (during execution of the analysis component 1441) as not accessible to be converted and compiled to produce a corresponding GPU task routine 1671, the processor 1350 can be caused to execute the task routine 1211 without compiling such a conversion to produce a corresponding CPU task routine 1571. For a task routine 1211 that has been determined by the processor 1350 to be amenable to conversion and compilation to produce a corresponding GPU task routine 1671, the processor 1350 can be caused to effect such conversion and compilation to produce the corresponding GPU task routine 1671 to produce.

In some embodiments where a task routine 1211 has been determined by the processor 1350 to be amenable to conversion and compilation to produce a corresponding GPU task routine 1671, the processor 1350 may be caused by the compiling component 1444 to perform more than one compilation of the same task routine 1211 to perform. For example, in some such embodiments, the processor 1350 may additionally compile the same task routine 1211 to also generate a corresponding CPU task routine 1571 in addition to the corresponding GPU task routine 1671 . This may be deemed desirable to address a situation where there may be an insufficient number of available node devices 1500 integrating one or more of the GPUs 1650 such that the task(s) of task routine 12,11 may be performed by one or more of the Node Devices 1500 must be performed based on the execution of the corresponding CPU Task Routine 1571 by one or more CPUs 1550. Alternatively or additionally, this may be deemed desirable to address a situation where a Node Device 1500 carrying one or integrates multiple GPUs 1650, fails to execute the corresponding GPU task routine 1650 and is assigned another node device 1500 to perform the task(s) of the task routine 1211 with the respective dataset partition(s) 1131 assigned to that node device 1500 must-be-that does not have a GPU 1650-integrated.

As another example of more than one compilation of the same task routine 1211, in some of these embodiments, the node device lattice 1005 may include a heterogeneous set of node devices 1500 that integrate different GPUs 1650 that do not share an instruction set, allowing the creation of a single corresponding GPU task routine 1671 for all different GPUs 1650 may not be possible. Thus, the processor 1350 can be caused to perform multiple conversions and compilations of the same task routine 1211 into each of the different versions of the GPU task routine 1671 required for each of the various GPUs 1650 required among the node devices 1500 of the node device trellis 1005.

However, in some of the embodiments in which the scheduler 1300 is to compile the task routines 1211 of the analysis routine 1210, whether analyzing the task routines 1211 to identify those responsible for a . Compilation accessible to one or more different GPUs 1650 takes place and/or whether such compilation takes place to generate corresponding GPU task routines 1671 are conditioned whether there are indications that there are any node devices 1500 that integrate a GPU 1650. In particular, in a situation where the current node statuses 1535 indicate that none of the node devices that integrate one or more of the GPUs 1650 are currently available to be assigned a task at all, the processor 1350 can be caused by the assignment component 1341 to refrain from performing an analysis of Task Routines 1211 to determine if any are amenable to compilation for execution by a GPU 1650. Alternatively or additionally, in embodiments where different node devices 1500 integrate different types of GPUs 1650 such that multiple compilations are required to generate GPU task routines 1671 for all of the different types of GPUs 1650, a situation may arise where the node states 1535 indicate that all of the 1650 GPUs of one of the types are currently unavailable to perform tasks. In such a situation, the processor 1350 may be caused by the allocation routine to desist from compiling any task routine 1211 in order to create a GPU task routine 1671 for execution by GPUs 1650 of that particular type.

The creation of the GPU 1671 task routines in a manner that includes the conversion of the task routine 1211 instructions into other instructions that take advantage of the highly parallel processing features of the GPUs 1650, followed by compilation may be deemed desirable to improve ease , which allows highly parallel features of the 1650 GPUs to be used. By using instruction transformations in this way, personnel may be enabled to utilize the GPUs 1650 that may be present in at least some of the node devices 1500 without the need to architect and write the task routines 1211 instructions in any way , designed for such highly parallel execution. In other words, these personnel may be spared the need to acquire the additional skills and experience to architect and write the instructions that implement the tasks of Analysis Routine 1210 in a manner designed for highly parallel execution by GPUs 1650 is. Additionally, if the node device lattice 1005 node devices 1500 . that integrate different ones of the GPUs 1650 that do not share an instruction set, such personnel are further spared the need to architect and write different versions of instructions that implement the tasks of the Analysis Routine 1210 in a manner appropriate for the different idiosyncrasies of the manner , in which highly parallel execution is provided by each of the different ones of the GPUs 1650.

5A and 5B 10 together illustrate one embodiment of performing tasks assigned to selected ones of the node devices 1500 of the node device lattice 1005. FIG. 5A illustrates aspects of preparing to perform, which may include compiling a task routine 1211 within an example node device 1500, and 5B illustrates aspects of performing a task within the example node device 1500 by executing the corresponding CPU task routine 1571 or corresponding GPU task routine 1671.

With reference to 5A As previously discussed, in embodiments in which the distributed processing system 1000 includes one or more different storage devices 1100 (such as the example distributed processing system 1000 from 1A ); Provide data sets 1130 (such as example distributed processing system 1000 in Fig 1B ), one or more data set partitions 1131 with which a task is to be performed within the example node device 1500 may already be stored within the example node device 1500 . Otherwise, such one or more dataset partitions 1131 may be provided to the example node device 1500 via the network 1999 from another one of the node devices 1500 .

As previously discussed, in embodiments where the coordinator 1300 does not compile the task routines 1211 of the analysis routine 1210, the coordinator 1300 may distribute the one or more task routines 1211 of the one or more assigned initial tasks to the selected node devices 1500. In such embodiments, and as illustrated, each of the node devices 1500 may store the compilation routine 1440 for execution by a CPU 1550 of each of the node devices 1500. Thus, upon receipt of a task routine 1211 from the coordinating device 1300, a CPU 1550 of the example node device 1500 may execute the compile routine 1440 to first analyze the task routine 1211 to determine whether it is amenable to conversion and compilation to a corresponding GPU - Create task routine 1671. If so, the CPU 1550 may be caused to perform such conversion and compilation of the task routine 1211 so as to generate the corresponding GPU task routine 1671 for execution by a GPU 1650 of the example node device 1500. However, if task routine 1211 is not so accessible, CPU 1550 can be caused to compile task routine 1211 without such conversion to produce the corresponding CPU task routine 1571.

Alternatively, in embodiments in which the coordinator 1300 compiles the task routines 1211 of the analysis routine 1210, the coordinator 1300 may distribute the one or more corresponding CPU task routines 1571 and/or GPU task routines 1671 that are generated within the coordinator 1300 from the one or more Task Routines 1211 have been generated. As a result, example node 1500 may receive a CPU task routine 1571 to be executed by CPU 1550 or a GPU task routine 1671 to be executed by GPU 1650, rather than the corresponding task routine 1211 from which one is generated .

As illustrated, the control routine 1640 executed by the GPU 1650 may include an execution component 1641 to cause the GPU 1650 to execute the corresponding GPU task routine 1671 if the corresponding GPU task routine 1671 differs from the task routine 1211 of the initial task associated with the example node device 1500 may be created (whether within the coordinating device 1300 or the example node device 1500). However, as also illustrated, the control routine 1540 executed by the CPU 1550 may include an execution component 1541 to cause the CPU 1550 to execute the corresponding CPU task routine 1571 if the corresponding GPU task routine 1671 is not dependent on the task routine 1211 of the initial task assigned to the example node device 1500, such that the creation of the corresponding CPU task routine 1571 is necessary.

With reference to 5B , and with brief reference to 2 , where the GPU 1650 is caused by the execution component 1641 to execute the GPU task routine 1671 to perform the assigned initial task with a data set partition 1131, pages from both the GPU task routine 1671 and the data set partition 1131 can be stored between the non-volatile memory 1563 and the volatile memory 1661 to which the GPU 1650 is coupled. Alternatively, where the CPU 1550 is caused by the execution component 1541 to execute the CPU task routine 1571 to perform the assigned initial task with a data set partition 1131, pages from both the CPU task routine 1571 and the data set partition 1131 can be switched between the non-volatile memory 1563 and the volatile memory 1561 to which the CPU 1550 is coupled. As previously related to 2 discussed above, each of volatile memory 1561 and 1661 may allow substantially faster access than non-volatile memory 1563 to data and/or routines stored therein, allowing CPU 1550 and GPU 1650 to more efficiently execute routines and perform operations on data stored within volatile memories 1561 and 1661, respectively. However, each of the volatile memories 1561 and 1661 may have a storage capacity much smaller than that of the non-volatile memory 1563. As a result, the situation may repeatedly arise when significantly more (if not all) of a relatively large dataset partition 1131 is storable within non-volatile memory 1563, while only a relatively small portion of that dataset partition 1131 is storable within one of volatile memories 1561 and 1661. which requires the use of a page swap.

As illustrated, the control routine 1540 may also include a status component 1543 to cause the CPU 1550 to operate the network interface 1590 of the example node 1500 to provide recurrently updated indications of the current status of the processing, memory, and/or network access resources of the example node 1500 as updated transmit instances of the node status 1535 to the coordination device 1300 . As previously related to 2 discussed above, the GPU 1650 within the graphics controller 1600 may not and/or cannot access the network interface 1590 do not run the necessary driver routines to operate the 1590 network interface directly. Thus, the CPU 1550 can be caused by the status component 1543 to periodically retrieve indications of the status of the GPU 1650 and/or the volatile memory 1661 from the GPU 1650 to add them to the indications of the status that are included in the transmitted node status 1535.

6 Figure 12 illustrates one embodiment of an assignment of next tasks to be performed as part of the execution of an analysis routine to selected ones of the node devices 1500 of the node device trellis 1005. As previously in connection with FIG 4A-B and 5B discussed, the processor 1350 of the coordinating device 1300 can be caused by the monitoring component 1545 to operate the network interface 1390 recurrently in order to receive the node statuses 1535, which are transmitted recurrently from the node devices 1500 of the node device grid 1005 over the network 1999.

In some embodiments, the processor 1350 may also be caused by the monitoring component 1545 to maintain and recurrently update indications of the lengths of time required to complete tasks assigned to the node devices 1500 . In some such embodiments, the task delay data 1335 may include a discrete per-task completion time that is recurrently calculated as an average of the lengths of time required for each of the multiple node devices 1500 to complete the same task. In embodiments where the task may have been assigned to different ones of the node devices 1500, where it can be executed with different processor types (e.g. a CPU 1550 vs. a GPU 1650 or different types of GPUs 1650), within the task delay data 1335 different average times are maintained for each different type of processor used to perform a task. Alternatively or additionally, as a mechanism for accounting for differences in resources (including processing resources) between Node Devices 1500, averages of relative differences in the amount of time it takes for each Node Device 1500 to complete assigned tasks compared to other Node Devices 1500 in the Task Delay Data 1335 can be included.

In such embodiments, the processor 1350 can be caused by the assignment component 1541 to use the stored indications of the durations for the . Use completion of assigned tasks within the task delay data 1335 to derive one or more periods of time by which the processor 1350 may delay the assignment of a next task with a particular data set partition 1131 to a node device 1500 that does not already have the particular data set partition 1131 stored therein. With multiple time periods of delay, each of them may be based on a different average of the completion times stored within the task delay data 1335 for a different task. If differences in resources between the node devices 1500 are relative by averages. Accounting for differences in time durations, such averages may be used by processor 1350 to modify each of the time periods for the delay for a particular node device 1500.

7A , 7B and 7C , taken together in the order of 7A until 7C 15, 15 illustrate one embodiment of performing tasks among selected ones of the node devices 1500 of the node device trellis 1005. In particular, Figs 7A-C 12 depicts an example triplet of node devices 1500x, 1500y, and 1500z performing a first task on an example data set 1130a and then performing a second task in conjunction with the first task on the same example data set 1130a. In this manner, each of the example node devices 1500x-z shall generate another example data set 1130b and then yet another example data set 1130c. It should be noted that this example, based on the only three node devices 1500x-z and including the three data sets 1130a-c, which may be small enough in size to be distributed among the only three node devices 1500, is a deliberately simplified one example and discussed herein for purposes of explanation and understanding and should not be taken as a limitation. In particular, it is envisioned that embodiments are likely to involve performing tasks with significantly larger data sets 1130 and thus likely involving the use of significantly more node devices 1500 .

Starting from 7A For example, each of the three node devices 1500x-z has been assigned to perform the first task on a corresponding one of three dataset partitions 1131ax-az of dataset 1130a to create a respective one of three dataset partitions 1131bx-bz of dataset 1130b. In addition, each of the three 1500x-z node devices was paired with its corresponding der three dataset partitions 1131ax-az of dataset 1130a from one or more storage devices 1100 via network 1999.

As illustrated, one or more processors of node device 1500x (e.g., one or more CPUs 1550 and/or one or more GPUs 1650) are still performing the first task with dataset partition 1131 ax, such that dataset partition 1131bx is still in progress is generated within the node device 1500x. Additionally, it may be that performing the first task with data set partition 1131ax consumes sufficient processing and/or memory resources of node device 1500x that node device 1500x is deemed to have potentially insufficient processing and/or memory resources to be allocated. perform another task until the first task with record partition 1131 ax has been completed. In performing the first task on data set partition 1131 ax to create data set partition 1131 bx, at least a subset of each of those partitions may be stored within non-volatile memory 1563 of node device 1500x. In addition, pages of these two partitions can be swapped between the non-volatile memory 1563 and one or the other of the volatile memory 1561 or 1661, depending on whether the first task is performed by one or more CPUs 1550 or one or more GPUs 1650 of the node device 1500x is carried out.

As also shown, one or more processors of node device 1500y have already completed performing the first task with dataset partition 1131ay, so dataset partition 1131by has already been created within node device 1500y and then transferred to the one or more storage devices 1100 for storage. In addition, upon completion of this performance of the first task, the node device 1500y was assigned to perform the second task on the data set partition 1131 ay to create a data set partition 1131 cy of the data set 1130c. As shown, one or more processors of node device 1500y are still in progress performing the second task with record partition 1131ay; so that record partition 1131cy is still created within node device 1500y. Additionally, performing the second task with dataset partition 1131ay may consume sufficient processing and/or memory resources of node device 1500y that node device 1500y is deemed to have potentially insufficient processing and/or memory resources to allocate to perform another task until the second task is completed with record partition 1131ay. Since the first task was performed on dataset partition 1131ay to create dataset partition 1131by, at least a subset of partition 1131by may remain stored within non-volatile memory 1563 of node device 1500y for a limited period of time. In performing the second task on dataset partition 1131ay to create dataset partition 1131cy, at least a subset of each of those partitions may be stored within non-volatile memory 1563 of node device 1500x. In addition, pages of these two data set partitions can be exchanged between the non-volatile memory 1563 and one or the other of the volatile memory 1561 or 1661, depending on whether the second task is performed by one or more CPUs 1550 or one or more GPUs 1650 of the node device 1500y is carried out. Furthermore, as a result of such a swap, it may be unlikely that any page of the record partition 1131by is still stored within the volatile memory 1561 or 1661.

As further illustrated, one or more processors of node device 1500z have already completed performing the first task with record partition 1131az, such that record partition 1131bz has already been created within node device 1500z and then transferred to the one or more storage devices 1100 for storage. Similarly, the one or more processors of node device 1500z have already completed performing the second task with dataset partition 1 131az, so dataset partition 1131cz is also already created within node device 1500z and then transferred to the one or more storage devices 1100 for storage became. Therefore, in contrast to node devices 1500x and 1500y, node device 1500z may be deemed to have sufficient available processing and memory resources for node device 1500z to be allocated to perform another task. As a result of performing the first task on data set partition 1131az to create data set partition 1131 bz and as a result of performing the second task on data set partition 1131 az to create data set partition 1131 cz, at least a subset of one or more of data set partitions 1131az, 1131bz and 1131cz may be a remain stored within non-volatile memory 1563 of node device 150Qz for a limited period of time. Also as a result of If the second task was performed more recently than the first task, pages from one or both of the data set partitions 1131az and 1131cz may still be stored within one or the other of the volatile memory 1561 or 1661 depending on whether the second task was completed one or more CPUs 1550 or one or more GPUs 1650 of node device 1500z. However, it may be unlikely that any page of partition 1131bz is still stored within volatile memory 1561 or 1661.

Thus, under the circumstances just described and as indicated by dashed lines in FIG 7B shown, available so that it could be assigned to perform the second task on dataset partition 1131ax of dataset 1130a to create dataset partition 1131cx of dataset 1130c. However, as discussed just above, it is the node device 1500x that was originally provisioned with the data set partition 1131ax from the one or more storage devices 1100 . Therefore, the data set partition 1131ax is already stored within the non-volatile memory 1563 of the node device 1500x, so that the data set partition 1131ax would not have to be retransmitted over the network 1999 from the one or more storage devices 1100 if the node device 1500x could be allocated. to perform the second task with the dataset partition 1131ax. Also, as previously mentioned, as a result of the ongoing performance of the first task with the data set partition 1131 ax within the node device 1500x, there is currently a swapping of sides of the data set partition 1131 ax between the non-volatile memory 1563 and either the volatile memory 1561 or 1661. Thus, there are one or more pages of data set partition 1131ax currently stored within volatile memory 1561 or 1661 of node device 1500x, and the relatively quick assignment of performing the second task with data set partition 1131ax to node device 1500x after node device 1500x completes its performance of the first task, may take advantage of the temporary storage of those one or more pages within volatile memory 1561 or 1661, which may allow node device 1500x to begin performing the second task much faster.

Therefore, as indicated by dashed lines in 7B shown, despite the availability of node device 1500z to be assigned to perform the second task with data set partition 1131ax, node device 1500z is not assigned to do so. Instead, processor 1350 of coordinator 1300 is prompted by allocation component 1341 (referring to FIG 6 ) to delay the assignment of performing the second task with dataset partition 1131ax to any one of node devices 1500x-z for a period of time to provide node device 1500x with an opportunity to complete its performance of the first task with dataset partition 1131ax.

As previously discussed, in some embodiments, the duration of the time period of such a delay may be based on indications in the recurrently received node status 1535 of how long it took for one or more nodes to complete the same task for which allocation is being delayed (e.g., (e.g. based on an average generated from the lengths of time required for one or more nodes to complete the same task). However, as has also been discussed, the time period of the delay may also be based on determinations of differences in the amount of time it takes the node when given an opportunity to complete a task versus the amount of time it takes other nodes to complete the task complete the same task. As previously mentioned, accounting for such differences in time durations may be deemed desirable in order to account for differences that exist in the various resources integrated in different node devices 1500. Thus, in the example 7A-C the time period of delay to allow node device 1500x a chance to complete the first task with dataset partition 1131ax, the amount of time it took for one or both of node devices 1500y and 1500z to complete the first task with their respective dataset partitions 1131ay and 1131az to complete (e.g., an average thereof), and/or differences in the amount of time it takes for node device 1500x to complete tasks compared to the amount of time it takes for node devices 1500y and/or 1500z to complete the same complete tasks based.

Regardless of the precise manner in which the time period of delay in assigning one of the node devices 1550x-z to perform the second task with the data set partition 1131 ax is derived, the processor 1350 of the coordinating device 1300 may use a clock 1351 to monitor of the passage of time can be integrated into the processor 1350 (refer to FIG 6 ) to determine when the delay time period has fully expired. If the time period of delay has fully elapsed before node device 1500x can complete its performance of the first task with data set partition 1131ax, then processor 1350 can be caused by allocation component 1341 to perform the second task with data set partition 1131ax to the node device assign 1500z. If, as in 7C shown, but node device 1500x completes its performance of the first task with dataset partition 1131ax before the time period of the delay expires, then processor 1350 can be caused to assign performance of the second task with dataset partition 1131ax to node device 1500x. As previously mentioned, such assignment of the second task with data set partition 1131ax to node device 1500x may utilize at least the storage of at least a portion of data set partition 1131ax within non-volatile memory 1563, if not also the storage of one or more pages of the data set partition 1131ax within volatile memory 1561 or 1661.

8A , 8B and 8C , taken together in order from 8A until 8C 15, 15 illustrate another embodiment of performing tasks among selected ones of the node devices 1500 of the node device trellis 1005. In particular, Figs 8A-C represents the same example triplet of node devices 1500x, 1500y, and 1500z performing a first task on example data set 1130a to create example data set 1130b, and then performing a second task on example data set 1130b generated by performing the first task is generated. In performing the second task on the example data set 1130b, each of the example node devices 1500x-z shall generate yet another example data set 1130c. As in 7A-C note that this example, based on only three node devices 1500 and including three data sets 1130a-c, which may be small enough in size to be distributed among only three node devices 1500x-z, is a deliberately simplified example and discussed herein for purposes of explanation and understanding and should not be taken as a limitation. As previously mentioned, it is envisioned that embodiments will likely involve performing tasks with significantly larger data sets 1130 and thus likely involving the use of significantly more of the node devices 1500 .

Starting from 8A For example, each of the three node devices 1500x-z has been assigned to perform the first task on a corresponding one of three dataset partitions 1131ax-az of dataset 1130a to create a respective one of three dataset partitions 1131bx-bz of dataset 1130b. As illustrated, all three of the node devices 1500x-z have completed their performances of the first task and each has transferred its corresponding one of the data set partitions 1131bx-bz to the one or more storage devices 1100 for storage as data set 1130b. To enable these performances of the first task, each of the three node devices 1500x-z was provided with its corresponding one of the three dataset partitions 1 131 ax-az of dataset 1130a from the one or more storage devices 1100 via network 1999.

After their completions of the first task, each of the node devices 1500y and 1500z was assigned to perform the second task with the data set partitions 1131by and 1131bz that the node devices 1500y and 1500z created as a result of their performances of the first task, respectively. However, upon completion of the first task, node device 1500x was assigned to perform a task of a different and unrelated analysis routine. As previously discussed, the assignment of tasks to different and unrelated analysis routines may occur in embodiments where the node device trellis 1005 is shared to the extent that multiple unrelated analysis routines are performed using the node devices 1500 concurrently.

As illustrated, one or more processors of node device 1500x (e.g., one or more CPUs 1550 and/or one or more GPUs 1650) are still performing the task of the unrelated analysis routine. Additionally, performing the unrelated analysis routine task may consume sufficient processing and/or memory resources of the node device 1500x that it is deemed that the node device 1500x may have insufficient processing and/or memory resources to be allocated a perform another task until the task of the unrelated parsing routine has completed. Since the first task was performed on dataset partition 1131ax to create dataset partition 1131bx, at least a subset of one or both of partitions 1131ax and 1131bx may remain stored within non-volatile memory 1563 of node device 1500x for a limited period of time. In performing the task of the unrelated analysis routine, an exchange of pages of an unrelated data set partition between the non-volatile memory 1563 and the one or other of volatile memories 1561 or 1661 depending on whether the unrelated analysis routine task was performed by one or more CPUs 1550 or one or more GPUs 1650 of node device 1500x. Furthermore, as a result of such an exchange, it may be unlikely that any page of the data set partitions 1131ax or 1131bx is still stored within the volatile memory 1561 or 1661.

As shown, one or more processors of node device 1500y is still in progress performing the second task with record partition 1131by, such that record partition 1131cy is still created within node device 1500y. In addition, it may be that performing the second task with the data set partition 1131by consumes sufficient processing and/or memory resources of the node device 1500y that the node device 1500y is deemed to have insufficient processing and/or memory resources to be allocated. perform another task until the second task is completed with record partition 1131ay. Since the first task was performed on data set partition 1131 ay to create data set partition 1131 by, at least a subset of partition 1131 ay may remain stored within non-volatile memory 1563 of node device 1500y for a limited period of time. In performing the second task on dataset partition 1131by to create dataset partition 1131cy, at least a subset of each of those partitions may be stored within non-volatile memory 1563 of node device 1500x. In addition, pages of these two data set partitions can be exchanged between the non-volatile memory 1563 and one or the other of the volatile memory 1561 or 1661, depending on whether the second task is performed by one or more CPUs 1550 or one or more GPUs 1650 of the node device 1500y is carried out. Furthermore, as a result of such an exchange, it may be unlikely that any page of the data set partition 1131ay is still stored within the volatile memory 1561 or 1661.

As further illustrated, one or more processors of node device 1500z have already completed performing the second task with record partition 1131bz, such that record partition 1131cz has already been created within node device 1500z and then transferred to the one or more storage devices 1100 for storage. Therefore, in contrast to node devices 1500x and 1500y, node device 1500z may be deemed to have sufficient available processing and memory resources for node device 1500z to be allocated to perform another task. As a result of performing the first task on dataset partition 1131az to create dataset partition 1131 bz and as a result of performing the second task on dataset partition 1131 bz to create dataset partition 1131cz, at least a subset of one or more of dataset partitions 1131az, 1131bz, and 1131cz may be a remain stored within non-volatile memory 1563 of node device 1500z for a limited period of time. Even as a result of the second task being performed more recently than the first task, pages from one or both of the data set partitions 1131bz and 1131cz may still be stored within one or the other of the volatile memory 1561 or 1661, depending on whether the second task was performed by one or multiple CPUs 1550 or one or more GPUs 1650 of the node device 1500z. However, it may be unlikely that any page of partition 1131az is still stored within volatile memory 1561 or 1661.

Thus, under the circumstances just described and as indicated by dashed lines in 8B , node device 1500z is available to perform the second task with data set partition 1131bx of data set 1130b. could be allocated to create record partition 1131 ex of record 1130c. However, as discussed just above, it is node device 1500x that originally created record partition 1131bx. Therefore, the data set partition 1131bx is already stored within the non-volatile memory 1563 of the node device 1500x, so that the data set partition 1131bx would not have to be transferred over the network 1999 from the one or more storage devices 1100 (or from the node device 1500x) if the Node device 1500x could be assigned to perform the second task with record partition 1131bx. Thus, the relatively quick assignment of performing the second task with the dataset partition 1131bx to the node device 1500x after the node device 1500x completes its performance of the task of the unrelated analysis routine, the temporary storage of the dataset partition 1131bx within the non-volatile memory 1563 of the node device 1500x, which may allow the node device 1500x to start performing the second task much faster.

Therefore, as indicated by dashed lines in 8B shown, despite the availability of node device 1500z to be assigned to perform the second task with dataset partition 1131bx, node device 1500z is not assigned to do so. Instead, processor 1350 of coordinator 1300 is prompted by allocation component 1341 (referring to FIG 6 ) to delay the assignment of performing the second task with dataset partition 1131bx to any one of node devices 1500x-z for a period of time to provide node device 1500x with an opportunity to complete its performance of the unrelated analysis routine task. Similar to the embodiment 7A-C , if the time period of delay has fully expired before node device 1500x is able to complete its performance of the unrelated analysis routine task, then processor 1350 may be prompted by allocation component 1341 (referring to FIG 6 ) to assign the performance of the second task with the dataset partition 1131bx to the node device 1500z. If, as in 8C shown, but node device 1500x completes its performance of the unrelated analysis routine task before the time period of the delay expires, then processor 1350 may be caused to assign performance of the second task to data set partition 1131bx to node device 1500x.

9 11 illustrates one embodiment of a logic flow 2100. Logic flow 2100 may be representative of some or all of the operations performed by one or more embodiments described herein. In particular, the logic flow 2100 may illustrate operations performed by the processor 1350 in executing the control routine 1340 and/or the compilation routine 1440 and/or other component(s) of the coordinator 1300.

At 2110, a processor of a coordinator of a distributed processing system (e.g., processor 1350 of coordinator 1300 of distributed processing system 1000) may analyze node states that the coordinator recurrently receives from node devices of a grid of node devices (e.g., node states 1535 of the node devices 1500 of the node device lattice 1005) to determine what processing resources are available within each of the node devices. As previously discussed, the node statuses 1535 that are recurrently transmitted by the node devices 1500 to the coordinating device 1300 may indicate what processing resources are integrated into each of the node devices 1500 (e.g., which CPU(s) 1500 and/or GPU(s ) 1600 are included, what processing features are provided by each, etc.) and/or to what extent these processing resources are currently available within each of the node devices 1500.

At 2120, the coordinating device processor may check whether the node statuses indicate any GPUs as sufficiently available within any of the node devices such that a task could be assigned to those node devices to be performed by such available GPUs. If no GPUs are so available, at 2122 the processor may compile all of the task routines of an analysis routine to be executed by the CPUs of the node devices (e.g., the task routines 1211 of an analysis routine 1210). At 2124, the processor may then prioritize the assignment of all of the analysis routine's tasks to node devices selected as integrating only one or more CPUs and/or integrating a combination of one or more CPUs and one or more GPUs.

However, if there are GPUs that are deemed sufficiently available at 2120, then at 2130 the processor may analyze each of the analysis routine's task routines to determine whether any of the task routines are amenable to conversion of their instructions and compilation required by the GPUs of the node devices of the node device lattice to run. If at 2140 there are no such accessible task routines, then at 2122 the processor may compile all of the task routines of the analysis routine to be executed by the CPUs of the node devices. Then, at 2124, the processor may re-prioritize the assignment of all of the analysis routine's tasks to node devices that have been selected as integrating only one or more CPUs and/or integrating a combination of one or more CPUs and one or more GPUs. As previously discussed, if none of the task routines 1211 of an analysis routine 1210 are accessible for compilation for execution by a GPU 1650, prioritizing the assignment of the tasks of such analysis routine 1210 to node devices 1500 that only integrate CPU(s) 1550, if possible, and/or Node Devices 1500 that integrate a combination of the CPU(s) 1550 and the GPU(s) 1650, if required, may be deemed desirable to facilitate the assignment of tasks to Node Devices 1500, the GPUs 1650 to minimize if none of the tasks be run by a GPU 1650. This may help keep the GPUs 1650 of the node devices 1500 of the node device lattice 1005 more readily available for use in performing tasks of another analysis routine that can be performed by GPUs.

However, if at 2140 there is at least one task routine of the analysis routine that is accessible for such conversion and compilation, a check may be made at 2150 whether all of the task routines of the analysis routine are so accessible. If so, at 2152 the processor may compile all of the analysis routine's task routines to be executed by the GPUs of the node devices. At 2154, the processor may then prioritize the assignment of all of the analysis routine's tasks to node devices selected as integrating only one or more GPUs and/or integrating a combination of one or more CPUs and one or more GPUs.

However, if at 2150 there is a mixture of task routines that are and are not accessible for such conversion and compilation, then at 2160 the processor may compile all of the analysis routine's task routines that are accessible to be executed by the GPUs of the node devices become. At 2162, the processor may compile all of the analysis routine's task routines that are not as accessible to be executed by the CPUs of the node devices. At 2164, the processor may then prioritize the assignment of all of the analysis routine's tasks to node devices that have been selected as integrating a combination of one or more CPUs and one or more GPUs.

10 12 illustrates one embodiment of a logic flow 2200. Logic flow 2200 may be representative of some or all of the operations performed by one or more embodiments described herein. In particular, the logic flow 2200 may illustrate operations performed by the one or more CPUs 1550 in executing the control routine 1540 and/or the compilation routine 1440 and/or other component(s) of the node devices 1500.

At 2210, a CPU of a node device of a distributed processing system (e.g., a CPU 1550 of one of the node devices 1500 of the distributed processing system 1000) may analyze a task routine (e.g., a task routine 1211 of an analysis routine 1210) to determine whether it is conversion of its instructions and compilation is amenable to execution by the one or more GPUs of the node device (e.g., one or more of GPUs 1650). As previously discussed, in embodiments where the coordinating device 1300 does not compile the task routines 1211, one or more CPUs 1550 of each of the node devices 1500 may do so. Also, such compilation may include an analysis of each task routine 1211 received from each of the node devices 1500 that integrates one or more of the GPUs 1650 to determine whether the compilation is for execution by one or more CPUs 1550 or one or more GPUs 1650 to run.

If the task routine at 2220 is not so accessible, then at 2222 the CPU may compile the task routine of the analysis routine for execution by the one or more CPUs of the node device. However, if the task routine is so accessible at 2220, then at 2230 the CPU may compile the analysis routine's task routine to be executed by the one or more GPUs of the node device.

11 11 illustrates one embodiment of a logic flow 2300. Logic flow 2300 may be representative of some or all of the operations performed by one or more embodiments described herein. In particular, the logic flow 2300 may illustrate operations performed by the processor 1350 in executing the control routine 1340 and/or other component(s) of the coordinator 1300.

At 2310, a processor of a coordinator of a distributed processing system (e.g., processor 1350 of coordinator 1300 of distributed processing system 1000) may receive metadata of a data set that is descriptive of a data set from one or more storage devices (e.g., metadata 1135 from one or more storage devices 1100). At 2312, the processor may receive an analysis routine that includes multiple task routines from the one or more storage devices and/or a viewing device (e.g., analysis routine 1210 that includes multiple task routines 1211 from the one or more storage devices 1100 or from the viewing device 1800 includes). At 2314, the processor may distribute the latest transmissions of node status data from node devices of a lattice of node devices th processing system (e.g., the node statuses 1535 of the node devices 1500 of the node device trellis 1005). As previously discussed, in embodiments in which the node devices 1500 also serve as storage devices of at least the one or more data sets 1130, the coordinating device 1300 may receive the metadata 1135 and/or the analysis routine from one of the node devices 1500.

At 2320, the processor may derive initial assignments of data set partitions (e.g., data set partitions 1131) and initial tasks, including a selection of a plurality of the node devices to which data set partitions are to be distributed, and initial assignments of the one or more initial tasks to be made . At 2322, the processor may transmit indications of the allocated distribution of the data set partitions to one or more storage devices and/or to the selected node devices. As previously discussed, the distribution of dataset partitions to the selected node devices from the one or more storage devices may be coordinated among the one or more storage devices, the selected node devices, and/or the coordinating device in any of a variety of ways, with any of a variety of protocols is used to cause the transfer of the data set partitions to the selected node devices.

At 2330, the processor may transmit to the selected node devices indications of the assignment of a first task to be performed on the selected node devices with corresponding ones of the data set partitions. As previously discussed, the assignment of task routines to the analysis routine or the assignment of CPU task routines and/or GPU task routines (e.g., task routines 1211, CPU task routines 1571, and/or GPU task routines 1671) to the selected node devices as part of the signaling of assignments of tasks to the selected node devices.

At 2340, the processor may receive from a first one of the selected node devices an indication of the first node device's completion of the first task with the first data set partition. As previously discussed, such indications may be transmitted as part of the periodic transmissions of node status 1535 to the coordinator.

At 2350, the processor may delay assignment of performance of a second task using a second data set partition to any of the selected node devices, including the first node device, for a predetermined time period of delay to allow a second one of the selected node devices in which the second record partition is already stored, giving a chance to become available. As previously discussed, such an imposition of a delay in the assignment of a next task with a particular dataset partition may be to allow a node device already storing that particular dataset partition to become available to store that particular dataset partition within that node device by assigning the next task with the particular dataset partition to that node device. As already mentioned, in this way the network bandwidth resources and the time needed for the transfer of the particular data set partition to another node device can be saved by waiting in this way for that node device to become available.

At 2360, the processor may check whether the second node device, in which the second data set partition is already stored, became available at the end of the delay time period. If the second node device became available within the time period of the delay, then the processor may transmit an indication of the assignment to perform the second task with the second dataset partition at 2362 to the second node device. However, if the second node device did not become available within the time period of the delay, then the processor may transmit an indication of the assignment to perform the second task with the second dataset partition at 2370 to the first node device.

12 12 illustrates one embodiment of a logic flow 2400. Logic flow 2300 may be representative of some or all of the operations performed by one or more embodiments described herein. In particular, the logic flow 2400 may illustrate operations performed by the processor 1350 in executing the control routine 1340 and/or other component(s) of the coordinator 1400.

At 2410, a processor of a distributed processing system coordinator (e.g., processor 1350 of coordinator 1300 of distributed processing system 1000) may receive an indication from a node device of a node device grid of the distributed processing system (e.g., one of node devices 1500 of node device grid 1005). indicates that this node device has completed its execution of a task routine involving a dataset partition of a dataset (e.g., one of dataset partitions 1131 of dataset 1130). At 2412, the processor may determine the amount of time that has elapsed between its transmitting a signal to the node device to perform the task with the data set partition and receiving the indication from the node device of the completion of that task performance.

At 2420, the processor may compare the new elapsed time to previous elapsed times required by other node devices to perform the same task with their corresponding dataset partitions of the same dataset. At 2422, the processor may check whether the new elapsed time is similar to the previous elapsed times for those performances of the same task, within a predetermined threshold of similarity. If not, then the processor can use the new elapsed time along with the previous elapsed times to derive again a time period of delay by which a new task assignment should be delayed in order to allow another node to complete the task make possible. As previously discussed, such determinations of a time period of delay may result in an averaging of lengths of time required by different ones of the node devices 1500 to perform the same task.

13 11 illustrates one embodiment of a logic flow 2500. Logic flow 2500 may be representative of some or all of the operations performed by one or more embodiments described herein. In particular, the logic flow 2500 may illustrate operations performed by the processor 1350 in executing the control routine 1340 and/or other component(s) of the coordinator 1300.

At 2510, a processor of a distributed processing system coordinator (e.g., processor 1350 of coordinator 1300 of distributed processing system 1000) may receive an indication from a first node device of a node device lattice of the distributed processing system (e.g., one of node devices 1500 of node device lattice 1005). completion of a first task with a first dataset partition of a dataset (e.g., one of dataset partitions 1131 of a dataset 1130) is received by the first node device. As previously discussed, such indications may be transmitted as part of the periodic transmissions of node status 1535 to the coordinator.

At 2520, the processor may delay assignment of performance of a second task using a second dataset partition of the same dataset to any node-devices of the node-device lattice, including the first node-device, for a predetermined period of delay for a second node-device of the node-device lattice in which the second record partition is already stored, giving a chance to become available. At 2530, the processor may check whether the second node device, in which the second data set partition is already stored, became available at the end of the delay time period. If the second node device became available within the time period of the delay, then the processor may transmit an indication of the assignment to perform the second task with the second data set partition at 2532 to the second node device.

However, if the second node device did not become available within the time period of the delay at 2530, then the processor may check at 2540 whether there was another node device of the node device grid that was provisioned with the second record partition to prepare the other node device to act as a backup - serve as a node device for a task involving the second dataset partition. If no such other device is present, then the processor may transmit an indication of the assignment to perform the second task with the second dataset partition at 2542 to the first node device.

However, if such another node device is present at 2540, then the processor may check at 2550 whether an indication has been received that the other node device is currently available to be assigned a task. If not, then the processor can again display an indication of the assignment to perform the second task with the second data set partition transmitted at 2542 to the first node device. However, if such an indication of the other currently available node device was received, then the processor may transmit an indication of the assignment of performing the second task with the second dataset partition at 2552 to the other node device.

In various embodiments, each of processors 1150, 1350, 1550, 1650, and 1850 may include any of a variety of commercially available processors. Furthermore, one or more of these processors may include multiple processors, a multi-threaded processor, a multi-core processor (whether the multiple cores co-exist on the same or separate dies), and/or a different type of multi-processor architecture by which multiple physically separate processors are connected are.

However, in a particular embodiment, the CPU 1550 of each of the one or more node devices 1500 may be selected to efficiently perform the analysis of multiple instances of job flows in parallel, at least in part. By way of example, the CPU 1550 may integrate a single-instruction-multiple-data (SIMD) architecture, may integrate multiple processing pipelines, and/or may integrate the ability to support multiple simultaneous threads of execution per processing pipeline.

Alternatively or additionally, in a particular embodiment, each GPU 1650 of the one or more node devices, which may include at least one of the GPUs 1650, may integrate multi-threaded capabilities and/or multiple processor cores to enable parallel execution of tasks. Illustratively, the GPU 1650 may incorporate an internal architecture designed to enable parallel execution of tasks that use a relatively limited instruction set across hundreds, thousands, tens of thousands, or even more threads of execution to support graphics applications that require relatively high-resolution images include.

In various embodiments, each of control routines 1140, 1340, 1540, 1840, and 1640, including the components that make up each, can be selected to be operable on any type of processor or processors selected to be applicable to the processors 1150, 1350, 1550, 1850 and/or 1650 within the corresponding one of the devices 1100, 1300, 1500 and/or the graphics controller 1600. In various embodiments, each of these routines may include one or more of an operating system, device drivers, and/or application-level routines (e.g., so-called "software suites" provided on disc media, "applets" provided by a remote servers are obtained, etc.). If an operating system is included, the operating system may be any of a variety of available operating systems suitable for the 1150, 1550, and/or 1850 processors. When one or more device files are included, these device drivers may provide support for any of a variety of other components, whether hardware or software components, of devices 1100, 1300, 1500, 1800, and/or 1600.

In various embodiments, each of the memories 1160, 1360, 1560, 1660, and 1860 may be based on any of a variety of information storage technologies, including volatile technologies that require the uninterrupted supply of electrical power and/or technologies that involve the use of machine-readable storage media, which may or may not be removable. Thus, each of these memories may include any of a variety of types (or combinations of types) of memory devices including, without limitation: read-only memory (ROM), random-access memory (RAM), Dynamic RAM (DRAM), Double-Data-Rate DRAM (DDR-DRAM), Synchronous DRAM (SDRAM), Static RAM (SRAM), Programmable ROM (PROM), Erasable Programmable ROM (EPROM). ), Electrically Erasable Programmable ROM (EEPROM), Flash memory, Polymer memory (e.g. ferroelectric polymer memory), Ovonic memory, Phase change or ferroelectric memory, Silicon Oxide Nitride Oxide Silicon (SONOS ) memory, magnetic or optical cards, one or more individual ferromagnetic disk drives, non-volatile storage class memory, or a plurality of storage devices organized into one or more arrays (e.g., B. multiple ferromagnetic disk drives organized in a redundant array of independent disks (Redundant Array of Independent Disks (RAID)). It should be noted that although each of these memories is illustrated as a single block, one or more of them may include multiple memory devices, which may be based on different memory technologies. Thus, for example, one or more of each of these memory shown is a combination of an optical drive or flash memory card reader, through which programs and/or data can be stored and transmitted on some form of machine-readable storage medium, a ferromagnetic disk drive, to store programs and/or data locally for a relatively long period of time , and one or more volatile solid-state memory devices that allow relatively fast access to programs and/or data (e.g., SRAM or DRAM). It should also be noted that each of these memories can be composed of several memory components based on an identical memory technology, but which can be managed separately due to the specialization in use (e.g. some DRAM devices are used as a main memory, while other DRAM devices are used as a different framebuffer of a graphics controller).

However, in a particular embodiment, in embodiments where the one or more of the storage devices 1100 provide for the storage of one or more data sets 1130, or in which the non-volatile memory 1563 in embodiments where the node devices 1500 provide for the storage of one or multiple data sets 1130 can be implemented with a redundant array of independent disks (RAID) of a RAID level selected to provide fault tolerance for the storage of one or more data sets 1130.

In various embodiments, each of the input devices 1320 and 1820 may each be any of a variety of types of input device, each of which may use any of a variety of input detection and/or reception technologies. Examples of such input devices include, but are not limited to, microphones, remote controls, styluses, card readers, fingerprint readers, virtual reality interactive gloves, graphical input tablets, joysticks, keyboards, retina scanners, the tactile input components of touch screens, trackballs, environmental sensors, and/or either cameras or camera arrays Monitoring the movement of individuals to accept commands and/or data provided by those individuals via gestures and/or facial expressions.

In various embodiments, each of the displays 1380 and 1880 can each be any of a variety of types of display device, each of which can use any of a variety of visual presentation technologies. Examples of such a display device include, but are not limited to, a cathode-ray tube (CRT), an electroluminescent (EL) panel, a liquid crystal display (LCD), a gas plasma display, etc. In some embodiments, the displays 1180 and/or or 1880 may each be a touch screen display such that input devices 1110 and/or 1810 may each be incorporated therein as touch-sensitive components thereof.

In various embodiments, each of network interfaces 1190, 1390, 1590, and 1890 may use any of a variety of communication technologies to enable these devices to interface with other devices as described. Each of these interfaces includes circuitry that provides at least some of the functionality required to enable such coupling. However, each of these interfaces may also be implemented, at least in part, with sequences of instructions executed by corresponding ones of the processors (e.g., to implement a protocol stack or other features). When electrically and/or optically conductive cabling is used, these interfaces may use timings and/or protocols that conform to any of a variety of industry standards including, without limitation, RS-232C, RS-422, USB, Ethernet (IEEE-802.3 ) or IEEE-1394. Where use of wireless transmissions is implied, these interfaces may use timings and/or protocols compliant with any of a variety of industry standards including, without limitation: IEEE 802.1 1a, 802.11 ad, 802.1 1ah, 802.1 1ax, 802.1 1b, 802.1 1g , 802.16, 802.20 (commonly referred to as "Mobile Broadband Wireless Access"); Bluetooth; ZigBee; or a cellular radiotelephone service such as GSM with General Packet Radio Service (GSM/GPRS), CDMA/1xRTT, Enhanced Data Rates for Global Evolution (EDGE), Evolution Data Only/Optimized (EV-DO), Evolution For Data and Voice (EV -DV), High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), 4G LTE, etc.

However, in a particular embodiment, one or more of network interfaces 1190, 1390, and/or 1590 may be implemented with multiple copper-based or fiber-optic-based network interface ports to provide redundant and/or parallel paths when interchanging one or more of the Record Partitions 1131, Task Routines 1211, CPU Task Routines 1571, and/or GPU Task Routines 1671.

In various embodiments, the distribution of processing and/or memory resources among the node devices 1300 and/or the API architectures used to support communication between the node devices and other devices may be designed and/or selected to work with any conform to a variety of distributed processing standards, including but not limited to IEEE P2413, AllJoyn, IoTivity, etc. Illustratively, a subset of API and/or other architectural features of one or more such standards may be used to provide the relatively minimal extent described herein Implement coordination to provide greater efficiency in parallelizing the processing of data, while at the same time minimizing exchange of coordinating information that can lead to undesirable instances of inter-process serialization. However, it should be noted that the parallelization of the storage, retrieval, and/or processing of portions of the records 1130 is not dependent on or limited by existing API architectures and/or supporting communication protocols. More generally, there is nothing in the manner in which data sets 1130 may be organized in storage, transmission, and/or distribution over network 1999 that is tied to existing API architectures or protocols.

Some systems may use Hadoop®, an open-source framework for storing and analyzing big data in a distributed computing environment. Some systems may use cloud computing, which can enable ubiquitous, on-demand, convenient network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that are rapidly provisioned with minimal management effort or minimal service provider interaction and can be released. Some grid systems may be implemented as multi-node Hadoop® clusters, as will be understood by those skilled in the art. Apache™ Hadoop® is an open source software framework for distributed computing.

Claims (14)

A computer-implemented method for performing analysis routine tasks, at least partially in parallel, using a plurality of node devices (1500), comprising: receiving an analysis routine comprising executable instructions for one or more tasks in the form of task routines for execution by at least one central processing unit (CPU) of a node device to be performed on at least one data set, analyzing a current status of resources of at least one node device of the plurality of node devices to determine an availability of at least one graphics processing unit (GPU) of the at least one node device, wherein said resources include at least one CPU and at least one GPU, and . if the determination reveals that at least one GPU of the at least one node device is available for assignment of a first task of the analysis routine in the form of a first task routine, Analyzing the first task routine to determine whether it can be compiled into a GPU task routine for execution by the at least one GPU, so that the at least one GPU can perform multiple instances of the first task of the analysis routine at least partially in parallel, wherein analyzing comprises whether the first task routine includes an instruction that prevents compilation to produce a GPU task routine and whether inputs and outputs of the first task routine have dependencies that prevent compilation to produce a GPU task routine, if the analyzing indicates that the first task routine can be compiled into a GPU task routine for execution by the at least one GPU, using a conversion rule to convert at least one instruction of the first task routine into at least one corresponding instruction of the GPU task routine; compiling the at least one corresponding GPU task routine instruction for execution by the at least one GPU; allocating a dataset partition of the dataset to the at least one node device to enable access to the dataset partition by the at least one GPU, and assigning the at least one node device to perform the GPU task routine to enable the at least one GPU to perform the multiple instances of the first task with the dataset partition. Computer-implemented method claim 1 , said resources comprising storage space within at least one memory of the at least one node device. Computer-implemented method claim 1 or 2 , wherein analyzing the first task routine for whether it can be compiled into a GPU task routine for execution by the at least one GPU comprises: determining whether the instruction of the first task routine is included in a set of instructions not included in at least one instruction executable by the at least one GPU; and in response to determining that the first task routine's instruction is not included in the set of instructions, determining whether the first task routine's instruction is used in the first task routine in a manner that prevents conversion to at least one instruction that is defined by which can run at least one GPU. Computer-implemented method claim 1 or 2 , wherein converting the at least one instruction of the first task routine into the at least one corresponding instruction of the GPU task routine comprises converting the at least one instruction of the first task routine from a first programming language into the at least one corresponding instruction in a second programming language according to the conversion rule used. Computer-implemented method claim 2 wherein: the at least one memory of the at least one node device comprises a first volatile memory communicatively coupled to the at least one CPU and a second volatile memory communicatively coupled to the at least one GPU; assigning the data set partition of the data set to the at least one node device to enable access to the data set partition by the at least one GPU comprises causing the data set partition to be stored within the second volatile memory. Computer-implemented method claim 5 , wherein: if the determining indicates that no GPU of the at least one node device is available for assignment of a first task of the analysis routine in the form of a first task routine, the method comprises: refraining from analyzing the first task routine to determine whether to convert it into a GPU task routine for execution - by which at least one GPU can be compiled; assigning the data set partition of the data set to the at least one node device to cause storage of the data set partition within the first volatile memory and to allow access to the data set partition by the at least one CPU; compiling the first task routine for execution by the at least one CPU; and assigning the at least one node device to perform the first task of the analysis routine on the data set partition to enable the at least one CPU to perform the first task on the data set partition. Computer-implemented method claim 1 , 2 or 6 wherein: a coordination device coordinates the operation of the plurality of node devices; the coordinating device recurrently receiving updates for the current status of each node device of the plurality of node devices; and analyzing a current status of resources to determine an availability of at least one GPU of the at least one node device comprises: identifying, by the coordinating device, a node device of the plurality of node devices that includes a GPU that is available in the current status is shown. Computer-implemented method claim 7 , wherein allocating a dataset partition of the dataset to the at least one node device comprises: analyzing, by the coordinator, metadata indicative of structural features of the dataset to identify a limitation of the manner in which the dataset is divided into the plurality of data set partitions can be divided, the constraint being an indication of a smallest atomic unit of data within the data set and/or a specification of a partitioning scheme; and deriving a partition of the data set into the plurality of data set partitions based at least in part on the constraint. Computer-implemented method claim 8 comprising: retrieving the metadata from at least one storage device in which the data set is stored; and transmitting to the at least one node device or the at least one storage device an indication of the allocation of the data set partition to cause a transfer of the data set partition from the at least one storage device to the at least one node device. Computer-implemented method claim 1 or 2 , comprising: analyzing a second task routine of the analysis routine to determine whether it can be compiled into a GPU task routine for execution by the at least one GPU such that the at least one GPU can perform multiple instances of a second task of the analysis routine at least partially in parallel, the analyzing the second task routine includes whether there is a dependency between inputs and outputs of the multiple instances of the second task that prevents compilation to generate a GPU task routine, the second task routine being generated for execution by the at least one CPU to perform a second task to perform the analysis routine; and if the analyzing determines that the second task routine cannot be compiled to generate the GPU task routine: compiling the second task routine for execution by the at least one CPU; and assigning the at least one node device to perform the second task of the analysis routine on the dataset partition to enable the at least one CPU to perform the second task on the dataset partitions. Computer-implemented method claim 10 , comprising analyzing the analysis routine to determine an order of tasks of the analysis routine, the order of the tasks including a relative order of the first and second tasks. A computer program product embodied in a non-transitory machine-readable storage medium, the computer program product comprising instructions capable of causing a processor to perform operations in accordance with a method according to any one of Claims 1 until 11 to perform. Apparatus comprising a processor and a memory for storing instructions which when executed by the processor cause the processor to perform operations according to a method according to any one of Claims 1 until 11 to perform. device after Claim 13 , wherein: analyzing the current status of resources of the at least one node device to determine an availability of at least one GPU of the at least one node device comprises analyzing, by a CPU of that node device, whether at least one GPU of that node device is currently available.

Images