WO2019202216A2 - Procédé de traitement d'informations distribuées, et système de traitement d'informations distribuées - Google Patents

Procédé de traitement d'informations distribuées, et système de traitement d'informations distribuées Download PDF

Info

Publication number
WO2019202216A2
WO2019202216A2 PCT/FI2019/050313 FI2019050313W WO2019202216A2 WO 2019202216 A2 WO2019202216 A2 WO 2019202216A2 FI 2019050313 W FI2019050313 W FI 2019050313W WO 2019202216 A2 WO2019202216 A2 WO 2019202216A2
Authority
WO
WIPO (PCT)
Prior art keywords
infrastructure
dataflow
determined
graphs
sub
Prior art date
Application number
PCT/FI2019/050313
Other languages
English (en)
Other versions
WO2019202216A3 (fr
Inventor
Michael Solomon
Original Assignee
Meeshkan Oy
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Meeshkan Oy filed Critical Meeshkan Oy
Publication of WO2019202216A2 publication Critical patent/WO2019202216A2/fr
Publication of WO2019202216A3 publication Critical patent/WO2019202216A3/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061—Partitioning or combining of resources
    • G06F9/5066—Algorithms for mapping a plurality of inter-dependent sub-tasks onto a plurality of physical CPUs
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/40—Transformation of program code
    • G06F8/41—Compilation
    • G06F8/43—Checking; Contextual analysis
    • G06F8/433—Dependency analysis; Data or control flow analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/40—Transformation of program code
    • G06F8/41—Compilation
    • G06F8/44—Encoding
    • G06F8/443—Optimisation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/40—Transformation of program code
    • G06F8/41—Compilation
    • G06F8/44—Encoding
    • G06F8/443—Optimisation
    • G06F8/4434—Reducing the memory space required by the program code
    • G06F8/4435—Detection or removal of dead or redundant code
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/40—Transformation of program code
    • G06F8/41—Compilation
    • G06F8/44—Encoding
    • G06F8/445—Exploiting fine grain parallelism, i.e. parallelism at instruction level
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/40—Transformation of program code
    • G06F8/51—Source to source
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/30—Arrangements for executing machine instructions, e.g. instruction decode
    • G06F9/38—Concurrent instruction execution, e.g. pipeline or look ahead
    • G06F9/3885—Concurrent instruction execution, e.g. pipeline or look ahead using a plurality of independent parallel functional units
    • G06F9/3889—Concurrent instruction execution, e.g. pipeline or look ahead using a plurality of independent parallel functional units controlled by multiple instructions, e.g. MIMD, decoupled access or execute
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/448—Execution paradigms, e.g. implementations of programming paradigms
    • G06F9/4494—Execution paradigms, e.g. implementations of programming paradigms data driven
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/5038—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/5044—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering hardware capabilities
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5083—Techniques for rebalancing the load in a distributed system
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/54—Interprogram communication
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/54—Interprogram communication
    • G06F9/547—Remote procedure calls [RPC]; Web services
    • G06F9/548—Object oriented; Remote method invocation [RMI]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/02—Capturing of monitoring data
    • H04L43/022—Capturing of monitoring data by sampling
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
    • H04L43/0805—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability
    • H04L43/0817—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability by checking functioning
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/10—Active monitoring, e.g. heartbeat, ping or trace-route
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/16—Threshold monitoring

Definitions

  • the present invention relates to a method for automatic distributed information processing using a dataflow graph and more particularly to a method according to the preamble of claim 1.
  • the present invention also relates to a method for automatic distributed information processing using a dataflow graph and more particularly to a method according to the preamble of claim 12.
  • the present invention further relates to an automatic distributed information processing system using a dataflow graph and more particularly to a distributed information processing system according to the preamble of claim 22.
  • the present invention also relates to an automatic distributed information processing system using a dataflow graph and more particularly to a distributed information processing system according to the preamble of claim 30.
  • a first prior art method for distributed information processing using a dataflow graph relates to the scheduler/executor pattern, the scheduler/executor pattern being a system in which a computational node called the scheduler sends units of work, called jobs, to a node called an executor that executes said jobs. The results of the jobs are then sent to a computational node that aggregates and further processes these results for use in a variety of industrial settings. Examples of this pattern comprise:
  • Apache Spark schedules jobs described using MLib
  • Apache Mesos acts as a broker between Spark and one or more devices that are executors.
  • a second prior art method for distributed information processing using a dataflow graph relates to the siloed execution pattern, the siloed execution pattern being an arbitrary number of instances of one dataflow graph, for example a face-recognition algorithm, being distributed over an arbitrary number of nodes, for example, mobile phones, that have minimal communication with a central server.
  • this pattern comprise:
  • Federated learning uses machine learning algorithms on devices that are producers of their own data in order to execute a modified gradient descent algorithm that allows for more infrequent communication with a central server.
  • a third prior art method for distributed information processing using a dataflow graph relates to the parallel hardware pattern, the parallel hardware pattern being the use of specialized hardware with parallel lanes for operations such as floating point addition being matched to parts of a dataflow graph that use those operations. Examples of this pattern comprise:
  • TPU Tensor Processing Unit
  • Kirin 970 have built- in machine learning primitives such as matrix multiplication, convolution and activation functions.
  • the Graphical Processing Unit has built-in matrix multiplication and parallel vertex manipulation, running parallel operations on pixels of an image or vectorized representations of geometric forms. These are also used by MLib, TensorFlow, Theano, Torch, Caffe and CNTK to execute parts of Machine Learning algorithms.
  • CPUs Central Processing Units
  • Arm NEON Modern Central Processing Units
  • FFTs FIR filters
  • distributing an MLib data flow graph via Apache Mesos and Apache Spark relies on an algorithm that uses approximations of available computational resources to make estimations about the real-time availability of computational units in a datacenter. While this allows for a certain degree of optimization, it does not allow for the distribution of dataflow graphs to the most suitable devices that can execute certain operations.
  • Distributed parallelization and distribution algorithm of Tensorflow is similar to Spark, but it improves upon this using the XLA Compiler, which fuses together certain sub-graphs, called "kernels”, of a dataflow graph and deploys them to nodes in a datacenter.
  • the XLA Compiler uses a just-in-time compilation strategy has mitigated results in improving the speed and accuracy of the distributed information processing.
  • the prior art methods also may result in excessive communication between devices, which leads to infrastructure degradation.
  • a machine learning algorithm doing processing on data generated by a group of mobile devices capturing real-time video of a football match would need to, using current machine learning solutions, either use siloed inference that does not take into account the feed of other devices, i.e. TensorFlow Lite, or transfer streamed video to a central server that aggregates all of the data into a model, draining battery of the devices and, if done repeatedly, lowering the longevity of the mobile devices due to intensive network operations.
  • No current solution can find an optimal way for mobile devices to work together in the execution of a machine learning algorithm, which would allow for more robust algorithms while lowering the dependency on mobile network traffic, thus preserving the battery longer and diminishing device degradation.
  • An object of the present invention is to provide a method and system for automatic distributed information processing such that the prior art disadvantages are solved or at least alleviated.
  • the objects of the invention are achieved by a method for automatic distributed information processing which is characterized by what is stated in claim 1.
  • the objects of the invention are also achieved by a method for automatic distributed information processing which is characterized by what is stated in claim 12.
  • the objects of the present invention are further achieved by an automatic distributed information processing system which is characterized by what is stated in claim 2.
  • the objects of the present invention are also achieved by an automatic distributed information processing system which is characterized by what is stated in claim 30.
  • the invention is based on the idea of providing a method for automatic distributed information processing using a dataflow graph comprising graph nodes and graph edges between the graph nodes.
  • the method comprises distributing a dataflow graph to an infrastructure comprising automatic infrastructure nodes and automatic infrastructure edges between the infrastructure nodes.
  • the method comprises segmenting the dataflow graph into dataflow sub-graphs, defining a pre-determined infrastructure having two or more infrastructure nodes and at least one infrastructure edge between the infrastructure nodes, projecting the dataflow sub-graphs to the pre-determined infrastructure by distributing the dataflow sub-graphs to the infrastructure nodes and infrastructure edges, and executing the dataflow graph by utilizing a communication protocol over the infrastructure edges between the infrastructure nodes in the execution of the dataflow graph and executing the distributed dataflow sub-graphs in the infrastructure nodes.
  • the method allows splitting the dataflow graph into dataflow sub-graphs and distributing the sub-graphs to infrastructure which is defined before distributing the sub-graphs. This allows forming the sub-graphs and distributing the sub-graphs such that the infrastructure properties may be taken into account for efficient processing and efficient use of the infrastructure.
  • the graph nodes of the dataflow graph comprise tensor operations.
  • the segmenting of the dataflow graph may comprise scalarization of the tensor operations, segmenting the graph nodes into scalar operations or cohorts of scalar operations, and grouping the scalar operations or cohorts of scalar operations into scalar operation groups based on the pre-determined infrastructure.
  • the scalar operation groups form the dataflow sub-graphs. This allows forming the dataflow sub-graphs based on the scalar operations and thus the forming of sub-graphs may take into account the scalar operations of the dataflow graph and also the infrastructure.
  • Segmenting the dataflow graph may comprise dividing the dataflow graph into the dataflow sub-graphs and grouping the sub-graphs into grouped dataflow sub-graphs based on the pre-determined infrastructure.
  • the sub graphs may be grouped taking into account the properties of the infrastructure.
  • the grouped dataflow sub-graphs may then be distributed to the pre-determined infrastructure.
  • the segmenting of the dataflow graph may comprise representing the tensor operations as scalar promises representing scalar operations to be performed, receiving input data, fulfilling the scalar promises based on the input data such that fulfilled scalar promises are formed from the scalar promises, segmenting the fulfilled scalar promises into scalar operations or cohorts of scalar operations, and grouping the scalar operations or cohorts of scalar operations into scalar operation groups based on the pre determined infrastructure, the scalar operation groups forming the dataflow sub graphs.
  • the segmenting further comprises repeating the following steps one or more times for fulfilling the scalar promises before the grouping of the scalar operations or cohorts of scalar operations: receiving input data, fulfilling the scalar promises based on the input data such that fulfilled scalar promises are formed from the scalar promises, and segmenting the fulfilled scalar promises into scalar operations or cohorts of scalar operations. This allows introducing information progressively into the dataflow graph and thus less time is required for the final step of graph segmentation due to the intermediary steps.
  • the segmenting of the dataflow graph also comprise merging redundant equivalent dataflow sub-graphs such that merged dataflow sub-graphs are formed. This means that two or more equivalent sub graphs or scalar operations or cohorts of scalar operations are merged into only one sub-graph or scalar operation or cohort of scalar operations which will be executed or performed only once, instead of executing or performing each separately.
  • Projecting the dataflow sub-graphs to the infrastructure comprises compiling the dataflow sub-graphs into a segmented dataflow graph comprising the dataflow sub-graphs.
  • the projecting of the dataflow sub-graphs to the infrastructure comprises compressing the dataflow sub-graphs into compressed dataflow sub-graphs and compiling the compressed dataflow sub-graphs into a segmented dataflow graph comprising the dataflow sub-graphs.
  • projecting the dataflow sub-graphs to the infrastructure may comprise compiling the dataflow sub-graphs into a segmented dataflow graph comprising the dataflow sub-graphs and compressing the segmented dataflow graph into compressed segmented dataflow sub-graphs. This allows forming the segmented dataflow graph which comprises the dataflow sub-graphs.
  • the segmented dataflow sub-graphs or the dataflow sub-graphs may be compressed using any known compression algorithm for grouping similar scalar operations. Thus, the projection and execution of the dataflow graphs may be more efficient.
  • defining the pre-determined infrastructure may comprise generating one or more sub-graph projection predictions based on the infrastructure, the dataflow sub-graph projection predictions representing the predicted optimal distribution of the dataflow sub-graphs to the infrastructure, and by defining the pre-determined infrastructure based on pre-defined infrastructure criteria, projecting the dataflow sub-graphs to the pre-determined infrastructure by projecting the one or more projection predictions to the infrastructure, executing the dataflow graph by executing the distributed dataflow sub-graphs according to one of the projection predictions.
  • the used infrastructure is defined as the pre-determined infrastructure, and further the one or more projection predictions are projected to the infrastructure.
  • One of the projection predictions is then decided to be executed in the infrastructure.
  • the one projection prediction to be executed may be decided based on the pre determined infrastructure or properties of the pre-determined infrastructure or the infrastructure nodes and infrastructure edges of the pre-determined infrastructure.
  • defining the pre-determined infrastructure may comprise generating the one or more projection predictions by creating a constraint optimization problem and utilizing a constraint optimization algorithm with pre-determined constraints, defining the pre-determined infrastructure from the one or more projection predictions by solving the constraint optimization problem, and projecting the dataflow sub-graphs to the hardware nodes and the network connections of the pre-determined infrastructure.
  • the determination of the pre-determined infrastructure may be accomplished by using the output of the constraint optimization algorithm as the target of the neural network with the infrastructure description as the training input, training the neural network and then using it to make projection predictions from infrastructure descriptions as the prediction input.
  • the step of defining the pre-determined infrastructure further comprises analysing the infrastructure and representing the infrastructure as one or more constraint optimization programs based on the analysing of the infrastructure, generating one or more projection predictions by utilizing the constraint optimization programs with constraints obtained from the analysing of the infrastructure, and defining the pre-determined infrastructure as the infrastructure used to generate the one or more constraint optimization programs.
  • the available infrastructure is known such that the available infrastructure is used as the pre-determined infrastructure.
  • the step of defining the pre-determined infrastructure further comprises analysing the infrastructure and representing the infrastructure as one or more constraint optimization programs based on the analysing of the infrastructure, generating one or more projection predictions by utilizing the constraint optimization programs with constraints obtained from the analysing of the infrastructure, and defining the pre-determined infrastructure based on the pre-defined infrastructure criteria.
  • properties of the available infrastructure are known, and defining the pre-determined infrastructure comprises defining the infrastructure nodes and edges of the available infrastructure which are used, based on the pre-defined infrastructure criteria.
  • the pre-defined infrastructure criteria may comprise properties of the infrastructure nodes and edges or the like.
  • the step of defining the pre-determined infrastructure further comprises analysing a potential infrastructure by generating synthesized representations of the infrastructure and representing the infrastructure as one or more constraint optimization programs based on the synthesized representations, generating one or more projection predictions based on the synthesized representations, and defining the pre-determined infrastructure based on the pre-determined criteria.
  • the available infrastructure is not known, and therefore synthesized representations of the infrastructure are generated based on the anticipations of bandwidth between the nodes, the computational power of the nodes, or the like.
  • the pre-determined infrastructure is defined based on the synthesized representations of the potential infrastructure or anticipations of the potential infrastructure.
  • the step of defining the pre determined infrastructure further comprises analysing a potential infrastructure by generating synthesized representations of the infrastructure and representing the infrastructure as one or more constraint optimization programs based on the synthesized representations, generating one or more projection predictions by utilizing a neural network trained with the results of the one or more constraint optimization programs as expected output values and the synthesized infrastructure representations as input values and making projection predictions using the same or different synthesized infrastructure representations as input, and defining the pre-determined infrastructure based on the pre-determined criteria.
  • the neural network is utilized for generating the projection predictions when the constraint optimization problem execution is a slow process.
  • the infrastructure may be first described.
  • a neural network is then further used to predict the distribution of the sub-graphs. The description of the infrastructure is received by the neural network as input and the output from the neural network is the projection prediction, or a prediction of the optimal distribution within a pre-determined threshold of accuracy.
  • the available infrastructure may be first analysed and the pre-determined infrastructure may be defined based on the pre-determined criteria such as computational power of available infrastructure nodes and/or available bandwidth of the infrastructure edges.
  • Projecting the dataflow sub-graphs to the infrastructure may comprise projecting the dataflow sub-graphs or the segmented dataflow graph to the pre determined infrastructure, or projecting the dataflow sub-graphs or the segmented dataflow graph to the pre-determined infrastructure such that each dataflow sub-graph is projected to a separate infrastructure node.
  • the dataflow sub-graphs may be distributed to most suitable infrastructure nodes and/or infrastructure edges. Accordingly, each dataflow sub-graph is projected to separate or different infrastructure node.
  • the execution of the dataflow graph comprises monitoring the infrastructure during the execution of the dataflow graph.
  • the execution of the dataflow graph may comprise monitoring the infrastructure during the execution of the dataflow graph and receiving a monitoring output, and comparing the monitoring output and a pre-determined expected monitoring output. If, based on the comparison, the monitoring output is within the pre-determined expected monitoring output, the execution of the dataflow graph continues utilizing the pre-determined infrastructure. If, based on the comparison, the monitoring output is outside the pre-determined expected monitoring output, the projection of the dataflow sub-graphs to the infrastructure is re-initiated.
  • the execution of the dataflow graph comprises monitoring the pre-determined infrastructure during the execution of the dataflow graph by spamming in the pre-determined infrastructure and receiving a spamming output.
  • the execution of the dataflow graph may comprise monitoring the pre-determined infrastructure nodes and the infrastructure edges of the pre-determined infrastructure during the execution of the dataflow graph and receiving a monitoring output, and comparing the monitoring output and a pre-determined expected monitoring output, and based on the comparison. If, based on the comparison, the monitoring output is within the pre-determined expected monitoring output, continuing the execution of the dataflow graph utilizing the infrastructure nodes and the infrastructure edges of the pre-determined infrastructure. If, based on the comparison, the monitoring output is outside the pre-determined expected monitoring output, re-initiating the projection of the dataflow sub-graphs to the infrastructure nodes and the infrastructure edges.
  • the execution of the dataflow graph comprises monitoring the pre-determined infrastructure during the execution of the dataflow graph by spamming in the pre-determined infrastructure and receiving a spamming output, and comparing the spamming output and a pre determined expected output. If, based on the comparison, the spamming output is within the pre-determined expected spamming output, the execution of the dataflow graph is continued. If, based on the comparison, the spamming output is outside the pre-determined expected spamming output, the projection of the dataflow sub-graphs to the infrastructure nodes and the infrastructure edges is re-initiated.
  • Monitoring the infrastructure enables dynamic adjustment of the distributed processing as the used infrastructure may be changed if properties of the pre-determined infrastructure change during the execution of the dataflow graph.
  • the execution step of the method according to the present invention further comprises progressively updating the dataflow sub graphs during the execution of the dataflow graph.
  • the updating is carried out by utilizing the communication protocol over the infrastructure edges of the pre determined infrastructure such that data may flow between the infrastructure nodes.
  • Progressively updating the dataflow sub-graphs during the execution of the dataflow graph comprises exchanging data between the sub-graphs executed in the infrastructure nodes of the pre-determined infrastructure and modifying the sub-graphs based on the data exchange.
  • the data exchange may be carried out by utilizing the communication protocol over the infrastructure edges of the pre determined infrastructure. Accordingly, during the execution of the dataflow graph, data flows through and between the sub-graphs, and the sub-graphs may be updated based on the data flowing through and between the sub-graphs.
  • the sub-graphs may modify during the execution of the dataflow graph based on the data.
  • the communication protocol may be configured to progressively update the dataflow sub-graphs during the execution of the dataflow graph, as disclosed above.
  • the invention is also based on the idea of providing a method for automatic distributed information processing using a dataflow graph comprising graph nodes and graph edges between the graph nodes.
  • the method comprises distributing a dataflow graph to an infrastructure comprising automatic infrastructure nodes and automatic infrastructure edges between the infrastructure nodes.
  • the method comprises segmenting the dataflow graph into dataflow sub-graphs, defining a pre-determined infrastructure having two or more infrastructure nodes and at least one infrastructure edge between the infrastructure nodes, projecting the dataflow sub-graphs to the pre-determined infrastructure by distributing the dataflow sub-graphs to the infrastructure nodes and infrastructure edges, and executing the dataflow graph by utilizing a communication protocol over the infrastructure edges between the infrastructure nodes in the execution of the dataflow graph and executing the distributed dataflow sub-graphs in the infrastructure nodes.
  • the method allows splitting the dataflow graph into dataflow sub-graphs and distributing the sub-graphs to infrastructure which is defined before distributing the sub-graphs. This allows forming the sub-graphs and distributing the sub-graphs such that the infrastructure properties may be taken into account for efficient processing and efficient use of the infrastructure.
  • the step of defining the pre determined infrastructure further comprises analysing a potential infrastructure by generating synthesized representations of the infrastructure and representing the infrastructure as one or more constraint optimization programs based on the synthesized representations, generating one or more projection predictions based on the synthesized representations, and defining the pre-determined infrastructure based on the pre-determined criteria.
  • the method further comprises projecting the dataflow sub-graphs to the pre-determined infrastructure by projecting the one or more projection predictions to the pre determined infrastructure, and executing the dataflow graph by executing the distributed dataflow sub-graphs according to one of the projection predictions.
  • the available infrastructure is not known, and therefore synthesized representations of the infrastructure are generated based on the anticipations of bandwidth between the nodes, the computational power of the nodes, or the like.
  • the pre-determined infrastructure is defined based on the synthesized representations of the potential infrastructure or anticipations of the potential infrastructure.
  • segmenting the dataflow graph comprises dividing the dataflow graph into the dataflow sub-graphs and grouping the sub graphs into grouped dataflow sub-graphs based on the pre-determined infrastructure.
  • the sub-graphs may be grouped taking into account the properties of the infrastructure.
  • the grouped dataflow sub-graphs may then be distributed to the pre-determined infrastructure.
  • the graph nodes of the dataflow graph comprise tensor operations.
  • the segmenting of the dataflow graph may comprise scalarization of the tensor operations, segmenting the graph nodes into scalar operations or cohorts of scalar operations, and grouping the scalar operations or cohorts of scalar operations into scalar operation groups based on the pre determined infrastructure.
  • the scalar operation groups form the dataflow sub graphs. This allows forming the dataflow sub-graphs based on the scalar operations and thus the forming of sub-graphs may take into account the scalar operations of the dataflow graph and also the infrastructure.
  • the segmenting of the dataflow graph may comprise representing the tensor operations as scalar promises representing scalar operations to be performed, receiving input data, fulfilling the scalar promises based on the input data such that fulfilled scalar promises are formed from the scalar promises, segmenting the fulfilled scalar promises into scalar operations or cohorts of scalar operations, and grouping the scalar operations or cohorts of scalar operations into scalar operation groups based on the pre determined infrastructure, the scalar operation groups forming the dataflow sub graphs.
  • the segmenting further comprises repeating the following steps one or more times for fulfilling the scalar promises before the grouping of the scalar operations or cohorts of scalar operations: receiving input data, fulfilling the scalar promises based on the input data such that fulfilled scalar promises are formed from the scalar promises, and segmenting the fulfilled scalar promises into scalar operations or cohorts of scalar operations. This allows introducing information progressively into the dataflow graph and thus less time is required for the final step of graph segmentation due to the intermediary steps.
  • the segmenting of the dataflow graph also comprises merging redundant equivalent dataflow sub-graphs such that merged dataflow sub-graphs are formed. This means that two or more equivalent sub graphs or scalar operations or cohorts of scalar operations are merged into only one sub-graph or scalar operation or cohort of scalar operations which will be executed or performed only once, instead of executing or performing each separately.
  • Projecting the dataflow sub-graphs to the infrastructure comprises compiling the dataflow sub-graphs into a segmented dataflow graph comprising the dataflow sub-graphs.
  • the projecting of the dataflow sub-graphs to the infrastructure comprises compressing the dataflow sub-graphs into compressed dataflow sub-graphs and compiling the compressed dataflow sub-graphs into a segmented dataflow graph comprising the dataflow sub-graphs.
  • projecting the dataflow sub-graphs to the infrastructure may comprise compiling the dataflow sub-graphs into a segmented dataflow graph comprising the dataflow sub-graphs and compressing the segmented dataflow graph into compressed segmented dataflow sub-graphs. This allows forming the segmented dataflow graph which comprises the dataflow sub-graphs.
  • the segmented dataflow sub-graphs or the dataflow sub-graphs may be compressed using any known compression algorithm for grouping similar scalar operations. Thus, the projection and execution of the dataflow graphs may be more efficient.
  • the step of defining the pre-determined infrastructure further comprises analysing a potential infrastructure by generating synthesized representations of the infrastructure and representing the infrastructure as one or more constraint optimization programs based on the synthesized representations, generating one or more projection predictions by utilizing a neural network trained with the results of the one or more constraint optimization programs as expected output values and the synthesized infrastructure representations as input values and making projection predictions using the same or different synthesized infrastructure representations as input, and defining the pre-determined infrastructure based on the pre-determined criteria.
  • the neural network is utilized for generating the projection predictions when the constraint optimization problem execution is a slow process.
  • the infrastructure may be first described.
  • a neural network is then further used to predict the distribution of the sub-graphs. The description of the infrastructure is received by the neural network as input and the output from the neural network is the projection prediction, or a prediction of the optimal distribution within a pre-determined threshold of accuracy.
  • the available infrastructure may be first analysed and the pre-determined infrastructure may be defined based on the pre-determined criteria such as computational power of available infrastructure nodes and/or available bandwidth of the infrastructure edges.
  • Projecting the dataflow sub-graphs to the infrastructure may comprise projecting the dataflow sub-graphs or the segmented dataflow graph to the pre determined infrastructure, or projecting the dataflow sub-graphs or the segmented dataflow graph to the pre-determined infrastructure such that each dataflow sub-graph is projected to a separate infrastructure node.
  • the dataflow sub-graphs may be distributed to most suitable infrastructure nodes and/or infrastructure edges. Accordingly, each dataflow sub-graph is projected to separate or different infrastructure node.
  • the execution of the dataflow graph comprises monitoring the infrastructure during the execution of the dataflow graph.
  • the execution of the dataflow graph may comprise monitoring the infrastructure during the execution of the dataflow graph and receiving a monitoring output, and comparing the monitoring output and a pre-determined expected monitoring output. If, based on the comparison, the monitoring output is within the pre-determined expected monitoring output, the execution of the dataflow graph continues utilizing the pre-determined infrastructure. If, based on the comparison, the monitoring output is outside the pre-determined expected monitoring output, the projection of the dataflow sub-graphs to the infrastructure is re-initiated.
  • the execution of the dataflow graph comprises monitoring the pre-determined infrastructure during the execution of the dataflow graph by spamming in the pre-determined infrastructure and receiving a spamming output.
  • the execution of the dataflow graph may comprise monitoring the pre-determined infrastructure nodes and the infrastructure edges of the pre-determined infrastructure during the execution of the dataflow graph and receiving a monitoring output, and comparing the monitoring output and a pre-determined expected monitoring output, and based on the comparison. If, based on the comparison, the monitoring output is within the pre-determined expected monitoring output, continuing the execution of the dataflow graph utilizing the infrastructure nodes and the infrastructure edges of the pre-determined infrastructure. If, based on the comparison, the monitoring output is outside the pre-determined expected monitoring output, re-initiating the projection of the dataflow sub-graphs to the infrastructure nodes and the infrastructure edges.
  • the execution of the dataflow graph comprises monitoring the pre-determined infrastructure during the execution of the dataflow graph by spamming in the pre-determined infrastructure and receiving a spamming output, and comparing the spamming output and a pre determined expected output. If, based on the comparison, the spamming output is within the pre-determined expected spamming output, the execution of the dataflow graph is continued. If, based on the comparison, the spamming output is outside the pre-determined expected spamming output, the projection of the dataflow sub-graphs to the infrastructure nodes and the infrastructure edges is re-initiated.
  • Monitoring the infrastructure enables dynamic adjustment of the distributed processing as the used infrastructure may be changed if properties of the pre-determined infrastructure change during the execution of the dataflow graph.
  • the execution step of the method according to the present invention further comprises progressively updating the dataflow sub graphs during the execution of the dataflow graph.
  • the updating is carried out by utilizing the communication protocol over the infrastructure edges of the pre determined infrastructure such that data may flow between the infrastructure nodes.
  • Progressively updating the dataflow sub-graphs during the execution of the dataflow graph comprises exchanging data between the sub-graphs executed in the infrastructure nodes of the pre-determined infrastructure and modifying the sub-graphs based on the data exchange.
  • the data exchange may be carried out by utilizing the communication protocol over the infrastructure edges of the pre determined infrastructure. Accordingly, during the execution of the dataflow graph, data flows through and between the sub-graphs, and the sub-graphs may be updated based on the data flowing through and between the sub-graphs.
  • the sub-graphs may modify during the execution of the dataflow graph based on the data.
  • the communication protocol may be configured to progressively update the dataflow sub-graphs during the execution of the dataflow graph, as disclosed above.
  • the present invention also relates to an automatic distributed information processing system using a dataflow graph comprising graph nodes and graph edges between the graph nodes.
  • the system comprises a distributed infrastructure comprising automatic hardware nodes and automatic network connections between the hardware nodes.
  • the system is configured to automatically distribute a dataflow graph to the infrastructure.
  • the system comprises a segmentation server comprising a segmentation module configured to segment the dataflow graph into dataflow sub-graphs, the segmentation server comprising a projection module configured to define a pre-determined infrastructure having two or more hardware nodes and at least one network connection between the hardware nodes, and to project the dataflow sub-graphs to the pre-determined infrastructure by distributing the dataflow sub-graphs to the hardware nodes and network connections for execution of the dataflow graph, and a communication protocol provided to the hardware nodes, the communication protocol being configured to execute the dataflow graph over the network connections by executing the distributed dataflow sub-graphs in the hardware nodes.
  • the system of the invention is configured to split the dataflow graph into dataflow sub-graphs and distribute the sub-graphs to the hardware nodes and network connections determined before the distribution of the sub-graphs. Then the sub-graphs are executed by the pre determined infrastructure in a distributed manner.
  • the segmentation server is configured to segment, by utilizing instructions of the segmentation module, the dataflow graph by scalarization of tensor operations, the graph nodes of the dataflow graph comprising the tensor operations, segmenting the graph nodes into scalar operations or cohorts of scalar operations, and grouping the scalar operations or cohorts of scalar operations into scalar operation groups based on the pre-determined infrastructure, the scalar operation groups forming the dataflow sub-graphs.
  • the system comprises the segmentation server comprising the segmentation module, or the segmentation server comprising a segmentation memory configured to store the segmentation module and a segmentation processor configured to execute instructions of the segmentation module.
  • the system comprises the segmentation server comprising the segmentation module and the projection module, the segmentation server comprising a segmentation memory configured to store the segmentation module and the projection module, and a segmentation processor configured to execute instructions of the segmentation module and the projection module.
  • the segmentation server is configured to segment, by utilizing instructions of the segmentation module executed by the segmentation processor, the dataflow graph by dividing the dataflow graph into the dataflow sub-graphs, and grouping the dataflow sub-graphs into grouped dataflow sub-graphs based on the pre-determined infrastructure.
  • the segmentation server is configured to segment, by utilizing instructions of the segmentation module executed by the segmentation processor, the dataflow graph by scalarization of tensor operations of the graph nodes into scalar operations, representing tensor operations as scalar promises representing scalar operations to be performed, the graph nodes of the dataflow graph comprising the tensor operations, receiving input data to the segmentation module, fulfilling the scalar promises based on the input data such that fulfilled scalar promises are formed from the scalar promises, segmenting the fulfilled scalar promises into scalar operations or cohorts of scalar operations, grouping the scalar operations or cohorts of scalar operations into scalar operation groups based on the pre-determined infrastructure, scalar operation groups forming the dataflow sub-graphs.
  • the segmentation server is configured to segment, by utilizing instructions of the segmentation module executed by the segmentation processor, the dataflow graph by merging redundant equivalent dataflow sub-graphs such that merged dataflow sub-graphs are formed. This enables decreasing the number operations to be performed as two or more equivalent sub-graphs or scalar operations or cohorts of scalar operations may be merged into only one sub-graph or scalar operation or cohort of scalar operations which will be executed or performed only once, instead of executing or performing each separately.
  • the segmentation server is further configured to project, by utilizing instructions of the projection module and executed by the segmentation processor, the dataflow sub-graphs to the infrastructure by defining the pre-determined infrastructure by generating one or more sub-graph projection predictions based on the infrastructure, the dataflow sub-graph projections representing the distribution of the dataflow sub-graphs to the infrastructure, and by defining the hardware nodes and network connections of the pre-determined infrastructure based on pre-defined infrastructure criteria, and projecting the dataflow sub-graphs to the hardware nodes and network connections of the pre-determined infrastructure by projecting the one or more projection predictions to the hardware nodes and network connections of the pre-determined infrastructure.
  • the communication protocol is configured to execute the dataflow graph over the network connections of the pre-determined infrastructure by executing the distributed dataflow sub-graphs according to one of the projection predictions in the hardware nodes of the pre determined infrastructure.
  • the projection module is configured to or the segmentation server is configured to project, by utilizing instructions of the projection module and executed by the segmentation processor, the dataflow sub-graphs to the infrastructure by analysing the infrastructure and representing the infrastructure as one or more constraint optimization programs based on the analysing of the infrastructure, generating one or more projection predictions by utilizing the constraint optimization programs with constraints obtained from the analysing of the infrastructure, and defining the pre-determined infrastructure as the infrastructure used to generate the one or more constraint optimization programs.
  • the projection module is configured to or the segmentation server is configured to project, by utilizing instructions of the projection module and executed by the segmentation processor, the dataflow sub-graphs to the infrastructure by analysing the infrastructure and representing the infrastructure as one or more constraint optimization programs based on the analysing of the infrastructure, generating one or more projection predictions by utilizing the constraint optimization programs with constraints obtained from the analysing of the infrastructure, and defining the pre-determined infrastructure based on the pre-defined infrastructure criteria.
  • the pre-determined infrastructure may be formed based on the available hardware nodes and network connections.
  • the projection module is configured to or the segmentation server is configured to project, by utilizing instructions of the projection module and executed by the segmentation processor, the dataflow sub-graphs to the infrastructure by analysing a potential infrastructure by generating synthesized representations of the infrastructure and representing the infrastructure as one or more constraint optimization programs based on the synthesized representations, generating one or more projection predictions based on the synthesized representations, and defining the pre-determined infrastructure based on the pre-determined criteria.
  • the projection module is configured to or the segmentation server is configured to project, by utilizing instructions of the projection module and executed by the segmentation processor, the dataflow sub-graphs to the infrastructure by analysing a potential infrastructure by generating synthesized representations of the infrastructure and representing the infrastructure as one or more constraint optimization programs based on the synthesized representations, training a neural using the results of the one or more constraint optimization programs as targets and the synthesized representations of the infrastructure as inputs, and generating one or more projection predictions by utilizing the trained neural network to generated one or more projection projections as output from the input of one or more synthesized infrastructure representations, and defining the pre-determined infrastructure based on the pre-determined criteria.
  • the projection module may be configured to or the segmentation server may configured to project, by utilizing instructions of the projection module and executed by the segmentation processor, the dataflow sub-graphs to the infrastructure by analysing a potential infrastructure by generating synthesized representations of the infrastructure and representing the infrastructure as one or more constraint optimization programs based on the synthesized representations, generating one or more projection predictions by utilizing a neural network trained with the results of the one or more constraint optimization programs as expected output values and the synthesised infrastructure representations as input values and making projection predictions using the same or different synthesised infrastructure representations as input, and defining the pre-determined infrastructure based on the pre-determined criteria.
  • the projection module is configured to or the segmentation server is configured to project, by utilizing instructions of the projection module and executed by the segmentation processor, the dataflow sub-graphs to the infrastructure by distributing the dataflow sub-graphs to the hardware nodes and network connections of the pre-determined infrastructure for execution of the dataflow graph, or by distributing the dataflow sub-graphs to the hardware nodes and network connections of the pre-determined infrastructure such that each dataflow sub-graph is projected to a separate hardware node.
  • system of the present invention further comprises a monitoring module configured to or the segmentation server comprises a monitoring module configured to monitor, by utilizing instructions of the monitoring module and executed by the segmentation processor, the infrastructure by monitoring the hardware nodes and the network connections of the pre-determined infrastructure during the execution of the dataflow graph.
  • the system of the present invention further comprises a monitoring module configured to or the segmentation server comprises a monitoring module configured to monitor, by utilizing instructions of the monitoring module and executed by the segmentation processor, the infrastructure by monitoring the hardware nodes and the network connections of the pre-determined infrastructure during the execution of the dataflow graph and receiving a monitoring output, comparing the monitoring output and a pre determined expected output. If, based on the comparison, the monitoring output is within the pre-determined expected output, continuing the execution of the dataflow graph utilizing the hardware nodes and the network connections of the pre-determined infrastructure. If, based on the comparison, the monitoring output is outside the pre-determined expected output, re-initiating the projection of the dataflow sub-graphs to the hardware nodes and the network connections.
  • a monitoring module configured to or the segmentation server comprises a monitoring module configured to monitor, by utilizing instructions of the monitoring module and executed by the segmentation processor, the infrastructure by monitoring the hardware nodes and the network connections of the pre-determined infrastructure during the execution of the dataflow graph and receiving a monitoring output,
  • system of the present invention further comprises a monitoring module configured to or the segmentation server comprises a monitoring module configured to monitor, by utilizing instructions of the monitoring module and executed by the segmentation processor, the infrastructure by monitoring the hardware nodes and the network connections of the pre-determined infrastructure during the execution of the dataflow graph by spamming in the hardware nodes and the network connections and receiving a spamming output.
  • the system of the present invention further comprises a monitoring module configured to or the segmentation server comprises a monitoring module configured to monitor, by utilizing instructions of the monitoring module and executed by the segmentation processor, the infrastructure by monitoring the hardware nodes and the network connections of the pre-determined infrastructure during the execution of the dataflow graph by spamming in the hardware nodes and the network connections of the pre determined infrastructure and receiving a spamming output, and comparing the spamming output and a pre-determined expected output, and based on the comparison. If, based on the comparison, the spamming output is within the pre determined expected output, continuing the execution of the dataflow graph utilizing the hardware nodes and the network connections of the pre-determined infrastructure. If, based on the comparison, the spamming output is outside the pre-determined expected output, re-initiating the projection of the dataflow sub graphs to the hardware nodes and the network connections.
  • the monitoring module enables changing the used infrastructure if properties of the pre-determined infrastructure change during the projection or execution of the dataflow graph.
  • the present invention further relates to an automatic distributed information processing system using a dataflow graph comprising graph nodes and graph edges between the graph nodes.
  • the system comprises a distributed infrastructure comprising automatic hardware nodes and automatic network connections between the hardware nodes.
  • the system is configured to automatically distribute a dataflow graph to the infrastructure.
  • the system comprises a segmentation server comprising a segmentation module configured to segment the dataflow graph into dataflow sub-graphs, the segmentation server comprising a projection module configured to define a pre-determined infrastructure having two or more hardware nodes and at least one network connection between the hardware nodes, and to project the dataflow sub-graphs to the pre-determined infrastructure by distributing the dataflow sub-graphs to the hardware nodes and network connections for execution of the dataflow graph, and a communication protocol provided to the hardware nodes, the communication protocol being configured to execute the dataflow graph over the network connections by executing the distributed dataflow sub-graphs in the hardware nodes.
  • the system of the invention is configured to split the dataflow graph into dataflow sub-graphs and distribute the sub-graphs to the hardware nodes and network connections determined before the distribution of the sub-graphs. Then the sub-graphs are executed by the pre determined infrastructure in a distributed manner.
  • the projection module is configured to or the segmentation server is configured to project, by utilizing instructions of the projection module and executed by the segmentation processor, the dataflow sub-graphs to the infrastructure by analysing a potential infrastructure by generating synthesized representations of the infrastructure and representing the infrastructure as one or more constraint optimization programs based on the synthesized representations, generating one or more projection predictions based on the synthesized representations, and defining the pre-determined infrastructure based on the pre-determined criteria.
  • the system comprises the segmentation server comprising the segmentation module, or the segmentation server comprising a segmentation memory configured to store the segmentation module and a segmentation processor configured to execute instructions of the segmentation module.
  • the system comprises the segmentation server comprising the segmentation module and the projection module, the segmentation server comprising a segmentation memory configured to store the segmentation module and the projection module, and a segmentation processor configured to execute instructions of the segmentation module and the projection module.
  • the segmentation server is configured to segment, by utilizing instructions of the segmentation module executed by the segmentation processor, the dataflow graph by dividing the dataflow graph into the dataflow sub-graphs, and grouping the dataflow sub-graphs into grouped dataflow sub-graphs based on the pre-determined infrastructure.
  • the segmentation server is configured to segment, by utilizing instructions of the segmentation module, the dataflow graph by scalarization of tensor operations, the graph nodes of the dataflow graph comprising the tensor operations, segmenting the graph nodes into scalar operations or cohorts of scalar operations, and grouping the scalar operations or cohorts of scalar operations into scalar operation groups based on the pre- determined infrastructure, the scalar operation groups forming the dataflow sub graphs.
  • the segmentation server is configured to segment, by utilizing instructions of the segmentation module executed by the segmentation processor, the dataflow graph by scalarization of tensor operations of the graph nodes into scalar operations, representing tensor operations as scalar promises representing scalar operations to be performed, the graph nodes of the dataflow graph comprising the tensor operations, receiving input data to the segmentation module, fulfilling the scalar promises based on the input data such that fulfilled scalar promises are formed from the scalar promises, segmenting the fulfilled scalar promises into scalar operations or cohorts of scalar operations, grouping the scalar operations or cohorts of scalar operations into scalar operation groups based on the pre-determined infrastructure, scalar operation groups forming the dataflow sub-graphs.
  • the segmentation server is configured to segment, by utilizing instructions of the segmentation module executed by the segmentation processor, the dataflow graph by merging redundant equivalent dataflow sub-graphs such that merged dataflow sub-graphs are formed. This enables decreasing the number operations to be performed as two or more equivalent sub-graphs or scalar operations or cohorts of scalar operations may be merged into only one sub-graph or scalar operation or cohort of scalar operations which will be executed or performed only once, instead of executing or performing each separately.
  • the segmentation server is further configured to project, by utilizing instructions of the projection module and executed by the segmentation processor, the dataflow sub-graphs to the infrastructure by defining the pre-determined infrastructure by generating one or more sub-graph projection predictions based on the infrastructure, the dataflow sub-graph projections representing the distribution of the dataflow sub-graphs to the infrastructure, and by defining the hardware nodes and network connections of the pre-determined infrastructure based on pre-defined infrastructure criteria, and projecting the dataflow sub-graphs to the hardware nodes and network connections of the pre-determined infrastructure by projecting the one or more projection predictions to the hardware nodes and network connections of the pre-determined infrastructure.
  • the communication protocol is configured to execute the dataflow graph over the network connections of the pre-determined infrastructure by executing the distributed dataflow sub-graphs according to one of the projection predictions in the hardware nodes of the pre determined infrastructure.
  • the projection module is configured to or the segmentation server is configured to project, by utilizing instructions of the projection module and executed by the segmentation processor, the dataflow sub-graphs to the infrastructure by analysing a potential infrastructure by generating synthesized representations of the infrastructure and representing the infrastructure as one or more constraint optimization programs based on the synthesized representations, training a neural using the results of the one or more constraint optimization programs as targets and the synthesized representations of the infrastructure as inputs, and generating one or more projection predictions by utilizing the trained neural network to generated one or more projection projections as output from the input of one or more synthesized infrastructure representations, and defining the pre-determined infrastructure based on the pre-determined criteria.
  • the projection module may be configured to or the segmentation server may configured to project, by utilizing instructions of the projection module and executed by the segmentation processor, the dataflow sub-graphs to the infrastructure by analysing a potential infrastructure by generating synthesized representations of the infrastructure and representing the infrastructure as one or more constraint optimization programs based on the synthesized representations, generating one or more projection predictions by utilizing a neural network trained with the results of the one or more constraint optimization programs as expected output values and the synthesised infrastructure representations as input values and making projection predictions using the same or different synthesised infrastructure representations as input, and defining the pre-determined infrastructure based on the pre-determined criteria.
  • the projection module is configured to or the segmentation server is configured to project, by utilizing instructions of the projection module and executed by the segmentation processor, the dataflow sub-graphs to the infrastructure by distributing the dataflow sub-graphs to the hardware nodes and network connections of the pre-determined infrastructure for execution of the dataflow graph, or by distributing the dataflow sub-graphs to the hardware nodes and network connections of the pre-determined infrastructure such that each dataflow sub-graph is projected to a separate hardware node.
  • system of the present invention further comprises a monitoring module configured to or the segmentation server comprises a monitoring module configured to monitor, by utilizing instructions of the monitoring module and executed by the segmentation processor, the infrastructure by monitoring the hardware nodes and the network connections of the pre-determined infrastructure during the execution of the dataflow graph.
  • the system of the present invention further comprises a monitoring module configured to or the segmentation server comprises a monitoring module configured to monitor, by utilizing instructions of the monitoring module and executed by the segmentation processor, the infrastructure by monitoring the hardware nodes and the network connections of the pre-determined infrastructure during the execution of the dataflow graph and receiving a monitoring output, comparing the monitoring output and a pre determined expected output. If, based on the comparison, the monitoring output is within the pre-determined expected output, continuing the execution of the dataflow graph utilizing the hardware nodes and the network connections of the pre-determined infrastructure. If, based on the comparison, the monitoring output is outside the pre-determined expected output, re-initiating the projection of the dataflow sub-graphs to the hardware nodes and the network connections.
  • a monitoring module configured to or the segmentation server comprises a monitoring module configured to monitor, by utilizing instructions of the monitoring module and executed by the segmentation processor, the infrastructure by monitoring the hardware nodes and the network connections of the pre-determined infrastructure during the execution of the dataflow graph and receiving a monitoring output,
  • system of the present invention further comprises a monitoring module configured to or the segmentation server comprises a monitoring module configured to monitor, by utilizing instructions of the monitoring module and executed by the segmentation processor, the infrastructure by monitoring the hardware nodes and the network connections of the pre-determined infrastructure during the execution of the dataflow graph by spamming in the hardware nodes and the network connections and receiving a spamming output.
  • the system of the present invention further comprises a monitoring module configured to or the segmentation server comprises a monitoring module configured to monitor, by utilizing instructions of the monitoring module and executed by the segmentation processor, the infrastructure by monitoring the hardware nodes and the network connections of the pre-determined infrastructure during the execution of the dataflow graph by spamming in the hardware nodes and the network connections of the pre determined infrastructure and receiving a spamming output, and comparing the spamming output and a pre-determined expected output, and based on the comparison. If, based on the comparison, the spamming output is within the pre determined expected output, continuing the execution of the dataflow graph utilizing the hardware nodes and the network connections of the pre-determined infrastructure. If, based on the comparison, the spamming output is outside the pre-determined expected output, re-initiating the projection of the dataflow sub graphs to the hardware nodes and the network connections.
  • the monitoring module enables changing the used infrastructure if properties of the pre-determined infrastructure change during the projection or execution of the dataflow graph.
  • An advantage of the invention is that it allows the dataflow graph to be distributed as sub-graphs to the infrastructure nodes and infrastructure edges based on the pre-determined infrastructure.
  • the pre-determined infrastructure may be defined based on the properties of the available infrastructure nodes and infrastructure edges. Therefore, a dynamic dataflow graph distribution may be achieved such that sub-graphs may be distributed to the infrastructure nodes and edges without sending the whole dataflow graph to each infrastructure node saving both data transfer amount infrastructure edges and computational power of the infrastructure nodes.
  • defining the pre-determined infrastructure before distributing the dataflow sub-graphs enables choosing the dataflow sub-graphs to be distributed to certain infrastructure nodes and edges based on the computational power and bandwidth of each infrastructure node and edge. Therefore, this enables efficient use of computation power and bandwidth and dataflow graph may be executed faster.
  • Figure 1 shows a schematic view of a dataflow graph
  • Figure 2 shows schematically steps of a method according to the present invention
  • Figure 3 shows a schematic view of a system according to the present invention
  • Figure 4 shows a schematic view of a segmentation server of a system according to the present invention
  • Figure 5 shows a schematic view of another segmentation server of a system according to the present invention.
  • Figure 6 shows a schematic view of a hardware device forming an infrastructure node.
  • Complex computations of information may be expressed as a dataflow through a directed graph, called a dataflow graph.
  • the dataflow graph comprises graph nodes and graph edges between the graph nodes.
  • the graph nodes represent the components of the computation or tensors of the dataflow graph.
  • the graph edges represent dataflows between the nodes of the graph.
  • e 2x data ® exponentiation ® squared
  • e 2x is a dataflow graph with two graph edges between three graph nodes: data, exponentiation, and squared, of which the last two are tensors and the first one is a matrix.
  • Tensors are geometric objects that describe linear relations between geometric vectors, scalars, and other tensors. Examples of such relations include the dot product, the cross product, linear maps, and other mathematical operations. Geometric vectors, often used in physics and engineering applications, and scalars themselves are also tensors.
  • the dataflow graph in the context of the present application be a tensor graph comprising the graph nodes and graph edges.
  • Figure 1 shows a schematic view a dataflow graph 50 comprising five nodes 52 each of which may comprise operations such as tensor operations.
  • the dataflow graph 50 further comprises graph edges 54 between the graph nodes 52.
  • the graph edges 54 represent the relationship of the graph nodes 52.
  • One graph node may comprise one or more operations or tensor operations.
  • Figure 2 shows schematically a diagram of the method for distributed information processing using the dataflow graph 50 according to the present invention.
  • the method comprises distributing the dataflow graph 50 to an infrastructure comprising infrastructure nodes and infrastructure edges between the infrastructure nodes.
  • the infrastructure 1 comprises hardware devices formed as the infrastructure nodes 6 of the infrastructure and carrying out the computation of the graph nodes 52 or operations of the graph nodes 52.
  • the infrastructure nodes 6 may be for example computers, mobile phones, servers or the like hardware devices. Therefore, the infrastructure nodes 6 may be processors or computing elements having computing capacity and power capable of executing the graph nodes 52 or operations of the graph nodes 52, such as tensor operations.
  • one infrastructure node 6 may be one hardware device or one computing element or alternatively one hardware device or one computing element may provide two or more infrastructure nodes 6. The latter may be accomplished by virtually dividing the computing element, the hardware device or a processor to two or more infrastructure sub-nodes.
  • one hardware device may comprise also two or more computing elements such as processors. Therefore, in the context of this application infrastructure node 6 or hardware node means one real hardware device, computing element or processor, or one virtually divided sub-element of the hardware device, computing element or processor. It may also comprise several connected real hardware devices that are, through an abstraction layer, treated as one single node. A single hardware node may contain both virtually divided sub-elements and be part of an abstraction that aggregates it onto a single virtual node.
  • the infrastructure may comprise one or more different kind of infrastructure nodes 6 or hardware devices. The present invention is not limited any type infrastructure node(s) or hardware device(s), but they may vary in different to embodiments.
  • FIG. 6 shows schematically one infrastructure node or hardware node 6.
  • the infrastructure node 6 may comprise an infrastructure node memory 32 configured to store information and instructions, such as information of the sub-graph which is projected to the infrastructure node 6 and instructions to execute the sub-graph.
  • the infrastructure node 6 may comprise an infrastructure node processor 34 configured to execute instructions of the sub-graph by utilizing the instructions stored in the infrastructure node memory 32.
  • the infrastructure node 6 may also comprise node application programming interface (API) 30 for communication with other infrastructure nodes 6 via the infrastructure edges 8.
  • API application programming interface
  • the node API 30 may be known as a kind of API module.
  • the infrastructure node 6 may further comprise a communication protocol provided to a communication protocol module 36.
  • the communication protocol may be any known communication protocol, such as a Transmission Control Protocol (TCP) or a peer-to-peer protocol, configured to execute the dataflow graph over the network connections or infrastructure edges 8 of the infrastructure 1 by executing the distributed dataflow sub-graphs in the infrastructure nodes 6.
  • TCP Transmission Control Protocol
  • peer-to-peer protocol configured to execute the dataflow graph over the network connections or infrastructure edges 8 of the infrastructure 1 by executing the distributed dataflow sub-graphs in the infrastructure nodes 6.
  • the infrastructure edges 8 comprise network connections between the infrastructure nodes 6 enabling data transfer between the infrastructure nodes 6.
  • the infrastructure edges 8 or the network connections may comprise any known network connection such as mobile phone network, local area network, wifi, wired network, fiber-optic network or the like. It should be noted that that the present invention is not limited to any specific network.
  • the infrastructure may comprise one or more different kind of infrastructure nodes 6 or hardware devices.
  • the present invention is not limited any type infrastructure edge(s) or network connection(s), but they may vary in different to embodiments.
  • the method comprises segmenting the dataflow graph 50 into dataflow sub-graphs. Segmenting means a process of splitting the dataflow graph 50 into sub-graphs based on a segmentation protocol. Accordingly, the dataflow graph 50 is divided into two or more sub-graphs in the segmentation step 100, as shown in figure 2.
  • One sub-graph may comprise one or more graph nodes 52 and one or more graph edges 8.
  • the one sub-graph may comprise one or more operations or tensor operations, or one or more tensors, and one or more graph edges 8.
  • the dataflow graph 50 is divided into smallest possible sub graphs such that the graph nodes of the sub-graph comprise only one operation, tensor operation or tensor.
  • the step 100 of the present invention may comprise segmenting two or more separate dataflow graphs 50 into sub-graphs. Continuing from that, the sub-graphs formed from the two or more dataflow graphs are processed further in the method of the present invention similarly manner together as when only one dataflow graph 50 is segmented in step 100.
  • the method further comprises defining a pre-determined infrastructure 1 having two or more infrastructure nodes and at least one infrastructure edge between the infrastructure nodes.
  • the step 200 of defining the pre-determined infrastructure 1 comprises defining the infrastructure nodes 6 and infrastructure edges 8 which will be used for processing or executing he dataflow graph 50 or the dataflow sub-graphs. Accordingly, the step 200 comprises choosing the pre-determined infrastructure 1, meaning pre determined infrastructure nodes 6 and infrastructure edges 8, from available infrastructure, meaning all available infrastructure nodes 6 and infrastructure edges.
  • the pre-determined infrastructure defined in step 200 may comprise the fixed infrastructure as such or it may comprise only part of the fixed infrastructure or part of the infrastructure nodes 6 and infrastructure edges.
  • pre-determined infrastructure 1 always comprises a separately defined number of infrastructure nodes 6 and infrastructure edges 8.
  • the method of the present invention further comprises projecting the dataflow sub-graphs onto the pre-determined infrastructure by distributing the dataflow sub-graphs to the infrastructure nodes and infrastructure edges in step 300, as shown in figure 2.
  • the formed dataflow sub-graphs, or the graph nodes and graph edges of the dataflow sub-graphs are distributed or transmitted to the pre-determined infrastructure 1, meaning the infrastructure nodes 6 and infrastructure edges 8 of the pre-determined infrastructure 1.
  • the process of nodel design and segmentation may be re-initiated with a new model so as to attain the desired targets before the actual projection to the infrastructure.
  • Distributing the sub-graphs to the pre-determined infrastructure 1 usually comprises assigning or transmitting the sub-graphs to the pre determined infrastructure 1 such that each sub-graph node is assigned to one infrastructure node 6 and each sub-graph edge is assigned to one infrastructure edge 8. Accordingly, the pre-determined infrastructure 1 may correspond the sub-graphs.
  • the method further comprises executing the dataflow graph in step 400 by utilizing a communication protocol over the infrastructure edges of the pre-determined infrastructure 1 between the infrastructure nodes of the pre determined infrastructure 1 in the execution of the dataflow graph and executing the distributed dataflow sub-graphs in the infrastructure nodes of the pre determined infrastructure 1.
  • the steps 100, 200, and 300 are usually carried out in central computing unit 2, which may be server, a cluster of servers connected to each other via a communication network or direct data transfer connections.
  • the central computing unit is defined as a segmentation server 2.
  • the segmentation server 2 may project the dataflow sub-graphs to the infrastructure nodes 6 utilizing projection network connections 4, as shown in figure 3.
  • the projection network connections 4 may be a known kind of network connection, similarly as disclosed above in connection with the infrastructure edges 8.
  • the present invention is not limited to particular type of projection network connections 4.
  • the segmentation server may comprise a segmentation memory or segmentation memories 10, as shown in figure 4.
  • the segmentation memory or memories 10 may be internal components of the segmentation server 2 or they may be external memories connected to the segmentation server 2.
  • the segmentation memory 10 may further be known kind of memory or physical memory component capable of storing information.
  • the segmentation server 2 may further comprise a segmentation processor or processor(s) 12.
  • the segmentation processor(s) 12 may be internal components of the segmentation server 2 or they may be external processors connected to the segmentation server 2.
  • the segmentation processor 12 may further be known kind of processor or physical computing component capable of executing operations.
  • the segmentation server application programming interface (SAP I) 14 for communication with the infrastructure nodes 6 via the projection network connections 4.
  • the segmentation SAPI 14 may be known kind of SAPI module such as Apache HTTP Server, Microsoft IIS, and Oracle iPlanet Web Server.
  • the segmentation server 2 may also comprise a segmentation module 20 configured to segment the dataflow graph 50 according to the step 100, as shown in figure 5.
  • the segmentation memory 10 may be configured to store the segmentation module 20 and the segmentation processor 12 may be configured to execute instructions of the segmentation module 20.
  • the segmentation server 2 may also comprise a projection module 22 configured to project the sub-graphs to pre-determined infrastructure 1 according to the step 300, as shown in figure 5.
  • the segmentation memory 10 may be configured to store the projection module 22 and the segmentation processor 12 may be configured to execute instructions of the segmentation module 20.
  • the segmentation server 2 may also comprise a projection module 22 configured to define the pre-determined infrastructure 1 and project the sub graphs to the pre-determined infrastructure 1 according to the step 300, as shown in figure 5.
  • the segmentation memory 10 may be configured to store the projection module 22 and the segmentation processor 12 may be configured to execute instructions of the segmentation module 20.
  • the segmenting 100 of the dataflow graph may further comprise dividing the dataflow graph into the dataflow sub-graphs and grouping the sub graphs into grouped dataflow sub-graphs based on the pre-determined infrastructure 1. This may be accomplished for example by dividing the dataflow graph 50 to separate operations or cohorts of operations, such as tensor operations, and grouping the operations into grouped dataflow sub-graphs based on the pre-determined infrastructure 1. Accordingly, the sub-graphs are formed based on the defined pre-determined infrastructure 1. This allows the grouping of the sub-graphs or operations to be carried out such that the pre-determined infrastructure 1 may be utilized based on the properties of the pre-determined infrastructure 1, meaning for example computation power of the infrastructure nodes 6 and bandwidth of the infrastructure edges 8. These properties may be expressed as known quantities or a range of possible quantities, in which case multiple projections may be made in order to acceptably cover the range of anticipated infrastructural mutations.
  • the segmentation server may configured to segment, by utilizing instructions of the segmentation module executed by the segmentation processor, the dataflow graph 50 by dividing the dataflow graph 50 into the dataflow sub-graphs, and grouping the dataflow sub-graphs into grouped dataflow sub-graphs based on the pre-determined infrastructure 1.
  • the graph nodes of the dataflow graph comprise tensor operations.
  • the segmenting of the dataflow graph in step 100 may comprise scalarization of the tensor operations whereby graph nodes are segmented into scalar operations or cohorts of scalar operations.
  • the method further may comprise grouping the scalar operations or cohorts of scalar operations into scalar operation groups based on the pre-determined infrastructure 1.
  • the scalar operation groups form the dataflow sub-graphs.
  • Scalarization means a process of representing tensor operations in a graph as scalar operations.
  • the segmentation server may be configured to segment, by utilizing instructions of the segmentation module executed by the segmentation processor, the dataflow graph by scalarization of tensor operations, segmenting the graph nodes into scalar operations or cohorts of scalar operations and grouping the scalar operations or cohorts of scalar operations into scalar operation groups based on the pre-determined infrastructure.
  • the scalar operation groups form the dataflow sub-graphs.
  • the graph nodes of the dataflow graph comprise tensor operations.
  • the segmenting of the dataflow graph in step 100 may comprise representing the tensor operations as scalar promises representing scalar operations to be performed, receiving input data and fulfilling the scalar promises based on the input data such that fulfilled scalar promises are formed from the scalar promises.
  • the segmentation may further comprise segmenting the fulfilled scalar promises into scalar operations or cohorts of scalar operations and grouping the scalar operations or cohorts of scalar operations into scalar operation groups based on the pre-determined infrastructure.
  • the scalar operation groups form the dataflow sub-graphs.
  • the segmentation server may be configured to segment, by utilizing instructions of the segmentation module executed by the segmentation processor, the dataflow graph by the scalarization of tensor operations of the graph nodes into scalar operations, representing tensor operations as scalar promises representing scalar operations to be performed, the graph nodes of the dataflow graph comprising the tensor operations, receiving input data to the segmentation module, and fulfilling the scalar promises based on the input data such that fulfilled scalar promises are formed from the scalar promises.
  • the segmentation server may be further configured, by utilizing instructions of the segmentation module executed by the segmentation processor, to segment the fulfilled scalar promises into scalar operations or cohorts of scalar operations, grouping the scalar operations or cohorts of scalar operations into scalar operation groups based on the pre-determined infrastructure, scalar operation groups forming the dataflow sub-graphs
  • the segmenting step 100 may further comprise repeating, by utilizing instructions of the segmentation module executed by the segmentation processor, the following steps one or more times for fulfilling the scalar promises before the grouping of the scalar operations or cohorts of scalar operations: receiving input data, fulfilling the scalar promises based on the input data such that fulfilled scalar promises are formed from the scalar promises, and segmenting the fulfilled scalar promises into scalar operations or cohorts of scalar operations.
  • Input data to scalar promises are the data that reduces ambiguity of the promises, such as the dimensionality of tensors or constant values, such that scalar promises can become fulfilled scalar promises, otherwise defined as scalar operations or cohorts of scalar operations such that result from using input data to resolve scalar promises.
  • the segmentation step 100 or the segmentation module are configured to represent tensors in two layers: scalar promises, which are formulas for generating operations that will be performed on the scalars that comprise tensors, and fulfilled scalar promises, which are the operations themselves.
  • scalar promises are an added level of indirection, relative to prior art, that represents the scalar arithmetic that will be executed once dimensions of the tensors are known.
  • the scalar promises become fulfilled and fulfilled scalar promises are formed. Once the dimensionality of the dataflow graph is entirely known, the scalar promises are fulfilled. The fulfillment of scalar promises represents the finest granularity into which the dataflow graph can be segmented, as every operation, or tensor, is decomposed into an unambiguous number of binary and unary scalar operations, or scalar tensors.
  • the method of the present invention may further comprise compiling the dataflow sub-graphs into a segmented dataflow graph comprised of the dataflow sub-graphs. This means that the sub-graphs are compiled together such that the segmented dataflow graph is formed.
  • the segmented dataflow graph comprises the sub-graphs and the relationship between the sub-graphs.
  • a scalar promise can be fulfilled either by in-memory representations of scalar operations to be performed (i.e. objects in a computer program that represent addition, subtraction, etc.) or an algorithm that will deterministically generate a representation of these operations (i.e. a transpiler that takes descriptions of fulfilled scalar promises as input and produces output in a given language like assembly or C).
  • in-memory representations of scalar operations to be performed i.e. objects in a computer program that represent addition, subtraction, etc.
  • an algorithm that will deterministically generate a representation of these operations i.e. a transpiler that takes descriptions of fulfilled scalar promises as input and produces output in a given language like assembly or C).
  • Scalar promises allow for graph currying.
  • Currying is progressive evaluation of a function with partial information instead of the traditional delayed evaluation of a function until all information is present.
  • scalar promises can be immediately fulfilled (i.e. sections of a dataflow graph where the dimensions of incoming and outgoing matrices or operations are entirely known)
  • the introduction of additional dimensional information progressively into the dataflow graph requires less time for the dataflow graph to be created. This allows for a compilation time of the dataflow graph to be spread out as information, meaning the input data, enters the dataflow graph.
  • the segmentation of the dataflow graph in step 100 may also comprise merging redundant equivalent dataflow sub-graphs such that merged dataflow sub-graphs are formed.
  • Merging means that equivalent or same sub-graphs or operations are first identified and these repetitive sub-graphs or operations are executed only once during the execution of the dataflow graph.
  • the segmentation server 2 may configured to segment, by utilizing instructions of the segmentation module 20 and executed by the segmentation processor 12, the dataflow graph 50 by merging redundant equivalent dataflow sub-graphs such that merged dataflow sub-graphs are formed.
  • Gains in data compression through the merging come from the detection of dataflow sub-graph substructures using traversals of the scalarized graph.
  • scalarized sub graphs do not represent aggregate operations (i.e. a dot product or a convolution) but rather the individual operations that comprise the aggregates behind this intention (i.e. addition, multiplication). Because of this, novel patterns in the dataflow graph that emerge beyond intention by analysing individual operations can be identified and grouped together to reduce size of the dataflow graph. Accordingly, sub-graphs having reduced size may be created.
  • Dataflow graph segmentation and merging via scalar promises and fulfilled scalar promises allow for greater gains in data compression than traditional means of dataflow graph representation. This, in turn, allows for dataflow sub-graphs to be transmitted over a network faster and more efficiently than traditional forms of compression.
  • the present invention may also comprise compressing the dataflow sub-graphs into compressed dataflow sub-graphs before projecting the dataflow sub-graphs onto the pre-determined infrastructure 1.
  • the dataflow sub-graphs may first be compiled to the to the compiled segmented dataflow graph, and then the compiled segmented dataflow graph is compressed.
  • the compression of the dataflow sub-graphs or the compiled segmented dataflow graph may be carried out by utilizing any known compression protocol, such as pickle, zip or tar.
  • the compression by utilizing known data compression protocols may be carried out before or after the merging step.
  • the step 100 of dataflow graph segmentation into to the subgraphs may achieve the segmentation of operations or tensors in two sub-steps - the formulation of the tensors as scalar promises, which represent a range of possible scalar operations to be performed, and the fulfilment of scalar promises, which represent the actual operations as scalar tensors.
  • dataflow graphs may be curried over time, they can be compressed by grouping similar operations, and redundant processes can be merged to make execution in the infrastructure more efficient.
  • segmentation allows for sub-graphs to be compiled or sescribed using arbitrary compilation or description processes and projected onto an infrastructure.
  • the invention may comprise defining the pre-determined infrastructure in step 200 by generating one or more sub-graph projection predictions based on the infrastructure, the dataflow sub-graph projections representing the distribution of the dataflow sub-graphs to the infrastructure, and by defining the pre-determined infrastructure based on pre-defined infrastructure criteria. Then in step 300 the dataflow sub-graphs may be projected to the pre-determined infrastructure by projecting the one or more projection predictions to the infrastructure, and in step 400 the dataflow graph may be executed by executing the distributed dataflow sub-graphs according to one of the projection predictions.
  • the execution step 400 may comprise sending the sub-graphs to the infrastructure nodes and edges of the pre-determined infrastructure.
  • the step 200 of defining the pre-determined infrastructure may comprise analysing the infrastructure and representing the infrastructure as one or more constraint optimization programs based on the analysing of the infrastructure, generating one or more projection predictions by utilizing the constraint optimization programs with constraints obtained from the analysing of the infrastructure, and defining the pre-determined infrastructure as the infrastructure used to generate the one or more constraint optimization programs. This may be carried out with the segmentation server by, utilizing instructions of the projection module executed by the segmentation processor.
  • step 200 of defining the pre-determined infrastructure may comprise analysing the infrastructure and representing the infrastructure as one or more constraint optimization programs based on the analysing of the infrastructure, generating one or more projection predictions by utilizing the constraint optimization programs with constraints obtained from the analysing of the infrastructure, and defining the pre-determined infrastructure based on the pre-defined infrastructure criteria. Also this may be carried out with the segmentation server by, utilizing instructions of the projection module executed by the segmentation processor.
  • the analysing may comprise analysing for example available computations capacity and power of the hardware nodes 6 and/or the available bandwidth or transmission speed of the network connections 8.
  • the pre-defined infrastructure criteria for defining the pre determined infrastructure from the one or more projection predictions may be based on instruction stored to the segmentation server memory 10 or the projection module of the segmentation server 2.
  • Analysing the available infrastructure 1 and creating one or more projection predictions allows choosing desired pre-determined infrastructure for each situation based on the properties of the hardware nodes 6 and network connections 8. Furthermore, when the pre-determined infrastructure is defined the details of the pre-determined infrastructure may be received in step 201 of figure 2 as input to the segmentation step 100. Accordingly, the details of the pre determined infrastructure may be utilized the segmentation step 100 and the sub-graphs may be formed based on the pre-determined infrastructure making the execution the dataflow graph and the use of the infrastructure efficient.
  • the step 200 of defining the pre-determined infrastructure may comprise analysing a potential infrastructure by generating synthesized representations of the infrastructure and representing the infrastructure as one or more constraint optimization programs based on the synthesized representations, generating one or more projection predictions based on the synthesized representations, and defining the pre-determined infrastructure based on the pre-determined criteria, by utilizing instructions of the projection module executed by the segmentation processor.
  • the step 200 of defining the pre-determined infrastructure may comprise predicting the range of possible mutations of an actual or anticipated infrastructure by creating a model capable of generating multiple projections based on a description of an infrastructure, generating one or more projection predictions using the model, defining the pre-determined infrastructure from the one or more projection predictions based on pre- determined criteria and projecting one or more of the projections of dataflow sub-graphs to the pre-determined infrastructure.
  • the synthesized representation of the infrastructure may be carried out by generating one or more projections predictions by first synthesizing several projections based on different infrastructure descriptions using alternative infrastructure descriptions as input to a predictive model such as a neural network in order to train the neural network to output projection predictions.
  • the neural network is then used to output projection predictions from synthesized infrastructure representations, and the pre-determined infrastructure is definedfrom the one or more projection predictions based on pre-determined criteria and projecting one or more outputs of the predictive model onto the infrastructure.
  • the projection predictions generated by the neural network based on the synthesized representation of the available infrastructure may be provided to the projection module.
  • the segmentation server may be configured to, by utilizing instructions of the projection module and executed by the segmentation processor, model the current or future infrastructure in one or more possible states by anticipating one or more configurations of the hardware nodes and the network connections, generating one or more projections based on the analysis of the hardware nodes and the network connections, using these configurations and projections to make predictions about additional projections defining the pre-determined infrastructure based on pre-determined criteria, and projecting one or more projections of dataflow sub-graphs to the hardware nodes and the network connections of the pre-determined infrastructure.
  • Generating the one or more projection predictions may be accomplished by feeding a representation of the hardware nodes and the network connections as well as the projection that results that representation from the segmentation server 2 or the projection module 22 to a neural network 16 and receiving the one or more projection predictions from the neural network 16 to the segmentation server 2 of the projection module 22.
  • the step 200 may comprise analysing a potential infrastructure by generating synthesized representations of the infrastructure and representing the infrastructure as one or more constraint optimization programs based on the synthesized representations, generating one or more projection predictions by utilizing a neural network trained with the results of the one or more constraint optimization programs as output and the synthesised infrastructure representation as input, and defining the pre determined infrastructure based on the pre-determined criteria.
  • the neural network 16 may be any known kind of neural network, such as many-to-many recurrent neural network.
  • the neural network 16 may be one neural network or number of interconnected neural networks, and the neural network 16 may be integral part of the segmentation server 2 or separate from the segmentation server 2 and connected to the segmentation server via a neural network connection 18, as shown in figure 5.
  • a neural network 16 any known predictive algorithm may be used for the same purpose.
  • the step 300 of projecting the dataflow sub-graphs, or the segmented dataflow graph, to the infrastructure comprises projecting the dataflow sub graphs or the segmented dataflow graph to the pre-determined infrastructure. In one embodiment, this is carried out by projecting the dataflow sub-graphs or the segmented dataflow graph to the pre-determined infrastructure such that each dataflow sub-graph is projected to a separate infrastructure node.
  • the segmentation server 2 may be configured to project, by utilizing instructions of the projection module 22 and executed by the segmentation processor 12, the dataflow sub-graphs to the infrastructure by distributing the dataflow sub-graphs to the hardware nodes 6 and network connections 8 of the pre-determined infrastructure 1 for execution of the dataflow graph. In one embodiment, this is carried out by distributing the dataflow sub-graphs to the hardware nodes and network connections of the pre-determined infrastructure such that each dataflow sub-graph is projected to a separate hardware node.
  • the projecting phase 300 and projecting may comprise sending or transmitting the sub-graphs to the infrastructure nodes 6.
  • the projecting may comprise sending or transmitting instructions to create the sub-graphs in the infrastructure nodes 6.
  • the sub- graph is first created in the infrastructure node 6 based on the instructions received from the segmentation server 2 or the projection module 22 and then the sub-graph may be executed.
  • the projecting may comprise sending or transmitting variables as input to pre-defined functions residing on the infrastructure nodes to create the sub-graphs in the infrastructure nodes 6.
  • the sub-graph is first created in the infrastructure node 6 using a pre-defined function whose input variables are received from the segmentation server 2 or the projection module 22 and then the sub-graph may be executed. All three of these alternatives, or any combination thereof, mean projecting the sub graphs to the infrastructure.
  • the method of the present invention may further comprise monitoring the infrastructure during the projection and/or execution of the dataflow graph or the sub-graphs.
  • the segmentation server 2 may comprise a monitoring module 24, as shown in figure 5, configured to monitor, by utilizing instructions of the monitoring module 24 and executed by the segmentation processor 12, the infrastructure by monitoring the hardware nodes 6 and the network connections 8 of the pre-determined infrastructure 1 during the execution of the dataflow and/or the projection of the sub-graphs.
  • the segmentation server 2 or the monitoring module may be configured to monitor the hardware nodes and the network connections of the pre-determined infrastructure during the execution of the dataflow graph and receiving a monitoring output and to compare the monitoring output and a pre-determined expected output. Based on the comparison, if the monitoring output is within the pre-determined expected output, the execution or projection may be continued by utilizing the hardware nodes and the network connections of the pre-determined infrastructure in step 300 or 400. If the monitoring output is outside the pre-determined expected output, based on the comparison, the projection of the dataflow sub-graphs to the hardware nodes and the network connections may be re-initiated in step 401, or alternatively defining of the pre-determined infrastructure 1 may be re-initiated in step 402. The step 402 may also result in re-initiating the segmenting of the segmentation of the data flowgraph.
  • Projection step 300 is the process of taking a segmented dataflow graph, or sub-graphs, and projecting it onto the pre-determined infrastructure.
  • Dataflow graph projection may take the segmented dataflow graph and recombine it in order using a polyglot approach to most efficiently use the set of hardware nodes and network connections onto which it is projected. Segmented dataflow graph projection results in more efficient computation, allowing for savings in cost of bandwidth, memory and computation power.
  • Projection of the segmented dataflow graph means the mapping of a dataflow graph onto the pre-determined infrastructure. For example, if there are two desktop computers connected on a local ethernet and both are connected to a server in the cloud over a 4G connection, all nodes of the segmented dataflow graph are projected onto these three devices and the edges of the segmented dataflow graph are projected onto the ethernet and the 4G network.
  • the dataflow graph projection module first identifies fixed and variable aspects of infrastructure onto which the projection occurs, such as the number of hardware nodes present or the throughput of bandwidth under different conditions. Different configurations of these variables may be used to generate a constraint optimization problem in which the unknown values represent binary (1/0) decisions for the hardware node onto which a fulfilled scalar promise is mapped.
  • six constraint optimization problems can be formulated for three configurations of hardware nodes (1, 100, or 1 000 for example) on a system with two levels of data throughput (peak and off-peak for example), each of which is used to determine an optimal projection of fulfilled scalar promises onto an infrastructure.
  • Constraint optimization problems are a category of problems where an optimal solution to a function is found given one or more constraints.
  • An example of a constraint optimization program that can be used by the projection module is a sparse mixed integer programming is a linear programming problem in which some of the variables are integers and most of the integers are 0. In this case, integers represent binary activations that a fulfilled scalar promise will be assigned to a particular hardware node.
  • Data synthesis is a technique in machine learning whereby data fed to a network is created synthetically.
  • Analysing the infrastructure, generating one or more projection predictions based on the analysed infrastructure, defining the pre-determined infrastructure from the one or more projection predictions based on pre determined criteria may be carried out only using a constraint optimization program. In this case, there is no need for utilizing a neural network.
  • the constraint optimization program with a constraint optimizer is used for generating the predictions, the constrain optimizer yields a local minimum of a deterministic function, which is always accurate insofar as the encoded constraints accurately represent the infrastructure.
  • the outputs from the constraint optimization program are also considered predictions.
  • the neural network could work for example in the following manner. Synthesized data is used to train a neural network, for example many-to-many recurrent neural network, that learns a generic model for how to project fulfilled scalar promises onto infrastructure.
  • the input to this neural network is the synthesized representation of physical infrastructure and the output is a sparse matrix representing the probability that a fulfilled scalar promise will be assigned to a hardware node.
  • the output of the neural network is compared to the output of a linear program using a categorical cross entropy loss function and, by doing this iteratively using classic forward and back propagation, the model "learns” how to project fulfilled scalar promises onto an infrastructure. It is then able to make projection predictions from new synthesised infrastructure representations.
  • the resiliency of the method is bounded by the synthetic data, and predictions of projections for infrastructures or fulfilled scalar promises that vary considerably from the synthesized data will be outside the scope of the model and generate noise.
  • the method learns to make reliable predictions for the most common real-world scenarios.
  • the provisioning of computation power may represent a hurdle for companies with distributed infrastructures, as an excess of computation power and, inversely, a dearth of computation power can lead throttling of data because of insufficient capacity.
  • This problem is solved using a method to project dataflow graphs onto infrastructure because the model can determine how many servers or devices need to be provisioned and terminated to scale with network demand, decreasing operational cost by using computational resources more efficiently.
  • the dataflow graph projection step 300 is the process step of taking a segmented dataflow graph and projecting it onto a given infrastructure comprised of hardware nodes and network connections. This is accomplished using the two-step protocol of synthesizing data about projections in various network configurations using mixed integer programming and feeding this synthesized data to the neural network that predicts, for a given infrastructure, the optimal projection of fulfilled scalar promises onto that infrastructure. Projected dataflow graphs can change for different bandwidth configurations, redistributing an algorithm as a function of increased or decreased network transmission time.
  • the dataflow graph execution step 400 is the process of executing an algorithm on a dataflow graph projection.
  • the pre-determined infrastructure onto which the dataflow graph is projected remains unchanged for the lifetime of the execution and all expected network conditions hold.
  • the execution must also deal with dynamic, unexpected features of the pre-determined infrastructure including failover, throughput volatility and erratic distributions of data.
  • the execution must be resilient to changes in the configuration of infrastructure nodes and edges, including the addition or subtraction of nodes and edges.
  • the execution of the projected dataflow graph must accommodate data being produced and consumed in a decentralized manner. The scheduling of data ingress and egress are often immediate, but can be delayed or throttled to accommodate irregularities in a projected tensor graph that would otherwise result in suboptimal performance.
  • Dataflow graph execution may be a queue-based producer/consumer messaging model implemented on top of a protocol negotiation library such as libp2p.
  • Each node of the dataflow graph projection represents data ingress and egress as queues with timeout and throttling mechanisms to accommodate failover and overload.
  • implementing dataflow graph execution on a transport-layer agnostic peer-to-peer protocol removes the single, central broker and allows for information to flow between hardware nodes in an unmediated fashion. Because there is no central broker of information, the execution of the distributed dataflow graph cannot be monitored by traditional logging mechanisms. Instead, the network must be periodically and stochastically sampled by monitoring for an expected outcome. Deviations from the expected outcome of the distributed dataflow graph becomes the subject of investigation into network failure. This is also called spamming the network.
  • a producer is a hardware node that produces data sent to another hardware node.
  • a consumer is a hardware node that consumes data.
  • Dataflow graph segmentation defining the pre-determined infrastructure, dataflow graph projection and dataflow graph execution, while presented as separable processes, are interdependent and are thus presented as part of a single process called dynamic distribution.
  • the dynamic part of the distribution process is manifested through the following cross correlations.
  • a dataflow graph may be segmented into fulfilled scalar promises based on a given amount of input data and output predictions in a distributed network. These fulfilled scalar promises are projected onto the infrastructure based on a constraint optimization program that outputs a sparse matrix representing the most efficient distribution of operations to hardware nodes given the constraints.
  • the inputs and results of this constraint optimization program are used to train to a neural network that learns features of projections and will automatically generate a best-fit projection for a given infrastructure.
  • the projected infrastructure may be executed via producer/consumer queues on a peer-to-peer network.
  • the actual performance of the execution monitored by network spamming, is used to re-evaluate the constants used in the projections.
  • the new projections require the genesis of different models that will contain different fulfilled scalar promises.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Environmental & Geological Engineering (AREA)
  • Health & Medical Sciences (AREA)
  • Cardiology (AREA)
  • General Health & Medical Sciences (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Multi Processors (AREA)

Abstract

La présente invention concerne un procédé et un système de traitement d'informations distribuées à l'aide d'un graphe de flux de données (50). Le procédé consiste à : distribuer un graphe de flux de données (50) à une infrastructure (1) comprenant des nœuds d'infrastructure (6) et des bords d'infrastructure (8) entre les nœuds d'infrastructure (6); segmenter le graphe de flux de données (50) en sous-graphes de flux de données, et définir une infrastructure prédéterminée (1) ayant au moins deux nœuds d'infrastructure (6) et au moins un bord d'infrastructure (8) entre les nœuds d'infrastructure (6); projeter des sous-graphes de flux de données à l'infrastructure prédéterminée (1) via la distribution des sous-graphes de flux de données aux nœuds d'infrastructure (6) et aux bords d'infrastructure (8), exécuter le graphe de flux de données (50) en utilisant un protocole de communication sur les bords d'infrastructure (8) entre les nœuds d'infrastructure (6) durant l'exécution du graphe de flux de données (50), et exécuter les sous-graphes de flux de données distribués dans les nœuds d'infrastructure (6).
PCT/FI2019/050313 2018-04-18 2019-04-17 Procédé de traitement d'informations distribuées, et système de traitement d'informations distribuées WO2019202216A2 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
FI20185365 2018-04-18
FI20185365A FI130232B (en) 2018-04-18 2018-04-18 Procedure for distributed processing of information and distributed information processing system

Publications (2)

Publication Number Publication Date
WO2019202216A2 true WO2019202216A2 (fr) 2019-10-24
WO2019202216A3 WO2019202216A3 (fr) 2019-12-19

Family

ID=68239958

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/FI2019/050313 WO2019202216A2 (fr) 2018-04-18 2019-04-17 Procédé de traitement d'informations distribuées, et système de traitement d'informations distribuées

Country Status (2)

Country Link
FI (1) FI130232B (fr)
WO (1) WO2019202216A2 (fr)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112099882A (zh) * 2020-09-11 2020-12-18 杭州海康威视数字技术股份有限公司 一种业务处理方法、装置及设备
CN112308317A (zh) * 2020-10-30 2021-02-02 中国地震台网中心 基于分布式架构的海量地震观测数据噪声功率谱计算方法及系统
CN113469322A (zh) * 2020-03-31 2021-10-01 杭州海康威视数字技术股份有限公司 确定模型的可执行程序的方法、装置、设备及存储介质
WO2022066639A1 (fr) * 2020-09-24 2022-03-31 SambaNova Systems, Inc. Logique de temps de compilation pour détecter des motifs d'accès à des données compatibles avec la diffusion en continu et compatibles avec la diffusion
US11392513B2 (en) 2020-10-15 2022-07-19 Dell Products L.P. Graph-based data flow control system
US11706076B2 (en) 2020-01-23 2023-07-18 Novnet Computing System Tech Co., Ltd. Computer system with computing devices, communication device, task processing device
US11709664B2 (en) 2020-06-02 2023-07-25 SambaNova Systems, Inc. Anti-congestion flow control for reconfigurable processors
RU2800966C1 (ru) * 2020-01-23 2023-08-01 Новнет Компютинг Систем Тек Ко., Лтд Компьютерная система
US11983509B2 (en) 2021-03-23 2024-05-14 SambaNova Systems, Inc. Floating-point accumulator

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8260598B2 (en) * 2004-09-20 2012-09-04 The Mathworks, Inc. Size vector sharing in code generated for variable-sized signals
US9733914B2 (en) * 2009-06-01 2017-08-15 National Instruments Corporation Loop parallelization analyzer for data flow programs
US9182957B2 (en) * 2012-07-10 2015-11-10 Loring Craymer Method and system for automated improvement of parallelism in program compilation
US8887154B2 (en) * 2012-09-06 2014-11-11 Imagine Communications Corp. Systems and methods for partitioning computing applications to optimize deployment resources
US9348560B2 (en) * 2013-06-04 2016-05-24 Qualcomm Incorporated Efficient execution of graph-based programs
WO2015139048A1 (fr) * 2014-03-14 2015-09-17 Concurrent, Inc. Règles de mise en correspondance de flux de données logiques à isomorphisme de (sous-)graphe de grappe
CN104504143B (zh) * 2015-01-04 2017-12-29 华为技术有限公司 一种流图优化方法及其装置
EP4242845A1 (fr) * 2015-10-28 2023-09-13 Google LLC Modification de graphes informatiques
US11151446B2 (en) * 2015-10-28 2021-10-19 Google Llc Stream-based accelerator processing of computational graphs
WO2018094087A1 (fr) * 2016-11-17 2018-05-24 The Mathworks, Inc. Systèmes et procédés de génération de code pour unités de traitement parallèles
US10452452B2 (en) * 2017-04-17 2019-10-22 Wave Computing, Inc. Reconfigurable processor fabric implementation using satisfiability analysis
CN111247513B (zh) * 2017-04-17 2021-10-08 迪普西格有限公司 无线电信号处理数据流操作的放置与调度

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11706076B2 (en) 2020-01-23 2023-07-18 Novnet Computing System Tech Co., Ltd. Computer system with computing devices, communication device, task processing device
RU2800966C1 (ru) * 2020-01-23 2023-08-01 Новнет Компютинг Систем Тек Ко., Лтд Компьютерная система
CN113469322A (zh) * 2020-03-31 2021-10-01 杭州海康威视数字技术股份有限公司 确定模型的可执行程序的方法、装置、设备及存储介质
CN113469322B (zh) * 2020-03-31 2023-07-18 杭州海康威视数字技术股份有限公司 确定模型的可执行程序的方法、装置、设备及存储介质
US11709664B2 (en) 2020-06-02 2023-07-25 SambaNova Systems, Inc. Anti-congestion flow control for reconfigurable processors
CN112099882A (zh) * 2020-09-11 2020-12-18 杭州海康威视数字技术股份有限公司 一种业务处理方法、装置及设备
CN112099882B (zh) * 2020-09-11 2023-08-25 杭州海康威视数字技术股份有限公司 一种业务处理方法、装置及设备
WO2022066639A1 (fr) * 2020-09-24 2022-03-31 SambaNova Systems, Inc. Logique de temps de compilation pour détecter des motifs d'accès à des données compatibles avec la diffusion en continu et compatibles avec la diffusion
US11645057B2 (en) 2020-09-24 2023-05-09 SambaNova Systems, Inc. Systems and methods for memory layout determination and conflict resolution
US11392513B2 (en) 2020-10-15 2022-07-19 Dell Products L.P. Graph-based data flow control system
CN112308317A (zh) * 2020-10-30 2021-02-02 中国地震台网中心 基于分布式架构的海量地震观测数据噪声功率谱计算方法及系统
US11983509B2 (en) 2021-03-23 2024-05-14 SambaNova Systems, Inc. Floating-point accumulator

Also Published As

Publication number Publication date
WO2019202216A3 (fr) 2019-12-19
FI20185365A1 (en) 2019-10-19
FI130232B (en) 2023-05-03

Similar Documents

Publication Publication Date Title
FI130232B (en) Procedure for distributed processing of information and distributed information processing system
US11989647B2 (en) Self-learning scheduler for application orchestration on shared compute cluster
Wang et al. Convergence of edge computing and deep learning: A comprehensive survey
US11868890B2 (en) Workflow optimization
WO2024114399A1 (fr) Procédé d'optimisation pour l'exécution distribuée d'une tâche d'apprentissage profond, et système distribué
CN111488177A (zh) 数据处理方法、装置、计算机设备和存储介质
JP7379668B2 (ja) 機械学習ワークロードのためのタスクスケジューリング
CN113568599A (zh) 用于处理计算作业的方法、电子设备和计算机程序产品
US20200044938A1 (en) Allocation of Shared Computing Resources Using a Classifier Chain
US11551095B2 (en) Sharing preprocessing, computations, and hardware resources between multiple neural networks
CN114118433A (zh) 一种设备的配置参数的推荐方法及装置
Pimpley et al. Towards Optimal Resource Allocation for Big Data Analytics.
Hogade et al. A survey on machine learning for geo-distributed cloud data center management
Betting et al. Oikonomos: An Opportunistic, Deep-Learning, Resource-Recommendation System for Cloud HPC
Arras et al. Dkpn: A composite dataflow/kahn process networks execution model
Li et al. Performance modelling and cost effective execution for distributed graph processing on configurable VMs
Liang et al. Communication-Efficient Large-Scale Distributed Deep Learning: A Comprehensive Survey
Guo et al. Hierarchical design space exploration for distributed CNN inference at the edge
CN114327925A (zh) 一种电力数据实时计算调度优化方法及系统
US11709662B2 (en) Sparsity uniformity enforcement for multicore processor
Ghannane et al. Diviml: A module-based heuristic for mapping neural networks onto heterogeneous platforms
Arkian Resource management for data stream processing in geo-distributed environments
EP4220501A1 (fr) Prédicteurs d'exécution pour réduction de calcul dans des calculs dépendants
Nallendran Predicting performance run-time metrics in fog manufacturing using multi-task learning
Mencagli A Control-Theoretic Methodology for Adaptive Structured Parallel Computations.

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19788384

Country of ref document: EP

Kind code of ref document: A2

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19788384

Country of ref document: EP

Kind code of ref document: A2