WO2017086986A1 - Traitement de graphes partitionnés - Google Patents

Traitement de graphes partitionnés Download PDF

Info

Publication number
WO2017086986A1
WO2017086986A1 PCT/US2015/061798 US2015061798W WO2017086986A1 WO 2017086986 A1 WO2017086986 A1 WO 2017086986A1 US 2015061798 W US2015061798 W US 2015061798W WO 2017086986 A1 WO2017086986 A1 WO 2017086986A1
Authority
WO
WIPO (PCT)
Prior art keywords
graph
processing
partitions
data
nodes
Prior art date
Application number
PCT/US2015/061798
Other languages
English (en)
Inventor
Asish GHOSHAL
Jun Li
Manish Marwah
Mijung Kim
Alexander Ulanov
Original Assignee
Hewlett Packard Enterprise Development Lp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hewlett Packard Enterprise Development Lp filed Critical Hewlett Packard Enterprise Development Lp
Priority to PCT/US2015/061798 priority Critical patent/WO2017086986A1/fr
Publication of WO2017086986A1 publication Critical patent/WO2017086986A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • G06F16/278—Data partitioning, e.g. horizontal or vertical partitioning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/901—Indexing; Data structures therefor; Storage structures
    • G06F16/9024—Graphs; Linked lists

Definitions

  • a graph is a representation of a set of data (e.g., Big Data).
  • An example graph may include a plurality of nodes and edges connecting the plurality of edges. The graph may be processed by executing nodes in accordance with characteristics of edges linked to the nodes and related nodes linked by the edges.
  • FIG. 1 is a schematic diagram of an example partitioned graph processing system including an example partition manager constructed in accordance with examples herein.
  • FIG. 2 is a block diagram of an example graph partition manager that may be used to implement the graph partition manager of FIG. 1 .
  • FIG. 3 is a flowchart representative of example machine readable instructions that may be executed to implement the graph partition manager of
  • FIG. 4 is another flowchart representative of other example machine readable instructions that may be executed to implement graph partition manager of FIG. 2.
  • FIG. 5 is a block diagram of an example processor platform capable of executing the instructions of FIGS. 3 and/or 4 to implement the graph partition manager of FIG. 2.
  • Examples disclosed herein involve processing a graph having graph data distributed across a plurality of partitions of a memory (e.g., a shared memory).
  • local graph data of the partitions is processed by local graph updaters.
  • Replicate node data within the graph data e.g., node data for a same node that is in a plurality of partitions
  • cluster-based computing environments include a relatively small number of processing units with relatively small amounts of memory in each cluster, in examples herein, a multi-core environment, in combination with a shared memory enable relatively short interprocess communication latency while holding increased amount of in-memory data structures that provide better data locality. Accordingly, examples herein enable increased speeds for graph processing over distributed partitions that run on multi-core and shared memory machines by having several local graph updaters process local graph data of corresponding data partitions of a shared memory.
  • An example method includes receiving node data for replicate nodes of a graph.
  • the example node data may be generated from first processing iterations of graph data distributed in a plurality of graph partitions received from respective local graph updaters of the plurality of graph partitions of a shared memory; aggregating beliefs of the processed replicate nodes; and instructing the local graph updaters of the graph partitions to update beliefs of the replicate nodes for second processing iterations of the graph partitions.
  • graph processing involves processing nodes of a graph until all nodes (or a threshold percentage of the nodes) have been processed, which is known as convergence of the graph processing. Convergence occurs when ail nodes (or a threshold percentage of the nodes) have been processed such that a particular characteristic of the nodes has reached a fixed point (or value), and, thus, does not change with subsequent iterations, in examples herein, parallel graph processing or processing graphs in parallel (e.g., synchronous graph processing) may involve several iterations of processing a set of nodes in parallel (at a same time or within a same time period).
  • a first set of nodes may be processed and in a second iteration a second set of nodes may be processed.
  • the second set of nodes may or may not include nodes related or linked to the first set of nodes.
  • asynchronous graph processing involves processing graphs based on priority of nodes of a graph.
  • the example priority or priority of (or associated with) a node refers to an importance that the node be processed sooner rather than later. For example, it is desirable that a node having a higher priority be processed before a node having a lower priority. Priority may be ranked or rated on any suitable scale, (e.g., from 1 to 10, 10 being highest priority, 1 being lowest priority (or vice versa).
  • hybrid graph processing or hybrid synchronous/asynchronous graph processing refers to a combination of synchronous graph processing and asynchronous graph processing to process a graph.
  • graph data of a graph includes edge data and node data corresponding to the edges and nodes of the graph, respectively.
  • the edge data includes edge information, such as message data, characteristic data, etc.
  • the node data includes node information, such as node characteristics, node attributes, priority, etc.
  • FIG. 1 is a schematic diagram of an example partitioned graph processing system 100 including an example graph partition manager 1 10 constructed in accordance with examples herein.
  • the example partitioned graph processing system 100 includes the graph partition manager 1 10, a memory fabric 120, a graph database 130, and a user interface 140.
  • the example memory fabric 120 includes a plurality of partitions 122 (which may be referred to collectively herein as the partitions 122).
  • Each of the plurality of partitions include local graph data 124 and a local graph updater 126 (e.g., an iteration processor, a virtual machine, etc.).
  • the graph partition manager 1 10 facilitates graph processing of graph data distributed among the plurality of partitions 122.
  • the example user interface 140 facilitates user interaction with the graph partition manager 1 10 and/or the memory fabric 120.
  • the user interface 140 may include user input(s) (e.g., a keyboard, a mouse, a trackball, a touchscreen, etc.) and/or user output(s) (e.g., a display, a touchscreen, speakers, indicators, etc.).
  • the user interface 140 enables a user to access and control settings of the graph partition manager 1 10.
  • the user interface 140 may request processing of a particular graph from the graph database 130.
  • the user interface 140 enables a user to indicate or set processing settings (e.g., synchronous settings, asynchronous settings, hybrid settings, etc.) to process the selected graph or particular types of graphs/graph data in accordance with examples herein.
  • the example graph database 130 stores graph data for graphs.
  • the example graph database 130 may be implemented by at least a storage device, computing device, network device, etc. that may store or provide graph data for graphs that are to be processed in accordance with examples herein.
  • the example graph partition manager 1 10 may receive/retrieve graph data for graphs from the graph database 130.
  • the graph partition manager 1 10 may partition the graph data into the plurality of partitions 122 of the memory fabric 120. Accordingly, the graph partition manager 1 10 may load the partitioned graph data into respective partitions of the plurality of partitions 122 using any suitable technique.
  • the example memory fabric 120 of FIG. 1 stores the graph data in the partitions 122 of the memory fabric.
  • the memory fabric 120 may be implemented by a non-volatile memory (NVM), such as a memristor, a phase change memory, a dynamic random access memory (DRAM), or any other suitable memory fabric/device.
  • NVM non-volatile memory
  • the memory fabric 120 may be a shared memory fabric that is accessible to a plurality of processing engines, devices, or systems (e.g., a plurality of graph partition managers 1 10, a plurality of shuffle engines, etc.).
  • the example partitions 122 of the memory fabric 120 store graph data for a graph or a plurality of graphs. For the sake of readability, though examples herein may refer to graph data for a single graph being distributed across the partitions 122, the partitions 122 may store graph data for a plurality of graphs.
  • the graph partition manager 1 10 may partition a graph from the graph database 130 such that edge data for edges of a graph is distributed across different partitions of the plurality of partitions 122.
  • edge data for a same edge may not be located in more than one partition of the plurality of partitions 122.
  • node data corresponding to nodes linked by edges of the edge data is included in the corresponding partitions 122.
  • node data corresponding the nodes linked by the first edge is also stored in the first partition.
  • node data for the same nodes may be replicated (i.e., duplicated) across multiple partitions of the plurality of partitions 122. Such nodes may be referred to herein as replicate nodes.
  • the graph partition manager 1 10 facilitates graph processing of graph data for a graph stored in the plurality of partitions 122 by accounting for replicated node data in the plurality of partitions 122.
  • the graph partition manager 1 10 may partition graph data for a graph from the graph database 130 into the plurality of partitions 122 in order to lessen (e.g., lower or minimize) a number of replicate nodes across the plurality of partitions 122. For example, some nodes may have an order of magnitude greater number of edges than other nodes (i.e., more edges may be associated with some nodes that others).
  • edge data for edges of the nodes may be distributed across the partitions 122 (along with node data corresponding to those edges) to relatively minimize the number of replicate nodes.
  • the example graph partition manager 1 10 may identify (e.g., flag, mark, etc.) replicate nodes so that the local graph updater 126 may identify replicate nodes when performing local processing of the local graph data 124).
  • the example partitions 122 each include local graph data 124 and a local graph updater 126.
  • the local graph updater 126 iterativeiy processes the example local graph data 124.
  • each process iteration may involve the local graph updater 126 processing the local graph data 124 (e.g., calculating beliefs of nodes of edges in the local graph data 124) and a reconciliation process implemented by the graph partition manager, as further described below to account for replicate nodes.
  • the local graph updater 126 may perform a belief propagation calculation on the local graph data 124,
  • the Iocal graph updater 126 may aggregate message data of edges for each node in the Iocal graph data.
  • the Iocal graph updater 126, for each node / may multiply a node potential p> with a product of all messages from neighbor nodes to calculate belief bi for each node, as follows:
  • the Iocal graph updater 126 may send messages from each node / to neighbor nodes j.
  • the example message may be:
  • ⁇ ⁇ is the potential defined on the edge from vertex / to vertex . and the summation is over the states of the node /.
  • the Iocal graph updater 126 may perform each partition locally within the corresponding partition 122. However, for replicate nodes (which may be identified by a node identifier, flag, mark, table, etc.), all edges may not be incident on the node locally within a partition (as the edges are distributed across the partitions 122). Accordingly, for replicate nodes, the Iocal graph updaters 126 may skip replicate nodes and but still mark such nodes as processed.
  • the graph partition manager 1 10 of FIG. 1 may determine whether convergence of a graph has been reached. For example, the Iocal graph updaters 126 may provide convergence information to the graph partition manager 1 10 when Iocal graph data 124 of the corresponding iocal graph updater 126 has converged. Accordingly, the graph partition manager 1 10 may monitor when the Iocal graph data 124 for all of the partitions 122 (or at least a threshold percentage of the partitions 122) has converged, and end graph processing of the graph.
  • FIG. 2 is a block diagram of an example graph partition manager 1 10 that may be used to implement the graph partition manager 1 10 of FIG. 1 .
  • the example graph partition manager 1 10 of FIG. 2 includes an update receiver 210, a replicate aggregator 220, and partition updaier 230.
  • graph partition manager 1 10 receives/retrieves sets of updated node data corresponding to replicate nodes marked as processed by the local graph updaters 126 during iterations of processing the local graph data 124.
  • the updated node data is received from the plurality of graph partitions 122 via the update receiver 210.
  • the example replicate aggregator 220 aggregates node data for the set of replicate nodes, and the partition updater 230 instructs the local graph updaters 126 to update the node data for the replicate nodes with the aggregated node data of the replicate nodes for subsequent iterations.
  • the example update receiver 210 may serve as a buffer for receiving updated graph data from the local graph updaters 126 of the plurality of graph partitions 122.
  • the replicate aggregator 220 aggregates the updated node data from the graph data that corresponds to replicate nodes.
  • replicate nodes may have been identified (e.g., following a partitioning of the graph by the graph partition manager 1 10 or other graph manager) and/or marked using any suitable techniques.
  • the graph partitioning manager 1 10 may dynamically partition the graph after each iteration.
  • the replicate nodes may be different for each iteration, and the replicate aggregator 220 may need to analyze the node data to identify replicate nodes (e.g., by identifying same node data corresponding to a same node in multiple partitions, by determining calculated hash codes from node data for nodes from separate partitions are the same, etc.). Accordingly, the graph partition manager 1 10 may determine and/or identify replicate nodes in the graph data form the local graph updaters 126 using any suitable techniques.
  • the replicate aggregator 220 of FIG. 2 may aggregate the node data for the replicate nodes by adding, combining, averaging, etc. the updated data such that different/separate updates to the node data in each partition for each iteration is considered/taken into account for a subsequent iteration. Once the node data for the replicate nodes is aggregated, the replicate aggregator 220 provides the aggregated replicate node data to the partition updater 230.
  • the replicate aggregator 220 may be implemented by a shuffle engine by retrieving (e.g., mapping into buckets) node data for replicate nodes (e.g., nodes marked as replicate nodes) from a processing iteration, and reducing the replicate nodes to generate by aggregating the replicate node data.
  • replicate nodes e.g., nodes marked as replicate nodes
  • the example partition updater 230 provides the aggregated replicate node data back to the iocal updaters 126 of the 122 of FIG. 1. From there, the local updaters 128 may update the node data corresponding to the replicate nodes in the partition for any subsequent iterations. For example, the Iocal updater 126 may recalculate the above belief propagation calculations of Equations (1 ) and Equation (2) for the above using the aggregated node data (e.g., aggregated beliefs) for the replicate nodes. Upon recalculating the above, the local updaters 126 may continue to a subsequent processing iteration of the Iocal graph data 124.
  • the aggregated node data e.g., aggregated beliefs
  • the partition updater 230 may provide update strategies (e.g., graph processing types) to be implemented by the local graph updaters.
  • the partition updater 230 may instruct the iocal graph updaters 126 to process the local graph data 122 using synchronous graph processing, asynchronous graph processing, or a hybrid
  • Examples herein may consider the following when selecting or providing instructions to the Iocal graph updaters 126 to use particular update strategies (graph processing types):
  • the synchronous graph processing strategy processes ail nodes once in each partition which may involve relatively increased communication volume with relatively less iterations to reach convergence.
  • the asynchronous processing strategy involves processing a subset of unconverged nodes in each partition.
  • the example set of nodes for asynchronous processing may be selected from a priority queue having the unconverqed nodes of the local graph data 122 in decreasing order of residue (a greatest difference between two successive messages received over all neighbor nodes).
  • the hybrid synchronous/asynchronous graph processing strategy combines the synchronous and asynchronous strategy which may involve communication volume and number of iterations between the synchronous and asynchronous strategies, but lower total time to reach convergence.
  • the partition updater 230 may instruct the local graph updaters 126 to switch from one update strategy used for a previous iteration to another updater strategy for a subsequent iteration.
  • a first number of iterations may be synchronous and the remaining iterations may be
  • the example number of synchronous operations may be tunable or adjustable depending on the graph type (which may be empirically determined).
  • empirical data may be generated for graph types to determine appropriate (e.g., optimal) update strategies for processing graph data distributed across the partitions 122 in accordance with examples herein.
  • processing or update scheduling may be tuned to selectively update a proportion (or percentage) of replicate nodes and non- replicated. For example, for a particular one of the partitions 122, in each iteration, a certain number of replicate nodes and non-replicate nodes may be processed for asynchronous updates where the number of updates is
  • a is a percentage of the replicate nodes processed
  • u r is the number of replicate nodes
  • u is the total number of nodes
  • K is the max value for the updates.
  • the percentage of replicate nodes processed ( ) may be empirically determined to find a suitable (e.g., optimal) update strategy.
  • a processing iteration to calculate belief propagation of nodes of a graph distributed among the partitions 122 includes the local updater 126 updating the local graph data 124 (e.g., calculating the beliefs of the nodes in the local graph data by processing the edge data of edges of the nodes and the node data for the nodes), providing the updated local graph data to the graph partition manager 1 10 to update the local graph data with aggregated node data for the replicate nodes, and continuing to the next iteration (and/or checking for convergence of the graph).
  • FIG. 2 While an example manner of implementing the graph partition manager 1 10 of FIG. 1 is illustrated in FIG. 2, at least one of the elements, processes and/or devices illustrated in FIG. 2 may be combined, divided, rearranged, omitted, eliminated and/or implemented in any other way. Further, the update receiver 210, the replicate aggregator 220, the partition updater 230, and/or, more generally, the example graph partition manager 1 10 of FIG. 2 may be implemented by hardware and/or any combination of hardware and executable instructions (e.g., software and/or firmware).
  • any of the update receiver 210, the replicate aggregator 220, the partition updater 230, and/or, more generally, the example graph partition manager 1 10 could be implemented by at least one of an analog or digital circuit, a logic circuit, a programmable processor, an application specific integrated circuit (ASIC), a programmable logic device (PLD) and/or a field programmable logic device (FPLD).
  • ASIC application specific integrated circuit
  • PLD programmable logic device
  • FPLD field programmable logic device
  • At least one of the update receiver 210, the replicate aggregator 220, and/or the partition updater 230 is/are hereby expressly defined to include a tangible machine readable storage device or storage disk such as a memory, a digital versatile disk (DVD), a compact disk (CD), a Blu-ray disk, etc. storing the executable
  • example graph partition manager 1 10 of FIG. 2 may include at least one element, process, and/or device in addition to, or instead of, those illustrated in FIG. 2, and/or may include more than one of any or all of the illustrated elements, processes and devices.
  • FIGS. 4 and 5 Flowcharts representative of example machine readable instructions for implementing the graph partition manager 1 10 of FIG. 2 is shown in FIGS. 4 and 5.
  • the machine readable instructions comprise a program/process for execution by a processor such as the processor 512 shown in the example processor platform 500 discussed below in connection with FIG. 5.
  • the program/process may be embodied in executable instructions (e.g., software) stored on a tangible machine readable storage medium such as a CD-ROM, a floppy disk, a hard drive, a digital versatile disk (DVD), a Blu-ray disk, or a memory associated with the processor 512, but the entire program/process and/or parts thereof could alternatively be executed by a device other than the processor 512 and/or embodied in firmware or dedicated hardware.
  • a tangible machine readable storage medium such as a CD-ROM, a floppy disk, a hard drive, a digital versatile disk (DVD), a Blu-ray disk, or a memory associated with the processor 512, but the entire program/process and/or parts thereof could alternatively be executed by a device other than the processor 512 and/or embodied in firmware or dedicated hardware.
  • a device such as a CD-ROM, a floppy disk, a hard drive, a digital versatile disk (DVD), a Blu-ray disk, or
  • the example process 300 of FIG. 3 begins with an initiation of the graph partition manager 1 10 (e.g., upon startup, upon instructions from a user, upon startup of a device implementing the graph partition manager 1 10 (e.g., the partitioned graph processing system 100), etc.).
  • the example process 300 may be executed to process a graph having graph data distributed across a plurality of partitions (e.g., the partitions 122) in accordance with examples herein.
  • the update receiver 210 receives replicated nodes of a graph in node data generated from first processing iterations of graph data distributed in the plurality of graph partitions 122.
  • the example first processing iterations of block 310 may be belief propagation calculations in each of the partitions 122.
  • the example replicate aggregator 220 aggregates (e.g., reduces) beliefs for the replicated nodes.
  • the partition updater 230 instructs the local graph updaters 126 of the graph partitions to update beliefs of the replicated nodes for second processing iterations of the graph partitions (e.g., a subsequent processing iteration of the graph).
  • the example process 300 ends, although the example process 300 may be iteratively executed for a subsequent iteration of processing the graph.
  • the example process 400 of FIG. 4 begins with an initiation of the graph partition manager 1 10.
  • the example process 400 may be executed to process an iteration of a graph having graph data distributed to the plurality of partitions 122,
  • the partition updater 230 instructs the local graph updaters 126 of the partitions 122 to perform next graph processing iterations of graph data in the graph partitions 122.
  • the replicate aggregator 220 determines whether node data includes node data for replicated nodes in the first graph processing iterations. If no replicate nodes are in the updated graph data, then control returns to block 410 and the next iteration may be executed. If replicate nodes are in the updated graph data, the replicate aggregator 220, at block 430, aggregates the node data (e.g., reduces beliefs of the replicate nodes) across the graph partitions that include the replicate nodes.
  • the partition updater 230 instructs the local graph updaters 126 to update node data of the replicate nodes with the aggregate node data in the graph partitions 122 for second graph processing iterations (e.g., subsequent execution of the process 400).
  • the example process 400 ends, in some examples, after block 440, control may iteratively return to block 410 until the graph manager 1 10 determines that the local graph data 124 for the graph has converged.
  • FIGS. 3 and 4 may be implemented using coded instructions (e.g., computer and/or machine readable instructions) stored on a tangible machine readable storage medium such as a hard disk drive, a flash memory, a read-only memory (ROM), a compact disk (CD), a digital versatile disk (DVD), a cache, a random-access memory (RAM) and/or any other storage device or storage disk in which information is stored for any duration (e.g., for extended time periods, permanently, for brief instances, for temporarily buffering, and/or for caching of the information).
  • coded instructions e.g., computer and/or machine readable instructions
  • a tangible machine readable storage medium such as a hard disk drive, a flash memory, a read-only memory (ROM), a compact disk (CD), a digital versatile disk (DVD), a cache, a random-access memory (RAM) and/or any other storage device or storage disk in which information is stored for any duration (e.g., for extended time periods, permanently, for brief instances,
  • tangible machine readable storage medium is expressly defined to include any type of machine readable storage device and/or storage disk and to exclude propagating signals and to exclude transmission media.
  • computer readable storage medium and “machine readable storage medium” are used interchangeably. Additionally or alternatively, the example processes of FIGS.
  • 3 and 4 may be implemented using coded instructions (e.g., computer and/or machine readable instructions) stored on a non-transitory computer and/or machine readable medium such as a hard disk drive, a flash memory, a read-only memory, a compact disk, a digital versatile disk, a cache, a random-access memory and/or any other storage device or storage disk in which information is stored for any duration (e.g., for extended time periods, permanently, for brief instances, for temporarily buffering, and/or for caching of the information).
  • a non-transitory computer and/or machine readable medium such as a hard disk drive, a flash memory, a read-only memory, a compact disk, a digital versatile disk, a cache, a random-access memory and/or any other storage device or storage disk in which information is stored for any duration (e.g., for extended time periods, permanently, for brief instances, for temporarily buffering, and/or for caching of the information).
  • FIG. 5 is a block diagram of an example processor platform 500 capable of executing the instructions of FIGS. 3 and 4 to implement the graph partition manager 1 10 of FIG. 2.
  • the example processor platform 500 may be or may be included in any type of apparatus, such as a server, a personal computer, or any other type of computing device.
  • the processor platform 500 of the illustrated example of FIG. 5 includes a processor 512.
  • the processor 512 of the illustrated example is hardware.
  • the processor 512 can be implemented by at least one integrated circuit, logic circuit, microprocessor or controller from any desired family or manufacturer.
  • the processor 512 of the illustrated example includes a local memory 513 (e.g., a cache).
  • the processor 512 of the illustrated example is in communication with a main memory including a volatile memory 514 and a nonvolatile memory 516 via a bus 518.
  • the volatile memory 514 may be
  • the non-volatile memory 518 may be implemented by flash memory and/or any other desired type of memory device. Access to the main memory 514, 516 is controlled by a memory controller.
  • the processor platform 500 of the illustrated example also includes an interface circuit 520.
  • the interface circuit 520 may be implemented by any type of interface standard, such as an Ethernet interface, a universal serial bus (USB), and/or a peripheral component interconnect (PCI) express interface.
  • At least one input device 522 is connected to the interface circuit 520.
  • the input device(s) 522 permit(s) a user to enter data and commands into the processor 512.
  • the input device(s) can be implemented by, for example, an audio sensor, a microphone, a camera (still or video), a keyboard, a button, a mouse, a touchscreen, a track-pad, a trackball, isopoint and/or a voice recognition system.
  • At least one output device 524 is also connected to the interface circuit 520 of the illustrated example.
  • the output device(s) 524 can be implemented, for example, by display devices (e.g., a light emitting diode (LED), an organic light emitting diode (OLED), a liquid crystal display, a cathode ray tube display (CRT), a touchscreen, a tactile output device, a light emitting diode (LED), a printer and/or speakers).
  • the interface circuit 520 of the illustrated example thus, may include a graphics driver card, a graphics driver chip or a graphics driver processor.
  • the interface circuit 520 of the illustrated example also includes a communication device such as a transmitter, a receiver, a transceiver, a modem and/or network interface card to facilitate exchange of data with external machines (e.g., computing devices of any kind) via a network 528 (e.g., an Ethernet connection, a digital subscriber line (DSL), a telephone line, coaxial cable, a cellular telephone system, etc.).
  • a communication device such as a transmitter, a receiver, a transceiver, a modem and/or network interface card to facilitate exchange of data with external machines (e.g., computing devices of any kind) via a network 528 (e.g., an Ethernet connection, a digital subscriber line (DSL), a telephone line, coaxial cable, a cellular telephone system, etc.).
  • DSL digital subscriber line
  • the processor platform 500 of the illustrated example also includes at least one mass storage device 528 for storing executable
  • mass storage device(s) 528 examples include floppy disk drives, hard drive disks, compact disk drives, Biu-ray disk drives, RAID systems, and digital versatile disk (DVD) drives,
  • the coded instructions 532 of FIGS. 3 and 4 may be stored in the mass storage device 528, in the local memory 513 in the volatile memory 514, in the non-volatile memory 518, and/or on a removable tangible machine readable storage medium such as a CD or DVD.
  • update strategies e.g., synchronous, asynchronous, hybrid synchronous/asynchronous

Abstract

L'invention concerne des procédés, un appareil, des systèmes et des articles de fabrication impliqués dans le traitement des données de graphes réparties dans les partitions d'une mémoire. Dans des exemples, un gestionnaire de partitions de graphes gère des itérations de traitement de données locales dans chaque partition exécutée par les dispositifs de mise à jour de graphes locaux des partitions. Les données de nœud répliquées sont agrégées par le gestionnaire de partitions de graphes et renvoyées aux dispositifs de mise à jour de graphes locaux pour les itérations suivantes.
PCT/US2015/061798 2015-11-20 2015-11-20 Traitement de graphes partitionnés WO2017086986A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/US2015/061798 WO2017086986A1 (fr) 2015-11-20 2015-11-20 Traitement de graphes partitionnés

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/US2015/061798 WO2017086986A1 (fr) 2015-11-20 2015-11-20 Traitement de graphes partitionnés

Publications (1)

Publication Number Publication Date
WO2017086986A1 true WO2017086986A1 (fr) 2017-05-26

Family

ID=58717551

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2015/061798 WO2017086986A1 (fr) 2015-11-20 2015-11-20 Traitement de graphes partitionnés

Country Status (1)

Country Link
WO (1) WO2017086986A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111241353A (zh) * 2020-01-16 2020-06-05 支付宝(杭州)信息技术有限公司 一种图数据的分区方法、装置以及设备
CN111615673A (zh) * 2018-01-22 2020-09-01 西门子股份公司 用于工业生产机器控制的技能匹配

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7216123B2 (en) * 2003-03-28 2007-05-08 Board Of Trustees Of The Leland Stanford Junior University Methods for ranking nodes in large directed graphs
US20080155656A1 (en) * 2006-12-22 2008-06-26 John Mark Agosta Authenticated distributed detection and inference
US20130013567A1 (en) * 2011-07-07 2013-01-10 International Business Machines Corporation Archiving de-duplicated data on tape storage media using graph partitions
US20140122422A1 (en) * 2012-10-25 2014-05-01 Hewlett-Packard Development Company, L.P. Data synchronization
US20140129320A1 (en) * 2011-04-05 2014-05-08 The Trustees Of Columbia University In The City Of New York B-matching using sufficient selection belief propagation

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7216123B2 (en) * 2003-03-28 2007-05-08 Board Of Trustees Of The Leland Stanford Junior University Methods for ranking nodes in large directed graphs
US20080155656A1 (en) * 2006-12-22 2008-06-26 John Mark Agosta Authenticated distributed detection and inference
US20140129320A1 (en) * 2011-04-05 2014-05-08 The Trustees Of Columbia University In The City Of New York B-matching using sufficient selection belief propagation
US20130013567A1 (en) * 2011-07-07 2013-01-10 International Business Machines Corporation Archiving de-duplicated data on tape storage media using graph partitions
US20140122422A1 (en) * 2012-10-25 2014-05-01 Hewlett-Packard Development Company, L.P. Data synchronization

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111615673A (zh) * 2018-01-22 2020-09-01 西门子股份公司 用于工业生产机器控制的技能匹配
US11747793B2 (en) 2018-01-22 2023-09-05 Siemens Aktiengesellschaft Skill matching for control of an industrial production machine
CN111615673B (zh) * 2018-01-22 2023-09-29 西门子股份公司 一种用于工业控制中的技能匹配的系统
CN111241353A (zh) * 2020-01-16 2020-06-05 支付宝(杭州)信息技术有限公司 一种图数据的分区方法、装置以及设备
CN111241353B (zh) * 2020-01-16 2023-08-22 支付宝(杭州)信息技术有限公司 一种图数据的分区方法、装置以及设备

Similar Documents

Publication Publication Date Title
US8904149B2 (en) Parallelization of online learning algorithms
US11868315B2 (en) Method for splitting region in distributed database, region node, and system
US10574752B2 (en) Distributed data storage method, apparatus, and system
AU2012217645B2 (en) Managing buffer overflow conditions
US10089761B2 (en) Graph processing using a shared memory
US20140366020A1 (en) System and method for managing virtual machine stock
US10621104B2 (en) Variable cache for non-volatile memory
US8768979B2 (en) In-memory data grid hash scheme optimization
CN109510852B (zh) 灰度发布的方法及装置
CN106909556B (zh) 内存集群的存储均衡方法及装置
WO2017086986A1 (fr) Traitement de graphes partitionnés
JP6189266B2 (ja) データ処理装置、データ処理方法及びデータ処理プログラム
US20170344607A1 (en) Apparatus and method for controlling skew in distributed etl job
CN104376047A (zh) 一种基于HBase的大表join方法
US20230132117A1 (en) Handling system-characteristics drift in machine learning applications
CN108021448B (zh) 一种内核空间的优化方法及装置
CN117763024A (zh) 一种数据分片抽取方法及装置
US20150347409A1 (en) Convert Command Into a BULK Load Operation
CN113986962A (zh) 排行榜生成方法、装置、设备及存储介质
CN110658999B (zh) 一种信息更新方法、装置、设备及计算机可读存储介质
CN109491594B (zh) 矩阵求逆过程中优化数据存储空间的方法和装置
US20120166728A1 (en) Systems and methods for performing parallel multi-level data computations
US11467971B2 (en) Systems and methods for accelerating data computation
US8782364B2 (en) Determining availability of data elements in a storage system
US20170075955A1 (en) Database system, database access method, and database access program

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15908973

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 15908973

Country of ref document: EP

Kind code of ref document: A1