CN113240089B - Graph neural network model training method and device based on graph retrieval engine - Google Patents

Graph neural network model training method and device based on graph retrieval engine Download PDF

Info

Publication number
CN113240089B
CN113240089B CN202110550314.9A CN202110550314A CN113240089B CN 113240089 B CN113240089 B CN 113240089B CN 202110550314 A CN202110550314 A CN 202110550314A CN 113240089 B CN113240089 B CN 113240089B
Authority
CN
China
Prior art keywords
node
graph
neural network
network model
nodes
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110550314.9A
Other languages
Chinese (zh)
Other versions
CN113240089A (en
Inventor
王贤明
吴志华
于佃海
马艳军
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Baidu Netcom Science and Technology Co Ltd
Original Assignee
Beijing Baidu Netcom Science and Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Baidu Netcom Science and Technology Co Ltd filed Critical Beijing Baidu Netcom Science and Technology Co Ltd
Priority to CN202110550314.9A priority Critical patent/CN113240089B/en
Publication of CN113240089A publication Critical patent/CN113240089A/en
Application granted granted Critical
Publication of CN113240089B publication Critical patent/CN113240089B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061—Partitioning or combining of resources
    • G06F9/5072—Grid computing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Abstract

The invention discloses a graph neural network model training method and device based on a graph retrieval engine, and relates to the field of artificial intelligence, in particular to the field of intelligent search and deep learning. The specific implementation scheme is as follows: in response to receiving a request from the graph neural network model computing device to obtain a node in the graph, sending a node identification of the node in at least a portion of the stored graph to the graph neural network model computing device; in response to receiving a request from a graph neural network model computing device to obtain neighboring nodes of a node, retrieving from the neighboring nodes of the node at least one neighboring node that satisfies a predetermined condition according to the stored graph; and sending the retrieved node identification of the at least one neighboring node to the graph neural network model computing device for training the graph neural network model. By the method, deep learning training of the large-scale image is improved, concurrency and expansibility are improved, and user experience is improved.

Description

Graph neural network model training method and device based on graph retrieval engine
Technical Field
The disclosure relates to the technical field of artificial intelligence, in particular to a graph neural network model training method and device based on a graph retrieval engine in the fields of intelligent search and deep learning.
Background
With the advent of the big data age, more and more businesses have accumulated a large amount of relational data from entity to entity. In order to effectively mine and utilize the information, the related algorithm is paid more attention and attention by researchers, and the graph neural network model is the most concerned one. In recent years, the graph neural network model is a big hotspot in the field of deep learning. The graph neural network model fully mines the similarity relation between the entities by combining the graph structure and the deep learning technology. Compared with the original pattern mining-based related data mining technology, the graph neural network model obtains better accuracy and generalization performance, so that the graph neural network model is more and more important to be applied in the recommendation field in the Internet industry.
Through the neural network model, an enterprise can mine similarity information between a user and other entities, so that various information recommendations such as interest recommendations and friend recommendations are achieved. It is expected that neural network models will play an increasingly important role in the future as research progresses. However, there are many problems to be solved in the training process of the graph neural network model.
Disclosure of Invention
The disclosure provides a graph neural network model training method and device based on a graph retrieval engine.
According to a first aspect of the present disclosure, a graph neural network model training method based on a graph retrieval engine is provided. The method comprises the following steps: in response to receiving a request from the graph neural network model computing device to obtain a node in the graph, sending a node identification of the node in at least a portion of the stored graph to the graph neural network model computing device; in response to receiving a request from a graph neural network model computing device to acquire a neighbor node of a node, retrieving at least one neighbor node satisfying a predetermined condition from the neighbor node of the node according to the stored graph; and sending the retrieved identification of the at least one neighboring node to a graph neural network model computing device for training a graph neural network model.
According to a second aspect of the present disclosure, a graph neural network model training method based on a graph search engine is provided. The method comprises the following steps: sending a request to a graph management device to obtain a node in a graph to receive a node identification of the node from the graph management device, the graph management device storing at least a portion of the graph including the node; sending a request for acquiring a neighboring node of the node to the graph management device to receive a node identifier of the neighboring node from the graph management device, the neighboring node being retrieved by the graph management device according to the stored graph; input data for the graph neural network model is determined based on the node identifications of the nodes and the node identifications of neighboring nodes for training the graph neural network model.
According to a third aspect of the present disclosure, a graph neural network model training device based on a graph retrieval engine is provided. The apparatus includes a node identification sending module configured to send, to a graph neural network model computing device, a node identification of a node in at least a portion of a stored graph in response to receiving a request from the graph neural network model computing device to obtain the node in the graph; a neighbor node acquisition module configured to, in response to receiving a request from a graph neural network model computing device to acquire neighbor nodes of a node, retrieve from the stored graph at least one neighbor node that satisfies a predetermined condition from among the neighbor nodes of the node; and a transmitting module configured to transmit the retrieved node identification of the at least one neighboring node to the graph neural network model computing device for training a graph neural network model.
According to a fourth aspect of the present disclosure, a graph neural network model training device based on a graph search engine is provided. The apparatus includes a node request sending module configured to send a request to a graph management device to obtain a node in a graph to receive a node identification of the node from the graph management device, the graph management device storing at least a portion of the graph including the node; a neighboring node request sending module configured to send a request for obtaining a neighboring node of a node to the graph management device to receive a node identifier of the neighboring node from the graph management device, the neighboring node being retrieved by the graph management device according to the stored graph; and an input data determination module configured to determine input data of the graph neural network model based on the node identifications of the nodes and the node identifications of the neighboring nodes for training the graph neural network model.
According to a fifth aspect of the present disclosure, an electronic device is provided. The electronic device includes at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a method according to the first aspect of the disclosure.
According to a sixth aspect of the present disclosure, an electronic device is provided. The electronic device includes at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a method according to the second aspect of the disclosure.
According to a seventh aspect of the present disclosure, there is provided a non-transitory computer readable storage medium having stored thereon computer instructions for causing a computer to perform the method according to the first aspect of the present disclosure.
According to an eighth aspect of the present disclosure, there is provided a non-transitory computer readable storage medium having stored thereon computer instructions for causing a computer to perform the method according to the second aspect of the present disclosure.
According to a ninth aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements the steps of the method according to the first aspect of the present disclosure.
According to a tenth aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements the steps of the method according to the second aspect of the present disclosure.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.
Drawings
The drawings are included to provide a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:
FIG. 1 illustrates a schematic diagram of an environment 100 in which embodiments of the present disclosure can be implemented;
FIG. 2 illustrates a flow diagram of a graph neural network model training method 200 based on a graph search engine, in accordance with some embodiments of the present disclosure;
FIG. 3 illustrates a flow diagram of a graph neural network model training method 300 based on a graph search engine in accordance with some embodiments of the present disclosure;
FIG. 4 illustrates a schematic diagram of a tree structure 400 constructed in accordance with some embodiments of the present disclosure;
fig. 5 illustrates a node schematic 500 for processing shards according to some embodiments of the present disclosure;
FIG. 6 shows a schematic diagram of an example of a distribution 600 of a graph neural network model, in accordance with some embodiments of the present disclosure;
FIG. 7 illustrates a block diagram of an apparatus 700 for training a neural network model according to some embodiments of the present disclosure;
FIG. 8 illustrates a block diagram of an apparatus 800 for training a neural network model according to some embodiments of the present disclosure; and
fig. 9 illustrates a block diagram of a device 900 capable of implementing multiple embodiments of the present disclosure.
Detailed Description
Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
In describing embodiments of the present disclosure, the terms "include" and "comprise," and similar language, are to be construed as open-ended, i.e., "including but not limited to. The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first," "second," and the like may refer to different or the same object. Other explicit and implicit definitions are also possible below.
The graph neural network algorithm is derived from deep learning, and the practical process is not independent of deep learning frameworks such as tensierflow, paddley and pyrrch. However, the graph neural network algorithm has a series of difficulties in the application process. Compared with the traditional machine learning task, the data of the graph neural network model not only needs to store the relevant information of each entity node, but also needs a large amount of storage resources to store the relationship or interaction information between the entities, so that the storage space of the training data depended on by the graph neural network model often exceeds the storage space of the traditional machine learning task by several orders of magnitude.
Therefore, training costs are increased. The limited storage resources and computing resources of a single machine often cannot meet the training of the neural network model. One conventional scheme utilizes a distributed key-value (kv) storage tool to store information of nodes and edges, where keys are nodes and values are edges or features of the nodes, and training is performed in cooperation with other deep learning frameworks. A user needs to define a node distributed storage rule by self, the node information is imported into a distributed kv storage tool of a server side in advance, and a related server is developed at the same time. After receiving a client query request, the server queries the kv table to obtain a corresponding node or neighbor, and then completes functions of distributed graph walking, graph sampling and the like by matching with a client code. The client side continuously acquires and collects training data in the mode, and then delivers the data to the deep learning framework to finish training. However, this solution lacks support for basic operations of the graph, and in order to implement these operations, a great development cost is added, which is not favorable for rapid online deployment. Meanwhile, since storage is completely dependent on a distributed kv storage architecture, developers cannot effectively build indexes for graphs to complete related graph operations, making such an architecture inefficient in distributed graph operations.
Another conventional approach is to utilize a specialized graph database. The graph databases can effectively store the information of the graph nodes, support some graph operations based on distribution, and save certain development burden of users. The client side realizes basic operations such as graph query, wandering, sampling and the like through a distributed graph database interface, so that training data is collected, and graph training is completed by matching with a deep learning framework. However, such architectures are not specifically designed for the graph deep learning algorithm. In order to meet the requirements of various scenes, a large amount of codes are added to the architecture to ensure data consistency, so that the architecture is relatively inefficient in processing the deep learning task of the graph. In addition, these architectures do not support graph neural network training by themselves.
In order to solve at least the above problems, according to an embodiment of the present disclosure, an improved scheme for training a neural network model of a graph is proposed. In this scheme, a request for acquiring a node in a graph is first sent to a graph management apparatus to receive a node identification of the node from the graph management apparatus. Then, a request for acquiring a neighboring node of the node is sent to the graph management device of the storage node to receive the node identification of the neighboring node from the graph management device. A node sequence including at least the node and the neighboring node is next generated using the node identification of the node and the node identification of the neighboring node. Input data for the graph neural network model is then determined based on the sequence of nodes for training the graph neural network model. By the method, deep learning training of the large-scale image is improved, concurrency and expansibility are improved, and user experience is improved.
Fig. 1 illustrates a schematic diagram of an environment 100 in which various embodiments of the present disclosure can be implemented. The example environment 100 includes graph management devices 102-1, 102-2, 102-3, \8230;, 102-N, where N is a positive integer and for ease of description is collectively referred to as graph management devices 102. The example environment 100 also includes a graph neural network model computing device 104-1, 104-2, \8230;, 104-M, where M is a positive integer, collectively referred to as the graph neural network model computing device 104 for ease of description.
The example graph management device 102 and the graph neural network model computing device 104 include, but are not limited to, personal computers, server computers, hand-held or laptop devices, mobile devices (such as mobile phones, personal Digital Assistants (PDAs), media players, and the like), virtual machines, multiprocessor systems, consumer electronics, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like. The Server may be a cloud Server, which is also called a cloud computing Server or a cloud host, and is a host product in a cloud computing service system, so as to solve the defects of high management difficulty and weak service expansibility in a traditional physical host and a VPS service ("Virtual Private Server", or simply "VPS"). The server may also be a server of a distributed system, or a server incorporating a blockchain.
The drawing management apparatus 102 is used to manage a drawing. Each of the graph management devices 102 stores at least a part of a graph. In some embodiments, the graph is partitioned into a plurality of shards, each shard consisting of a portion of the nodes and edges of the graph. In one example, there is one slice per graph management device 102. In another example, each graph management device 102 includes a plurality of shards. For each fragment, the access operations to the fragment, including but not limited to sampling, finding, adding, etc., are performed sequentially by queuing in an operation queue.
The graph neural network model computing device 104 is used to run a graph neural network model. In one example, each graph neural network model computing device 104 runs a neural network model separately and trains a portion of the input data with the running neural network model. In the training process, the graph neural network model computing device 104 requests the graph management device 102 for relevant information of the nodes to form input data of the graph neural network model to train the graph neural network model. Training the graph neural network model with the input data may result in a trained graph neural network model 106 for later use.
By the method, deep learning training of the large-scale image is improved, concurrency and expansibility are improved, and user experience is improved.
Fig. 1 above illustrates a schematic diagram of an environment 100 in which various embodiments of the present disclosure can be implemented. A flow diagram of a graph neural network model training method 200 based on a graph search engine according to some embodiments of the present disclosure is described below in conjunction with fig. 2. Method 200 in fig. 2 is performed by diagram management device 102 in fig. 1 or any suitable computing device.
At block 202, it is determined whether a request to acquire a node in the graph is received from the graph neural network model computing device 104. For example, the graph management device 102 detects whether a request to acquire a node in the graph is received from the graph neural network model computing device 104. If a request is received, at block 204, node identifications of nodes in at least a portion of the stored graph are sent to the neural network model computing device 104.
In one example, the graph is stored in one graph management device 102. In one example, a graph is divided into a plurality of slices, each slice comprising a portion of the graph, the slices being stored by a plurality of graph management devices 102. Each graph management device has one or more shards. The above examples are intended to be illustrative of the present disclosure, and are not intended to be limiting of the present disclosure.
In some embodiments, when the graph management device 102 receives a request to acquire a node, the node identification of the randomly selected node is sent to the graph neural network model computing device 104. In some embodiments, the graph management apparatus 102 stores therein a node queue queued in the order of the reception time of the nodes, and then sequentially fetches the identifications of the nodes from the node queue to transmit to the graph neural network model calculation apparatus 104. By the method, the node identification of the node can be quickly and accurately acquired.
At block 206, it is determined whether a request to acquire a neighbor of a node is received from the graph neural network model computing device 104. Upon determining that a request to acquire the neighbor nodes of the node is received from the graph neural network model computing device 104, at block 208, at least one neighbor node that satisfies a predetermined condition is retrieved from the neighbor nodes of the node according to the stored graph.
In some embodiments, a graph management device obtains at least a portion of the graph, the at least a portion including a plurality of tiles, each tile including at least a portion of a sub-portion, each tile corresponding to a task queue maintained by a task execution thread, tasks in the task queue being executed in order. If it is determined that the identity of the node in the request to acquire the neighboring node is located in a group of fragments, performing a retrieval operation of the neighboring node of the node concurrently by a group of task execution threads for the group of fragments. In some embodiments, other operations, including read, modify, etc., operations may also be performed concurrently. In this way, various operations can be quickly processed.
In some embodiments, if the graph management device 102 receives a request from the graph neural network model computing device 104 to bulk modify a node in the graph, bulk modifications are made to the node in the graph. In this way, requests can be processed in batches.
In some embodiments, the graph management apparatus 102 stores nodes in at least a portion of the graph, and sets a neighbor list for each of the nodes to store its neighbors and their weights. The weight value represents the magnitude of the degree of association of the node with the corresponding neighboring node. Thus, a tree structure, e.g., a binary tree, may be constructed for each node's neighboring nodes based on each node's neighboring nodes and node weights. Leaf nodes of the tree structure correspond to neighbors of the node and have corresponding weight values. The weight value of a parent node in the tree structure is determined by the weight value of its child node. In some embodiments, the tree structure is a balanced binary tree, and the tree structure may dynamically perform addition, deletion, and adjustment of node weight values for nodes.
FIG. 4 illustrates a schematic diagram of a constructed tree structure 400 according to some embodiments of the present disclosure. As shown in fig. 4, a node has 4 neighbor nodes, wherein the four neighbor nodes correspond to 4 leaf nodes in the tree structure 400 with weights of 0.6, 0.8, 0.2, 0.4, respectively. A parent node is then formed above every two leaf nodes, the parent node having a weight value that is the sum of the weight values of the two leaf nodes, in this way a tree structure 400 is constructed that includes nodes 402, 404 and 406 and four leaf nodes 408, 410, 412, 414.
Next, returning to the description of fig. 2, when selecting a neighboring node of a node, the graph management apparatus 102 acquires a tree structure for the node. Then, a random number is generated for each layer of the tree structure of the nodes, and the size of the random number is not larger than the weight value of the node of each layer. Then, a neighbor node to transmit is selected based on the random number. In this way, a suitable neighbor node can be selected quickly.
Taking fig. 4 as an example, at the level of the root node 402, a random number not greater than 2.0 is generated, the left sub-tree is selected if the random number is between 0-1.4 (including 1.4), and the right sub-tree is selected if the random number is between 1.4-2.0 (not including 1.4). If the left sub-tree is selected based on a random number, a random number is generated at the level of node 404, the size of the random number is not greater than 1.4, if the random number is between 0-0.6 (including 0.6), node 408 is selected as the neighbor node to be transmitted, and if the random number is between 0.6-1.4 (not including 0.6), node 410 is selected as the neighbor node to be transmitted. If a plurality of nodes need to be transmitted, the weight of the selected node is set to be 0 at the moment, the weight of each level is readjusted, and then the next adjacent node to be transmitted is reselected. After the selection of the neighbor nodes is completed for a request of a graph neural network model computing device, the original tree structure can be restored.
In some embodiments, one thread is assigned to each tile in the graph management device 102 and one task queue is maintained to enable access to the tile. All access operations to the same slice are added to the task queue in the form of tasks, and the thread responsible for the slice continuously takes out tasks from the task queue and executes them, for example, as shown in fig. 5, for the slice 502, a sampling operation 506, a query operation 508 and an add operation 508 for accessing it are sequentially executed in the task queue 504. This ensures that there is no problem of data inconsistency caused by simultaneous read and write operations of multiple threads on one data slice.
In some embodiments, when sending a node operation request (e.g., a query), the graph neural network model computing device 104 may collect a batch of nodes that are sent to the same graph management device 102 in advance, and batch process the operations of multiple nodes with one request. After receiving the batch requests, the graph management device 102 quickly distributes the requests to task queues of different fragments for concurrent processing. Therefore, network communication overhead caused by frequently and repeatedly sending network requests can be effectively avoided.
At block 210, the retrieved node identification of the at least one neighboring node is sent to the graph neural network model computing device for training a graph neural network model.
In some embodiments, the graph management device 102 sends the parameters to the graph neural network model computing device upon receiving a request to obtain the parameters of the graph neural network model. When the graph management apparatus 102 receives the adjustment value for the parameter, the parameter of the graph neural network model is adjusted based on the adjustment value. By the method, parameters can be quickly transmitted and adjusted.
In some embodiments, the communication of the graph neural network model computing device with the graph management device is implemented through a dedicated communication protocol, and the information of the transmitted nodes is implemented through a template. For example, different types of features may be freely defined for different nodes by the user through template programming, and efficient transfer communication of the different types of node features between the graph management device 102 and the graph neural network computational model computing device 104 is enabled through a protocol such as the brpc protocol.
In some embodiments, where the graph neural network model computing device and the graph management device are located on the same physical device, the graph neural network model computing device and the graph management device interact with the same physical device. For example, the graph management device 102 and the graph neural network computational model computing device 104 are virtual machines that may be placed in the same physical machine, in which case some of the communication overhead may be reduced.
By the method, deep learning training of the large-scale image is improved, concurrency and expansibility are improved, and user experience is improved.
A flowchart of a method 300 for training a neural network model according to some embodiments of the present disclosure is described above in connection with fig. 2 and 4-5. A flow diagram of a graph neural network model training method 300 for a graph-based search engine according to some embodiments of the present disclosure is described below in conjunction with fig. 3. The method 300 in FIG. 3 is performed by the graphical neural network computational model computing device 104 in FIG. 1 or any suitable computing device.
At block 302, a request to obtain a node in a graph is sent to a graph management device to receive a node identification for the node from the graph management device, the graph management device storing at least a portion of the graph including the node. For example, the graph neural network model computing device 104 in fig. 1 may send a request to obtain a node to the graph management device 102 in order to obtain an identification of the node from the graph management device 102 that stores the node of the graph.
In some embodiments, the graph management device is a plurality of graph management devices, each graph management device storing a portion including a graph. In one example, a graph is divided into a plurality of tiles, each tile corresponding to a portion of the graph. Each graph management device may store one or more shards. In some embodiments, the graph management device is a graph management device in which the entire graph is stored. The above examples are intended to be illustrative of the present disclosure, and are not intended to be limiting of the present disclosure.
In some embodiments, a request to obtain a node in a graph is sent to one graph management device to receive one or more nodes from the graph management device. In some embodiments, a request to obtain a node in a graph is sent to a plurality of graph management devices to receive one or more nodes from each of the plurality of graph management devices. The above examples are intended to be illustrative of the present disclosure, and are not intended to be limiting of the disclosure in any way.
At block 304, a request is sent to the graph management device to obtain a neighbor node of the node to receive a node identification of the neighbor node from the graph management device, the neighbor node retrieved by the graph management device from the stored graph. For example, the graph neural network model computing device 104 in fig. 1, after obtaining the node identification, may send a request to obtain the neighboring node of the node using the obtained node identification to receive the identification of the neighboring node of the node from the graph management device 102.
In some embodiments, the graph neural network model computing device 104 hashes the node identifications to determine the graph management device 102 in which the node is located. In one example, a shard in which the node is located may be determined by hashing the node identification, and then the graph management device 102 in which the shard is located is determined based on the shard identification. In another example, each graph management device 102 includes a shard, and the graph management device 102 in which the node is located is directly determined by hashing the node identification. The above examples are intended to be illustrative of the present disclosure, and are not intended to be limiting of the present disclosure. The graph neural network model computing device 104 sends a request to the determined graph management device 102 to obtain neighboring nodes, the request including at least a node identification of the node and an identification of the corresponding graph management device 102. By the method, the graph management device 102 of the storage node can be quickly determined, the adjacent node can be accurately and quickly acquired, and the consumption of network resources is reduced.
At block 306, input data for the graph neural network model is determined for training the graph neural network model based on the node identifications of the nodes and the node identifications of the neighboring nodes.
In some embodiments, the graph neural network model computing device 104 generates a sequence of nodes including at least the node and the neighboring nodes based on the node identification of the node and the node identification of the neighboring node. For example, the graph neural network model computing device 104 forms a sequence of nodes with the obtained nodes and their neighbors. Input data for the graph neural network model is then determined based on the sequence of nodes for training the graph neural network model. For example, the graphical neural network model computing device 104 will utilize the sequence of nodes to form the input data. In this way, input data can be generated quickly.
In some embodiments, the input data may be generated using the nodes and neighboring nodes of the graph by any suitable graph algorithm to interoperate between the graph neural network model computing device 104 and the graph management device 102. The above examples are intended to be illustrative of the present disclosure, and are not intended to be limiting of the present disclosure.
In some embodiments, in generating the input data using the sequence of nodes, the graph neural network model computing device 104 determines the graph management device 102 that stores the neighboring nodes using the obtained identification of the neighboring nodes. Then, a request for acquiring a next neighbor node of the neighbor nodes is sent to the determined graph management apparatus 102 to acquire the next neighbor node. The graph neural network model computing device 104 adds the next neighbor node to the sequence of nodes. The graphical neural network model computing device 104 expands the node sequence by continuously acquiring neighboring nodes of the neighboring nodes, and if the length of the direct node sequence satisfies a predetermined threshold, the adding of the node sequence is not continued. By the method, the nodes with the association relation can be quickly acquired, and the acquisition efficiency of the nodes is improved.
In some embodiments, a node is a node, and a sequence of nodes is formed. In some embodiments, the node is a plurality of nodes, each of the plurality of nodes forming a corresponding sequence of nodes.
In some embodiments, positive sample data is determined based on two nodes if it is determined that the difference in position between the two nodes in the sequence of nodes is not greater than a threshold size. For example, if the position difference between two nodes in the node sequence is not greater than the threshold, the two nodes are associated in the table, and the class mark of the node pair is determined to be 1 to form positive sample data. If the difference of the positions between two nodes in the node sequence is larger than the threshold value, determining negative sample data based on the two nodes. For example, when the position difference of two nodes in the node sequence is smaller than a threshold value, the class mark of the node pair is determined to be 0 to form negative sample data. Input data is then generated using the positive and negative sample data. For example, the input data will be formed by class labels between a predetermined number of node teams. By this way, input data can be generated quickly and accurately.
In some embodiments, the graph management device 102 also stores parameters of the graph neural network model. The graph neural network model computing device 104 sends a request to the graph management device 102 for obtaining parameters of the graph neural network model to receive parameters, such as initial parameters, from the graph management device 10. The parameters and input data are then applied to the graph neural network model to obtain node representations for the nodes. By the method, parameters and input data can be rapidly applied to the graph neural network model.
In some embodiments, the graph neural network model computing device 104 determines adjustment values for parameters of the graph neural network model based on the node representations of the nodes. In one example, cosine transforming node representations of node pairs of the nodes determines similarity values for the two nodes, then determines differences between the similarity values and their actual landmarks, and then determines whether to continue training based on a comparison of the sum of the differences for all node pairs to a predetermined threshold. If greater than the predetermined threshold, indicates that the parameters need to be adjusted to retrain. In another example, the node representations of the obtained node pairs are non-linearly transformed, e.g., processed using a neural network model, to obtain a similarity value for the vector representations of the two node pairs, and then a difference between the similarity value and a landmark between the two nodes is determined. It is then determined whether to continue training based on a comparison of the sum of the differences for all node pairs to a predetermined threshold. If greater than the predetermined threshold, indicates that an adjustment to the parameter is required to retrain. At this point the graphical neural network model computing device 104 determines an adjusted value for the parameter.
The graph neural network model calculation device 104 transmits the adjustment values of the parameters to the graph management device 102 to adjust the parameters of the graph neural network model. In some embodiments, if the graph management device 102 receives an adjustment value sent by the graph neural network model computing device 104, the parameter will be adjusted with the adjustment value for the parameter. In some embodiments, if the graph management device 102 receives adjustment values sent by multiple graph neural network model computing devices 104. The parameter may be adjusted using a plurality of adjustment values. In one example, an average of a plurality of adjustment values is calculated. In another example, a weighted average of the plurality of adjustment values is calculated. The above examples are intended to be illustrative of the present disclosure, and are not intended to be limiting of the present disclosure. Through the mode, the parameters can be quickly adjusted.
In some embodiments, the graph neural network model computing device 104 may collect multiple access operations for multiple nodes of a graph in the same graph management device. Then, a processing request including the plurality of access operations is sent to the same graph management apparatus to perform batch processing on the plurality of access operations. By the method, a plurality of operations on a plurality of nodes can be processed in batch, and the use of network resources is reduced.
In some embodiments, in the case where the graph neural network model computing device and the graph management device are located on the same physical device, the graph neural network model computing device and the graph management device interact using the same physical device. At this time, the network resource is not needed to be used for communication, so that the processing speed is improved, and the network resource is saved.
In some embodiments, the communication of the graph neural network model computing device with the graph management device is implemented through a dedicated communication protocol, and the information of the transmitted nodes is implemented through a template. For example, different types of features may be freely defined for different nodes by the user through template programming, and efficient transfer communication of the different types of node features between the graph management device 102 and the graph neural network computational model computing device 104 is enabled through a protocol such as the brpc protocol.
By the method, deep learning training of the large-scale image is improved, concurrency and expansibility are improved, and user experience is improved.
Fig. 3 above shows a flowchart describing a graph neural network model training method 300 based on a graph search engine, in accordance with some embodiments of the present disclosure. A schematic diagram of a distribution example 600 of a graph neural network model according to some embodiments of the present disclosure is described below in conjunction with fig. 6.
In FIG. 6, it includes a data store 602 that corresponds to the partitioning of a graph into slices for storage into a plurality of different graph management devices. A plurality of node sequences may then be generated by performing graph sampling 604. Input data is then generated using the sequence of nodes for transmission to a plurality of graph neural network models forming a distributed model. The obtained parameters are then stored in parameter server 608. By the method, deep learning training of the large-scale image is improved, concurrency and expansibility are improved, and user experience is improved.
Fig. 7 shows a schematic block diagram of an apparatus 700 for training graph neural network model based on graph search engine according to an embodiment of the present disclosure. As shown in fig. 8, the apparatus 700 includes a node identification sending module 702 configured to send a node identification of a node in at least a portion of a stored graph to a neural network model computing device in response to receiving a request from the neural network model computing device to obtain the node in the graph. The apparatus 700 further includes a neighboring node acquisition module 704 configured to, in response to receiving a request from a graph neural network model computing device to acquire neighboring nodes of a node, retrieve from the stored graph at least one neighboring node from among the neighboring nodes of the node that satisfies a predetermined condition. The apparatus 700 further includes a sending module 706 configured to send the retrieved node identification of the at least one neighboring node to the graph neural network model computing device for training a graph neural network model.
In some embodiments, wherein the node identification sending module 702 comprises one of: a random transmission module configured to transmit the node identification of the randomly selected node to the graph neural network model computing device; or an in-order sending module configured to send node identifications of sequentially selected nodes from the ordered queue of nodes to the graph neural network model computing device.
In some embodiments, the neighboring node acquisition module comprises: a graph acquisition module configured to acquire at least a portion of the graph, the at least a portion including a plurality of tiles, each tile including a sub-portion of the at least a portion, each tile corresponding to a task queue maintained by a task execution thread, tasks in the task queue being executed in order; a concurrent execution module configured to concurrently perform, by a set of task execution threads for the set of shards, a retrieval operation of a neighboring node of a node if it is determined that the identity of the node in the request to obtain the neighboring node is located in the set of shards of the plurality of shards.
In some embodiments, the apparatus 700 further comprises: a bulk modification module configured to bulk modify nodes in a graph in response to receiving a request from a graph neural network model computing device to bulk modify a node in the graph.
In some embodiments, the neighboring node obtaining module 704 includes a random number generating module configured to generate a random number for each layer of a tree structure of nodes, a leaf node of the tree structure corresponding to a neighboring node of the node and having a corresponding weight value, the weight value of a parent node in the tree structure being determined by its child node weight value, the weight value of the leaf node representing the magnitude of the degree of association of the node with the corresponding neighboring node; and a neighbor selection module configured to select a neighbor node to transmit based on the random number.
In some embodiments, wherein the tree structure is a binary tree, the tree structure may dynamically perform addition, deletion, and adjustment of node weight values for nodes.
In some embodiments, the apparatus 700 further comprises: a parameter sending module configured to send a parameter to a neural network model computing device in response to receiving a request to obtain the parameter; and an adjustment module configured to adjust a parameter of the graphical neural network model based on the adjustment value in response to receiving the adjustment value for the parameter.
In some embodiments, the graph neural network model computing device is located on the same physical device as a graph management device running the method, and the graph neural network model computing device and the graph management device interact with the same physical device.
In some embodiments, the communication of the graph neural network model computing device with the graph management device running the method is implemented by a dedicated communication protocol, and the transmitted information of the nodes is implemented by a template.
Fig. 8 shows a schematic block diagram of an apparatus 800 for training graph neural network model based on graph search engine according to an embodiment of the present disclosure. As shown in fig. 8, apparatus 800 includes a node request sending module 802 configured to send a request to a graph management device to obtain a node in a graph to receive a node identification of the node from the graph management device, the graph management device storing at least a portion of the graph including the node. The apparatus 800 further comprises a neighbor node request sending module 804 configured to send a request to a graph management device of the storage node for obtaining a neighbor node of the node to receive a node identification of the neighbor node from the graph management device. The apparatus 800 further comprises a first input data determining module 806 configured to determine input data of the graph neural network model based on the node identifications of the nodes and the node identifications of the neighboring nodes for training the graph neural network model.
In some embodiments, the neighboring node request sending module 804 includes a hash operation module configured to perform a hash operation on the node identifier to determine a graph management device where the node is located; and an acquisition neighbor node module configured to send a request for acquiring a neighbor node to the determined graph management device.
In some embodiments, the first input data determining module 806 comprises a node sequence generating module configured to generate a node sequence comprising at least a node and a neighboring node based on a node identification of the node and a node identification of the neighboring node; and a second input data determination module configured to determine input data of the graph neural network model based on the sequence of nodes for training the graph neural network model.
In some embodiments, wherein the second input data determination module comprises a positive sample data determination module configured to determine positive sample data based on two nodes in the sequence of nodes if it is determined that the difference in position between the two nodes is not greater than a threshold size; a negative sample data determination module configured to determine negative sample data based on two nodes in the sequence of nodes if it is determined that a difference in position between the two nodes is greater than a threshold magnitude; and an input data generation module configured to generate input data based on the positive and negative sample data.
In some embodiments, the apparatus 800 further comprises a graph management device determination module configured to determine a graph management device storing the neighboring node based on the identification of the neighboring node; a next-neighbor-node-request transmitting module configured to transmit a request for acquiring a next neighbor node of the neighbor nodes to the determined graph management apparatus to acquire the next neighbor node; and an adding module configured to add the next neighbor node to the sequence of nodes.
In some embodiments, the apparatus 800 further comprises a parameter request sending module configured to send a request to the graph management device for obtaining parameters of the graph neural network model to receive the parameters from the graph management device; a node representation acquisition module configured to apply the parameters and the input data to the graph neural network model to acquire a node representation for the node.
In some embodiments, the apparatus 800 further comprises an adjustment value determination module configured to determine an adjustment value for a parameter of the graph neural network model based on the node representation of the node; and an adjustment module configured to send the adjustment values to the graph management device to adjust parameters of the graph neural network model.
The present disclosure also provides an electronic device, a readable storage medium, and a computer program product according to embodiments of the present disclosure.
FIG. 9 illustrates a schematic block diagram of an example electronic device 900 that can be used to implement embodiments of the present disclosure. The example electronic device 900 may be used to implement the example graph management device 102 and the graph neural network model computing device 104 of FIG. 1. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 9, the apparatus 900 includes a computing unit 901, which can perform various appropriate actions and processes in accordance with a computer program stored in a Read Only Memory (ROM) 902 or a computer program loaded from a storage unit 908 into a Random Access Memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the device 900 can also be stored. The calculation unit 901, ROM 902, and RAM 903 are connected to each other via a bus 904. An input/output (I/O) interface 905 is also connected to bus 904.
A number of components in the device 900 are connected to the I/O interface 905, including: an input unit 906 such as a keyboard, a mouse, and the like; an output unit 907 such as various types of displays, speakers, and the like; a storage unit 908 such as a magnetic disk, optical disk, or the like; and a communication unit 909 such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 909 allows the device 900 to exchange information/data with other devices through a computer network such as the internet and/or various telecommunication networks.
The computing unit 901 may be a variety of general and/or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and so forth. The computing unit 901 performs the various methods and processes described above, such as the methods 200 and 300. For example, in some embodiments, methods 200 and 300 may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 908. In some embodiments, part or all of the computer program may be loaded and/or installed onto device 900 via ROM 902 and/or communications unit 909. When the computer program is loaded into RAM 903 and executed by computing unit 901, one or more steps of methods 200 and 300 described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to perform the methods 200 and 300 in any other suitable manner (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the Internet.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, and the present disclosure is not limited herein.
The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the scope of protection of the present disclosure.

Claims (34)

1. A graph neural network model training method based on a graph retrieval engine comprises the following steps:
at a graph management device, in response to receiving a request from a graph neural network model computing device of a plurality of graph neural network model computing devices to obtain a node in a graph, sending a stored node identification for the node in at least a portion of the graph to the graph neural network model computing device;
in response to receiving a request from the graph neural network model computing device to obtain neighboring nodes of the node, retrieving at least one neighboring node from the neighboring nodes of the node that satisfies a predetermined condition according to the stored graph; and
sending the retrieved node identification of the neighboring node to the graph neural network model computing device for training the graph neural network model;
in response to receiving a request to obtain the graph neural network model parameters, sending the graph neural network model parameters to the graph neural network model computing device, the graph management device storing the graph neural network model parameters;
in response to receiving a plurality of adjustment values for the graph neural network model parameters from the plurality of graph neural network model computing devices, adjusting the graph neural network model parameters based on the plurality of adjustment values;
wherein retrieving the at least one neighboring node comprises:
obtaining at least a portion of the graph, the at least a portion including a plurality of slices, each slice including a sub-portion of the at least a portion, each slice corresponding to a task queue maintained by a task execution thread, tasks in the task queue being executed in order;
if it is determined that the identity of the node in the request to obtain the neighboring node is located in a set of fragments in the plurality of fragments, a retrieval operation for the neighboring node of the node is concurrently performed by a set of task execution threads for the set of fragments.
2. The method of claim 1, wherein sending the node identification of the node comprises one of:
sending node identifications of randomly selected nodes to the graph neural network model computing device; or
Sending node identifications of sequentially selected nodes from the ordered queue of nodes to the graph neural network model computing device.
3. The method of claim 1, further comprising: in response to receiving a processing request including a plurality of access operations from a graph neural network model computing device, the plurality of access operations are batch processed.
4. The method of claim 1, retrieving the at least one neighbor node comprises:
obtaining a tree structure for the node, the tree structure being constructed based on neighboring nodes of the node, leaf nodes of the tree structure corresponding to neighboring nodes of the node and having corresponding weight values, the weight value of a parent node in the tree structure being determined by the weight values of its children nodes, the weight value of the leaf node representing the magnitude of the degree of association of the node with the corresponding neighboring node;
generating a random number for each level of the tree structure of the node; and
selecting a neighboring node to transmit based on the random number.
5. The method of claim 4, wherein the tree structure is a balanced binary tree.
6. The method of claim 4, wherein the tree structure can dynamically perform addition, deletion, and adjustment of node weight values for nodes.
7. The method of claim 1, wherein the graph neural network model computing device and the graph management device that run the method interact with the same physical device with which the graph neural network model computing device and the graph management device are located.
8. The method of claim 1, wherein the communication of the graph neural network model computing device with a graph management device running the method is achieved through a proprietary communication protocol, and the information of the nodes transmitted is achieved through a template.
9. A graph neural network model training method based on a graph retrieval engine comprises the following steps:
at a graph neural network model computing device of a plurality of graph neural network model computing devices, sending a request to a graph management device to obtain a node in a graph to receive a node identification for the node from the graph management device, the graph management device storing at least a portion of the graph including the node;
sending a request for acquiring a neighboring node of the node to the graph management device to receive a node identifier of the neighboring node from the graph management device, the neighboring node being retrieved by the graph management device according to the stored graph;
determining input data for the graph neural network model based on the node identifications of the nodes and the node identifications of the neighboring nodes for training the graph neural network model;
sending a request to the graph management device for obtaining graph neural network model parameters to receive the graph neural network model parameters from the graph management device, the graph management device storing the graph neural network model parameters;
applying the graph neural network model parameters and the input data to the graph neural network model to obtain node representations for the nodes;
determining adjustment values for the graph neural network model parameters based on the node representations of the nodes; and
sending the adjustment values to the graph management device to adjust the graph neural network model parameters, the graph management device adjusting the parameters of the graph neural network model based on a plurality of adjustment values for the graph neural network model parameters received from the plurality of graph neural network model computing devices;
wherein sending the request for acquiring the neighboring node comprises:
performing hash operation on the node identification to determine the graph management equipment where the node is located; and
sending a request for acquiring the neighboring node to the determined graph management device.
10. The method of claim 9, wherein determining the input data comprises:
generating a node sequence including at least the node and the neighboring node based on the node identification of the node and the node identification of the neighboring node;
input data for the graph neural network model is determined based on the sequence of nodes.
11. The method of claim 10, determining input data for the graph neural network model based on the sequence of nodes comprises:
determining positive sample data based on two nodes in the sequence of nodes if it is determined that the difference in position between the two nodes is not greater than a threshold magnitude;
if the difference value of the positions between two nodes in the node sequence is larger than the threshold value, determining negative sample data based on the two nodes; and
generating the input data based on the positive sample data and the negative sample data.
12. The method of claim 10, further comprising:
determining a graph management device storing the neighboring node based on the identifier of the neighboring node;
sending a request for acquiring a next neighbor node of the neighbor nodes to the determined graph management device to acquire the next neighbor node; and
adding the next neighbor node to the sequence of nodes.
13. The method of claim 9, further comprising:
collecting a plurality of access operations for a plurality of nodes of a graph in the same graph management device;
sending a processing request including the plurality of access operations to the graph management device to batch process the plurality of access operations.
14. The method of claim 9, wherein the graph neural network model computing device and the graph management device that run the method interact with the same physical device with which the graph neural network model computing device and the graph management device are located.
15. The method of claim 9, wherein the communication of the graph neural network model computing device running the method with the graph management device is achieved through a proprietary communication protocol and the information of the transmitted nodes is achieved through a template.
16. An apparatus for training a neural network model based on a graph search engine, comprising:
a node identification sending module configured to send, at a graph management device, in response to receiving a request from a graph neural network model computing device of a plurality of graph neural network model computing devices to obtain a node in a graph, a stored node identification for the node in at least a portion of the graph to the graph neural network model computing device;
a neighbor acquisition module configured to, in response to receiving a request from the graph neural network model computing device to acquire neighbors of the nodes, retrieve at least one neighbor satisfying a predetermined condition from the neighbors of the nodes according to the stored graph; and
a sending module configured to send the retrieved node identifications of the neighboring nodes to the graph neural network model computing device for training the graph neural network model;
a parameter sending module configured to send the graph neural network model parameters to the graph neural network model computing device in response to receiving a request to obtain the graph neural network model parameters, the graph management device storing the graph neural network model parameters; and
an adjustment module configured to adjust the graph neural network model parameters based on a plurality of adjustment values in response to receiving the plurality of adjustment values for the graph neural network model parameters from the plurality of graph neural network model computing devices;
wherein the neighboring node acquisition module comprises:
a graph acquisition module configured to acquire at least a portion of the graph, the at least a portion including a plurality of tiles, each tile including a sub-portion of the at least a portion, each tile corresponding to a task queue maintained by a task execution thread, tasks in the task queue being executed in order;
a concurrent execution module configured to concurrently perform, by a set of task execution threads for a set of the shards, a retrieval operation of a neighboring node of a node if it is determined that the identity of the node in the request to obtain the neighboring node is located in the set of shards.
17. The apparatus of claim 16, wherein the node identification sending module comprises one of:
a random transmission module configured to transmit node identifications of randomly selected nodes to the graph neural network model computing device; or
An in-order sending module configured to send node identifications of nodes selected in-order from the ordered queue of nodes to the graph neural network model computing device.
18. The apparatus of claim 16, further comprising: a batch processing module configured to batch process a plurality of access operations in response to receiving a processing request including the plurality of access operations from a graph neural network model computing device.
19. The apparatus of claim 16, the neighboring node acquisition module comprising:
a tree structure obtaining module configured to obtain a tree structure for the node, the tree structure being constructed based on neighboring nodes of the node, leaf nodes of the tree structure corresponding to neighboring nodes of the node and having corresponding weight values, a weight value of a parent node in the tree structure being determined by a child node weight value thereof, the weight value of the leaf node representing a magnitude of a degree of association of the node with the corresponding neighboring node;
a random number generation module configured to generate a random number for each layer of the tree structure; and
a neighbor selection module configured to select a neighbor to transmit based on the random number.
20. The apparatus of claim 19, wherein the tree structure is a balanced binary tree.
21. The apparatus of claim 19, wherein the tree structure can dynamically perform addition, deletion, and adjustment of node weight values.
22. The apparatus of claim 16, wherein the graph neural network model computing device and the graph management device interact with the same physical device if the graph neural network model computing device and the graph management device are located on the same physical device.
23. The apparatus of claim 16, wherein the communication of the graph neural network model computing device with the graph management device is achieved through a proprietary communication protocol and the information of the nodes transmitted is achieved through a template.
24. A graph neural network model training device based on a graph retrieval engine comprises:
a node request sending module configured to send, at a graph neural network model computing device of a plurality of graph neural network model computing devices, a request to a graph management device to obtain a node in a graph to receive a node identification of the node from the graph management device, the graph management device storing at least a portion of the graph including the node;
a neighboring node request transmission module configured to transmit a request for acquiring a neighboring node of the node to the graph management device to receive a node identifier of the neighboring node from the graph management device, the neighboring node being retrieved by the graph management device according to the stored graph; and
a first input data determination module configured to determine input data of the graph neural network model for training the graph neural network model based on node identifications of the nodes and node identifications of the neighboring nodes;
a parameter request sending module configured to send a request for obtaining graph neural network model parameters to the graph management device to receive the graph neural network model parameters from the graph management device, the graph management device storing the graph neural network model parameters;
a node representation acquisition module configured to apply the graph neural network model parameters and the input data to the graph neural network model to acquire node representations for the nodes;
an adjustment value determination module configured to determine an adjustment value for the graph neural network model parameter based on a node representation of the node; and
an adjustment module configured to send the adjustment values to the graph management device to adjust the graph neural network model parameters, the graph management device adjusting the parameters of the graph neural network model based on a plurality of adjustment values for the graph neural network model parameters received from the plurality of graph neural network model computing devices;
wherein the neighboring node request sending module comprises:
the hash operation module is configured to perform hash operation on the node identification to determine the graph management device where the node is located; and
an acquire neighboring node module configured to send a request for acquiring the neighboring node to the determined graph management device.
25. The apparatus of claim 24, wherein the first input data determination module comprises:
a node sequence generation module configured to generate a node sequence including at least the node and the neighboring node based on the node identification of the node and the node identification of the neighboring node; and
a second input data determination module configured to determine input data of the graph neural network model based on the sequence of nodes for training the graph neural network model.
26. The apparatus of claim 25, wherein the second input data determination module comprises:
a positive sample data determination module configured to determine positive sample data based on two nodes in the sequence of nodes if it is determined that a difference in position between the two nodes is not greater than a threshold size;
a negative sample data determination module configured to determine negative sample data based on two nodes in the sequence of nodes if it is determined that a difference in position between the two nodes is greater than a threshold magnitude; and
an input data generation module configured to generate the input data based on the positive sample data and the negative sample data.
27. The apparatus of claim 25, further comprising:
a graph management device determination module configured to determine a graph management device storing the neighboring node based on the identity of the neighboring node;
a next neighboring node request transmission module configured to transmit a request for acquiring a next neighboring node of the neighboring nodes to the determined graph management device to acquire the next neighboring node; and
an adding module configured to add the next neighbor node to the sequence of nodes.
28. The apparatus of claim 24, further comprising:
a collection module configured to collect a plurality of access operations for a plurality of nodes of a graph in the same graph management apparatus;
a batch processing request sending module configured to send a processing request including the plurality of access operations to the graph management device to perform batch processing on the plurality of access operations.
29. The apparatus of claim 24, wherein the graph neural network model computing device and the graph management device interact with the same physical device if the graph neural network model computing device and the graph management device are located on the same physical device.
30. The apparatus of claim 24, wherein the communication of the graph neural network model computing device with the graph management device is achieved through a proprietary communication protocol and the information of the nodes transmitted is achieved through a template.
31. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8.
32. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein, the first and the second end of the pipe are connected with each other,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 9-15.
33. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-8.
34. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 9-15.
CN202110550314.9A 2021-05-20 2021-05-20 Graph neural network model training method and device based on graph retrieval engine Active CN113240089B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110550314.9A CN113240089B (en) 2021-05-20 2021-05-20 Graph neural network model training method and device based on graph retrieval engine

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110550314.9A CN113240089B (en) 2021-05-20 2021-05-20 Graph neural network model training method and device based on graph retrieval engine

Publications (2)

Publication Number Publication Date
CN113240089A CN113240089A (en) 2021-08-10
CN113240089B true CN113240089B (en) 2022-11-25

Family

ID=77137861

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110550314.9A Active CN113240089B (en) 2021-05-20 2021-05-20 Graph neural network model training method and device based on graph retrieval engine

Country Status (1)

Country Link
CN (1) CN113240089B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114004358B (en) * 2021-12-29 2022-06-14 粤港澳大湾区数字经济研究院(福田) Deep learning model training method

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11494637B2 (en) * 2019-03-28 2022-11-08 International Business Machines Corporation Layer-wise distillation for protecting pre-trained neural network models
CN112631549A (en) * 2019-10-08 2021-04-09 橙载(上海)信息技术有限公司 Cross-platform reconstruction method for pseudo-random number generator in FTS random algorithm
CN110705709B (en) * 2019-10-14 2021-03-23 支付宝(杭州)信息技术有限公司 Method and device for training neural network model of graph
CN110888880A (en) * 2019-11-19 2020-03-17 武汉光谷信息技术股份有限公司 Proximity analysis method, device, equipment and medium based on spatial index
CN111461004B (en) * 2020-03-31 2023-08-22 北京邮电大学 Event detection method and device based on graph attention neural network and electronic equipment
CN112085172B (en) * 2020-09-16 2022-09-16 支付宝(杭州)信息技术有限公司 Method and device for training graph neural network

Also Published As

Publication number Publication date
CN113240089A (en) 2021-08-10

Similar Documents

Publication Publication Date Title
CN109564568B (en) Apparatus, method and machine-readable storage medium for distributed dataset indexing
CN103631657B (en) A kind of method for scheduling task based on MapReduce
US20210374542A1 (en) Method and apparatus for updating parameter of multi-task model, and storage medium
CN108875955B (en) Gradient lifting decision tree implementation method based on parameter server and related equipment
EP3688681A1 (en) Gradient-based auto-tuning for machine learning and deep learning models
CN102915347A (en) Distributed data stream clustering method and system
US20220374776A1 (en) Method and system for federated learning, electronic device, and computer readable medium
KR20210124109A (en) Methods and apparatuses for information processing, and information recommendation, electronic device, storage medium and computer program product
CN111966495B (en) Data processing method and device
CN114329201A (en) Deep learning model training method, content recommendation method and device
US20200118033A1 (en) Method for approximate k-nearest-neighbor search on parallel hardware accelerators
EP3449428A1 (en) Machine learning aggregation
CN110322931A (en) A kind of base recognition methods, device, equipment and storage medium
WO2022047335A1 (en) Systems and methods for artificial intelligence-based data system optimization
CN113240089B (en) Graph neural network model training method and device based on graph retrieval engine
CN112784102A (en) Video retrieval method and device and electronic equipment
Hai et al. A performance comparison of big data processing platform based on parallel clustering algorithms
CN113868434A (en) Data processing method, device and storage medium for graph database
CN115472257A (en) Method and device for recruiting users, electronic equipment and storage medium
CN116362955A (en) Graph data storage, access and processing methods, training methods, equipment and media
CN114217933A (en) Multi-task scheduling method, device, equipment and storage medium
CN113961720A (en) Method for predicting entity relationship and method and device for training relationship prediction model
Kamdar et al. A survey: classification of huge cloud datasets with efficient map-reduce policy
Zhao et al. Parallel K-Medoids Improved Algorithm Based on MapReduce
CN114066278B (en) Method, apparatus, medium, and program product for evaluating article recall

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant