JP7527535B1 - Method, system and storage medium for predicting evolution of multi-value chains - Google Patents

Abstract

The present invention provides a method, system and storage medium for predicting the evolution of multi-value chains, which relates to the field of artificial intelligence, and under the premise that each entity (e.g. each client) in the value chain does not share local original data, based on the client's local original data, the server interacts with each client participating in the distributed collaboration time series graph neural network model training, trains a distributed collaboration time series graph neural network model, and the client dynamically predicts the state of each entity and the relationship between each entity based on the model.

Description

The present invention belongs to the field of artificial intelligence, specifically, a method for predicting the evolution of multi-value chains;
This invention relates to a system and a storage medium.

Due to the significant improvement in the performance of machine hardware, deep learning methods based on neural networks have achieved great success in fields such as intelligent recommendation and network performance prediction. However, existing prediction methods rarely take dynamic time delays into account, and mostly make predictions by considering the topology of the neural network, and are unable to output information that changes over time.
As various stakeholders in society become concerned with issues such as data compliance, trade secrets, information security, and business competition, companies and
Companies and individuals are reluctant to disclose their own original data, resulting in low data sharing in the upstream, midstream, and downstream, and underutilization of large amounts of data, making it difficult for companies, businesses, and individuals in multi-value chains to understand the relationships between other entities. Therefore, how to predict the relationships between each entity without obtaining the original data of each entity has become an urgent issue for companies involved in the value chain.

The present invention has been made in consideration of the problems with the conventional technology described above, and provides a method, system, and storage medium for predicting the evolution of a multi-value chain, which can dynamically predict the state of each entity in the value chain and the relationships between the entities, under the premise that each entity (e.g., each client) does not share local original data.
The present invention provides the following solutions:
In one embodiment, the present invention provides a method for predicting the evolution of a multi-value chain, including: for each client among clients participating in training a distributed collaborative time series graph neural network model, each of the clients generates a feature vector of a node where each of the clients is located based on the distributed time series graph data of each of the clients at a historical time point and a feature vector of a node where an adjacent client of each of the clients is located, the feature vector is used to characterize information of an edge between the node where each of the clients is located and a node where an adjacent client of each of the clients is located, the distributed time series graph data of each of the clients includes node information of the node where each of the clients is located and information of an edge between the node where each of the clients is located and a node where an adjacent client of each of the clients is located, each of the clients generates local network parameters of the distributed collaborative time series graph neural network model of each of the clients based on the feature vector of the node where each of the clients is located, and transmits the local network parameters to a server; and further, the server performs a process based on the local network parameters transmitted from each of the clients:
Global network parameters of the distributed collaborative time-series graph neural network model are obtained, and the global network parameters are broadcast to each of the clients. Finally, each of the clients updates network parameters of the distributed collaborative time-series graph neural network model of each of the clients using the global network parameters, inputs the distributed time-series graph data of each of the clients at the current time point into the updated distributed collaborative time-series graph neural network model, and outputs a predicted value of an edge between a node where each of the clients is located and a node where an adjacent client of each of the clients is located.
In the method for predicting the evolution of multi-value chains provided by the present invention, each client participating in the training of a distributed collaborative time series graph neural network model only obtains edge information between itself and the node where its adjacent clients are located (the node where the adjacent client of one client is located may also be called the adjacent node of the client), and the server only obtains information of the nodes where all clients are located. When the client does not share local original data (i.e., the distributed time series graph data generated locally by the client), each client interacts with the server;
A global distributed collaborative time series graph neural network model applicable to each client is trained, and further a prediction of edges between the node where each client in the multi-value chain is located and the node where its adjacent client is located is dynamically completed. Based on the edge prediction result, the state of each entity and the relationship between each entity are dynamically predicted.
In another embodiment, the local network parameters of the distributed collaborative time-series graph neural network include local weights, a local feature transformation matrix, and a local loss, and the global network parameters of the distributed collaborative time-series graph neural network include global weights, a global feature transformation matrix, and a global loss.
In another embodiment, the distributed federated time series graph neural network model includes a feature learning network and a label prediction network, where the feature learning network is used to output a feature vector of a node on which each of the clients is located based on the distributed time series graph data of each of the clients at a historical point in time and the feature vector of a node on which an adjacent client of each of the clients is located, the label prediction network is used to output a predicted value of an edge between the node on which each of the clients is located and a node on which an adjacent client of each of the clients is located based on the feature vector of the node on which each of the clients is located, the feature learning network is composed of a message passing layer, and the label prediction network is composed of a fully connected layer.
In another embodiment, the feature vector of the node in which each client i is located is
where
teeth
The client at the time
The client output from the l-th layer of the feature learning network corresponding to
denotes the feature vector of the node in which
is the client at time t
Neighboring clients of
The first feature learning network corresponding to
Denote the feature vector of the node in which the neighboring client j is located, output from the layer;
teeth
The client at the time
The first feature learning network corresponding to
The client output from the layer
denotes the feature vector of the node in which
indicates an aggregation operation,
denotes a linear operation,
denotes the local feature transformation matrix.
In another embodiment, the feature vector of the node in which each client i is located is
where
teeth
The client at the time
The client output from the l-th layer of the feature learning network corresponding to
denotes the feature vector of the node in which
teeth
The client at the time
Neighboring clients of
The first feature learning network corresponding to
The adjacent clients output from the layer
denotes the feature vector of the node in which
teeth
The client at the time
The first feature learning network corresponding to
The client output from the layer
denotes the feature vector of the node in which
indicates an aggregation operation,
denotes linear operation,
denotes the local feature transformation matrix,
The client
The node where the client is located and the
Neighboring clients of
indicates the correlation coefficient between the nodes where
In another embodiment, the generating of local network parameters of the distributed federated time-series graph neural network model of each of the clients based on a feature vector of a node where the client is located includes: each of the clients inputting the feature vector of the node where the client is located to a label prediction network corresponding to the client, and generating a predicted value of an edge between the node where the client is located and a node where an adjacent client of the client is located; each of the clients determining a local loss of the distributed federated time-series graph neural network model based on the predicted value of the edge between the node where the client is located and a node where an adjacent client of the client is located and a true value of the edge; and each of the clients performing back-propagation based on the local loss to generate local network parameters of the distributed federated time-series graph neural network model of each of the clients.
In another embodiment, the method for predicting the evolution of a multi-value chain provided by the present invention includes:
The method further includes each of the clients sending a local loss of the distributed federated time-series graph neural network model to the server, and then the server determining a global loss of the distributed federated time-series graph neural network model based on the local loss sent from each of the clients, whereby the server obtains a global network parameter of the distributed federated time-series graph neural network model based on the local network parameter sent from each of the clients, and the server performs back-propagation based on the local network parameters and the global loss sent from each of the clients to obtain a global network parameter of the distributed federated time-series graph neural network model.
In one embodiment, the present invention provides a client device, which interacts with a server to implement the operations performed by each client in any embodiment of the above multi-value chain evolution forecasting method.
In one embodiment, the present invention provides a server that interacts with each client among the plurality of clients and is used to realize the operations performed by the server in any embodiment of the above method for predicting the evolution of a multi-value chain.
In one embodiment, the present invention provides a system for predicting the evolution of a multi-value chain, comprising a server and a plurality of clients, the plurality of clients participating in training a distributed collaborative time series graph neural network model, the server interacting with each of the plurality of clients, and performing any one of the above-mentioned methods for predicting the evolution of a multi-value chain.
The distributed and linked time-series graph neural network model is trained by executing the above-mentioned process, and edges between a node where each client in a multi-value chain is located and a node where another client is located are predicted based on the distributed and linked time-series graph neural network model.
In one embodiment, the present invention provides a computer-readable storage medium having computer program instructions stored thereon, the computer program instructions, when executed by a multi-core processor, causing the multi-core processor to perform the above-described method for predicting the evolution of multi-value chains.

1 is a schematic diagram of a system architecture provided by the present invention; FIG. 1 is a schematic diagram of the architecture of a distributed, collaborative time-series graph neural network provided by the present invention.

To clearly describe the embodiments of the present application, we first introduce some concepts that may appear in subsequent embodiments.
Original data refers to actual data, including features and samples, obtained by each entity in a multi-value chain in a specific period of time, where the original data is collected according to rules and methods defined by those skilled in the art, and the original data records the original data of clients in the form of tensors or tables, and includes features and samples, for example, feature 1 is automobile sales volume, and the samples include January automobile sales volume data, February automobile sales volume data, March automobile sales volume data, ... corresponding to each of different clients; feature 2 is automobile inventory volume, and the samples include January automobile inventory volume data, February automobile inventory volume data, March automobile inventory volume data, ... corresponding to each of different clients; feature 3 is accessory sales volume, and the samples include January accessory sales volume data, February accessory sales volume data, March accessory sales volume data, ... corresponding to each of different clients; feature 4 is accessory inventory volume, ... and so on.
It will be clear to those skilled in the art that the features and samples referred to in the present invention are not limited to those mentioned above, and the features may be part or all of the features of the data actually collected or recorded by each client, and the samples may be part or all of the data actually collected or recorded by each client. In addition, the features, dimensions, types, etc. of the original data of different clients of the data that can be input may be different, and the features, dimensions, types, etc. of the original data between different groups may also be different, and the client may collect original data on a monthly basis, may collect original data on a weekly basis, or may collect or record original data in real time.
Those skilled in the art can arbitrarily set this according to actual needs.
Metadata is data that describes objects such as information resources and data. Its uses include identifying resources, evaluating resources, tracking changes in resource usage, achieving simple and efficient management of large amounts of network data, and effectively discovering information resources.
The objective of metadata is to enable searching, unifying organization, and effectively managing the use of resources. Therefore, metadata is publicly available data. Metadata includes information about the characteristics of the input data,
It includes size, dimensions, attributes, creation time, type, shape, client flags, variable names, dimensions, etc. Metadata is data describing other data or structured data used to provide information about a particular resource, and can be generated by the client or server side based on each client's input data or original data. It should be understood that the metadata of the present invention refers to data describing the client's original data, i.e., data for describing the attributes or characteristics of the original data.
Graph Neural Network (GNN)
is a neural network for processing graph data (graph data is data with graph structure characteristics), which can learn from graph data, extract and discover features and patterns in graph data, and meet the needs of graph learning tasks such as convergence, classification, prediction, partitioning, generation, etc. In the present invention, the clients interact with each other and generate some data with the characteristics of graph structure data, where each client in the clients corresponds to a node (also called a graph node);
The nodes corresponding to the clients form a graph.
According to the method for predicting the evolution of a multi-value chain proposed by the present invention, clients share only metadata without sharing original data, thereby protecting the data privacy and security of enterprises and enabling specific collaboration to be safely carried out between enterprises in the multi-value chain without sharing original data.
The present invention provides a distributed and coordinated time series graph neural network (model), and
—A "multi-client" distributed architecture is adopted, in which the server mainly manages parameters and adjusts clients to form a distributed linked time series graph neural network. By adopting the distributed linked time series graph neural network, the distributed linked time series graph neural network corresponding to each client is a component of the entire distributed linked time series graph neural network, and the parameters of the distributed linked time series graph neural network obtained by local training of each client are different.
Optionally, each client's local distributed collaborative time series graph neural network may be homogeneous or heterogeneous, i.e., the structure of each client's local distributed collaborative time series graph neural network may be the same or different.
In addition, the distributed linked time series graph neural network corresponds to a graph at each time point, the nodes in the diagram refer to clients or companies, and the clients correspond to companies, the edges between the nodes in the diagram indicate the relationships between clients or between companies, including cooperation relationships, trading relationships, supply and demand relationships, etc., and as time changes, the topological relationships in the graph also change, and the edges in the graph also change. The present invention builds a linked time series graph neural network that allows companies to predict future edges of the graph and predict whether they will go bankrupt without sharing their own original data.
The present invention will now be described in detail with reference to the accompanying drawings in conjunction with embodiments.
Referring initially to FIG. 1, a schematic diagram illustrates a system architecture 100 in which exemplary embodiments according to the present disclosure may be used.
Fig. 1 is a schematic diagram showing an example of a system architecture 100 according to an embodiment of the present disclosure. It should be noted that Fig. 1 is also referred to as an architecture schematic diagram of the hardware operating environment of the multi-value chain evolution forecasting method and device architecture. The embodiment of the present invention is based on a client device, which may be a terminal device such as a PC, a portable computer, etc.
The system architecture shown in Fig. 1 may be a "server-multi-client" distributed architecture, and only one client is illustrated in Fig. 1. As shown in Fig. 1, in the "server-multi-client" distributed architecture, the client is used to train a sub-model (i.e., a local distributed federated time-series graph neural network model) that characterizes the current nodes and connection edges locally on the client using the original data.
Preferably, each client comprises an original data module, a local parameter module, a data collection module, a data labeling module, a message passing module, a local training module and a blockchain, where the original data module is used to store the original data of the client, the data collection module is used to collect and record the original data of the client, the local parameter module is used to store all parameters of the local distributed federated time series graph neural network of the client, the data labeling module is used to label data to train the distributed federated time series graph neural network, the message passing module is used to support the client-server communication module multiple parallel communication and information upload (i.e. upload messages to the blockchain), the local training module is used by the client to train the distributed federated time series graph neural network, and the blockchain is used to record all communication between the server and the client, traceability of information leakage and non-repudiation. The server is used to manage the global parameters of the distributed federated time series graph neural network.
Preferably, the server comprises a metadata module, a parameter database module,
The system includes a parameter management module, a communication module, a global training module, and a blockchain, where the metadata module is used to manage and store metadata, the parameter database module is used to store all parameters of the distributed federated time series graph neural network, the parameter management module manages and updates the parameters of the distributed federated time series graph neural network, the communication module is used to support multiple parallel communications and information uploads between the server and the multi-client message passing module, the global training module is used to fuse data sent from multiple clients, and the blockchain is used to record all communications between the server and the clients, traceability of information leakage, and non-repudiation. In one embodiment of the present invention, each distributed federated time series graph neural network performs training locally on the client, and the server mainly performs parameter processing (i.e., processes the parameters of the distributed federated time series graph neural network obtained by training of each client), and the processing includes averaging (i.e.,
The server averages the parameters of the distributed and federated time series graph neural networks obtained from each client.) In another embodiment of the present invention, each distributed and federated time series graph neural network is trained locally on the client, and the obtained local network parameters of the distributed and federated time series graph neural network are sent to the server, and the server processes the received local network parameters using a neural network, the processing including parameter fitting and/or prediction by the neural network.
The hardware parts of the client and server are composed of a processor, e.g., a CPU, a network interface, a user interface, a memory, and a communication bus.
The communication bus is used to realize the connection communication between the different components. The user interface includes a display, an input unit, e.g., a keyboard,
The user interface may include a standard wired interface, a wireless interface, and optionally the network interface may include a standard wired interface, a wireless interface (e.g., a WI-FI interface, a Bluetooth interface, a 5G interface). The memory may be a high-speed RAM memory, or a non-volatile memory, such as a disk memory. The memory may optionally be a storage device independent of the processor.
It will be appreciated by those skilled in the art that the structure of the servers and clients shown in FIG. 1 is not intended to limit the servers or clients, and that the servers and clients may be comprised of more or fewer components than those shown, or a combination of the specified components, or a different arrangement of the components.
As shown in FIG. 2, the memory as a computer storage medium may include an operating system program, a network communication module program, a user interface module program, and a federated time series graph neural network program. For multiple clients in the multi-value chain, a node is set for each client, and the node and the client have a one-to-one correspondence. Here, the operating system is used to manage and control the programs of the hardware and software resources of the client device to support the operation of the distributed federated time series graph neural network program and other software or programs. The distributed federated time series graph neural network corresponds to the topological structure of the graph at each time point when it is assumed that the nodes are constant, and the edges of the graph change with the change in time. FIG.
In the server shown in FIG. 1, the communication module is mainly used for transmitting and receiving requests, data, etc. between the server and multiple clients such as client A, client B, client C, client D, and client E, and the message passing module in the multiple clients is mainly used for transmitting and receiving requests, data, etc. between each client and the server, where the number of clients may be more or less. Preferably, the communication module and the message passing module perform data communication via their respective network interfaces, and the processor can be used to call a distributed collaborative time series graph neural network program stored in the memory and execute the method for predicting the evolution of multi-value chains provided by the embodiment of the present invention.
The method for predicting the evolution of a multi-value chain provided by an embodiment of the present invention is as follows:
Includes 01 to S104.
S101, a client establishes a distributed collaborative time series graph neural network model.
The distributed federated time-series graph neural network model includes a feature learning network and a label prediction network. The feature learning network mainly includes a message passing layer, in which a client receives messages from its neighboring clients, and performs graph convolution, graph attention, graph transformation, etc. through the message passing layer.
The label prediction network consists of a fully connected layer and is used to predict edges between nodes.
The present invention provides a distributed and linked time series graph neural network architecture with a server-multi-client structure, and
and
clients
Each company corresponds to one client, and each client has one node.
,The nodes corresponding to the clients in the multi-value chain are connected to one mesh structure time series graph data.
In the multi-value chain, each client participating in the training of the distributed federation graph neural network establishes a distributed federation time series graph neural network. In the present invention, the time series graph data
Figure
A collection of time-series graph data at multiple points in time is called multi-time graph data, and a graph
exists in the entire network space, or logically, and each client
is a graph
One node in
Corresponds to.
In the present invention, a distributed and linked time series graph neural network is adopted, and the distributed and linked time series graph neural network established by each client is a component of the entire distributed and linked time series graph neural network, and the parameters of the distributed and linked time series graph neural network obtained by the local training of each client are different, and the complete distributed time series graph data is not known to any entity in the network, and any entity includes any client or server. Here, each client
is the client
Neighboring clients of
only allow communication with the server,
is a natural number, or
is a positive integer,
indicates the serial number of the client or node,
is 0～
Take the value between
denotes the serial number of the neighboring client or the node on which the neighboring client is located (also called the neighboring node),
is 0～
In one embodiment, the neighboring client or neighbor node in the present invention can take a value between:
It refers to a first-order neighboring client or a first-order neighboring node. In another embodiment, the neighboring node in the present invention includes a first-order neighboring node and/or a second-order neighboring node. In another embodiment, the neighboring node in the present invention includes a first-order neighboring node and/or a second-order neighboring node and/or other higher-order neighboring node. Any node
For the adjacency list on the graph
constitutes the first feature, i.e. the structural relationship of the graph.
In an alternative embodiment, the server sends a request to the client to establish a distributed federated time series graph neural network, the request message including a request for data sources required for training. Preferably, these data are metadata, e.g., descriptive information about value flows, demand flows, business flows, etc. The client
If the client is willing to participate, the server will feed back a message indicating that it will receive the request.
can actively send a request message to the server to participate in graph neural network training, and the server can
Choose whether or not you agree to participate in the
If the client agrees to participate in
Establish a distributed collaborative time series graph neural network model.
In the present invention, the local parameters (local network parameters) of the distributed linked time-series graph neural network model corresponding to the client include a local feature change matrix, a local weight and a local loss, and the global parameters (global network parameters) of the distributed linked time-series graph neural network model include at least a global weight, a global feature transformation matrix and a global loss.
About distributed and linked time series graph neural network model,
The global loss function of the model at time is
and the local loss function is
It is expressed as
The global loss function of the model at time is
and the local loss function is
That is,
The global loss of the distributed time series graph neural network model at time is
and the local loss is
and
The global loss of the model at time
and the local loss function is
It is.
As shown in FIG. 2, nodes A, B, C, D, and E correspond to clients A, B, C, D, and E, respectively.
The local network parameters at time are at least the local feature transformation matrix
, local weights
and local losses
In the present invention, the local loss
indicates the loss of the distributed, linked time-series graph neural network model corresponding to client A, and the loss is calculated by the loss function of the model.
The local network parameters at time are at least the local feature transformation matrix
, local weights
and local losses
, and correspondingly, each of the other nodes, such as node B, node C, node D, node E, etc., includes a corresponding local network parameter (not shown) at a corresponding time.
At this point, the global network parameters in the server are at least the global feature transformation matrix
, global weight
and Global Loss
Including,
At this point, the global feature transformation matrix in the server
, global weight
and Global Loss
where
indicates a point in time, time, or timing. Referring to Fig. 2, node B can communicate with only three nodes, namely, the server, node A, and node C, and node D can communicate with only four nodes, namely, the server, node A, node C, and node E. Similarly, the description of other nodes will be omitted in this specification and later.
S102, data for training the distributed linked time series graph neural network model is acquired.
The server is the client
When a client agrees to participate in training a distributed federated time-series graph neural network, the server
a consent token to the client,
is used to participate in training the graph neural network. In an alternative embodiment, the client
The neighbor list is a list of neighbors who are willing to participate in the modeling of neighbors with related data, i.e., to train a distributed federated time series graph neural network model of a multi-value chain.
where the neighbor list is determined by the client
Excluding itself,
is the client's serial number,
is a natural number, or
is a positive integer.
It should be understood that each client can obtain node information of the node where the client is located and edge information between the node where the client is located and the node where the adjacent client of the client is located, and thus the distributed time series graph data of the client is formed. That is, the distributed time series graph data of each client includes node information of the node where the client is located and edge information between the node where the client is located and the node where the adjacent client of the client is located.
Optionally, the node information of the node where the client is located in the distributed time series graph data may include node identification information or other information, for example, the node identification information may be a node serial number, name, etc. In different application scenarios, the specific content of the node information may be different. The edge information between the node where the client is located and the node where the client's adjacent client is located may include edge identification information or other information, for example, the edge identification information may be an edge name or number, etc. Similarly, in different application scenarios, the specific content of the edge information may be different.
Logically aggregate the distributed time series graph data of all clients to generate the entire distributed time series graph data
where:
indicates a point in time, time or timing,
denotes a set of nodes, and each node
indicates one client, i.e. one company,
denotes the set of edges at time t, and each edge
is a label vector
(the edge label vector is also called the edge label),
is a node
and
It indicates whether there is cooperation, demand flow, business flow, and value flow between them.
In addition, the above
is a logical distributed time series graph data, and the server is a complete distributed time series graph data
The server does not know about the node set.
However, the set of edges
, the edge label vector
It is not possible to obtain
Based on the above, in the present invention, the feature learning network in the distributed linked time series graph neural network model is used to output the feature vector of the node where each client is located based on the distributed time series graph data of each client at a historical point in time and the feature vector of the node where each client's adjacent clients are located.
The label prediction network in the distributed collaborative time-series graph neural network model is used to output predicted values of edges between the node in which each client is located and the nodes in which each client's adjacent clients are located, based on the feature vector of the node in which each client is located.
The functions of each client and server are as follows:
Each client
is a graph
The client in
The node corresponding to
Related calculations of,
and the associated calculations include:
1) Node
Calculate the feature vector of node
and its adjacent nodes
Weight of the connecting edge of
where:
is a node
Indicates the serial number of
is the adjacent node
Indicates the serial number of the
2) Send and/or receive feature vectors of nodes, e.g., a client
Client to client adjacent to
The client transmits the feature vector of the node where the client is located, or receives the feature vector of the node where the neighboring client is located from the neighboring client.
3) Perform backpropagation based on the local loss (backpropagation is prior art in the field).
client
is the node where the neighboring client is located.
You can get the client
The edge between the node where the client is located and the node where the client is located
, and the edge labels
Preferably, the node
and adjacent nodes
The edge of
Time point weighting
is the edge label
It is.
The function of the server is to coordinate the entire training process of the distributed federated time-series graph neural network model, including collecting the losses of each client, calculating the global loss (i.e., receiving the local losses of the distributed federated time-series graph neural network model from each client), updating the global parameters using the Adam algorithm, and initiating backpropagation on all client sides. As shown in FIG. 2, the server may be a third-party server, the server may be one or more, and the related functions may be implemented by a distributed server cluster.
In one embodiment, the neural network may reside on a server.
In one embodiment, the neural network on the server is different from the neural network on the client, and preferably the neural network on the server is a Recurrent Neural Network (RNN), a Long Short Term Memory (LSTM), or a similar network.
The network may be implemented by any one or more combinations of neural networks such as a long short-term memory network (LOSM), a type of special recurrent neural network, a graph neural network (GNN), a graph attention network (GAT), or a graph transformer. In some embodiments, when the server generates the current time parameters (global network parameters), it can take into account the parameter data in the server at the historical time, and broadcast the generated current time parameters to all clients or designated clients.
In another embodiment, the neural network on the server is similar to the neural network on the client and is part of a distributed neural network, with the server corresponding to one node of the distributed neural network.
In another embodiment, there is no neural network on the server, but simply processes the data uploaded by all clients at each point in time. For privacy purposes, the third-party server does not
Without knowing the global topology structure information of the graph, i.e.,
However, we can know the edges of the graph.
Since the nodes cannot know the identity of the node, the privacy of each node is protected.
S103, each client interacts with the server to train a distributed collaborative time series graph neural network model.
(1) The client updates the feature vector of the node based on the feature learning network. The process of updating the feature vector of the node by the client is as follows: For each client participating in the distributed federated time series graph neural network model training, each client updates the feature vector of the node based on the feature learning network.
generating a feature vector of a node where each client is located based on the distributed time series graph data of each client at a time instant (a point in time) and a feature vector of a node where an adjacent client of each client is located, where the feature vector of the node is used to characterize information of an edge between the node where each client is located and a node where an adjacent client of each client is located.
First, based on the feature learning network, node feature learning is performed using graph snapshots at each time point, where the graph snapshots at each time point refer to distributed time series graph data at a certain time point, and then the multi-time graph data is used to learn the temporal evolution characteristics of the node features, and the feature vectors of the nodes obtained by learning are used to predict the edges of the graph. Note that the feature learning network includes multiple layers (or feature learning layers), for example:
layers, where the first
The layer is used to generate a feature vector of the node where the client is located based on the client's local distributed time series graph data (i.e., the client's local original data), and the remaining layers are ...
It is used to update the feature vector obtained from the layer.
Specifically, the serial numbers of multiple clients participating in the distributed collaborative time series graph neural network model training
Client (Client i or Client
The value of
For example, the client
is an arbitrary client among multiple clients.
The client further
Neighboring clients of
From adjacent clients
Receive the feature vector generated by, e.g.,
via the Internet
Send a request to
Obtain the feature vector of the node where
is all neighboring clients
The feature vector of the node in which is located is received.
For example, during the process of training a distributed time series graph neural network model,
In the repetition of the round,
The first distributed and connected time series graph neural network at
About the layer, client
is a client
A kind of update rule for updating the feature vector of the node where is located is (i.e., the client
The feature vector of the node where is located is:
where
teeth
Client at the time
The first feature learning network corresponding to
Clients output from layers
denotes the feature vector of the node in which
teeth
Client at the time
Neighboring clients of
The first feature learning network corresponding to
Neighboring clients output from the layer
indicates the feature vector of the node where
is a client
is a client
received from
teeth
Client at the time
The first feature learning network corresponding to
Clients output from layers
denotes the feature vector of the node in which
indicates an aggregation operation,
denotes linear operation,
denotes the local feature transformation matrix.
Selectable,
Aggregation methods include any one or combination of summing, averaging, maximizing, etc.
Client i:
Global feature transformation matrix for time points
Send a request to the server to get
Global feature transformation matrix for time points
, i.e., we calculate the timing effect of multi-temporal graph neural network training with weight matrices.
Using
Linearly transform the client
The node where
The final feature vector in the layer
Get the.
By repeating the above steps, the feature vectors of the nodes in each layer in the feature learning network are calculated.
The first distributed and connected time series graph neural network at
About the layer, client
is a client
Another update rule for updating the feature vector of the node where is located is (i.e., the client
The feature vector of the node where is located is:
where
teeth
Client at the time
The first feature learning network corresponding to
Clients output from layers
denotes the feature vector of the node in which
teeth
Client at the time
Neighboring clients of
The first feature learning network corresponding to
The output from the layer is the feature vector of the node where client j is located,
teeth
Client at the time
The first feature learning network corresponding to
Clients output from layers
denotes the feature vector of the node in which
indicates an aggregation operation,
denotes linear operation,
denotes the local feature transformation matrix,
is a client
The node and client where
Neighboring clients of
indicates the correlation coefficient between the nodes where
In addition, during the training process of the distributed and linked time series graph neural network, message passing and back propagation are performed via the Internet, which requires a large amount of communication and takes a long time. Nodes in a graph have degrees, and the degree of a node indicates the number of edges connected to a node. In some cases, there is a large difference between the degrees of nodes in a graph, for example, the degrees of nodes in a scale-free network have an exponential distribution, with a small number of nodes having very large degrees and most of the nodes having very small degrees. The number of messages sent/received by a node is proportional to the degree of the node, and a node with a large degree becomes the performance bottleneck of the entire distributed and linked time series graph neural network model. To address this issue, a client may use the feature vectors of the nodes where neighboring clients are located to aggregate the feature vectors of the nodes.
is each adjacent node
Correlation coefficient of
The method learns the correlation coefficients and combines them to aggregate feature vectors.
In one embodiment, the final global loss function
By regularizing , the correlation coefficients of most neighboring nodes tend to be zero, while the absolute values of the correlation coefficients of a few neighboring nodes tend to be much larger than zero.
Preferably, the regularization method is
This is regularization.
or
This is regularization.
Through regularization, the correlation coefficients of most adjacent nodes are zero, and the absolute values of the correlation coefficients of a small number of adjacent nodes are much larger than zero. Thus, when aggregating the feature vectors of adjacent nodes, the client can select adjacent nodes, and the client does not aggregate the feature vectors of all adjacent nodes, but rather aggregates the feature vectors of some of the most relevant adjacent nodes, i.e., the adjacent node set
Most relevant
It is possible to update neighbors, retain a small number of important neighbors, and remove unimportant neighbors, thereby reducing the number of communication messages generated by forward propagation and backward propagation during the training process of the distributed linked time series graph neural network, and greatly improving communication efficiency.
Optionally, in this embodiment, each client among the multiple clients can select an adjacent node, or some clients among the multiple clients can select an adjacent node, which is not limited in this embodiment of the present invention.
In one embodiment, the server records the response time of each client until the forward propagation is complete, sorts all clients in descending order according to their response time, and selects the client with the longest response time.
Select nodes for regularization, with the longest response time.
This client may be notified to reduce the number of neighbors.
(2) The client generates local network parameters of a distributed, collaborative time-series graph neural network model.
Each client generates local network parameters of its distributed, linked time-series graph neural network model based on the feature vector of the node where the client is located, and transmits the local network parameters to the server.
A specific process of each client generating local network parameters of the distributed-linked time-series graph neural network model for each client based on the feature vector of the node where each client is located includes: each client inputs the feature vector of the node where each client is located into a label prediction network corresponding to each client, generates a predicted value of the edge between the node where each client is located and the node where each client's adjacent client is located; each client determines a local loss of the distributed-linked time-series graph neural network model based on the predicted value of the edge between the node where each client is located and the node where each client's adjacent client is located and the true value of the edge; and each client performs back propagation based on the local loss to generate local network parameters of the distributed-linked time-series graph neural network model for each client.
The above clients
For example, the client
The label prediction network uses a fully connected layer to predict the client
The node and client where
Clients adjacent to
The edge between the nodes where
Calculate the predicted value of the label of (i.e. edge
(Predict the label vector of the client output from the feature learning network.)
The feature vector of the node where is located is input to the label prediction network, and the client
The node and client where
Neighboring clients of
The client outputs the predicted edge of the node where
is the global weight of the edge prediction fully connected layer from the server.
and obtain the global weight
Based on the edge
Calculate the predicted value of the label of edge
The predicted values of the labels are expressed as a label vector
Let,
is determined by the following formula:
here,
The client
The node and client where
is a distance function indicating the distance between the nodes where are located. Optionally, the distance function may adopt the cosine distance, the Euclidean distance or the Mahalanobis distance.
In addition, the client i determines a local loss (also referred to as a local edge loss) of the distributed federated time-series graph neural network model corresponding to the client i according to the following local loss function (also referred to as a local edge loss function), where the local loss function is:
where:
denotes the local loss,
Edge
indicates the true value of the label,
Edge
The predicted value of the label is shown.
Finally, the client
are the local network parameters and the local loss
to the server.
(3) The server updates the global network parameters. The server obtains the global network parameters of the distributed federated time-series graph neural network model based on the local network parameters sent by each client, and broadcasts the global network parameters to each client.
The server
Global weight matrix for time points
Global weight matrix for time points
is used to calculate the RNN, GRU, LSTM or Transf
Global weight matrix using ORMER
Optionally, the server may be configured to calculate the
to all clients, reducing the latency for clients to receive the global network parameters.
First, the server determines a global loss of the distributed federated time-series graph neural network model based on the local losses sent by each client, and the global loss is the sum of the local losses of each client and a regularization term, i.e.,
here,
is the sum of the local losses of each client,
is the regularization term.
Then, the server performs backpropagation based on the local network parameters and the global loss sent by each client to obtain the global network parameters of the distributed federated time-series graph neural network model.
The dam algorithm is used to optimize the parameters of the distributed, linked time series graph neural network model, and the server updates the global network parameters:
,
and
denotes the number of global network parameters.
In one embodiment, the server broadcasts global network parameters to the clients to initiate backpropagation for all clients.
In another embodiment, the server broadcasts the global network parameters to the clients, and the clients then decide whether to start backpropagation. The clients update the local network parameters and the correlation coefficient matrix
Responsible for updating.
In an embodiment of the present invention, each client updates the network parameters of its distributed, collaborative time-series graph neural network model using the global network parameters broadcast by the server.
Optionally, in an embodiment of the present invention, a blockchain may be established, and communication traffic such as feature vectors, parameters, and gradients of nodes between client-client and server-client may be deposited in the blockchain to prevent repudiation and support auditing of privacy violations of the server and client.
S104, the client predicts the labels of the edges.
In an embodiment of the present invention, after completing the training of the distributed-linked time-series graph neural network model by the above S101 to S103, each client inputs the distributed time-series graph data of each client at the current time point into the updated distributed-linked time-series graph neural network model, and outputs a predicted value of the edge between the node where each client is located and the node where each client's adjacent client is located.
In one embodiment,
Edge at a time
To predict the label of
Time series graph data distribution
The distributed and linked time series graph neural network model of S101 to S103 is trained using the above, and the distributed and linked time series graph neural network model is
Distributed time series graph data, i.e.
Inform the client that they will enter the following information.
is the edge to be predicted from the distributed time series graph data of the client.
Finally, the edge
It represents outputting the predicted value of the label.
Similarly,
Edge of Time
To predict the label of
Dispersion graph data of time
We train a distributed,connected time series graph neural network model using
Distributed time series graph data, i.e.
The client is notified that it will enter the
Output the predicted value of the label.
In summary, in the method for predicting the evolution of a multi-value chain provided by the present invention, each client participating in the training of the distributed federated time series graph neural network model can only obtain edge information between the nodes where the client and its adjacent clients are located, and the server can only obtain information of the nodes where all clients are located, and if the client does not share local original data (i.e., the distributed time series graph data generated locally by the client, the distributed time series graph data includes node information and edge information), each client interacts with the server to train a global distributed federated time series graph neural network model applicable to each client, and further dynamically predicts the edges between the nodes where each client is located in the multi-value chain and the nodes where its adjacent clients are located.
In an embodiment of the present invention, after obtaining prediction results of edges between nodes where each client in a multi-value chain is located based on a distributed collaborative time series graph neural network model, the state of each entity and the relationship between each entity can be dynamically predicted based on the edge prediction results.
In one embodiment, the method predicts future value flows between companies based on the edge prediction results.
Network traffic such as demand flow, business flow, and technology flow can be dynamically predicted, for example, product sales volume. In a multi-value chain digital ecosystem, when predicting the sales of an enterprise, accurate prediction results cannot be obtained only with the enterprise's internal data, and it is necessary to comprehensively utilize the data of related enterprises upstream and downstream of the enterprise's value chain. For example, clients such as retailers, distributors, and manufacturing plants input local data (data related to sales volume) into a distributed federation time series graph neural network model, predict the feature vector of the node through the feature learning network, and obtain the predicted value of product sales volume through the label prediction network (i.e., the predicted value of the edge of the model prediction is the predicted value of product sales volume). As can be seen, a distributed federation time series graph neural network model is established based on the local privacy data of each client to predict product sales volume, and the data of related enterprises upstream and downstream of the target enterprise's value chain is comprehensively utilized for distributed training to achieve more accurate sales volume prediction.
In another embodiment, the method can dynamically predict the relationship between each entity and the state of each entity based on the edge prediction result, that is, predict the evolution state of the enterprise node on the multi-value chain. For example,
About the node
If the predicted value of an edge connected to
or
If it is predicted that the trend will change continuously towards
This means that there is no cooperative relationship between any node and the other end of the edge. For example,
About the node
The predicted values of all edges connected to
or
is predicted to have a continuous tendency to change toward
The evolutionary state of is "extinct", i.e., the corresponding client
The evolutionary state of is "extinction".

Claims (9)

For each of the clients participating in the training of the distributed federated time series graph neural network model, the client generates a feature vector of the node where the client is located based on the distributed time series graph data of the client at the historical time point and the feature vector of the node where the adjacent client of the client is located, the feature vector is used to characterize information of the edge between the node where the client is located and the node where the adjacent client of the client is located, the distributed time series graph data of the client includes node information of the node where the client is located and information of the edge between the node where the client is located and the node where the adjacent client of the client is located;
Each of the clients generates local network parameters of the distributed collaborative time-series graph neural network model of each of the clients based on the feature vector of the node where the each of the clients is located, and transmits the local network parameters to a server;
The server obtains global network parameters of the distributed federated time-series graph neural network model based on the local network parameters transmitted from each client, and broadcasts the global network parameters to each client;
each of the clients updates a network parameter of the distributed collaborative time series graph neural network model of the each of the clients using the global network parameter, inputs the distributed time series graph data of the each of the clients at a current time point into the updated distributed collaborative time series graph neural network model, and outputs a predicted value of an edge between a node where the each of the clients is located and a node where an adjacent client of the each of the clients is located. The local network parameters include at least a local weight, a local feature transformation matrix, and a local loss;
The method of claim 1 , wherein the global network parameters include at least a global weight, a global feature transformation matrix, and a global loss. The distributed federated time series graph neural network model includes a feature learning network and a label prediction network,
The method according to claim 1 or 2, wherein the feature learning network is used to output a feature vector of a node in which each of the clients is located based on the distributed time series graph data of each of the clients at a historical point in time and the feature vector of a node in which an adjacent client of each of the clients is located, and the label prediction network is used to output a predicted value of an edge between the node in which each of the clients is located and a node in which an adjacent client of each of the clients is located based on the feature vector of the node in which each of the clients is located, the feature learning network being composed of a message passing layer, and the label prediction network being composed of a fully connected layer. Each of the clients is a client
If so, the client
The feature vector of the node where is located is
where
teeth
The client at the time
The first feature learning network corresponding to
Denote the feature vector of the node where the client i is located, output from the layer,
teeth
The client at the time
Neighboring clients of
The first feature learning network corresponding to
The adjacent clients output from the layer
denotes the feature vector of the node in which
teeth
The client at the time
The first feature learning network corresponding to
The client output from the layer
denotes the feature vector of the node in which
indicates an aggregation operation,
denotes linear operation,
4. The method of claim 3, wherein: denotes a local feature transformation matrix. Each of the above clients
The feature vector of the node where is located is
where:
teeth
The client at the time
The first feature learning network corresponding to
The client output from the layer
denotes the feature vector of the node in which
teeth
The client at the time
Neighboring clients of
The first feature learning network corresponding to
The adjacent clients output from the layer
denotes the feature vector of the node in which
teeth
The client at the time
The first feature learning network corresponding to
The client output from the layer
denotes the feature vector of the node in which
indicates an aggregation operation,
denotes linear operation,
denotes the local feature transformation matrix,
The client
The node where the client is located and the
Neighboring clients of
4. The method of claim 3, wherein: t denotes a correlation coefficient between the nodes in which t is located. Each of the clients generates local network parameters of the distributed collaborative time-series graph neural network model of the each of the clients based on the feature vector of the node in which the each of the clients is located.
Each of the clients inputs a feature vector of the node in which the client is located into a label prediction network corresponding to the client, and generates a predicted value of an edge between the node in which the client is located and a node in which an adjacent client of the client is located;
Each of the clients determines a local loss of the distributed federated time-series graph neural network model based on a predicted value of an edge between a node where the each client is located and a node where an adjacent client of the each client is located and a true value of the edge;
4. The method of claim 3, further comprising: each of the clients performing backpropagation based on the local loss to generate local network parameters of the distributed federated time-series graph neural network model of the each of the clients. The method comprises:
each of the clients sending a local loss of the distributed federated time-series graph neural network model to the server;
The server determines a global loss of the distributed federated time-series graph neural network model based on the local losses sent from each client;
The server obtains global network parameters of the distributed federated time-series graph neural network model based on the local network parameters transmitted from each client,
3. The method of claim 1, further comprising: performing backpropagation on the local network parameters and the global loss sent from each client to obtain global network parameters of the distributed federated time-series graph neural network model. A system for predicting the evolution of a multi-value chain, comprising: a server and a plurality of clients, the plurality of clients participating in training a distributed-linked time-series graph neural network model, the server interacting with each of the plurality of clients to train the distributed-linked time-series graph neural network model by executing the method of claim 7, and predicting edges between a node in which each client in a multi-value chain is located and a node in which an adjacent client of each of the clients is located based on the distributed-linked time-series graph neural network model. 10. A computer readable storage medium having stored thereon computer program instructions which, when executed by a multi-core processor, cause the multi-core processor to perform the method of claim 7.