CN110728376A - Federated learning method and device based on tree topology structure - Google Patents
Federated learning method and device based on tree topology structure Download PDFInfo
- Publication number
- CN110728376A CN110728376A CN201911000573.3A CN201911000573A CN110728376A CN 110728376 A CN110728376 A CN 110728376A CN 201911000573 A CN201911000573 A CN 201911000573A CN 110728376 A CN110728376 A CN 110728376A
- Authority
- CN
- China
- Prior art keywords
- node
- layer
- model
- training
- nodes
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
Abstract
The application provides a federated learning method and a federated learning device based on a tree topology structure, relates to the technical field of machine learning, and is used for solving the problems of overlong federated aggregation time and high network transmission pressure. The tree topology of the method comprises at least two layers of structures, each layer of structure comprises at least one node, and each node corresponds to a training model, and the method comprises the following steps: if the preset triggering condition is met, sampling the nodes of the current layer of the tree-shaped topological structure; and determining the training model corresponding to the current layer node after sampling treatment, and performing federal aggregation on the determined training model. The method reduces the network transmission pressure of the layered tree topology structure during the federal polymerization and accelerates the speed of the federal polymerization.
Description
Technical Field
The application relates to the technical field of machine learning, in particular to a federated learning method and device based on a tree topology structure.
Background
In order to solve the problems of large storage pressure of a server and large network transmission pressure caused by excessive clients, a layered tree topology structure is often adopted to alleviate the problems, see fig. 1, fig. 1 shows a three-layer tree topology structure, when federal model aggregation is triggered, according to a known federal aggregation relationship, each training model of a third layer is firstly subjected to federal aggregation to obtain each training model of a second layer, for example, the training models 131, 132 and the training models corresponding to the training model 132 are subjected to federal aggregation to obtain the training model corresponding to the training model 121, and the federal aggregation of other training models can refer to the description.
However, when the federated aggregation is performed at the current layer, the hierarchical tree topology must wait for the completion of the federated aggregation at the next layer of the current layer, and as long as any federated aggregation at the next layer is not completed, the federated aggregation at the current layer may be waited indefinitely.
In summary, when the federated aggregation is performed based on the hierarchical tree topology, the federated aggregation time is too long, the network transmission pressure is high, and the success of the federated aggregation cannot be guaranteed.
Disclosure of Invention
The embodiment of the application provides a federated learning method and a federated learning device based on a tree-shaped topological structure, so that the network transmission pressure of a layered tree-shaped topological structure during federated aggregation is at least reduced, and the federated aggregation speed is accelerated.
In a first aspect of the present application, a federated learning method based on a tree-like topology is provided, where the tree-like topology includes at least two layers of structures, each layer of structure includes at least one node, and each node corresponds to a training model, including:
if the preset triggering condition is met, sampling the current layer node of the tree-shaped topological structure;
and determining the training model corresponding to the current layer node after sampling treatment, and performing federal aggregation on the determined training model.
In one possible implementation, the determined training models are federated aggregated, including:
and carrying out federal aggregation on the determined training models to obtain at least one training model corresponding to the node of the current parent layer, wherein the current parent layer is the layer where the parent node of the current layer is located.
In a possible implementation manner, the sampling processing on the current level node of the tree topology includes:
sampling at least one current layer node from the current layer nodes;
deleting the training model corresponding to the at least one current layer node, or marking the at least one current layer node, so that the training model corresponding to the at least one current layer node does not participate in the federated aggregation.
In a possible implementation manner, the preset trigger condition includes one or more of the following:
the number of the nodes of the current layer is greater than a first preset value;
and the training model corresponding to the current layer node does not accord with a first preset federal model condition.
In one possible implementation, the sampling at least one current layer node from the current layer nodes includes:
randomly sampling at least one current layer node from the current layer nodes; or
And sampling current layer nodes with preset number of nodes from the current layer nodes.
In a possible implementation manner, before sampling a current layer node of a preset number of nodes from the current layer node, the method further includes:
and determining the number of nodes of the current layer node, and determining the number of preset nodes corresponding to the number of nodes of the current layer node according to the preset mapping relation between the number of nodes and the number of preset nodes.
In a possible implementation manner, before performing federated model aggregation sequentially from the bottom layer to the top layer of the tree topology, the method further includes:
and receiving a training model corresponding to a bottom node of the tree-shaped topological structure, wherein the training model is obtained by training a local model through local training data for a client corresponding to the bottom node.
In one possible implementation manner, the method further includes:
if the top-level federated model obtained by federated aggregation does not meet second preset federated model conditions, the top-level federated model is sent to a client corresponding to the bottom-level node, so that the client can conduct the next round of training by taking the top-level federated model as the local model;
and receiving a training model obtained by the client side for the next round of training, and sequentially carrying out the next round of federal model aggregation from the bottom layer of the tree-shaped topological structure to the top layer.
In a second aspect of the present application, a federated learning apparatus based on a tree-like topology is provided, where the tree-like topology includes at least two layers of structures, each layer of structure includes at least one node, and each node corresponds to a training model, and the apparatus includes:
the node sampling unit is used for sampling the current layer node of the tree-shaped topological structure if a preset trigger condition is met;
and the federal aggregation unit is used for determining the training model corresponding to the current layer node after sampling processing and performing federal aggregation on the determined training model.
In a possible implementation manner, the federation aggregation unit is configured to perform federation aggregation on the determined training models to obtain a training model corresponding to at least one node of a current parent layer, where the current parent layer is a layer where a parent node of the current layer of nodes is located.
In one possible implementation, the node sampling unit is configured to:
sampling at least one current layer node from the current layer nodes;
deleting the training model corresponding to the at least one current layer node, or marking the at least one current layer node, so that the training model corresponding to the at least one current layer node does not participate in the federated aggregation.
In a possible implementation manner, the preset trigger condition includes one or more of the following:
the number of the nodes of the current layer is greater than a first preset value;
and the training model corresponding to the current layer node does not accord with a first preset federal model condition.
In one possible implementation, the node sampling unit is configured to:
randomly sampling at least one current layer node from the current layer nodes; or
And sampling current layer nodes with preset number of nodes from the current layer nodes.
In one possible implementation, the node sampling unit is configured to:
determining the number of nodes of the current layer node before sampling the current layer node with the preset number of nodes from the current layer node, and determining the preset number of nodes corresponding to the number of nodes of the current layer node according to the preset mapping relation between the number of nodes and the preset number of nodes.
In a possible implementation manner, the federation aggregation unit is configured to perform federation model aggregation from the bottom layer of the tree-like topology structure to the top layer in sequence, where the training model corresponding to the bottom layer node of the tree-like topology structure is received before performing federation model aggregation from the bottom layer of the tree-like topology structure to the top layer in sequence, and the training model is obtained by training a local model through local training data for a client corresponding to the bottom layer node.
In one possible implementation, the federal aggregation unit is further configured to:
if the top-level federated model obtained by federated aggregation does not meet second preset federated model conditions, the top-level federated model is sent to a client corresponding to the bottom-level node, so that the client can conduct the next round of training by taking the top-level federated model as the local model;
and receiving a training model obtained by the client side for the next round of training, wherein the bottom layer of the tree-shaped topological structure starts, and the next round of federal model aggregation is sequentially carried out towards the top layer.
In a third aspect, a computer device is provided, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method of any one of the first aspect and one possible implementation when executing the program.
In a fourth aspect, a computer-readable storage medium is provided, which stores computer instructions that, when executed on a computer, cause the computer to perform the method according to any one of the first aspect and one possible implementation.
The embodiment of the application at least has the following beneficial effects:
in the scheme of the application, when the tree-shaped topological structure is subjected to horizontal federal learning, training models of part of nodes are discarded, and the network transmission pressure during the transmission of the training models is reduced; and because the training models of part of nodes are abandoned, the time required for carrying out the federal polymerization is also reduced; meanwhile, the storage pressure of the server side for carrying out the federal aggregation is correspondingly reduced due to the fact that the training models of part of the nodes are abandoned.
On the other hand, random discarding can be selected in the training models of which part of nodes are discarded, so that federate aggregation can be performed according to the rest training models, and the time for obtaining the top-level federate model by the tree-shaped topological structure is reduced.
Drawings
FIG. 1 is a schematic diagram of a tree topology provided in the background of the present application;
FIG. 2 is a diagram illustrating a standard federated learning process provided by an embodiment of the present application;
fig. 3 is a flowchart of another tree topology provided in an embodiment of the present application;
fig. 4 is a schematic diagram of a federation learning process based on a tree topology according to an embodiment of the present application;
fig. 5 is a schematic diagram of another federal learning process based on a tree topology according to an embodiment of the present application;
fig. 6 is a structural diagram of a federal learning device provided in an embodiment of the present application;
fig. 7 is a block diagram of a computer device according to an embodiment of the present application.
Detailed Description
In order to better understand the technical solutions provided by the embodiments of the present application, the following detailed description is made with reference to the drawings and specific embodiments.
In order to facilitate those skilled in the art to better understand the technical solutions of the present application, the following description refers to the terms/technical terms of the present application.
The client may be a terminal device such as a mobile phone, a computer, a Personal Digital Assistant (PDA), a smart television, or a vehicle-mounted device, or a network-side device such as a server.
Federal model learning thought: the machine learning model is built by encrypting a technical algorithm, a plurality of clients in the federal learning framework do not need to give own data during model training, but train a local model according to a global federal model and client local training data encrypted by parameters sent by a server, the local model is returned to be aggregated and updated by the server, the updated federal model is sent to the clients again, and the operation is repeated until convergence.
As shown in fig. 2, a standard federal model learning procedure is given below, which specifically includes:
step S201, each client 21 trains a local model locally to obtain a training model;
step S202, each client 21 sends the training model obtained by training to a server;
step S203, the server 22 receives a plurality of training models and performs federal aggregation on the plurality of training models to obtain a federal model;
step S204, the server side sends the federal model obtained by federal aggregation to each client side;
and step S205, each client receives the federal model and updates the local model into the federal model.
However, the standard federal model learning procedure in fig. 2 clearly suffers from the following disadvantages:
1) when the number of clients in the federal structure is large, the storage pressure of a server is very large after a plurality of clients send training models;
2) when the number of clients in the federal structure is large, the pressure of network transmission is large when a plurality of clients send training models to the server.
The above disadvantages in the standard-based federal model training are often alleviated by adopting a layered tree topology structure, as shown in fig. 1, a three-layer tree topology structure is provided, and when federal model aggregation is triggered, according to a known federal aggregation relationship, each training model in the third layer is subjected to federal aggregation to obtain each training model in the second layer.
However, when the hierarchical tree topology is federated aggregated at the current level, the hierarchical tree topology must be executed when federated aggregation at the next level of the current level is completed, and as long as any federated aggregation at the next level is not completed, it may cause unlimited waiting of federated aggregation at the current level, as shown in fig. 3, a four-level tree topology is provided, in fig. 3, each circle represents a node, and each node corresponds to a training model, wherein the federated aggregation result of the node 321 needs to wait for completion of federated aggregation of the nodes 331 and 332, and similarly, federated aggregation of the node 331 needs to wait for completion of training of the training models of the nodes 31 and 342, federated aggregation of the node 332 needs to wait for completion of training of the training models of the nodes 343 and 344, as long as federated training or model training of a node in the middle is not completed, the federated aggregation of node 421 may wait indefinitely.
In view of this, the present application provides a method, an apparatus, and a medium for horizontal federal learning based on a tree topology, which are characterized in that the tree topology is known and includes at least two layers of structures, each layer of structure includes at least one node, and each node corresponds to a training model, and specifically includes: and when a preset trigger condition is met, sampling the current layer node of the tree-shaped topological structure, further performing the training model corresponding to the current layer node after sampling, and performing federal aggregation on the determined training model.
And further, carrying out federal aggregation on the determined training models to obtain at least one training model corresponding to the node of the current parent layer, wherein the current parent layer is the layer of the parent node shuttle of the node of the current layer.
The scene applied by the method can comprise a server and a plurality of clients, the training model obtained by training the local model by the clients is used as the training model of the bottom node of the tree-shaped topological structure, and the server performs transverse federal learning according to the known tree-shaped topological structure.
The method can also be applied to a scene comprising a plurality of servers and a plurality of clients, wherein the plurality of servers and the plurality of clients construct a tree-shaped network topology structure according to the known tree-shaped topology structure, each client is used as a bottom node of the tree-shaped topology structure, each server is used as a node except the bottom node, and the servers of all layers carry out federal aggregation on training models of the next layer.
The methods provided herein are further described below:
in the process of performing horizontal federal learning based on the tree-shaped topological structure in this embodiment, federal model aggregation may be performed sequentially from the bottom layer to the top layer of the tree-shaped topological structure, and when the process is performed to any layer, if a preset trigger condition is met, the nodes on the layer are sampled.
The preset trigger condition comprises one or more of the following conditions:
first trigger condition: the number of the nodes of the current layer is larger than a first preset value.
The first preset value may be, but is not limited to, 1 or 2, that is, when the number of nodes of the current layer model is 1, the training model corresponding to the node may be directly used as the final federal model; when the number of the nodes of the current layer model is 2, the training models corresponding to the 2 nodes can be directly subjected to federal aggregation, and the federal model obtained by federal aggregation is used as a final federal model.
The second trigger condition is: and the training model corresponding to the current layer node does not accord with the first preset federal model condition.
If the training models corresponding to the nodes of the current layer have a first preset number, the training models do not accord with the first preset federal model condition; the first preset federal model condition is used for measuring the prediction accuracy of the top-level federal model, so that the preset federal model condition can be set as follows: the model prediction accuracy is greater than a second preset value, and a person skilled in the art can also set the preset federal model condition according to other actual requirements; the first preset number and the second preset number can be set by those skilled in the art according to actual requirements.
In this embodiment, federate model aggregation is performed sequentially from the bottom layer to the top layer of the tree-like topology structure, wherein a training model corresponding to a bottom node of the tree-like topology structure is further received before federate model aggregation is performed sequentially from the bottom layer to the top layer of the tree-like topology structure.
The training model corresponding to the bottom node can be obtained by training a local model through local training data for a client corresponding to the bottom node.
After the method is used for carrying out federal model polymerization on the top layer to obtain a top layer federal model, whether the obtained top layer federal model meets second preset federal model conditions or not can be judged, if not, the top layer federal model is sent to a client corresponding to a bottom node, and the client is instructed to carry out next round of training by taking the top layer federal model as the local model; and
and receiving a training model obtained by the client side for the next round of training, taking the received training model as a training model corresponding to the bottom node, and sequentially carrying out the next round of federal model polymerization from the bottom of the tree-shaped topological structure to the top layer until the obtained top layer federal model meets the second preset federal model condition.
The second preset federal model condition is used for measuring the prediction accuracy of the top federal model, and the second preset federal model condition and the first preset federal model condition may be set to be the same condition or different conditions, for example, the second preset federal model condition is set to be: the model prediction accuracy is greater than a third preset value, which may be the same as or different from the first preset value in the first preset federal model conditions.
In this embodiment, the sampling processing may be performed on the current layer node of the tree topology by, but not limited to, the following sampling method:
sampling at least one current layer node from the current layer nodes;
deleting the training model corresponding to the sampled at least one current layer node, or marking the sampled at least one current layer node, so that the training model corresponding to the sampled at least one current layer node does not participate in federated aggregation.
Several methods for sampling at least one current level node from a current level node are provided as follows:
the first sampling method comprises the following steps: at least one current level node is randomly sampled from the current level nodes.
In the sampling method, the current layer node is randomly sampled, so that the number of the sampled nodes is uncertain, and the specific sampled current layer node is uncertain.
The second sampling method comprises the following steps: and sampling the current layer nodes with the preset number of nodes from the current layer nodes.
In the sampling mode, the node number range to which the node number of the current layer node belongs can be determined firstly;
and determining the number of preset nodes corresponding to the node number range, and sampling the current layer nodes with the preset nodes from the current layer nodes, wherein the number of the sampled preset nodes is determined at the moment, and the sampled specific current layer points are not determined.
The number of the node number range and the specific numerical value range are not limited too much, and those skilled in the art can set the node number range according to actual requirements, and only one illustrative example is given below:
if the number of nodes of the bottom node of the known tree topology is 15, three node number ranges, such as 2-6, 7-11, and 12-15, may be set, and since only one node is left, the training model corresponding to the node is determined as the final federal model, the node number range may not include 1.
The preset number of nodes corresponding to the range of the number of nodes is determined, which may be, but is not limited to, the following two methods:
the first preset node number determining mode is as follows:
and determining the preset node number corresponding to the node number range of the current layer node according to the preset mapping relation between the node number range and the preset node number.
Optionally, a person skilled in the art can obtain appropriate preset node numbers corresponding to different node number ranges according to an experiment and other manners, and further construct a preset mapping relationship between the node number range and the preset node number; if the number range of the nodes is 7-11, the number of the corresponding preset nodes is 3, and the like.
The second preset node number determining mode is as follows:
determining a sampling node number range corresponding to the node number range according to a preset mapping relation between the node number range and the sampling node number range; and randomly determining the number of one sampling node in the sampling node number range as the preset node number corresponding to the node number range of the current layer node.
Optionally, a person skilled in the art can obtain appropriate sampling node number ranges corresponding to different node number ranges according to an experiment or the like, and further construct a preset mapping relationship between the node number ranges and the sampling node number ranges according to the appropriate sampling node number ranges; if the node number range is 7-11, the corresponding sampling node number range can be 2-5, and the like, if the node number range is 7-11, the sampling node number range is 2-5, and a preset node number with a value corresponding to the node number range of the current layer node can be randomly determined from 2, 3, 4 and 5.
Examples of 2 tree topology based horizontal federated learning approaches are given below:
example 1: randomly deleting training model corresponding to at least one node in current layer
The tree topology is shown in fig. 4, the top layer includes a node 411, the second layer includes a node 421 and a node 422, the third layer includes a node 431 to a node 434, the fourth layer includes a node 441 to a node 449, and the fifth layer includes a node 4501 to a node 4517, and the federate aggregation process of the tree topology is as follows:
the first step is as follows: and randomly discarding part of nodes at the fifth layer, and carrying out federal aggregation on the rest training models to obtain the training models corresponding to the nodes at the fourth layer.
In fig. 4, the shaded circles indicate nodes that are selected for discarding, and the dashed circles indicate nodes for which the training model cannot be obtained because all nodes of the factor layer are discarded or marked for discarding.
In fig. 4, the training models corresponding to nodes 4501, 4503, 4504, 4508, 4511, 4512, 4513, and 4517 are discarded at the fifth level, and further, since the training models corresponding to node 4501 and node 4502 are both discarded, node 442 does not get a training model, and similarly, node 446 does not get a model.
The second step is that: and partially discarding the obtained training models at the fourth layer, and carrying out federal aggregation on the rest training models to obtain the training models corresponding to the nodes at the third layer.
As shown in fig. 4, the training models corresponding to the nodes 444 and 445 are discarded in the third layer, and since the training model corresponding to the node 445 is discarded and the node 446 does not obtain a training model, the node 433 also does not obtain a corresponding training model.
And thirdly, partially discarding the obtained training models in the third layer, and carrying out federal aggregation on the rest training models to obtain the training models corresponding to the nodes in the second layer.
As shown in fig. 4, the training models corresponding to the node 431 are discarded in the third layer.
And fourthly, partially discarding the obtained training models in the second layer, and carrying out federal aggregation on the rest training models to obtain a federal model corresponding to the node 411 in the first layer.
As shown in fig. 4, in the second layer, the training models corresponding to the nodes 422 are discarded, and if only the training model corresponding to the node 421 remains after discarding the training model corresponding to the node 422, the training model corresponding to the node 421 is directly used as the federal model corresponding to the node 411.
Optionally, in the fourth step, the training models may not be discarded, and the federate model corresponding to the node 411 may be obtained by directly performing federate aggregation on the 2 training models in the second layer.
Example 2: discarding at least one training model according to the node number range to which the node number of the current layer node belongs
The tree topology is shown in fig. 5, the top layer includes node 511, the second layer includes node 521 and node 522, the third layer includes node 531 to node 534, the fourth layer includes node 541 to node 549, and the fifth layer includes node 5501 to node 5517;
the mapping relationship between the preset node number range and the sampling node number range is as follows:
TABLE 1
The federated aggregation process for this tree topology is as follows:
the first step is as follows: and discarding part of nodes at the fifth layer, and carrying out federal aggregation on the rest training models to obtain training models corresponding to the nodes at the fourth layer.
In fig. 5, the shaded circles indicate nodes that are selected for discarding, and the dashed circles indicate nodes for which the training model cannot be obtained because all nodes of the factor layer are discarded or marked for discarding.
The number of the nodes on the fifth layer is 17, the nodes belong to the preset node number range of 12-18, the number range of the sampling nodes corresponding to the nodes is 5-9, at the moment, a numerical value of 6 is randomly determined from numbers 5-9, the nodes 5502, 5505, 5508, 5509, 5513 and 5517 are randomly selected from the nodes on the fifth layer, training models corresponding to the selected 6 nodes are discarded, and the rest training models are subjected to federal aggregation to obtain training models corresponding to the nodes on the fourth layer.
The second step is that: and partially discarding the obtained training models at the fourth layer, and carrying out federal aggregation on the rest training models to obtain the training models corresponding to the nodes at the third layer.
The number of the nodes on the fourth layer is 8, the nodes belong to the preset node number range of 7-11, the number range of the corresponding sampling nodes is 3-5, the numerical value 3 is randomly determined from the numbers 3-5, the nodes 541, 545 and 547 are randomly selected from the nodes on the fourth layer, the training models corresponding to the selected 3 nodes are discarded, and the residual training models are subjected to federal aggregation to obtain the training models corresponding to the nodes on the third layer.
And thirdly, partially discarding the obtained training models in the third layer, and carrying out federal aggregation on the rest training models to obtain the training models corresponding to the nodes in the second layer.
The number of the nodes on the third layer is 4, the nodes belong to the preset node number range of 3-6, the number range of the corresponding sampling nodes is 1-2, the numerical value 2 is randomly determined from the numbers 1 and 2, the nodes 531 and 534 are randomly selected from the nodes on the third layer, the training models corresponding to the selected 2 nodes are discarded, and the residual training models are subjected to federal aggregation to obtain the training models corresponding to the nodes on the second layer.
Fourthly, because the number of the nodes corresponding to the second layer is 2, the number of the nodes discarded at the layer is 0, and the 2 training models at the layer are directly subjected to federal aggregation to obtain the training model corresponding to the node 511.
As shown in fig. 6, based on the same technical concept, an embodiment of the present application further provides a horizontal federal learning apparatus 600, where the tree-shaped topology structure includes at least two layers, each layer includes at least one node, and each node corresponds to a training model, and the apparatus includes:
a node sampling unit 601, configured to sample a current layer node of the tree topology structure if a preset trigger condition is met;
a federal aggregation unit 602, configured to determine the training model corresponding to the current layer node after the sampling processing, and perform federal aggregation on the determined training model.
Optionally, the federal aggregation unit is configured to perform federal aggregation on the determined training models to obtain a training model corresponding to at least one node of a current parent layer, where the current parent layer is a layer where a parent node of the current layer node is located.
Optionally, the node sampling unit is configured to:
sampling at least one current layer node from the current layer nodes;
deleting the training model corresponding to the at least one current layer node, or labeling the at least one current layer node, so that the training model corresponding to the at least one current layer node does not participate in the federated aggregation.
Optionally, the preset trigger condition includes one or more of the following:
the number of the current layer nodes is larger than a first preset value;
and the training model corresponding to the current layer node does not accord with the first preset federal model condition.
Optionally, the node sampling unit is configured to:
randomly sampling at least one current layer node from the current layer nodes; or
And sampling current layer nodes with preset number of nodes from the current layer nodes.
Optionally, the node sampling unit is configured to:
before sampling the current layer nodes with the preset number of nodes from the current layer nodes, determining the number of the nodes of the current layer nodes, and determining the preset number of nodes corresponding to the number of the nodes of the current layer nodes according to the preset mapping relation between the number of the nodes and the preset number of the nodes.
Optionally, the federation aggregation unit is configured to perform federation model aggregation sequentially from a bottom layer to a top layer of the tree-like topology, where the receiving unit receives a training model corresponding to a bottom node of the tree-like topology before performing federation model aggregation sequentially from the bottom layer to the top layer, where the training model is obtained by training a local model through local training data for a client corresponding to the bottom node.
Optionally, the federal polymerization unit mentioned above is further configured to:
if the top layer federal model obtained by the federal polymerization does not meet a second preset federal model condition, sending the top layer federal model to a client corresponding to the bottom node, so that the client can conduct the next round of training by taking the top layer federal model as the local model;
and receiving a training model obtained by the client for the next round of training, wherein the bottom layer of the tree-shaped topological structure starts, and the next round of federal model aggregation is sequentially carried out towards the top layer.
Based on the same technical concept, an embodiment of the present application further provides a computer device 700, please refer to fig. 7, the computer device includes a processor 701 and a memory 702, wherein:
the memory 702 has stored therein a computer program;
the processor 701, when executing the computer program, implements the above discussed federated learning method based on tree topology, and will not be described again here.
Fig. 7 illustrates an example of one processor 701, but the number of processors 701 is not limited in practice.
The memory 702 may be a volatile memory (volatile memory), such as a random-access memory (RAM); the memory 702 may also be a non-volatile memory (non-volatile) such as, but not limited to, a read-only memory (rom), a flash memory (flash memory), a hard disk (HDD) or a solid-state drive (SSD), or the memory 702 may be any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. Memory 702 may be a combination of the above.
The functions of the various modules of the lateral federal learning device 600 in fig. 6 can be implemented, as one embodiment, by the processor 701 in fig. 7.
Based on the same technical concept, the embodiment of the present application further provides a computer-readable storage medium, which stores computer instructions that, when executed on a computer, cause the computer to execute the tree topology-based federal learning method as discussed above.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While the preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all alterations and modifications as fall within the scope of the application.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.
Claims (10)
1. A federated learning method based on a tree-shaped topological structure is characterized in that the tree-shaped topological structure comprises at least two layers of structures, each layer of structure comprises at least one node, and each node corresponds to a training model, and the method comprises the following steps:
if the preset triggering condition is met, sampling the current layer node of the tree-shaped topological structure;
and determining the training model corresponding to the current layer node after sampling treatment, and performing federal aggregation on the determined training model.
2. The method of claim 1, wherein the determined training models are federated aggregated, comprising:
and carrying out federal aggregation on the determined training models to obtain at least one training model corresponding to the node of the current parent layer, wherein the current parent layer is the layer where the parent node of the current layer is located.
3. The method of claim 1, wherein the sampling a current level node of the tree topology comprises:
sampling at least one current layer node from the current layer nodes;
deleting the training model corresponding to the at least one current layer node, or marking the at least one current layer node, so that the training model corresponding to the at least one current layer node does not participate in the federated aggregation.
4. The method of claim 1, wherein the preset trigger condition comprises one or more of:
the number of the nodes of the current layer is greater than a first preset value;
and the training model corresponding to the current layer node does not accord with a first preset federal model condition.
5. The method of claim 3, wherein said sampling at least one current level node from said current level nodes comprises:
randomly sampling at least one current layer node from the current layer nodes; or
And sampling current layer nodes with preset number of nodes from the current layer nodes.
6. The method according to claim 1, wherein federated model aggregation is performed from the bottom layer of the tree topology to the top layer in sequence, and wherein before federated model aggregation is performed from the bottom layer of the tree topology to the top layer in sequence, the method further comprises:
and receiving a training model corresponding to a bottom node of the tree-shaped topological structure, wherein the training model is obtained by training a local model through local training data for a client corresponding to the bottom node.
7. The method of claim 6, further comprising:
if the top-level federated model obtained by federated aggregation does not meet second preset federated model conditions, the top-level federated model is sent to a client corresponding to the bottom-level node, so that the client can conduct the next round of training by taking the top-level federated model as the local model;
and receiving a training model obtained by the client side for the next round of training, and sequentially carrying out the next round of federal model aggregation from the bottom layer of the tree-shaped topological structure to the top layer.
8. The federated learning aggregation device based on the tree-shaped topological structure is characterized in that the tree-shaped topological structure comprises at least two layers of structures, each layer of structure comprises at least one node, and each node corresponds to one training model, and the federated learning aggregation device comprises:
the node sampling unit is used for sampling the current layer node of the tree-shaped topological structure if a preset trigger condition is met;
and the federal aggregation unit is used for determining the training model corresponding to the current layer node after sampling processing and performing federal aggregation on the determined training model.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the steps of the method of any of claims 1-7 are implemented when the program is executed by the processor.
10. A computer-readable storage medium having stored thereon computer instructions which, when executed on a computer, cause the computer to perform the method of any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911000573.3A CN110728376A (en) | 2019-10-21 | 2019-10-21 | Federated learning method and device based on tree topology structure |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911000573.3A CN110728376A (en) | 2019-10-21 | 2019-10-21 | Federated learning method and device based on tree topology structure |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110728376A true CN110728376A (en) | 2020-01-24 |
Family
ID=69220489
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911000573.3A Pending CN110728376A (en) | 2019-10-21 | 2019-10-21 | Federated learning method and device based on tree topology structure |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110728376A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112288094A (en) * | 2020-10-09 | 2021-01-29 | 武汉大学 | Federal network representation learning method and system |
| CN113112307A (en) * | 2021-04-30 | 2021-07-13 | 欧冶云商股份有限公司 | Steel price prediction method, device, equipment and medium based on federal learning |
| CN113282933A (en) * | 2020-07-17 | 2021-08-20 | 中兴通讯股份有限公司 | Federal learning method, device and system, electronic equipment and storage medium |
| CN113515760A (en) * | 2021-05-28 | 2021-10-19 | 平安国际智慧城市科技股份有限公司 | Horizontal federal learning method, device, computer equipment and storage medium |
| WO2022066089A1 (en) * | 2020-09-23 | 2022-03-31 | Chalmers Ventures Ab | System and method for scalable machine learning in a communication network |
| CN114666274A (en) * | 2022-03-17 | 2022-06-24 | 广州广电运通金融电子股份有限公司 | Federal learning method, device, system and readable medium for asynchronous mode training |
-
2019
- 2019-10-21 CN CN201911000573.3A patent/CN110728376A/en active Pending
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113282933A (en) * | 2020-07-17 | 2021-08-20 | 中兴通讯股份有限公司 | Federal learning method, device and system, electronic equipment and storage medium |
| WO2022012621A1 (en) * | 2020-07-17 | 2022-01-20 | 中兴通讯股份有限公司 | Federated learning method, apparatus and system, electronic device and storage medium |
| CN113282933B (en) * | 2020-07-17 | 2022-03-01 | 中兴通讯股份有限公司 | Federal learning method, device and system, electronic equipment and storage medium |
| WO2022066089A1 (en) * | 2020-09-23 | 2022-03-31 | Chalmers Ventures Ab | System and method for scalable machine learning in a communication network |
| CN112288094A (en) * | 2020-10-09 | 2021-01-29 | 武汉大学 | Federal network representation learning method and system |
| CN112288094B (en) * | 2020-10-09 | 2022-05-17 | 武汉大学 | Federal network representation learning method and system |
| CN113112307A (en) * | 2021-04-30 | 2021-07-13 | 欧冶云商股份有限公司 | Steel price prediction method, device, equipment and medium based on federal learning |
| CN113515760A (en) * | 2021-05-28 | 2021-10-19 | 平安国际智慧城市科技股份有限公司 | Horizontal federal learning method, device, computer equipment and storage medium |
| CN113515760B (en) * | 2021-05-28 | 2024-03-15 | 平安国际智慧城市科技股份有限公司 | Horizontal federal learning method, apparatus, computer device, and storage medium |
| CN114666274A (en) * | 2022-03-17 | 2022-06-24 | 广州广电运通金融电子股份有限公司 | Federal learning method, device, system and readable medium for asynchronous mode training |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110728376A (en) | Federated learning method and device based on tree topology structure | |
| CN110942154B (en) | Data processing method, device, equipment and storage medium based on federal learning | |
| CN108416327B (en) | Target detection method and device, computer equipment and readable storage medium | |
| KR102180994B1 (en) | Adaptive artificial neural network selection techniques | |
| US20060003823A1 (en) | Dynamic player groups for interest management in multi-character virtual environments | |
| CN109919313B (en) | Gradient transmission method and distributed training system | |
| CN108124489B (en) | Information processing method, apparatus, cloud processing device and computer program product | |
| CN111898484A (en) | Method and device for generating model, readable storage medium and electronic equipment | |
| CN104134203A (en) | Fast dense matching method based on close-range photogrammetry | |
| CN111275054A (en) | Image processing method, image processing device, electronic equipment and storage medium | |
| CN113705520A (en) | Motion capture method and device and server | |
| CN109859314A (en) | Three-dimensional rebuilding method, device, electronic equipment and storage medium | |
| CN109657801B (en) | Shunting method and device of recommendation system and readable storage medium | |
| CN111222553A (en) | Training data processing method and device of machine learning model and computer equipment | |
| CN109948803A (en) | Algorithm model optimization method, device and equipment | |
| CN115829064B (en) | Federal learning acceleration method, device, equipment and storage medium | |
| CN116800671A (en) | Data transmission method, apparatus, computer device, storage medium, and program product | |
| CN108898527A (en) | A kind of traffic data fill method based on the generation model for having loss measurement | |
| CN103503358B (en) | The method of a kind of group resource interaction message and device thereof | |
| TWI772932B (en) | Image processing and neural network training method, electronic device and computer storage medium | |
| CN107800738B (en) | Data updating method and device | |
| CN111260706B (en) | Dense depth map calculation method based on monocular camera | |
| CN112866504B (en) | Air three encryption method and system | |
| CN110399881B (en) | End-to-end quality enhancement method and device based on binocular stereo image | |
| CN107644055B (en) | communication guarantee capability generation method based on training data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |