CN117808127B

CN117808127B - Image processing method, federal learning method and device under heterogeneous data condition

Info

Publication number: CN117808127B
Application number: CN202410230103.0A
Authority: CN
Inventors: 范宝余; 李仁刚; 王立; 张润泽; 郭振华; 赵雅倩; 曹芳; 赵坤; 鲁璐; 贺蒙
Original assignee: Inspur Electronic Information Industry Co Ltd
Current assignee: Inspur Electronic Information Industry Co Ltd
Priority date: 2024-02-29
Filing date: 2024-02-29
Publication date: 2024-05-28
Anticipated expiration: 2044-02-29
Also published as: CN117808127A

Abstract

The invention discloses an image processing method, a federal learning method and a federal learning device under a data heterogeneous condition, which relate to the technical field of image processing, edge computing equipment is clustered according to data distribution similarity, the edge computing equipment in the cluster has similar data distribution, a model can better capture the characteristics of data, and the problem of data heterogeneous is effectively solved. The edge computing devices in the clusters perform model parameter aggregation according to the tree-shaped aggregation network in the clusters, and the edge computing devices in the lower layer only send model parameters to the corresponding edge computing devices in the upper layer, but not send model parameters to other edge computing devices, so that communication expenditure can be greatly reduced. The edge computing equipment and the edge cloud server conduct two-layer model parameter aggregation in the federal learning process to obtain an accurate and reliable image processing model, and finally the edge computing equipment uses the accurate and reliable image processing model to conduct image processing, so that the accuracy and reliability of image processing can be improved.

Description

Image processing method, federal learning method and device under heterogeneous data condition

Technical Field

The present invention relates to the field of image processing technologies, and in particular, to an image processing method, federal learning method, apparatus, system, device, and medium under heterogeneous data conditions.

Background

Federal learning is a distributed machine learning method. Local data is not interacted among all participants of federal learning, and local model parameters are interacted, wherein the local model parameters refer to model parameters obtained by using a local data training model by edge computing equipment. The edge cloud server aggregates local model parameters of all edge computing devices to obtain global model parameters, and sends the global model to all edge computing devices, wherein the global model is a model taking the global model parameters as model parameters. However, conventional federal learning schemes often do not take into account differences in data distribution among edge computing devices, which can result in poor adaptation of the model to the data distribution of certain edge computing devices, and for federal learning environments where data is heterogeneous, which can result in impaired performance of the model. In addition, the traditional federal learning scheme generally adopts a centralized mode to aggregate model parameters, namely, all edge computing devices send own model parameters to an edge cloud server for aggregation, so that larger communication overhead is brought. Under the condition that the global model obtained through training is an image processing model, the accuracy and reliability of image processing by using the trained image processing model are not high.

In view of this, how to solve the problem of data isomerism in federal learning, reducing communication overhead and improving accuracy and reliability of image processing has become a technical problem to be solved by those skilled in the art.

Disclosure of Invention

The invention aims to provide an image processing method, a federal learning method, a device, a system, equipment and a medium under the heterogeneous condition of data, which can solve the problem of data isomerism in federal learning and reduce communication overhead.

In order to solve the technical problems, the invention provides an image processing method under a heterogeneous data condition, which comprises the following steps:

dividing the edge computing equipment into a plurality of data identity clusters according to the data distribution similarity of the edge computing equipment;

Selecting edge computing equipment from the data homography cluster as a cluster head of the data homography cluster;

Receiving an intra-cluster model parameter aggregation result of the data homopolar cluster uploaded by the cluster head; the cluster head is a root node of the cluster tree-shaped aggregation network, a child node in the cluster tree-shaped aggregation network sends model parameters to a corresponding father node, and model parameter aggregation is carried out with the local model parameters of the father node;

And aggregating cluster model parameter aggregation results of the data homopolar clusters to obtain a second-layer model parameter aggregation result, and when an image processing model taking the second-layer model parameter aggregation result as a model parameter converges, issuing the image processing model to each edge computing device so that the edge computing device processes an image by using the image processing model.

In some embodiments, selecting an edge computing device from the data-like clusters as a cluster head for the data-like clusters comprises:

and selecting the edge computing device which is closest to other edge computing devices in the cluster or has the largest communication rate with other edge computing devices in the cluster in the data homography cluster as the cluster head of the data homography cluster.

In some embodiments, the cluster head selects a preset number of edge computing devices from other edge computing devices in the cluster, and the preset number of edge computing devices are used as layer 1 sub-nodes of the tree-shaped aggregation network in the cluster, and the layer 1 sub-nodes are edge computing devices with the maximum data sending rate when communicating with the cluster head; selecting a preset number of edge computing devices from the rest edge computing devices in the cluster by the child nodes of the ith layer to serve as the child nodes of the (i+1) th layer of the tree-shaped aggregation network in the cluster; the (i+1) th layer child node is edge computing equipment with the maximum data transmission rate when communicating with the (i) th layer child node; and i takes a value from 1 until the intra-cluster tree-shaped aggregation network is constructed.

In some embodiments, aggregating intra-cluster model parameter aggregation results for each of the data-like clusters, the obtaining a second-layer model parameter aggregation result comprises:

Determining the weight coefficient of each data homopolar cluster in the current training round;

And carrying out weighted aggregation on the cluster model parameters of each data homopolar cluster according to the weight coefficient of each data homopolar cluster to obtain the second layer model parameter aggregation result of the current training round.

In some embodiments, determining the weight coefficient for each of the data-like clusters for the current training round comprises:

counting the local data testing precision of each edge computing device of the data homography cluster to obtain the local data testing precision of the data homography cluster; the local data testing precision of the edge computing equipment is the data testing precision of the edge computing equipment for testing the global model obtained by the previous training round by using the local data;

determining a weight coefficient of the data homography cluster according to the local data testing precision of the data homography cluster; the weight coefficient of the data homography cluster is inversely related to the local data testing precision of the data homography cluster.

In some embodiments, determining the weight coefficient of the data-like cluster according to the local data testing accuracy of the data-like cluster comprises:

According to Obtaining a weight coefficient of the data homopolar cluster; k _c represents the weight coefficient of the c-th data-like cluster, and μ represents the local data test accuracy of the c-th data-like cluster.

In some embodiments, one training round includes three phases of local model training, first layer model parameter aggregation, and second layer model parameter aggregation; after each edge computing device in each data like character cluster completes model parameter updating for a first preset time, first layer model parameter aggregation is carried out, and after each data like character cluster completes first layer model parameter aggregation for a second preset time, second layer model parameter aggregation is carried out.

In some embodiments, further comprising:

And clustering the edge computing equipment according to the data distribution similarity of the edge computing equipment every time after model training of the preset training round is completed, and carrying out model training of the preset training round again until the global model converges.

In some embodiments, dividing the edge computing device into a number of data-like clusters according to data distribution similarities of the edge computing device comprises:

Constructing a weighted undirected graph according to the data distribution similarity of the edge computing devices; the value of the connecting edge between two edge computing devices in the weighted undirected graph is the value of the data distribution similarity of the two edge computing devices;

And dividing the edge computing equipment into a plurality of data identity clusters according to the weighted undirected graph.

In some embodiments, constructing the weighted undirected graph based on the similarity of the data distribution of the edge computing devices includes:

Searching public data from a public network, and constructing a test data set based on the public data;

Issuing the test data set to each edge computing device so that each edge computing device uses a local model to infer the test data set;

receiving the reasoning results uploaded by the edge computing devices and computing the similarity of the reasoning results;

And constructing the weighted undirected graph according to the similarity of the reasoning results.

In some embodiments, said constructing the weighted undirected graph based on the similarity of the inference results comprises:

comparing the similarity of the reasoning results of the edge computing devices with a preset threshold value;

If the similarity of the reasoning results of the two edge computing devices is greater than a preset threshold, establishing a connection relation between the two edge computing devices, wherein the value of the similarity of the reasoning results of the two edge computing devices is used as the value of the connection edge between the two edge computing devices.

In some embodiments, partitioning the edge computing device into a number of data-like clusters according to the weighted undirected graph includes:

Initializing labels of all edge computing devices in the weighted undirected graph;

Iteratively updating labels of all edge computing devices; wherein iteratively updating the labels of the edge computing devices comprises: setting the labels of the edge computing devices connected with the connecting edges with values larger than the set threshold value as the same labels; counting the number of times of occurrence of the label of the neighbor edge computing device of the target edge computing device, and selecting the label with the largest number of occurrence in the neighbor edge computing device as the label of the target edge computing device; the target edge computing device is an edge computing device with the value of each connected edge not larger than a set threshold value;

Judging whether an iteration update stop condition is met;

If the iteration update stop condition is met, the edge computing devices with the same label are divided into the same data identity cluster.

In some embodiments, determining whether an iterative update stop condition is satisfied comprises:

After each iteration update is completed, calculating the change quantity of the labels of all edge computing devices after the iteration update and the labels of all edge computing devices after the previous iteration update;

comparing the variation with a preset value;

And if the variation is smaller than the preset value, an iteration update stopping condition is met.

In order to solve the technical problem, the invention also provides a federal learning method under the heterogeneous condition of data, which comprises the following steps:

And aggregating cluster model parameter aggregation results of the data homopolar clusters to obtain a second-layer model parameter aggregation result, and issuing the global model to the edge computing devices when the global model with the second-layer model parameter aggregation result as a model parameter converges.

In order to solve the above technical problem, the present invention further provides an image processing apparatus under heterogeneous data conditions, including:

the dividing module is used for dividing the edge computing equipment into a plurality of data identity clusters according to the data distribution similarity of the edge computing equipment;

A selecting module, configured to select an edge computing device from the data homography cluster as a cluster head of the data homography cluster;

The receiving module is used for receiving the cluster model parameter aggregation result of the data homopolar clusters uploaded by the cluster head; the cluster head is a root node of the cluster tree-shaped aggregation network, a child node in the cluster tree-shaped aggregation network sends model parameters to a corresponding father node, and model parameter aggregation is carried out with the local model parameters of the father node;

The aggregation module is used for aggregating cluster model parameter aggregation results of the data homopolar clusters to obtain a second-layer model parameter aggregation result, and when an image processing model taking the second-layer model parameter aggregation result as a model parameter converges, the image processing model is issued to the edge computing devices so that the edge computing devices can process images by using the image processing model.

In order to solve the technical problem, the invention also provides a federal learning device under the heterogeneous condition of data, which comprises:

The dividing unit is used for dividing the edge computing equipment into a plurality of data identity clusters according to the data distribution similarity of the edge computing equipment;

A selecting unit, configured to select an edge computing device from the data homography cluster as a cluster head of the data homography cluster;

The receiving unit is used for receiving the cluster model parameter aggregation result of the data homopolar clusters uploaded by the cluster head; the cluster head is a root node of the cluster tree-shaped aggregation network, a child node in the cluster tree-shaped aggregation network sends model parameters to a corresponding father node, and model parameter aggregation is carried out with the local model parameters of the father node;

The aggregation unit is used for aggregating cluster model parameter aggregation results of the data homopolar clusters to obtain a second-layer model parameter aggregation result, and issuing the global model to the edge computing equipment when the global model with the second-layer model parameter aggregation result as a model parameter converges.

In order to solve the technical problem, the present invention further provides an apparatus, including:

a memory for storing a computer program;

and a processor for implementing the steps of the image processing method under the data heterogeneous condition or the step of the federal learning method under the data heterogeneous condition when the computer program is executed.

In order to solve the technical problem, the present invention further provides an image processing system under heterogeneous data conditions, including:

The edge computing equipment is used for training an image processing model by using the local data to obtain local model parameters, and processing an image by using an image processing module issued by the edge cloud server;

The edge cloud server is used for dividing the edge computing equipment into a plurality of data identity clusters according to the data distribution similarity of the edge computing equipment; selecting edge computing equipment from the data homography cluster as a cluster head of the data homography cluster; receiving an intra-cluster model parameter aggregation result of the data homopolar cluster uploaded by the cluster head; the cluster head is a root node of the cluster tree-shaped aggregation network, a child node in the cluster tree-shaped aggregation network sends model parameters to a corresponding father node, and model parameter aggregation is carried out with the local model parameters of the father node; and aggregating cluster model parameter aggregation results of the data homopolar clusters to obtain a second-layer model parameter aggregation result, and when an image processing model taking the second-layer model parameter aggregation result as a model parameter converges, issuing the image processing model to each edge computing device so that the edge computing device processes an image by using the image processing model.

In order to solve the technical problem, the invention also provides a federal learning system under the heterogeneous condition of data, which comprises:

the edge computing equipment is used for training a model by using the local data to obtain local model parameters;

The edge cloud server is used for dividing the edge computing equipment into a plurality of data identity clusters according to the data distribution similarity of the edge computing equipment; selecting edge computing equipment from the data homography cluster as a cluster head of the data homography cluster; receiving an intra-cluster model parameter aggregation result of the data homopolar cluster uploaded by the cluster head; the cluster head is a root node of the cluster tree-shaped aggregation network, a child node in the cluster tree-shaped aggregation network sends model parameters to a corresponding father node, and model parameter aggregation is carried out with the local model parameters of the father node; and aggregating cluster model parameter aggregation results of the data homopolar clusters to obtain a second-layer model parameter aggregation result, and issuing the global model to the edge computing devices when the global model with the second-layer model parameter aggregation result as a model parameter converges.

To solve the above technical problem, the present invention further provides a medium on which a computer program is stored, the computer program implementing the steps of the image processing method under the data heterogeneous condition or the federal learning method under the data heterogeneous condition as described above when executed by a processor.

According to the image processing method under the heterogeneous data condition, the edge computing equipment is clustered according to the similarity of the data distribution, the edge computing equipment in the cluster has similar data distribution, and the model can better capture the characteristics of the data due to the similar data distribution, so that model training accuracy of each cluster can be improved through model parameter aggregation in the cluster, and the heterogeneous data problem is effectively solved. And the edge computing devices in the cluster perform model parameter aggregation according to the tree aggregation network in the cluster, and the child nodes only send model parameters to the corresponding parent nodes, namely, the edge computing devices in the lower layer only send model parameters to the corresponding edge computing devices in the upper layer, and do not send model parameters to other edge computing devices, so that the communication cost can be greatly reduced. Meanwhile, after the edge computing devices are clustered, if the edge computing device in a certain cluster fails or goes offline, other clusters can still normally perform local model training and model parameter aggregation, so that the overall fault tolerance and expandability of the federal training system can be improved. The edge computing equipment and the edge cloud server conduct two-layer model parameter aggregation in the federal learning process to obtain an accurate and reliable image processing model, and finally the edge computing equipment uses the accurate and reliable image processing model to conduct image processing, so that the accuracy and reliability of image processing can be improved.

According to the federation learning method under the heterogeneous data condition, the edge computing equipment is clustered according to the similarity of the data distribution, the edge computing equipment in the cluster has similar data distribution, and the model can better capture the characteristics of the data due to the similar data distribution, so that model training accuracy of each cluster can be improved through model parameter aggregation in the cluster, and the heterogeneous data problem is effectively solved. And the edge computing devices in the cluster perform model parameter aggregation according to the tree aggregation network in the cluster, and the child nodes only send model parameters to the corresponding parent nodes, namely, the edge computing devices in the lower layer only send model parameters to the corresponding edge computing devices in the upper layer, and do not send model parameters to other edge computing devices, so that the communication cost can be greatly reduced. Meanwhile, after the edge computing devices are clustered, if the edge computing device in a certain cluster fails or goes offline, other clusters can still normally perform local model training and model parameter aggregation, so that the overall fault tolerance and expandability of the federal training system can be improved.

The image processing device, the federal learning equipment, the federal learning system and the federal learning medium under the heterogeneous data conditions have the technical effects.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required in the prior art and the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic flow chart of a federal learning method under heterogeneous data conditions according to an embodiment of the present invention;

FIG. 2 is a diagram of a weighted undirected graph according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of a clustering result provided in an embodiment of the present invention;

Fig. 4 is a schematic diagram of an intra-cluster tree aggregation network according to an embodiment of the present invention;

FIG. 5 is a schematic diagram of a federal learning device under heterogeneous data conditions according to an embodiment of the present invention;

Fig. 6 is a schematic flow chart of an image processing method under heterogeneous data conditions according to an embodiment of the present invention;

Fig. 7 is a schematic diagram of an image processing apparatus under heterogeneous data conditions according to an embodiment of the present invention.

Detailed Description

The invention provides an image processing method, a federal learning method, a device, a system, equipment and a medium under the heterogeneous condition of data, which can solve the problem of data isomerism in federal learning, reduce communication overhead and improve the accuracy and reliability of image processing.

For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

Referring to fig. 1, fig. 1 is a flow chart of a federal learning method under heterogeneous data conditions according to an embodiment of the present invention, and referring to fig. 1, the method includes:

s101: dividing the edge computing equipment into a plurality of data identity clusters according to the data distribution similarity of the edge computing equipment;

The data distribution of edge computing devices within the same data-like cluster is similar.

According to the embodiment, a weighted undirected graph is constructed according to the data distribution similarity of the edge computing equipment, and the edge computing equipment is divided into a plurality of data identity clusters according to the weighted undirected graph. The value of the connecting edge between two edge computing devices in the weighted undirected graph is the value of the similarity of the data distribution of the two edge computing devices.

Wherein in some embodiments, constructing the weighted undirected graph according to the data distribution similarity of each edge computing device comprises:

Each edge computing device performs model training based on the local data to obtain a local model. The edge cloud server searches public data from the public network and builds a test data set based on the public data. The edge cloud server sends the test data set to the respective edge computing device. The edge computing device stores the test data set, and uses the local model obtained based on the local data training to infer the test data set, so as to obtain an inference result. And uploading the reasoning result to the edge cloud server by each edge computing device. And the edge cloud server calculates the similarity of each reasoning result and constructs a weighted undirected graph of each edge computing device according to the similarity of the reasoning results.

The edge cloud server can calculate the similarity of each reasoning result by adopting a vector similarity calculation method. The edge cloud server calculates the similarity of each reasoning result by using a Jaccard similarity coefficient calculation method. Jaccard similarity coefficient calculation can be used for calculating the similarity between sets and also can be used for calculating the similarity of binary vectors. For two binary vectors A and B, the calculation formula of the Jaccard similarity coefficient calculation is as follows: similarity= |a n b|/|a u b|. Where A.u.B represents the intersection of vectors A and B and A.u.B represents the union of vectors A and B.

Assume that the inference result of edge computing device C is a binary vector [1,0, … …,1,0], and the inference result of edge computing device D is a binary vector [0,1, 0, … …,1,0]. The binary vectors 1,0, … …,1,0] and binary vector [0,1, 0, … …,1,0].

In the embodiment, the similarity of data distribution among edge computing devices is represented by the similarity of the reasoning results of the edge computing devices on the same test data set, the more similar the reasoning results of the edge computing devices for reasoning the test data set by using the local model, the more similar the local model obtained by the edge computing devices through local data training, and the more similar the local model obtained by the edge computing devices through local data training, the more similar the data distribution of the edge computing devices. By using the data-based homopolar clustering method provided by the embodiment, the edge computing device only needs to upload the reasoning result, and does not need to upload local data, so that the privacy of the local data can be ensured.

In some embodiments, constructing the weighted undirected graph based on the similarity of the inference results comprises:

If the similarity of the reasoning results of the two edge computing devices is greater than a preset threshold, a connection relationship between the two edge computing devices is established, and the value of the connection edge between the two edge computing devices is equal to the value of the similarity of the reasoning results between the two edge computing devices. If the similarity of the reasoning results of the two edge computing devices is not greater than a preset threshold, the connection relationship between the two edge computing devices is not established. For example, the preset threshold is set to 0.7, and the weighted undirected graph shown in fig. 2 is constructed according to the preset threshold, and in fig. 2, the devices 1 to 6 represent six edge computing devices, and the numerical values on the connection edges of the two devices are the values of the similarity of the prediction results of the two devices.

Judging whether an iteration update stop condition is met;

The labels of the edge computing devices in the weighted undirected graph are initialized to different labels. And the edge cloud server carries out iterative updating on the labels of the edge computing devices. The process of one iteration update comprises the following steps: comparing the sizes of the connecting edges with the set threshold, and setting the labels of the edge computing devices connected with the connecting edges with the values larger than the set threshold as the same labels. And for the edge computing equipment with the value of each connected connection edge not larger than the set threshold value, further counting the occurrence times of the labels of the neighbor edge computing equipment of the edge computing equipment, and selecting the label with the largest occurrence times in the neighbor edge computing equipment as the label of the edge computing equipment. Edge computing devices with connection relationships in the weighted undirected graph are neighbors of each other. When the iterative update stop condition is satisfied, the edge computing devices with the same label are divided into the same data-like clusters, and the same data-like clusters are the sets of the edge computing devices with the same label.

For example, referring to the weighted undirected graph shown in fig. 2, assuming that the threshold is set to 0.9 and the value of the connection side of the device 1 and the device 2 is 0.94>0.9, the tags of the device 1 and the device 2 are set to the same tag, for example, to the tag a. The value of the connection side of the device 3 and the device 4 is 0.91>0.9, and thus the tags of the device 3 and the device 4 are set to the same tag, for example, to the tag B. The value of the connection edge of device 5 and device 4 and the value of the connection edge of device 5 and device 3 are not greater than 0.9, thus counting the labels of the neighbor edge computing devices (device 3 and device 4) of device 5. Since the labels of the device 3 and the device 4 are label B, and the device 5 has no other neighboring edge computing device, the label with the largest occurrence number in the neighboring edge computing device of the device 5 is label B (two occurrences), so that label B is used as the label of the device 5. The value of the connection edge of device 6 with device 4 and the value of the connection edge of device 6 with device 3 are no greater than 0.9, thus counting the labels of the neighboring edge computing devices (device 3 and device 4) of device 6. Since the labels of the device 3 and the device 4 are label B, and the device 6 has no other neighboring edge computing device, the label with the largest occurrence number in the neighboring edge computing device of the device 6 is label B (two occurrences), so that label B is used as the label of the device 6.

After the iteration update stop condition is met, the labels of the equipment 1 and the equipment 2 are the label A, and the labels of the equipment 3 and the equipment 6 are the label B, so that the equipment 1 and the equipment 2 are divided into the same data like clusters, and the equipment 3 and the equipment 6 are divided into the same data like clusters, and a clustering result shown in the figure 3 is obtained.

In some embodiments, the determining whether the iterative update stop condition is satisfied comprises:

After each iteration update is completed, calculating the variation of the labels of the edge computing devices after the iteration update and the labels of the edge computing devices after the previous iteration update;

comparing the variation with a preset value;

And after each iteration update, the edge cloud server calculates the change quantity of the labels of the edge computing devices after the iteration update and the labels of the edge computing devices after the previous iteration update. If the variation is smaller than the preset value, namely the label is basically stable, no significant variation occurs any more, at the moment, the algorithm is considered to be converged, and iterative updating is stopped. If the variation is not less than the preset value, namely the label is unstable, the label is still changed remarkably, and at the moment, the iterative updating is continued.

After clustering, the edge cloud server sends the device number of each edge computing device in the partitioned data homography cluster to each edge computing device in the data homography cluster so that the edge computing device can communicate with other edge cloud servers in the same cluster according to the device number.

S102: selecting edge computing equipment from the data homography cluster as a cluster head of the data homography cluster;

the cluster head is responsible for uploading cluster model parameter aggregation results of the cluster where the cluster head is located to the edge cloud server.

In some embodiments, the selecting an edge computing device from the data-like clusters as a cluster head of the data-like clusters comprises:

and selecting the edge computing equipment with the closest distance to other edge computing equipment or the maximum communication rate with other edge computing equipment in the data identity cluster as a cluster head of the data identity cluster.

In this embodiment, the cluster head is selected by using the principle of maximum communication rate or closest communication distance as the selection principle, and the edge computing device with the closest distance from other edge computing devices or the maximum communication rate with other edge computing devices in the cluster is specifically selected as the cluster head, so that the communication distance and delay can be reduced, and the communication efficiency can be improved. After the cluster head is selected, the edge cloud server sends the device number of the cluster head to edge computing devices in the cluster.

S103: receiving an intra-cluster model parameter aggregation result of the data homopolar cluster uploaded by the cluster head; the cluster-in-cluster model parameter aggregation result is a first-layer model parameter aggregation result obtained by each edge computing device in the data-like cluster according to the cluster-in-cluster tree aggregation network aggregation local model parameters, each edge computing device in the data-like cluster is constructed according to a communication optimal strategy to obtain the cluster-in-cluster tree aggregation network, the cluster head is a root node of the cluster-in-cluster tree aggregation network, a child node in the cluster-in-cluster tree aggregation network sends model parameters to a father node, and model parameter aggregation is carried out with the local model parameters of the father node.

Referring to fig. 3, the intra-cluster tree aggregation network includes several layers, each edge computing device being a node in the intra-cluster tree aggregation network. The cluster head is positioned at the layer 1 and is a root node of the tree-shaped aggregation network in the cluster. The child node of the cluster head (layer 1 cluster head shown in fig. 3) is located at the second layer; the child nodes of the cluster head (layer 2 cluster head shown in fig. 3) are located at the third layer; and so on. The layer 1 cluster head is a parent node of the layer 2 cluster head, and the layer 1 cluster head is also a child node of the cluster head. Similarly, the layer 2 cluster head is a parent node of the layer 3 cluster head, and the layer 2 cluster head is also a child node of the layer 1 cluster head. And so on. The layer 1 cluster head sends model parameters only to the cluster head, and the layer 2 cluster head sends model parameters only to the layer 1 cluster head, and so on.

The aggregation mode of the cluster model parameters is as follows: the edge computing device of the N layer (the last layer), namely the leaf node, uploads the local model parameters obtained by self training based on the local data to the father node of the edge computing device of the N-1 layer, the edge computing device of the N-1 layer conducts parameter aggregation on the local model parameters uploaded by the child nodes of the edge computing device and the local model parameters of the edge computing device, and the parameter aggregation result is uploaded to the father node of the edge computing device of the N-2 layer, and the like, and finally the cluster head conducts aggregation on the parameter aggregation result uploaded by the child nodes of the edge computing device and the local model parameters of the edge computing device of the N-1 layer to obtain the cluster model parameter aggregation result.

The mode of obtaining the parameter aggregation result by aggregating the local model parameters by the non-leaf nodes and the model parameters uploaded by the child nodes of the non-leaf nodes can be as follows: the non-leaf node averages the model parameters uploaded by its child nodes with its own local model parameters. For example, the cluster head averages the parameter aggregation result uploaded by the child node thereof with the local model parameter thereof to obtain the cluster model parameter aggregation result.

The tree-shaped aggregation network in the cluster is constructed by all edge computing devices in the data homopolar cluster according to the optimal communication strategy.

The edge cloud server selects one cluster head serving as the data homography cluster from edge computing devices of the data homography cluster. The cluster heads of the data-like cluster select a preset number of edge computing devices with the maximum data transmission rate when communicating with the cluster heads from the rest of edge computing devices in the cluster as layer 1 sub-nodes, namely layer 1 cluster heads, the layer 1 cluster heads select the preset number of edge computing devices with the maximum data transmission rate when communicating with the layer 1 cluster heads from the rest of edge computing devices in the cluster as second layer sub-nodes, namely layer 2 cluster heads, and so on until all the edge computing devices in the cluster acquire the serial numbers of the tree-shaped aggregation network in the cluster, namely until each edge computing device in the cluster knows to which edge computing device the model parameters are uploaded. The cluster head only receives the model parameters of the layer 1 cluster head and does not receive the model parameters of other edge computing devices in the cluster, the layer 1 cluster head only receives the model parameters of the layer 2 cluster head and does not receive the model parameters of other edge computing devices in the cluster, and accordingly, the layer N-1 cluster head only receives the model parameters of the layer N cluster head and does not receive the model parameters of other edge computing devices in the cluster.

In the process of constructing the tree-shaped aggregation network in the cluster, the cluster head communicates with all edge computing devices in the cluster through a wireless link, and the communicated data can be model parameters. The cluster head collects model parameters uploaded by all edge computing devices in the cluster and calculates the sending rate of each edge computing device. The calculation mode of the sending rate can be as follows: transmission rate = total communication data amount/(total transmission time + total reception time).

And the cluster head selects a preset number of edge computing devices with the maximum transmission rate as the layer 1 cluster head according to the transmission rate of each edge computing device in the cluster. After receiving an instruction for designating the cluster head, the layer 1 cluster head communicates with other edge computing devices in the cluster except the cluster head and the layer 1 cluster head, calculates the sending rate of the other edge computing devices in the cluster, and selects a preset number of edge computing devices with the largest sending rate as the layer 2 cluster head; after receiving the designation instruction of the layer 1 cluster head, the layer 2 cluster head communicates with other edge computing devices in the cluster except the cluster head, the layer 1 cluster head and the layer 2 cluster head, calculates the sending rate of the other edge computing devices in the cluster, and selects a preset number of edge computing devices with the largest sending rate as the layer 3 cluster head; and so on.

Illustratively, the cluster head selects 2 edge computing devices with the largest sending rate as the layer 1 cluster head according to the sending rate of each edge computing device in the cluster. After receiving an instruction for designating the cluster head, the layer 1 cluster head communicates with other edge computing devices in the cluster except the cluster head and the layer 1 cluster head, calculates the sending rate of the other edge computing devices in the cluster, and selects 2 edge computing devices with the largest sending rate as layer 2 cluster heads; after receiving the designation instruction of the layer 1 cluster head, the layer 2 cluster head communicates with other edge computing devices in the clusters except the cluster head, the layer 1 cluster head and the layer 2 cluster head, calculates the sending rate of the other edge computing devices in the cluster, and selects 2 edge computing devices with the largest sending rate as the layer 3 cluster head; and so on.

S104: and aggregating cluster model parameter aggregation results of the data homopolar clusters to obtain a second-layer model parameter aggregation result, and issuing the global model to the edge computing devices when the global model with the second-layer model parameter aggregation result as a model parameter converges.

And the edge cloud server aggregates cluster model parameter aggregation results of all the data homopolar clusters to obtain global model parameters, namely second-layer model parameter aggregation results. And when the global model taking the second-layer model parameter aggregation result as the model parameter converges, issuing the converged global model to each edge computing device. The edge computing device uses the global model to perform corresponding detection, analysis, etc. Illustratively, if the global model obtained by federal learning is an image processing model, the edge computing device uses the global model for image processing. If the global model obtained by federal learning is a network attack detection model, the edge computing device uses the global model to detect network attacks.

According to the embodiment, the corresponding weight coefficient is set for each data like-nature cluster, and the cluster model parameter aggregation result uploaded by the cluster heads of each data like-nature cluster is subjected to weighted aggregation according to the weight coefficient of each data like-nature cluster, so that the model can better reflect real global data distribution, and the generalization capability of the model is improved.

After broadcasting the global model obtained by the previous training round to all edge computing devices by the edge cloud server, each edge computing device tests by using local data, and the testing method can be as follows: the edge computing device randomly selects 1/10 of the local data as test data to test the global model. And after the test is completed, the edge computing equipment sends the test result of the equipment to the edge cloud server. The edge cloud server traverses each data identity cluster, and performs statistics on the local data testing precision of each data identity cluster, wherein the statistical method is an average value of the local data testing precision of each edge computing device in the cluster. If the local data test precision of a certain data homography cluster is low, the fact that the data training of the data homography cluster is insufficient or the global model is not suitable for the data of the data homography cluster is indicated, and the weight coefficient of the data homography cluster is increased.

According to the embodiment, the weight coefficient of the data homography cluster is determined according to the local data testing precision of the data homography cluster, if the local data testing precision of the data homography cluster is different, the weight coefficient of the data homography cluster is different, and the edge computing equipment with large data quantity or good quality can be prevented from being ignored by endowing different data homography clusters with different weight coefficients. Meanwhile, the weight coefficient of the data homography cluster is determined according to the local data testing precision of the data homography cluster, and the weight coefficient of the data homography cluster is dynamically adjusted, so that the model can be prevented from being transitionally adapted to specific edge computing equipment, and the generalization capability of the model is improved.

Wherein, in some embodiments, determining the weight coefficient of the data-like cluster according to the local data testing accuracy of the data-like cluster comprises:

According to Obtaining a weight coefficient of the data homopolar cluster; k _c represents the weight coefficient of the c-th data-like cluster, and μ represents the local data test accuracy of the c-th data-like cluster. /(I)E is a base number and-mu is an index.

Assuming that all edge computing devices are partitioned into C data-like clusters, by collectionMeaning that the kth data-like cluster S _k contains/>And an edge computing device. In the federal learning system, the edge computing device i trains to obtain a local model based on its own local data set D _i. The local empirical loss function of the data distribution at the edge computing device i is: /(I); Wherein/>For local model parameters,/>For data samples participating in iterative training,/>For quantifying data samples/>, as a sample loss functionPrediction error on the same.

The main goal of federal learning is to optimize global model parameters to minimize the global loss functions associated with all edge computing devices while ensuring highest aggregation efficiency. The global loss function is:

。

The federal learning model training process is divided into three stages of local model training, first-layer model parameter aggregation, i.e. cluster model parameter aggregation, and second-layer model parameter aggregation, i.e. global aggregation, which are combined into one training round.

Local model training: each edge computing device updates the local model. In some embodiments, the edge computing devices within the cluster update the local model using an SGD (Stochastic GRADIENT DESCENT, random gradient descent) algorithm. In the training t-th round process, the first iterative update process is expressed as:

; wherein/> For the local model parameters updated for the first iteration of the t-th round,/>The learning rate of the first iteration of the t-th round.

Intra-cluster model parameter aggregation: edge computing device within a clusterAnd after the iterative updating, performing once cluster model parameter aggregation. The polymerization mode can be as follows: and the layer k cluster head receives the model parameters uploaded by the child nodes of the layer k cluster head, averages the model parameters with the local model parameters of the layer k cluster head, and obtains a cluster model parameter aggregation result by receiving the model parameters uploaded by the child nodes of the layer k cluster head and averaging the local model parameters of the layer k cluster head. Wherein edge computing devices within a cluster are every/>Training for the second time, and storing the model. And when receiving an aggregation instruction sent by the upper-layer cluster head, sending the stored model to the upper-layer cluster head connected with the stored model.

Global aggregation: and after the cluster model parameters are aggregated for tau times by all the data homopolar clusters, the edge cloud server executes global aggregation in a synchronous mode. The edge cloud server receives model parameters uploaded by the C cluster heads, and updates global model parameters in a parameter averaging mode as follows:

；/> Model parameters for the t training round of cluster c,/> And updating global model parameters for the t training round.

After the global model parameters are updated, the edge cloud server broadcasts the global model to all edge computing devices, and each edge computing device tests by using local data, wherein the testing method can be as follows: the edge computing device randomly selects 1/10 of the local data as test data to test the global model. And after the test is completed, the edge computing equipment sends the test result of the equipment to the edge cloud server. The edge cloud server traverses each data identity cluster, and performs statistics on the local data testing precision of each data identity cluster, wherein the statistical method is an average value of the local data testing precision of each edge computing device in the cluster.

The global aggregation for the next training round is expressed as:

；/> model parameters of the (t+1) th training round of the (c) th cluster,/> And training global model parameters updated for the t+1st round.

K _c represents the weight coefficient of the c-th data-like cluster, and μ represents the local data test accuracy of the c-th data-like cluster.

In some embodiments, further comprising:

After the model training of the h training round is completed, the edge computing equipment is re-executed to carry out the data homography cluster, and the model training (including local model training, cluster model parameter aggregation and global aggregation) is carried out again after the data homography cluster is re-divided. Repeating the steps of dividing the data homography clusters and carrying out model training after the data homography clusters are divided until the global model converges.

In summary, according to the federal learning method under the heterogeneous data condition provided by the invention, the edge computing devices are clustered according to the similarity of the data distribution, the edge computing devices in the clusters have similar data distribution, and the model can better capture the characteristics of the data due to the similar data distribution, so that model training accuracy of each cluster can be improved by model parameter aggregation in the clusters, and the heterogeneous data problem is effectively solved. And the edge computing devices in the cluster perform model parameter aggregation according to the tree aggregation network in the cluster, and the child nodes only send model parameters to the corresponding parent nodes, namely, the edge computing devices in the lower layer only send model parameters to the corresponding edge computing devices in the upper layer, and do not send model parameters to other edge computing devices, so that the communication cost can be greatly reduced. Meanwhile, after the edge computing devices are clustered, if the edge computing device in a certain cluster fails or goes offline, other clusters can still normally perform local model training and model parameter aggregation, so that the overall fault tolerance and expandability of the federal training system can be improved.

The invention also provides a federal learning device under the heterogeneous condition of data, and the device can be correspondingly referred to the method. Referring to fig. 5, fig. 5 is a schematic diagram of a federal learning device under heterogeneous data conditions according to an embodiment of the present invention, and in combination with fig. 5, the device includes:

a dividing unit 11, configured to divide an edge computing device into a plurality of data-like clusters according to data distribution similarity of the edge computing device;

a selection unit 12 for selecting an edge computing device from the data-like clusters as a cluster head of the data-like clusters;

The receiving unit 13 is used for receiving the cluster model parameter aggregation result of the data homopolar cluster uploaded by the cluster head; the cluster head is a root node of the cluster tree-shaped aggregation network, a child node in the cluster tree-shaped aggregation network sends model parameters to a corresponding father node, and model parameter aggregation is carried out with the local model parameters of the father node;

and the aggregation unit 14 is configured to aggregate intra-cluster model parameter aggregation results of the data-like clusters to obtain a second-layer model parameter aggregation result, and send the global model to each edge computing device when the global model with the second-layer model parameter aggregation result as a model parameter converges.

In some embodiments, the selection unit 12 is specifically configured to:

In some embodiments, the aggregation unit 14 includes:

The determining subunit is used for determining the weight coefficient of each data homopolar cluster of the current training round;

And the aggregation subunit is used for carrying out weighted aggregation on the in-cluster model parameters of each data homopolar cluster according to the weight coefficient of each data homopolar cluster to obtain the second-layer model parameter aggregation result of the current training round.

In some embodiments, the determining subunit is specifically configured to:

In some embodiments, further comprising:

and the repeating unit is used for clustering the edge computing equipment again according to the data distribution similarity of the edge computing equipment after model training of the preset training round is completed, and carrying out model training of the preset training round again until the global model converges.

In some embodiments, the dividing unit 11 includes:

A construction subunit, configured to construct a weighted undirected graph according to the data distribution similarity of each edge computing device; the value of the connecting edge between two edge computing devices in the weighted undirected graph is the value of the data distribution similarity of the two edge computing devices;

And the dividing subunit is used for dividing the edge computing equipment into a plurality of data identity clusters according to the weighted undirected graph.

In some embodiments, constructing the subunit comprises:

the searching subunit is used for searching public data from the public network and constructing a test data set based on the public data;

A transmitting subunit, configured to issue the test data set to each of the edge computing devices, so that each of the edge computing devices uses a local model to infer the test data set;

The computing subunit is used for receiving the reasoning results uploaded by the edge computing devices and computing the similarity of the reasoning results;

and the weighted undirected graph construction subunit is used for constructing the weighted undirected graph according to the similarity of the reasoning results.

In some embodiments, the weighted undirected graph construction subunit includes:

the comparison subunit is used for comparing the similarity of the reasoning results of the edge computing devices with a preset threshold value;

And the establishing subunit is used for establishing a connection relation between the two edge computing devices if the similarity of the reasoning results of the two edge computing devices is greater than a preset threshold value, wherein the value of the similarity of the reasoning results of the two edge computing devices is used as the value of the connection edge between the two edge computing devices.

In some embodiments, dividing the subunits comprises:

an initializing subunit, configured to initialize labels of each edge computing device in the weighted undirected graph;

An iteration updating subunit, configured to iteratively update the labels of the edge computing devices; wherein iteratively updating the labels of the edge computing devices comprises: setting the labels of the edge computing devices connected with the connecting edges with values larger than the set threshold value as the same labels; counting the number of times of occurrence of the label of the neighbor edge computing device of the target edge computing device, and selecting the label with the largest number of occurrence in the neighbor edge computing device as the label of the target edge computing device; the target edge computing device is an edge computing device with the value of each connected edge not larger than a set threshold value;

a judging subunit, configured to judge whether an iteration update stop condition is satisfied;

And the clustering subunit is used for dividing the edge computing devices with the same label into the same data identity cluster if the iteration update stop condition is met.

In some embodiments, the judging subunit is specifically configured to:

comparing the variation with a preset value;

The invention also provides a federal learning system under the heterogeneous condition of data, which comprises: an edge computing device and an edge cloud server.

For the description of the system provided by the present invention, please refer to the above method embodiment, and the description of the present invention is omitted here.

Referring to fig. 6, fig. 6 is a flowchart of an image processing method under heterogeneous data conditions according to an embodiment of the present invention, and referring to fig. 6, the method includes:

s201: dividing the edge computing equipment into a plurality of data identity clusters according to the data distribution similarity of the edge computing equipment;

s202: selecting edge computing equipment from the data homography cluster as a cluster head of the data homography cluster;

S203: receiving an intra-cluster model parameter aggregation result of the data homopolar cluster uploaded by the cluster head; the cluster head is a root node of the cluster tree-shaped aggregation network, a child node in the cluster tree-shaped aggregation network sends model parameters to a corresponding father node, and model parameter aggregation is carried out with the local model parameters of the father node;

S204: and aggregating cluster model parameter aggregation results of the data homopolar clusters to obtain a second-layer model parameter aggregation result, and when an image processing model taking the second-layer model parameter aggregation result as a model parameter converges, issuing the image processing model to each edge computing device so that the edge computing device processes an image by using the image processing model.

In some embodiments, further comprising:

Judging whether an iteration update stop condition is met;

comparing the variation with a preset value;

In summary, according to the image processing method under the heterogeneous data condition provided by the invention, the edge computing devices are clustered according to the similarity of the data distribution, the edge computing devices in the clusters have similar data distribution, and the model can better capture the characteristics of the data due to the similar data distribution, so that model training accuracy of each cluster can be improved by the aggregation of model parameters in the clusters, and the heterogeneous data problem is effectively solved. And the edge computing devices in the cluster perform model parameter aggregation according to the tree aggregation network in the cluster, and the child nodes only send model parameters to the corresponding parent nodes, namely, the edge computing devices in the lower layer only send model parameters to the corresponding edge computing devices in the upper layer, and do not send model parameters to other edge computing devices, so that the communication cost can be greatly reduced. Meanwhile, after the edge computing devices are clustered, if the edge computing device in a certain cluster fails or goes offline, other clusters can still normally perform local model training and model parameter aggregation, so that the overall fault tolerance and expandability of the federal training system can be improved. The edge computing equipment and the edge cloud server conduct two-layer model parameter aggregation in the federal learning process to obtain an accurate and reliable image processing model, and finally the edge computing equipment uses the accurate and reliable image processing model to conduct image processing, so that the accuracy and reliability of image processing can be improved.

The invention also provides an image processing device under the heterogeneous condition of data, and the device can be referred to correspondingly with the method. Referring to fig. 7, fig. 7 is a schematic diagram of an image processing apparatus under heterogeneous data conditions according to an embodiment of the present invention, and in combination with fig. 7, the apparatus includes:

A dividing module 21, configured to divide an edge computing device into a plurality of data-like clusters according to data distribution similarity of the edge computing device;

a selection module 22, configured to select an edge computing device from the data-like clusters as a cluster head of the data-like clusters;

The receiving module 23 is configured to receive an intra-cluster model parameter aggregation result of the data like-character cluster uploaded by the cluster head; the cluster head is a root node of the cluster tree-shaped aggregation network, a child node in the cluster tree-shaped aggregation network sends model parameters to a corresponding father node, and model parameter aggregation is carried out with the local model parameters of the father node;

And the aggregation module 24 is configured to aggregate intra-cluster model parameter aggregation results of the data homopolar clusters to obtain a second-layer model parameter aggregation result, and when an image processing model using the second-layer model parameter aggregation result as a model parameter converges, issue the image processing model to each edge computing device, so that the edge computing device processes an image using the image processing model.

In some embodiments, the selection module 22 is specifically configured to:

In some embodiments, aggregation module 24 includes:

the determining submodule is used for determining the weight coefficient of each data homography cluster in the current training round;

And the aggregation sub-module is used for carrying out weighted aggregation on the in-cluster model parameters of each data homopolar cluster according to the weight coefficient of each data homopolar cluster to obtain the second-layer model parameter aggregation result of the current training round.

In some embodiments, the determination submodule is specifically configured to:

In some embodiments, further comprising:

And the repeating module is used for clustering the edge computing equipment again according to the data distribution similarity of the edge computing equipment after model training of the preset training round is completed, and carrying out model training of the preset training round again until the global model converges.

In some embodiments, the partitioning module 21 includes:

the construction submodule is used for constructing a weighted undirected graph according to the data distribution similarity of the edge computing devices; the value of the connecting edge between two edge computing devices in the weighted undirected graph is the value of the data distribution similarity of the two edge computing devices;

and the dividing sub-module is used for dividing the edge computing equipment into a plurality of data identity clusters according to the weighted undirected graph.

In some embodiments, the building sub-module comprises:

The searching sub-module is used for searching public data from the public network and constructing a test data set based on the public data;

a transmitting sub-module, configured to issue the test data set to each of the edge computing devices, so that each of the edge computing devices uses a local model to infer the test data set;

the computing sub-module is used for receiving the reasoning results uploaded by the edge computing devices and computing the similarity of the reasoning results;

And the weighted undirected graph construction sub-module is used for constructing the weighted undirected graph according to the similarity of the reasoning results.

In some embodiments, the weighted undirected graph construction submodule includes:

the comparison sub-module is used for comparing the similarity of the reasoning results of the edge computing devices with a preset threshold value;

And the establishing sub-module is used for establishing a connection relation between the two edge computing devices if the similarity of the reasoning results of the two edge computing devices is greater than a preset threshold value, wherein the value of the similarity of the reasoning results of the two edge computing devices is used as the value of the connection edge between the two edge computing devices.

In some embodiments, the partitioning submodule includes:

An initialization sub-module, configured to initialize labels of each edge computing device in the weighted undirected graph;

an iteration update sub-module, configured to iteratively update labels of the edge computing devices; wherein iteratively updating the labels of the edge computing devices comprises: setting the labels of the edge computing devices connected with the connecting edges with values larger than the set threshold value as the same labels; counting the number of times of occurrence of the label of the neighbor edge computing device of the target edge computing device, and selecting the label with the largest number of occurrence in the neighbor edge computing device as the label of the target edge computing device; the target edge computing device is an edge computing device with the value of each connected edge not larger than a set threshold value;

the judging submodule is used for judging whether iteration update stopping conditions are met or not;

And the clustering sub-module is used for dividing the edge computing devices with the same label into the same data identity cluster if the iteration update stop condition is met.

In some embodiments, the judging submodule is specifically configured to:

comparing the variation with a preset value;

The invention also provides an apparatus comprising a memory and a processor.

A memory for storing a computer program;

A processor for executing a computer program to perform the steps of any embodiment of the method as federal learning or to perform the steps of any embodiment of the method as image processing.

For the description of the apparatus provided by the present invention, refer to the above method embodiment, and the description of the present invention is omitted herein.

The present invention also provides a medium having stored thereon a computer program which, when executed by a processor, performs steps such as any embodiment of the federal learning method or performs steps such as any embodiment of the image processing method.

The medium may include: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

For the description of the medium provided by the present invention, refer to the above method embodiments, and the description of the present invention is omitted here.

In the description, each embodiment is described in a progressive manner, and each embodiment is mainly described by the differences from other embodiments, so that the same similar parts among the embodiments are mutually referred. The apparatus, device and medium disclosed in the embodiments are relatively simple to describe, and the relevant points refer to the description of the method section since they correspond to the methods disclosed in the embodiments.

Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative elements and steps are described above generally in terms of functionality in order to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. The software modules may be disposed in Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of medium known in the art.

The federal learning method, the image processing method and the device under the heterogeneous data condition provided by the invention are described in detail above. The principles and embodiments of the present invention have been described herein with reference to specific examples, the description of which is intended only to facilitate an understanding of the method of the present invention and its core ideas. It should be noted that it will be apparent to those skilled in the art that the present invention may be modified and practiced without departing from the spirit of the invention.

Claims

1. An image processing method under heterogeneous data conditions, comprising:

selecting edge computing equipment which is closest to other edge computing equipment in the cluster or has the largest communication rate with other edge computing equipment in the cluster in the data homography cluster as a cluster head of the data homography cluster;

2. The image processing method according to claim 1, wherein the cluster head selects a preset number of edge computing devices from other edge computing devices in the cluster, and the preset number of edge computing devices are used as layer 1 child nodes of the tree-shaped aggregation network in the cluster, and the layer 1 child nodes are edge computing devices with the maximum data transmission rate when communicating with the cluster head; selecting a preset number of edge computing devices from the rest edge computing devices in the cluster by the child nodes of the ith layer to serve as the child nodes of the (i+1) th layer of the tree-shaped aggregation network in the cluster; the (i+1) th layer child node is edge computing equipment with the maximum data transmission rate when communicating with the (i) th layer child node; and i takes a value from 1 until the intra-cluster tree-shaped aggregation network is constructed.

3. The image processing method according to claim 2, wherein aggregating intra-cluster model parameter aggregation results for each of the data homography clusters to obtain a second-layer model parameter aggregation result comprises:

4. The image processing method of claim 3, wherein determining the weight coefficient for each of the data-like clusters for the current training round comprises:

5. The image processing method of claim 4, wherein determining the weight coefficient of the data-like cluster according to the local data testing accuracy of the data-like cluster comprises:

6. The image processing method according to claim 5, wherein one training round includes three stages of local model training, first layer model parameter aggregation and second layer model parameter aggregation; after each edge computing device in each data like character cluster completes model parameter updating for a first preset time, first layer model parameter aggregation is carried out, and after each data like character cluster completes first layer model parameter aggregation for a second preset time, second layer model parameter aggregation is carried out.

7. The image processing method according to claim 6, characterized by further comprising:

8. The image processing method of claim 1, wherein dividing the edge computing device into a number of data-like clusters based on data distribution similarities of the edge computing device comprises:

9. The image processing method according to claim 8, wherein constructing a weighted undirected graph based on the data distribution similarity of each of the edge computing devices comprises:

10. The image processing method according to claim 9, wherein constructing the weighted undirected graph based on the similarity of the inference results comprises:

11. The image processing method of claim 10, wherein dividing the edge computing device into a number of data-like clusters according to the weighted undirected graph comprises:

Judging whether an iteration update stop condition is met;

12. The image processing method according to claim 11, wherein determining whether an iterative update stop condition is satisfied comprises:

comparing the variation with a preset value;

13. A federal learning method under heterogeneous data conditions, comprising:

Receiving an intra-cluster model parameter aggregation result of the data homopolar cluster uploaded by the cluster head; the cluster head is a root node of the cluster tree-shaped aggregation network, a child node of the cluster tree-shaped aggregation network sends model parameters to a father node, and model parameter aggregation is carried out with the local model parameters of the father node;

14. An image processing apparatus under heterogeneous data conditions, comprising:

The selecting module is used for selecting the edge computing device with the closest distance to other edge computing devices in the cluster or the maximum communication rate with other edge computing devices in the cluster in the data like cluster as the cluster head of the data like cluster;

15. A federal learning device under heterogeneous data conditions, comprising:

16. An apparatus, comprising:

a memory for storing a computer program;

A processor for implementing the steps of the image processing method under the data heterogeneous condition according to any one of claims 1 to 12 or the step of the federal learning method under the data heterogeneous condition according to claim 13 when executing the computer program.

17. An image processing system under heterogeneous data conditions, comprising:

The edge cloud server is used for dividing the edge computing equipment into a plurality of data identity clusters according to the data distribution similarity of the edge computing equipment; selecting edge computing equipment which is closest to other edge computing equipment in the cluster or has the largest communication rate with other edge computing equipment in the cluster in the data homography cluster as a cluster head of the data homography cluster; receiving an intra-cluster model parameter aggregation result of the data homopolar cluster uploaded by the cluster head; the cluster head is a root node of the cluster tree-shaped aggregation network, a child node in the cluster tree-shaped aggregation network sends model parameters to a corresponding father node, and model parameter aggregation is carried out with the local model parameters of the father node; and aggregating cluster model parameter aggregation results of the data homopolar clusters to obtain a second-layer model parameter aggregation result, and when an image processing model taking the second-layer model parameter aggregation result as a model parameter converges, issuing the image processing model to each edge computing device so that the edge computing device processes an image by using the image processing model.

18. A federal learning system under heterogeneous data conditions, comprising:

19. A medium having stored thereon a computer program which, when executed by a processor, implements the steps of the image processing method under heterogeneous conditions of data according to any one of claims 1 to 12 or the federal learning method under heterogeneous conditions of data according to claim 13.