CN115577360A - A Gradient-Independent Clustering Federated Learning Method and System - Google Patents

Abstract

The invention discloses a gradient-independent clustering federal learning method and a gradient-independent clustering federal learning system, wherein the method comprises the following steps: the client respectively calculates data distribution information and intersection similarity of the data distribution information and the intersection similarity and constructs intersection similarity vectors; and the server collects the intersection similarity vectors and constructs a similarity matrix, clusters the clients and executes model training, and selects the clients to form a verification committee after the server detects that the precision of the model is reduced and determines a malicious cluster, verifies and votes to exclude the malicious model and reserve a benign model. The server does not need to rely on gradient information of the client for clustering but carries out clustering according to intersection similarity among data distributions of the client, thereby avoiding the problem of gradient information leakage of the client, protecting the gradient safety of the client, enhancing the safety and reliability in the process of clustering federal learning and improving the training precision.

Description

A method and system for clustering federated learning that does not depend on gradient

technical field

The invention relates to the technical field of clustering federated learning of artificial intelligence, in particular to a gradient-independent clustering federated learning method and system.

Background technique

Although with the development of informatization, information is becoming more and more abundant, but information essentially exists in the form of isolated islands because they are highly sensitive. A very typical application field is the medical field. Data in the medical industry is very sensitive because it may involve important privacy of patients, and these data are usually kept by different hospitals. The focus of the data owned by each hospital may be different (for example, some hospitals are good at treating heart disease, and some hospitals are good at treating kidney, etc.), that is, there is a problem of non-independent and identical distribution of data. In recent years, federated learning has attracted attention in resolving the conflict between model training and data privacy protection. However, traditional federated learning cannot well solve the problem of non-independent and identical distribution of data between clients. In response to the above problems, the prior art proposes clustering federated learning, which uses gradients to measure the similarity of data distribution between clients and clusters them to solve the non-IID problem. However, recent studies have shown that customers' private information and even original training data can be recovered through gradients, and the gradient dimension tends to explode with the increase of model complexity. At the same time, existing clustering federated learning schemes cannot group clients with diverse data into multiple clusters, resulting in the inability to fully utilize the diverse data owned by some clients. Moreover, compared with federated learning, the cluster structure in clustering federated learning provides opportunities for malicious clients to conspire to gather in a cluster and poison the aggregated cluster models by initiating model poisoning attacks locally, resulting in failure of model training. . Therefore, how to protect the privacy of clients in clustering federated learning and make full use of the diversity and availability of client data will have a crucial impact on the development of the industry. At the same time, how to improve the detection efficiency of malicious models, reduce the detection overhead, and improve the security of the training process are also important issues that need to be solved.

Contents of the invention

The technical problem to be solved by the present invention is to provide a gradient-independent clustering federated learning method and system for the above-mentioned problems in the prior art. In the present invention, the server does not need to rely on the gradient information of the client for clustering but according The intersection similarity between data distributions is used for clustering, which avoids the gradient information leakage problem of the client, protects the gradient security of the client, enhances the security and reliability of the clustering federated learning process, and improves the training accuracy.

In order to solve the problems of the technologies described above, the technical solution adopted in the present invention is:

A clustering federated learning method that does not rely on gradients, including:

S1, the client calculates the data distribution information of its own label samples separately, obtains the intersection similarity between the data distribution information of itself and other clients, and constructs an intersection similarity vector;

S2, the server collects the intersection similarity vectors of each client and constructs a similarity matrix;

S3, the server clusters the clients based on the similarity matrix using a clustering method that guarantees diversity, and executes the model training step, and jumps to the next step when the server detects that the accuracy of the model has decreased;

S4, the server detects the malicious cluster, and after determining the malicious cluster, selects the client that has the most similar data distribution to the client in the malicious cluster and is not in the malicious cluster to form a verification committee;

S6, use the members of the verification committee to verify the models of the members in the malicious cluster and vote for benign models and malicious models, exclude the malicious models and keep the benign models.

Optionally, in step S1, the function expression for the client to calculate the data distribution information of its own label samples is:

以及

分别表示第1、2以及第i个客户端的单一标签的数据分布信息，且有任意第i个客户端的单一标签的数据分布信息的计算函数表达式为：In the above formula,

as well as

Represent the data distribution information of the single label of the 1st, 2nd and i-th clients respectively, and the calculation function expression of the data distribution information of a single label of any i-th client is:

表示第i个客户端的第i个索引的数据数量，idx_i表示标签i的索引，Q_max表示一个预先定义的任意标签数量的最大值，且任意一个标签的数量都不可以超过这个最大值，X_i表示第i个客户端构建的数据分布信息，j为第i个客户端的第j个索引的序号。in,

Indicates the data quantity of the i-th index of the i-th client, idx _i indicates the index of label i, Q _max indicates a predefined maximum number of any label, and the number of any label cannot exceed this maximum value, X _i represents the data distribution information constructed by the i-th client, and j is the serial number of the j-th index of the i-th client.

Optionally, the functional expression of the intersection similarity vector constructed in step S1 is:

In the above formula, ISM _i is the intersection similarity vector of the i-th client, ISM _i [1]~ISM _i [j] represent the data distribution similarity between the i-th client and the 1st-j clients, |X _i ∩X _j | represents the intersection of the data distribution between the i-th client and the j-th client, X _i represents the data distribution information constructed by the i-th client, and X _j represents the data distribution constructed by the j-th client information.

Optionally, the functional expression of the similarity matrix constructed in step S2 is:

In the above formula, M _sim is a similarity matrix, and any i-th row represents the intersection similarity vector formed by the data distribution similarity between the i-th client and the 1st to nth clients.

Optionally, in step S3, the server clustering the clients based on the similarity matrix using a clustering method that guarantees diversity includes: aggregating all clients whose intersection similarity is higher than the threshold α into a candidate cluster set and deduplication, so that The candidate cluster set contains all possible clustering results; in the candidate cluster set, calculate the weight of each candidate cluster, and use the greedy algorithm, each selection can make the candidate cluster with the smallest load be added to the final cluster set , until all clients are assigned to the final cluster set.

Optionally, the function expression for calculating the weight of each candidate cluster is:

In the above formula, Cost(S' _i ) represents the cost of calculating the candidate cluster S' _i , ID is the serial number of the client, and M _sim [i][ID] is the element of the i-th row ID column of the similarity matrix; The calculation function expression of the load is:

In the above formula, Payload represents the load, S'\I represents the candidate cluster set that has not been selected into the final cluster, S' represents the candidate cluster set, and I represents the number of the client that has been selected into the final cluster.

Optionally, the detection of malicious clusters by the server in step S4 means that the server detects the accuracy of each cluster in the final cluster set according to its own local data, and selects the cluster with the lowest accuracy as the malicious cluster.

Optionally, in step S6, using members of the verification committee to verify the models of the members in the malicious cluster and voting for benign models and malicious models includes: using members of the verification committee to verify the model of each member in the malicious cluster according to their own local data. Model accuracy, the model members whose accuracy is lower than the average are regarded as malicious models and voted, and finally all the voting results will be summed up, and the model with the number of votes higher than the average number of votes will be identified as a malicious model by the verification committee, otherwise it will be identified by the verification committee benign model.

In addition, the present invention also provides a gradient-independent clustering federated learning system, which includes multiple clients connected to each other, the clients include microprocessors and memories connected to each other, and the microprocessors are programmed or configured To implement the gradient-independent clustering federated learning method.

In addition, the present invention also provides a computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and the computer program is used to be programmed or configured by a microprocessor to perform the gradient-independent clustering Federated learning methods.

Compared with the prior art, the present invention mainly has the following advantages:

1. In the clustering process of the present invention, the server does not need to rely on the gradient information of the client for clustering, but performs clustering according to the intersection similarity between the data distributions of the clients, thus avoiding the leakage of the gradient information of the client , which protects the gradient security of the client, enhances the security and reliability of the clustering federated learning process, and improves the training accuracy.

2. In the process of clustering, the present invention innovatively allows the same client to appear in multiple clusters, so that the most suitable cluster can be found for each client, and the diversity of client data can be fully utilized , to enhance model accuracy.

3. The present invention is different from the pre-detection commonly used in the existing federated learning, and innovatively uses a post-event detection mechanism to detect malicious clusters existing in the system, allowing detection after the attack starts to save overhead, and further Improve system security.

Description of drawings

Fig. 1 is a schematic flow diagram of the basic process of the method of the embodiment of the present invention.

Fig. 2 is a schematic diagram of the principle of an embodiment of the present invention.

Fig. 3 is the test accuracy of the embodiment of the present invention on the MNIST data set.

Fig. 4 is the test accuracy of the embodiment of the present invention on the FMNIST data set.

Fig. 5 is the test accuracy of the embodiment of the present invention on the CIFAR10 data set.

Fig. 6 is a comparison of test accuracy of the embodiment of the present invention in three methods.

Fig. 7 shows the main overhead when calculating the intersection similarity according to the embodiment of the present invention.

Fig. 8 shows the test accuracy of each cluster when there is a malicious client doing evil in the embodiment of the present invention.

FIG. 9 shows the test accuracy of the global model after detection using different detection methods according to the embodiment of the present invention.

Fig. 10 shows the accuracy rate of the global model in different protection noises according to the embodiment of the present invention.

detailed description

The present invention can be applied in medical industry scenarios. Each hospital uses the present invention as a client of federated learning to efficiently utilize the diversity of hospital sensitive data and ensure the availability of sensitive data, which has solved the non-independent and identical distribution among hospitals problem to efficiently obtain several cluster models applicable to each hospital. The following will take each hospital as the client of federated learning, and take the machine learning of cancerous cell identification based on CT images on the client as an example, and further describe the present invention in conjunction with the accompanying drawings and specific preferred embodiments, but it is not limited thereby protection scope of the present invention. In this example, the system includes a server and several clients, and the server and the device communicate through a secure channel to realize the interaction of information and data. It is not limited to the federated learning system in this embodiment, and ordinary people in the field can implement the deployment of the federated learning system according to actual conditions.

As shown in Figure 1, the gradient-independent clustering federated learning method of this embodiment includes:

S1, the client calculates the data distribution information of its own label samples separately, obtains the intersection similarity between the data distribution information of itself and other clients, and constructs an intersection similarity vector;

S2, the server collects the intersection similarity vectors of each client and constructs a similarity matrix;

S3, the server clusters the clients based on the similarity matrix using a clustering method that guarantees diversity, and executes the model training step, and jumps to the next step when the server detects that the accuracy of the model has decreased;

S4, the server detects the malicious cluster, and after determining the malicious cluster, selects the client that has the most similar data distribution to the client in the malicious cluster and is not in the malicious cluster to form a verification committee;

S6, use the members of the verification committee to verify the models of the members in the malicious cluster and vote for benign models and malicious models, exclude the malicious models and keep the benign models.

Based on the content disclosed in this embodiment, those of ordinary skill in the art can understand that data transmission is carried out between the server and the client, and between the client and the client through a secure channel, such as publishing the global model and uploading the local model , Upload similarity vectors, use RSA-PSI scheme to calculate intersection and other processes. In step S1 of this embodiment, the method for the client to obtain the intersection of data distribution between itself and other clients includes: using the RSA-PSI scheme to obtain the intersection between itself and other clients. As mentioned above, the existing clustering federated learning schemes rely on the use of gradients for clustering. This gradient-dependent scheme may leak the customer's private information or even the original training data. Therefore, in order to avoid this problem, the RSA-PSI scheme is used in this embodiment to allow the client to obtain the intersection with other client data without disclosing the training data, and calculate the distance between itself and other clients based on this intersection. Intersection similarity vector.

In this embodiment, each client needs to separately count the number of samples of each label in the local data, and process the quantity information to prevent leakage of its own real data distribution information. In step S1, the client calculates its own label samples respectively The function expression of the data distribution information of is:

以及

分别表示第1、2以及第i个客户端的单一标签的数据分布信息，且有任意第i个客户端的单一标签的数据分布信息的计算函数表达式为：In the above formula,

as well as

Represent the data distribution information of the single label of the 1st, 2nd and i-th clients respectively, and the calculation function expression of the data distribution information of a single label of any i-th client is:

表示第i个客户端的第i个索引的数据数量，idx_i表示标签i的索引，Q_max表示一个预先定义的任意标签数量的最大值，且任意一个标签的数量都不可以超过这个最大值，X_i表示第i个客户端构建的数据分布信息，j为第i个客户端的第j个索引的序号。经过上述变换之后，客户端可以在不暴露自身真实数据分布信息的情况下获得交集。in,

Indicates the data quantity of the i-th index of the i-th client, idx _i indicates the index of label i, Q _max indicates a predefined maximum number of any label, and the number of any label cannot exceed this maximum value, X _i represents the data distribution information constructed by the i-th client, and j is the serial number of the j-th index of the i-th client. After the above transformation, the client can obtain the intersection without exposing its own real data distribution information.

In this embodiment, the method for the client to obtain the intersection of data distribution with other clients in step S1 includes: using the RSA-PSI scheme (RSA-based privacy set intersection scheme) to calculate the data distribution with other clients distributions, and construct a similarity vector. As an optional implementation, the functional expression of the intersection similarity vector constructed in step S1 of this embodiment is:

In the above formula, ISM _i is the intersection similarity vector of the i-th client, ISM _i [1]~ISM _i [j] represent the data distribution similarity between the i-th client and the 1st-j clients, |X _i ∩X _j | represents the intersection of the data distribution between the i-th client and the j-th client, X _i represents the data distribution information constructed by the i-th client, and X _j represents the data distribution constructed by the j-th client information. Based on the content disclosed in this embodiment, those skilled in the art can also use different privacy set intersection schemes to realize intersection calculation, and this specific embodiment does not limit the technical scope of protection claimed in this application.

In this embodiment, the functional expression of the similarity matrix constructed in step S2 is:

In the above formula, M _sim is a similarity matrix, and any i-th row represents the intersection similarity vector formed by the data distribution similarity between the i-th client and the 1st to nth clients.

In this embodiment, in step S3, the server clusters the clients based on the similarity matrix using a clustering method that guarantees diversity includes: gathering all clients whose intersection similarity is higher than the threshold α into a candidate cluster set and deduplication, Make the candidate cluster set contain all possible clustering results; in the candidate cluster set, calculate the weight of each candidate cluster, and use the greedy algorithm, each selection can make the candidate cluster with the smallest load be added to the final cluster set , until all clients are assigned to the final cluster set.

In this embodiment, the function expression for calculating the weight of each candidate cluster is:

In the above formula, Cost(S′ _i ) represents the cost of calculating the candidate cluster S′ _i , ID is the serial number of the client, M _sim [i][ID] is the element of row i and column ID of the similarity matrix; the load The calculation function expression is:

In the above formula, Payload represents the load, S'\I represents the candidate cluster set that has not been selected into the final cluster, S' represents the candidate cluster set, and I represents the number of the client that has been selected into the final cluster.

In this embodiment, the detection of malicious clusters by the server in step S4 means that the server detects the accuracy of each cluster in the final cluster set according to its own local data, and selects the cluster with the lowest accuracy as the malicious cluster.

In this embodiment, the method for detecting whether there is a malicious cluster includes: whether the accuracy of the global model detected by the server is obviously decreased (decreased by more than a set value). The method for determining a malicious cluster in this embodiment includes: the server detects the accuracy of each cluster according to its own local data, and selects the cluster with the lowest accuracy as the malicious cluster. Of course, those skilled in the art can also choose different methods to detect malicious clusters according to needs; for example, whether the cluster model meets the preset standard or not. The technical scope claimed in this application is not limited by this specific embodiment.

In this embodiment, using the members of the verification committee to verify the models of the members in the malicious cluster in step S6 and voting for the benign model and the malicious model includes: using the members of the verification committee to verify each member in the malicious cluster according to their own local data The model accuracy of the model, the model members whose accuracy is lower than the average are regarded as malicious models and voted, and finally all the voting results will be summed up, and the model with the number of votes higher than the average number of votes will be identified as a malicious model by the verification committee, otherwise it will be recognized as a malicious model by the verification committee identified as a benign model. The method for selecting a verification committee in step S6 of this embodiment includes: for each client in the malicious cluster, the server selects a client that has the most similar data distribution but is not in the malicious cluster. The method of voting by the verification committee includes: the members of the verification committee verify the accuracy of each model in the malicious cluster according to their own local data, and the model whose accuracy is lower than the average will be regarded as a malicious model by this committee member and voted for. The voting method of the verification committee also includes: after each member of the verification committee votes, sum up all the voting results, and the model with a vote higher than the average number of votes will be identified as a malicious model by the committee.

In this embodiment, this embodiment is verified through specific simulation experiments, and the simulation of federated learning is performed based on Pytorch. In the simulation experiment, set up 1 server and 100 clients. The server uses the FederatedAveraging algorithm to aggregate the local models in each cluster, and each client iterates once on the local data during training. The server has a part of independent and identically distributed data that can be used to detect malicious clusters. And set three different training methods of GCFL, FedAvg, GICFL. In the GCFL method, the system uses a gradient-based clustering method to cluster clients. In the FedAvg method, the system does not cluster clients, and only uses the most primitive method for federated learning training. In GICFL, clients will be clustered using the method described above. For each training method, the MNIST, FMNIST, and CIFAR-10 datasets are used as benchmark datasets, respectively. For each training set, two different initialization methods of ill-conditioned non-IID and Dirichlet non-IID are used to initialize the data of the client. In the ill-conditioned non-IID initialization method, each client can obtain random data of two labels, that is, the parameter of the ill-conditioned non-IID is set to k=2. In the Dirichlet non-IID initialization method, the data of each client obeys the Dirichlet distribution of β=0.5. The test accuracy of the model under the three training methods is shown in Fig. 3, Fig. 4 and Fig. 5. In Figure 3, (a) is the test accuracy on the ill-conditioned initialization distribution, (b) is the test accuracy on the Dirichlet initialization distribution; in Figure 4, (a) is the test accuracy on the ill-conditioned initialization distribution, ( b) is the test accuracy on the Dirichlet initialization distribution; in Figure 5, (a) is the test accuracy on the ill-conditioned initialization distribution, and (b) is the test accuracy on the Dirichlet initialization distribution. It can be seen from Fig. 3, Fig. 4 and Fig. 5 that under the GICFL training method of this embodiment, for various data sets and data distributions, the test accuracy of the model is better than that of the other two training methods.

In the simulation experiment of this embodiment, the efficiency of the technical solution of the present invention is also tested by setting a more extreme data initialization mode. In Figure 6, when the parameter of the Dirichlet initialization method is set to β=0.1, the test accuracy of three different training methods under the MNIST and FMNIST data sets after convergence, where (a) is on the MNIST data set The test accuracy of , (b) is the test accuracy on the FMNIST dataset. It can be seen from FIG. 6 that, in a more extreme data initialization mode, the test accuracy of this embodiment is still excellent.

In the simulation experiment of this embodiment, FIG. 7 shows the main overhead required to calculate the intersection of data distributions between clients when using different data distribution initialization methods, where (a) is the overhead on the ill-conditioned initialization distribution, (b) is the overhead on the Dirichlet initialization distribution. It can be seen from FIG. 7 that in this embodiment, it takes some time to calculate the intersection similarity. But since this phase precedes the training phase, it is negligible compared to the training time. In addition, when a new member joins, it is not necessary to calculate the intersection between the original members again. Considering the time required for training and the need for privacy protection, the overhead of this protocol is acceptable.

In the simulation experiment of this embodiment, a situation in which someone does evil in the client is simulated. A malicious client adds Gaussian noise with a variance of 1 to its local model, that is, δ=1. The test accuracy of each cluster model after the malicious client is evil is shown in Figure 8, where (a) is the test accuracy on the ill-conditioned initialization distribution, and (b) is the test accuracy on the Dirichlet initialization distribution. It can be seen from Figure 8 that after the malicious client does evil, the server can still detect malicious clusters through its own local data.

In the simulation experiment of this embodiment, three methods are used to form the verification committee. In the first method, the server selects clients other than the malicious cluster that are most similar to the client data distribution in the cluster to form the verification committee, which is the method selected in this embodiment. In the second method, the server selects random clients outside the malicious cluster to form the verification committee. In the third method, the server selects the clients outside the malicious cluster that are most dissimilar to the clients in the cluster to form the verification committee. After the verification committee votes, the clients identified as malicious clusters are eliminated, and the remaining clients are re-clustered. The model accuracy after re-clustering is shown in Figure 9, where (a) is the test accuracy on the ill-conditioned initialization distribution, and (b) is the test accuracy on the Dirichlet initialization distribution. It can be seen from FIG. 9 that under the detection of the method of this embodiment, excellent detection effects of malicious clusters and malicious clients can be obtained.

In the simulation experiment of this embodiment, the robustness of our test method is also verified by gradually adding different degrees of Gaussian noise to the model of the client, as shown in Figure 10, where (a) is on the ill-conditioned initialization distribution Test accuracy, (b) is the test accuracy on the Dirichlet initialization distribution. It can be seen from Figure 10 that when Gaussian noise with a variance of 0.006 is added to the Dirichlet initialization distribution, the malicious model in the malicious cluster can also be accurately identified. In an ill-conditioned initialization distribution, even Gaussian noise with a variance of 0.01 can be tolerated.

To sum up, the gradient-independent clustering federated learning method of this embodiment does not need to rely on the gradient information of the client to perform clustering during the clustering process, but to perform clustering according to the intersection similarity between data distributions of the client Clustering avoids the gradient information leakage problem of the client, protects the gradient security of the client, enhances the security and reliability of the clustering federated learning process, and improves the training accuracy. The gradient-independent clustering federated learning method of this embodiment innovatively allows the same client to appear in multiple clusters during the clustering process, so that the most suitable cluster can be found for each client, and fully Make full use of the diversity of client data to enhance model accuracy. The gradient-independent clustering federated learning method in this embodiment is different from the pre-detection commonly used in existing federated learning. It innovatively uses a post-event detection mechanism to detect malicious clusters existing in the system, allowing the attack to be carried out after the start of the attack. detection to save overhead and further improve system security. The gradient-independent clustering federated learning method of this embodiment can be applied in medical industry scenarios. Each hospital uses the invention as a federated learning client to efficiently utilize the diversity of hospital sensitive data and ensure the availability of sensitive data. Solve the non-independent and identical distribution problem between hospitals, and efficiently obtain several cluster models suitable for each hospital.

In addition, this embodiment also provides a gradient-independent clustering federated learning system, which includes multiple clients connected to each other, the client includes a microprocessor and a memory connected to each other, and the microprocessor is programmed or configured to Implement the aforementioned gradient-independent clustering federated learning method. In addition, this embodiment also provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and the computer program is used to be programmed or configured by a microprocessor to perform the aforementioned gradient-independent clustering federated learning method.

Those skilled in the art should understand that the embodiments of the present application may be provided as methods, systems, or computer program products. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein. The present application is described with reference to flowcharts and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It should be understood that each procedure and/or block in the flowchart and/or block diagram, and a combination of procedures and/or blocks in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions may be provided to a general purpose computer, special purpose computer, embedded processor, or processor of other programmable data processing equipment to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing equipment produce a An apparatus for realizing the functions specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram. These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture comprising instruction means, the instructions The device realizes the function specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram. These computer program instructions can also be loaded onto a computer or other programmable data processing device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby The instructions provide steps for implementing the functions specified in the flow chart or blocks of the flowchart and/or the block or blocks of the block diagrams.

The above descriptions are only preferred implementations of the present invention, and the protection scope of the present invention is not limited to the above-mentioned embodiments, and all technical solutions under the idea of the present invention belong to the protection scope of the present invention. It should be pointed out that for those skilled in the art, some improvements and modifications without departing from the principles of the present invention should also be regarded as the protection scope of the present invention.

Claims (10)

1. A gradient-independent clustered federation learning method, comprising:

s1, respectively calculating data distribution information of a label sample of a client by the client, obtaining intersection similarity between the data distribution information of the client and data distribution information of other clients, and constructing an intersection similarity vector;

s2, the server collects intersection similarity vectors of the clients and constructs a similarity matrix;

s3, clustering the clients by using a clustering method for ensuring diversity based on the similarity matrix by the server, executing a model training step, and jumping to the next step when the server detects that the precision of the model is reduced;

s4, the server detects the malicious clusters, and selects the clients which have the data distribution most similar to that of the clients in the malicious clusters and are not in the malicious clusters to form an authentication committee after the malicious clusters are determined;

and S6, verifying by using the model of which the member of the verification committee is a member in the malicious cluster, voting to determine the model into a benign model and a malicious model, and removing and reserving the benign model from the malicious model.

2. The gradient-independent clustering federation learning method of claim 1, wherein in step S1, the functional expression of the client respectively calculating the data distribution information of its label sample is as follows:

in the above formula, the first and second carbon atoms are,

and

the data distribution information of the single tags of the 1 st, 2 nd and ith clients are respectively represented, and the calculation function expression of the data distribution information of the single tag of any ith client is as follows:

wherein,

data quantity, idx, representing the ith index of the ith client _i Index, Q, representing tag i _max Representing a predefined maximum of any number of tags, where none of the tags exceeds the maximum, X _i And j is the sequence number of the jth index of the ith client.

3. The gradient-independent clustering federated learning method of claim 2, wherein the functional expression of the intersection similarity vector constructed in step S1 is:

in the above formula, ISM _i For the intersection similarity vector of the ith client, ISM _i [1]～ISM _i [j]Represents the data distribution similarity of the ith client to the 1 st to the j th clients, | X _i ∩X _j | represents the intersection of the data distributions between the ith and jth clients, X _i Representing data distribution information, X, constructed by the ith client _j And representing the data distribution information constructed by the jth client.

4. The gradient-independent clustering federated learning method according to claim 3, wherein the functional expression of the similarity matrix constructed in step S2 is:

in the above formula, M _sim And any ith row represents an intersection similarity vector formed by the data distribution similarities of the ith client to the 1 st to nth clients.

5. The gradient-independent clustering federated learning method according to claim 4, wherein the server clustering the clients using a diversity-guaranteed clustering method based on the similarity matrix in step S3 includes: aggregating all clients with intersection similarity higher than a threshold value alpha into a candidate cluster set and removing duplication, so that the candidate cluster set comprises all possible clustering results; and calculating the weight of each candidate cluster in the candidate cluster set, and adding the candidate cluster with the minimum load into the final cluster set by using a greedy algorithm in each selection until all the clients are distributed into the final cluster set.

6. The gradient-independent clustering federal learning method as claimed in claim 5, wherein the functional expression for calculating the weight of each candidate cluster is:

of the above formula, cost (S' _i ) Denotes calculated candidate Cluster S' _i ID is the number of the client, M _sim [i][ID]The element is the ith row and the ID column of the similarity matrix; the calculation function expression of the load is as follows:

in the above formula, payload represents the load, S '\ I represents the set of candidate clusters that have not been selected into the final cluster, S' represents the set of candidate clusters, and I represents the number of the client that has been selected into the final cluster.

7. The gradient-independent cluster federal learning method as claimed in claim 6, wherein the step S4 of detecting the malicious cluster by the server means that the server detects the accuracy of each cluster in the final cluster set according to local data of the server, and selects a cluster with the lowest accuracy as the malicious cluster.

8. The gradient-independent cluster federated learning method of claim 7, wherein the step S6 of validating and voting for the decision as a benign model and a malicious model using the models of the members of the validation committee as members of the malicious cluster comprises: and verifying the model accuracy of each member in the malicious cluster by using the members of the verification committee according to local data of the members, regarding the model members with the accuracy lower than the average value as malicious models and voting, and finally summing up all voting results, wherein the models with the votes higher than the average votes are identified as malicious models by the verification committee, and otherwise, the models are identified as benign models by the verification committee.

9. A gradient-independent clustered federated learning system, comprising a plurality of interconnected clients, the clients comprising an interconnected microprocessor and memory, wherein the microprocessor is programmed or configured to perform the gradient-independent clustered federated learning method of any one of claims 1 to 8.

10. A computer readable storage medium having a computer program stored thereon, wherein the computer program is adapted to be programmed or configured by a microprocessor to perform the gradient independent clustering federal learning method as claimed in any of claims 1-8.

Images