CN115238806A

CN115238806A - Sample class imbalance federal learning method and related equipment

Info

Publication number: CN115238806A
Application number: CN202210908867.1A
Authority: CN
Inventors: 李泽远; 王健宗; 曹康养
Original assignee: Ping An Technology Shenzhen Co Ltd
Current assignee: Ping An Technology Shenzhen Co Ltd
Priority date: 2022-07-29
Filing date: 2022-07-29
Publication date: 2022-10-25

Abstract

The method comprises the steps of obtaining a sample data set, wherein each sample data in the sample data set corresponds to at least one classification type, determining the sample type distribution of the sample data set according to the classification type of the sample data, then dividing the sample data set according to the sample type distribution to obtain a majority type sample set and a minority type sample set, actively adjusting the type balance coefficient of the majority type sample, and carrying out iterative training on a local model based on a target sample set and the minority type sample set, which are extracted from the majority type sample set and have the category balance coefficient ratio, until the current training round of the local model reaches a preset training number, so that the learning capacity of the model on the type unbalanced sample in federated learning can be improved, and the problem of model performance reduction caused by the occurrence of the type unbalance of the sample is solved.

Description

Sample class imbalance federal learning method and related equipment

Technical Field

The application relates to the technical field of computers, in particular to a sample class unbalanced federal learning method and related equipment.

Background

Federal learning is a distributed machine learning technology which breaks the data island and protects the data privacy. But in real-world applications, more obstacles are encountered, such as unbalanced sample classes. For example, in a Multi-class Classification (Multi-class Classification) task, the class distribution of the sample data may be long-tail distribution (long-tail distribution), and the model focuses on the class with a large amount of data and ignores the class with a small amount of data.

The federal learning has more complex challenges, and not only sample data in the client is unbalanced in category, but also the category distribution of the sample data among different clients is different, which leads the server to be unable to learn the information of the small sample category.

Disclosure of Invention

The main purpose of the embodiments of the present application is to provide a federate learning method, an apparatus, an electronic device, and a computer-readable storage medium for sample class imbalance, which can improve the learning ability of a model for a class imbalance sample in federate learning, and reduce the problem of performance degradation of the model due to class imbalance of the sample.

In order to achieve the above object, a first aspect of an embodiment of the present application provides a federated learning method with unbalanced sample classes, where the method is applied to a client participating in federated learning, and the client is communicatively connected to a server, and the method includes:

acquiring a sample data set, wherein each sample data in the sample data set corresponds to at least one classification category;

determining the sample category distribution of the sample data set according to the classification category corresponding to the sample data;

dividing the sample data set according to the sample category distribution to obtain a majority sample set and a minority sample set;

acquiring a dynamic class balance coefficient;

extracting sample data of the class balance coefficient ratio from the majority of sample sets to obtain a target sample set;

training a local model based on the minority class sample set and the target sample set, and returning to the step of obtaining a dynamic class balance coefficient until the current training round of the local model reaches a preset training number, so as to obtain a trained local model;

and uploading the trained local model to the server.

According to the federal learning method for unbalanced sample classes provided in some embodiments of the present invention, after the sample data set is divided according to the sample class distribution to obtain a majority class sample set and a minority class sample set, before the obtaining of the dynamic class balance coefficient, the method further includes:

dividing the majority of sample sets to obtain a first sample set and a second sample set, wherein the number of samples in the second sample set is greater than that of the samples in the first sample set;

training a local model based on the minority sample set and the first sample set to obtain a trained local model;

inputting the second sample set into the trained local model to obtain a classification predicted value corresponding to each sample data in the second sample set through the trained local model;

determining information entropy corresponding to each sample data in the second sample set according to the classification predicted value;

sequencing each sample data in the second sample set according to the information entropy from large to small;

the extracting sample data of the class balance coefficient ratio from the majority class sample set to obtain a target sample set includes:

and extracting the sample data of the class balance coefficient ratio from the second sample set according to the sequence to obtain a target sample set.

According to the federal learning method for sample class imbalance provided by some embodiments of the invention, the information entropy is determined by the following formula:

wherein H (x) is the information entropy of sample data x, n is the number of classification categories, and P is _i (x) And the classification predicted value corresponding to the sample data x.

According to the federal learning method for sample class imbalance provided by some embodiments of the present invention, before the obtaining of the dynamic class balance coefficient, the method further includes:

acquiring preset training times;

constructing a value function of a category balance coefficient according to the preset training times;

the obtaining of the dynamic class balance coefficient includes:

acquiring a current training round;

and acquiring a class balance coefficient from the value function based on the current training turn.

According to the federate learning method for the sample class imbalance provided by some embodiments of the present invention, the value function of the class balance coefficient is determined by the following formula:

θ _k ＝|sin(kπ/T)|；

wherein, the theta _k And the class balance coefficient is the class balance coefficient in the k training round, and the T is the preset training times.

According to a federate learning method for sample class imbalance provided by some embodiments of the present invention, the minority sample set and the target sample set are input to a local model, so as to obtain a classification prediction value corresponding to each sample data in the minority sample set and the target sample set through the local model;

constructing a loss function according to the classification predicted value and the classification category corresponding to the sample data;

training the local model based on the loss function.

According to a federal learning method for sample class imbalance provided by some embodiments of the invention, the loss function is determined by the following formula:

wherein the EFL (pt) is the loss function, C is the number of samples, a _t As a balance factor, the

Is a weighting factor of class j, said y _b As a focusing factor, the

And the pt is a classification predicted value corresponding to the sample data as the class specific parameter.

In order to achieve the above object, a second aspect of the embodiments of the present application provides a federated learning apparatus with unbalanced sample classes, where the apparatus is applied to a client participating in federated learning, and the client is communicatively connected to a server, and the apparatus includes:

the system comprises a sample acquisition module, a classification module and a classification module, wherein the sample acquisition module is used for acquiring a sample set, and each sample data in the sample set corresponds to at least one classification category;

the distribution determining module is used for determining the sample class distribution of the sample data set according to the classification class corresponding to the sample data;

the sample dividing module is used for dividing the sample data set according to the sample category distribution to obtain a majority sample set and a minority sample set;

the coefficient acquisition module is used for acquiring a dynamic class balance coefficient;

the sample extraction module is used for extracting sample data of the class balance coefficient ratio from the majority of sample sets to obtain a target sample set;

the model training module is used for training a local model based on the minority class sample set and the target sample set and returning to the step of obtaining a dynamic class balance coefficient until the current training round of the local model reaches a preset training number, so that a trained local model is obtained;

and the model uploading module is used for uploading the trained local model to the server.

To achieve the above object, a third aspect of the embodiments of the present application provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when executed by the processor, the computer program implements the method of the first aspect.

To achieve the above object, a fourth aspect of the embodiments of the present application proposes a storage medium, which is a computer-readable storage medium for computer-readable storage, and the storage medium stores one or more computer programs, which are executable by one or more processors to implement the method of the first aspect.

The application provides a federal learning method, a federal learning device, electronic equipment and a computer readable storage medium for sample class unbalance, wherein the federal learning method for sample class unbalance comprises the steps of obtaining a sample data set, determining sample class distribution of the sample data set according to classification classes corresponding to each sample data in the sample data set, dividing the sample data set according to the sample class distribution to obtain a majority sample set and a minority sample set, carrying out iterative training on a local model until the current training round of the local model reaches a preset training number, obtaining a dynamic class balance coefficient in each training round, extracting sample data with the class balance coefficient being in proportion from the majority sample set to obtain a target sample set, training the local model based on the minority sample set and the target sample set to obtain a trained local model, and uploading the trained local model to a server. According to the method and the device, the dynamic class balance coefficient is obtained, iterative training is carried out on the local model on the basis of the target sample set and the minority sample set, the class balance coefficient proportion of which is extracted from the majority sample set, the number of samples of the majority sample data is close to the minority sample data in the training process of the local model, the class imbalance condition of the sample data in the client side is reduced, the learning capacity of the model on the class imbalance sample in federal learning is improved, and the problem of model performance reduction caused by class imbalance of the sample is reduced.

Drawings

FIG. 1 is a flow chart of a federated learning method with unbalanced sample classes provided in an embodiment of the present application;

FIG. 2 is a schematic flow chart diagram illustrating a sample class imbalance federated learning method according to another embodiment of the present application;

FIG. 3 is a flow chart illustrating a sample class imbalance federated learning method provided in another embodiment of the present application;

FIG. 4 is a flow chart illustrating a sample class imbalance federated learning method provided in another embodiment of the present application;

FIG. 5 is a flow chart illustrating a sample class imbalance federated learning method provided in another embodiment of the present application;

FIG. 6 is a flow chart illustrating a sample class imbalance federated learning method provided in another embodiment of the present application;

FIG. 7 is a schematic flow chart diagram illustrating a sample class imbalance federated learning method according to another embodiment of the present application;

FIG. 8 is a schematic diagram illustrating a sample class distribution of a sample data set according to an embodiment of the present application;

FIG. 9 is a diagram of an environment for implementing a sample class unbalanced Federal learning method according to an embodiment of the present application;

FIG. 10 is a structural diagram of a sample class imbalance federal learning device provided in an embodiment of the present application;

fig. 11 is a schematic hardware structure diagram of an electronic device according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of and not restrictive on the broad application.

It is to be noted that, unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of the present application only and is not intended to be limiting of the application.

The federal learning is a distributed machine learning technology which breaks a data island and protects data privacy. But in real-world applications, more obstacles are encountered, such as unbalanced sample classes. For example, in a multi-classification task, the class distribution of the sample data may be a long-tailed distribution, and the model focuses on the class with a large amount of data and ignores the class with a small amount of data.

The federal learning faces more complicated challenges, and not only sample data in the client side has unbalanced categories, but also the category distribution of the sample data among different client sides has differences, which leads the server side to be unable to learn the information of the small sample category.

Based on this, the embodiment of the application provides a federate learning method and device for sample class imbalance, an electronic device and a computer readable storage medium, which can improve the learning ability of a model for a sample with class imbalance in federate learning, and reduce the problem of performance reduction of the model due to the occurrence of class imbalance in the sample.

The federal learning method, apparatus, electronic device and computer-readable storage medium for sample class imbalance provided in the embodiments of the present application are specifically described in the following embodiments.

The embodiment of the application can acquire and process related data based on an artificial intelligence technology. Among them, artificial Intelligence (AI) is a theory, method, technique and application system that simulates, extends and expands human Intelligence using a digital computer or a machine controlled by a digital computer, senses the environment, acquires knowledge and uses the knowledge to obtain the best result.

The artificial intelligence infrastructure generally includes technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a robot technology, a biological recognition technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and the like.

The application is operational with numerous general purpose or special purpose computing system environments or configurations. For example: personal computers, server computers, hand-held or portable devices, tablet-type devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like. The application may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The application may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.

First, a federal learning method with unbalanced sample classes in an embodiment of the present application is described, please refer to fig. 9, fig. 9 shows an implementation environment diagram of a federal learning method with unbalanced sample classes provided in an embodiment of the present application, the method is applied to a client participating in federal learning, and the client is in communication connection with a server, please refer to fig. 1, fig. 1 shows a flow chart of the federal learning method with unbalanced sample classes provided in an embodiment of the present application, as shown in fig. 1, the method includes, but is not limited to, steps S110 to S170:

step S110, a sample data set is obtained, and each sample data in the sample data set corresponds to at least one classification category.

And step S120, determining the sample category distribution of the sample data set according to the classification category corresponding to the sample data.

And step S130, dividing the sample data set according to the sample category distribution to obtain a majority sample set and a minority sample set.

Step S140, a dynamic class balance coefficient is obtained.

And S150, extracting the sample data of the class balance coefficient ratio from the majority of sample sets to obtain a target sample set.

And step S160, training a local model based on the minority class sample set and the target sample set, and returning to the step of dynamically acquiring class balance coefficients until the current training round of the local model reaches a preset training number, so as to obtain the trained local model.

And S170, uploading the trained local model to the server.

It is understood that, in step S110, for the multi-classification task, each sample data in the sample data set corresponds to one classification category, and the total number of classification categories in the sample data set is greater than or equal to 2, whereas in the multi-classification task, each sample data in the sample data set corresponds to at least one classification category, and the total number of classification categories in the sample data set is greater than or equal to 2.

It can be understood that, in step S120, the sample data in the sample data set is divided according to the classification category corresponding to each sample data in the sample data set, so as to obtain the sample category distribution of the sample data set. For example, in the multi-classification task of federal learning, the sample data in the sample data set collectively correspond to the following classification categories: {1,2,3,4,5,6,7,8,9}, with reference to fig. 8 for sample class distribution, fig. 8 shows the sample class distribution of the sample data set provided by the embodiment of the present application, and as shown in fig. 8, the classification classes with the sample number greater than 10 have: {1,2,3}, classification categories for which the number of samples is less than or equal to 10 are: {4,5,6,7,8,9}.

It is to be understood that, in step S130, by setting a number threshold, the sample data of the cluster is divided into a majority-class sample set and a minority-class sample set according to the number of samples. Illustratively, as shown in FIG. 8, for the classification category: {1,2,3,4,5,6,7,8,9}, dividing the sample data set by using 10 sample data as a threshold value to obtain a majority sample set and a minority sample set, wherein the majority sample set is the majority sample data with a classification category of {1,2,3} and the minority sample set is the minority sample data with a classification category of {4,5,6,7,8,9}.

It can be understood that, in steps S140 to S160, acquiring a dynamic class balance coefficient refers to acquiring different class balance coefficients in each round of training of the local model, so as to extract sample data with different class balance coefficient ratios from the majority sample set to obtain target data sets containing different numbers of majority sample data, that is, the proportion of majority sample data selected from the majority sample set is dynamically adjusted, so that iterative training is performed on the local model based on the minority sample set and the majority sample data with different data ratios until the current training round of the local model reaches a preset training number, so that the local model can fully learn the minority sample data while learning the majority sample data, the learning capability of the model on class unbalanced samples in federal learning is improved, and the problem of model performance degradation caused by class imbalance of the samples is reduced.

Illustratively, the preset dynamic class balance coefficients are an array {0.1,0.2,0.3 \8230;, 0.9}, so that in each local model training round, the class balance coefficients are sequentially obtained from the array, thereby adjusting the sample number of most sample data in each local model training round.

It can be understood that the local model is trained based on the minority class sample set and the target sample set, if the current training round of the local model does not reach the preset training times, the step of dynamically obtaining the class balance coefficient is returned, and if the current training round of the local model reaches the preset training times, the training is ended, and the trained local model is obtained.

In a specific embodiment, referring to fig. 6, fig. 6 shows a federate learning method for sample class imbalance provided in the embodiment of the present application, as shown in fig. 6, in a round of training of a local model, a dynamic class balance coefficient is obtained, then sample data in a class balance coefficient ratio is extracted from a majority class sample set to obtain a target sample set, the local model is trained based on a minority class sample set and the target sample set, and finally, a current training round of the local model is determined, if the current training round of the local model does not reach a preset training number, the step of obtaining the dynamic class balance coefficient is returned, and if the current training round of the local model reaches the preset training number, the training is ended, and a trained local model is obtained.

In a specific embodiment, referring to fig. 7, fig. 7 shows a federal learning method for unbalanced sample classes provided in an embodiment of the present application, and as shown in fig. 7, in a round of training of a local model, a current training round of the local model is first judged, if the current training round of the local model does not reach a preset training number, the step of obtaining a dynamic class balance coefficient is returned, then, sample data in proportion to the class balance coefficient is extracted from a majority of sample sets to obtain a target sample set, the local model is trained based on a minority of sample sets and the target sample set, and if the current training round of the local model reaches the preset training number, the training is ended to obtain a trained local model.

In some embodiments, please refer to fig. 2, fig. 2 shows a flow chart of a federal learning method for sample class imbalance according to an embodiment of the present application, and as shown in fig. 2, after the step S130, before the step S140, the method further includes, but is not limited to, steps S210 to S250:

step S210, dividing the majority sample sets to obtain a first sample set and a second sample set, where the number of samples in the second sample set is greater than the number of samples in the first sample set.

Step S220, training a local model based on the minority sample set and the first sample set to obtain a trained local model.

Step S230, inputting the second sample set to the trained local model, so as to obtain a predicted classification value corresponding to each sample data in the second sample set through the trained local model.

Step S240, determining an information entropy corresponding to each sample data in the second sample set according to the classification prediction value.

And step S250, sequencing each sample data in the second sample set according to the information entropy from large to small.

The step S150 includes:

It should be noted that, in steps S210 and S220, the majority sample set is first divided to obtain a first sample set and a second sample set, and exemplarily, the number of samples in the first sample set is the same as the number of samples in the minority sample set, and then the local model is trained based on the minority sample set and the first sample set to obtain the local model for learning the minority sample data and the minority sample data.

It should be noted that, in steps S230 to S240, the second sample set, that is, the sample data in the majority sample set except the first sample set, is input to the trained local model, so as to output the classification predicted value corresponding to each sample data in the second sample set through the trained local model, and then the information entropy corresponding to each sample data in the second sample set is determined according to the classification predicted value. It can be understood that the information entropy represents the classification difficulty of the sample data in the classification task of the model, that is, the local model which has learned part of the majority sample data performs preliminary classification prediction on the majority sample data in the second sample set, so as to determine the classification difficulty of the majority sample data in the second sample set.

It should be further noted that, in step S250, each sample data in the second sample set is sorted from large to small according to the information entropy, so that after the dynamic class balance coefficient is obtained, sample data in which the class balance coefficient is compared is extracted from the second sample set according to the sorting to obtain the target sample set, that is, sample data with a larger information entropy is selected from the second sample set as the target sample set. The method has the advantages that the majority of sample data with high entropy is selected for subsequent local model training, namely, the majority of representative sample data with high classification difficulty is selected for training, and the model performance of the local model on the majority of sample data can be guaranteed under the condition of reducing the number of the samples of the majority of sample data, so that the local model can fully learn the minority of sample data and the majority of sample data, and the learning capability of the model on the unbalanced samples in the federal learning is remarkably improved.

Exemplarily, the second sample set includes sample data and information entropy corresponding to the sample data as follows:

{ (sample data a, information entropy 1), (sample data B, information entropy 3), (sample data C, information entropy 5), (sample data D, information entropy 4), (sample data E, information entropy 2), (sample data F, information entropy 6) }, and sorts the sample data in the second sample set according to the information entropy from large to small:

{ (sample data F, information entropy 6), (sample data C, information entropy 5), (sample data D, information entropy 4), (sample data B, information entropy 3), (sample data E, information entropy 2), (sample data a, information entropy 1) }, and sample data with a category balance coefficient (e.g., 0.5) in proportion is selected from the second sample set according to the ranking to obtain a target sample set:

{ sample data F, sample data C, sample data D }.

In some embodiments, the information entropy is determined by the following formula:

For example, if the classification category in the sample data set is { category a, category B, category C }, then the predicted classification value P corresponding to the sample data x is obtained _i (x) If the result is { (category a, predicted value 0.1), (category B, predicted value 0.8), (category C, predicted value 0.2) }, 3 classification predicted values corresponding to the sample data x are calculated, and the information entropy corresponding to the sample data x is obtained.

In some embodiments, please refer to fig. 3, fig. 3 shows a flow chart of a federal learning method for sample class imbalance provided in the embodiment of the present application, and as shown in fig. 3, before the step S140, the method includes, but is not limited to, steps S310 to S320:

step S310, obtaining a preset training frequency.

And S320, constructing a value function of the category balance coefficient according to the preset training times.

Referring to fig. 4, fig. 4 is a schematic flow chart illustrating a federal learning method for sample class imbalance according to an embodiment of the present application, and as shown in fig. 4, the step S140 includes, but is not limited to, steps S410 to S420:

step S410, acquiring a current training round.

And step S420, acquiring a class balance coefficient from the value function based on the current training round.

It can be understood that the preset training times are obtained and the value function of the category balance coefficient is constructed, so that in each round of training of the local model, the dynamic category balance coefficient can be obtained from the value function based on the obtained current training times. On one hand, the client can actively adjust the class balance coefficient according to the preset training times only by the acquired preset training times, and the learning of the model on the unbalanced sample class is completed; on the other hand, value functions of different types of balance coefficients can be constructed according to the types of the models or training tasks, the number or the class distribution of sample data, the value of the type balance coefficient in each training round is further adjusted, and the adaptability of model learning in different application scenes in the federal learning with unbalanced sample classes is improved.

It should be noted that, according to different application scenarios, value functions of different category balance coefficients are constructed, and the value trend of the category balance coefficients may be incremental, decremental, or randomly changed.

In some embodiments, the value function of the class balance coefficient is determined by the following formula:

θ _k ＝|sin(kπ/T)|；

wherein, the theta _k And the number of training rounds is the category balance coefficient of the kth training round, and T is the preset training number.

For example, the preset training time T is 10, the class balance coefficient θ is calculated in 10 local training rounds _k Respectively as follows:

and 1, round: theta ₁ = sin (π/10) |; and 2, round 2: theta ₂ = sin (2 π/10) |; and (3) round: theta ₃ ＝|sin(3π/10)|……。

It can be understood that the value range of the cosine function is [ -1,1], so that the value function of the class balance coefficient is constructed based on the cosine function, and the class balance coefficient which is in the range of [0,1] and is increased can be intuitively and quickly obtained from the value function according to the previous training round of the local model.

In some embodiments, referring to fig. 5, fig. 5 illustrates a federal learning method for category imbalance provided in an embodiment of the present application, and as shown in fig. 5, the local model is trained based on a minority sample set and the target sample set, including but not limited to steps S510 to S530:

step S510, inputting the minority sample set and the target sample set into a local model, so as to obtain a predicted classification value corresponding to each sample data in the minority sample set and the target sample set through the local model.

And S520, constructing a loss function according to the classification predicted value and the classification category corresponding to the sample data.

Step S530, training the local model based on the loss function.

In some embodiments, the loss function is determined by the following equation:

Is a weighting factor of class j, said y _b As a focusing factor, the

It can be understood that the loss function trained by the local model adopts Equalized loss local which is specially responsible for mining the difficult samples, and the learning capacity of the local model on the category imbalance samples is further improved by dynamically adjusting the attention degree of the loss function on the difficult samples.

In some embodiments, the uploading the trained local model to the server includes:

and uploading the trained local model to the server so that the server receives a plurality of local models uploaded by the client, and integrating the plurality of local models by the server to obtain a federal learning model.

It can be understood that, the client of the federal learning system adjusts the majority of sample data in the local model training by obtaining the dynamic class balance coefficient, that is, the proportion of the majority of sample data in the training sample in the model training process is reduced, so that the sample number of the majority of sample data is close to the minority of sample data in the local model training process of the same client, and the class imbalance of the sample data in the client is reduced.

In some embodiments, the method further comprises: downloading the federated learning model from the server.

In some embodiments, the server and the client are pre-configured with the same local model, and the uploading the trained local model to the server includes:

and uploading the model parameters of the trained local model to the server so that the server receives the plurality of model parameters uploaded by the client, and integrating the plurality of model parameters by the server to obtain the federal learning model parameters.

In some embodiments, the method further comprises: downloading the federated learning model parameters from the server and updating the local model based on the federated learning model parameters.

The federal learning method for sample class imbalance provided by the present application is described below by using a specific embodiment, as shown in fig. 9, the method is applied to a client participating in federal learning, and the client is in communication connection with a server, and the method specifically includes the following steps:

step 1, acquiring a sample data set, wherein each sample data in the sample data set corresponds to at least one classification category;

step 2, determining the sample category distribution of the sample data set according to the classification category corresponding to the sample data;

step 3, dividing the sample data set according to the sample class distribution to obtain a majority sample set and a minority sample set;

step 4, dividing the majority sample sets to obtain a first sample set and a second sample set, wherein the number of samples in the second sample set is greater than that of the samples in the first sample set;

step 5, training the local model based on the minority class sample set and the first sample set to obtain a trained local model;

step 6, inputting the second sample set into the trained local model, so as to obtain a classification predicted value corresponding to each sample data in the second sample set through the trained local model;

step 7, determining the information entropy corresponding to each sample data in the second sample set according to the classification predicted value;

step 8, sequencing each sample data in the second sample set according to the information entropy from large to small;

step 9, acquiring preset training times;

step 10, constructing a value function of a class balance coefficient according to preset training times;

step 11, obtaining a dynamic class balance coefficient:

step 11.1, obtaining the current training round;

step 11.2, based on the current training round, acquiring a class balance coefficient from a value taking function;

step 12, extracting sample data with the class balance coefficient ratio from the second sample set according to the sequence to obtain a target sample set;

step 13, training the local model based on the minority sample set and the target sample set, returning to the step 11 until the current training round of the local model reaches the preset training times, and obtaining the trained local model;

and step 14, uploading the trained local model to a server.

The application provides a federal learning method with unbalanced sample classes, which comprises the steps of obtaining a sample data set, determining the sample class distribution of the sample data set according to the class corresponding to each sample data in the sample data set, dividing the sample data set according to the sample class distribution to obtain a majority sample set and a minority sample set, performing iterative training on a local model until the current training round of the local model reaches a preset training number, obtaining a target sample set by obtaining a dynamic class balance coefficient and extracting sample data with the class balance coefficient in proportion from the majority sample set in each training round, and training the local model based on the minority sample set and the target sample set to obtain a trained local model. According to the method and the device, the dynamic class balance coefficient is obtained, iterative training is carried out on the local model based on the target sample set and the minority sample set, which are extracted from the majority sample set and account for the class balance coefficient, the sample number of the majority sample data is close to the minority sample data in the training process of the local model, the class imbalance condition of the sample data in the client side is reduced, the learning capacity of the model on the class imbalance sample in federal learning is improved, and the problem of model performance reduction caused by class imbalance of the sample is reduced.

The embodiment of the application also provides a federated learning method with unbalanced sample classes, which is applied to a server participating in federated learning, wherein the server is in communication connection with a client, and the method comprises the following steps:

receiving a local model uploaded by a plurality of clients according to the sample class unbalance federal learning method provided by the embodiment of the application;

and integrating a plurality of local models to obtain a federal learning model.

In some embodiments, the server integrates The local models based on The federal averaging Algorithm (FedAvg) method.

It can be understood that the model parameter uploaded by the server receiving m clients (set is V) is w _t The local models are integrated, and the integration formula is as follows:

wherein n is _k Is the number of samples on the kth client, n is the total number of samples on m clients,

model parameters of the trained local model for the kth client.

Referring to fig. 10, an embodiment of the present application further provides a federal learning apparatus 100 with unbalanced sample classes, where the apparatus is applied to a client participating in federal learning, and the client is communicatively connected to a server, and the apparatus 100 includes:

a sample obtaining module 110, configured to obtain a sample set, where each sample data in the sample set corresponds to at least one classification category;

a distribution determining module 120, configured to determine, according to the classification category corresponding to the sample data, a sample category distribution of the sample data set;

a sample dividing module 130, configured to divide the sample data set according to the sample category distribution to obtain a majority sample set and a minority sample set;

a coefficient obtaining module 140, configured to obtain a dynamic class balance coefficient;

a sample extracting module 150, configured to extract sample data in proportion to the class balance coefficients from the multiple sample sets to obtain a target sample set;

the model training module 160 is configured to train a local model based on the minority class sample set and the target sample set, and return to the step of dynamically obtaining the class balance coefficient until the current training round of the local model reaches a preset training number of times, so as to obtain a trained local model;

a model uploading module 170, configured to upload the trained local model to the server.

The application provides a federal learning device with unbalanced sample classes, the federal learning device with unbalanced sample classes obtains a sample data set through a sample obtaining module, a distribution determining module determines sample class distribution of the sample data set according to a classification class corresponding to each sample data in the sample data set, then a sample dividing module divides the sample data set according to the sample class distribution to obtain a majority sample set and a minority sample set, iterative training is conducted on a local model until the current training round of the local model reaches a preset training number, in each training round, a dynamic class balance coefficient is obtained through a coefficient obtaining module, a sample extracting module extracts sample data with a class balance coefficient ratio from the majority sample set to obtain a target sample set, a model training module trains the local model based on the minority sample set and the target sample set to obtain a trained local model, and finally a model uploading module uploads the trained local model to a server. According to the method and the device, the dynamic class balance coefficient is obtained, iterative training is carried out on the local model based on the target sample set and the minority sample set, which are extracted from the majority sample set and account for the class balance coefficient, the sample number of the majority sample data is close to the minority sample data in the training process of the local model, the class imbalance condition of the sample data in the client side is reduced, the learning capacity of the model on the class imbalance sample in federal learning is improved, and the problem of model performance reduction caused by class imbalance of the sample is reduced.

It should be noted that, for the information interaction, execution process, and other contents between the modules of the apparatus, the specific functions and technical effects of the embodiments of the method are based on the same concept, and thus reference may be made to the section of the embodiments of the method specifically, and details are not described here.

Referring to fig. 11, fig. 11 shows a hardware structure of an electronic device according to an embodiment of the present application, where the electronic device includes:

the processor 210 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an Application Specific Integrated Circuit (ASIC), or one or more Integrated circuits, and is configured to execute a related computer program to implement the technical solution provided in the embodiments of the present Application;

the Memory 220 may be implemented in the form of a Read Only Memory (ROM), a static storage device, a dynamic storage device, or a Random Access Memory (RAM). The memory 220 may store an operating system and other application programs, and when the technical solution provided by the embodiments of the present disclosure is implemented by software or firmware, the relevant program codes are stored in the memory 220 and called by the processor 210 to execute the federal learning method for unbalanced sample classes of the embodiments of the present disclosure;

an input/output interface 230 for implementing information input and output;

the communication interface 240 is configured to implement communication interaction between the device and another device, and may implement communication in a wired manner (e.g., USB, network cable, etc.) or in a wireless manner (e.g., mobile network, WIFI, bluetooth, etc.); and a bus 250 that transfers information between each of the components of the device (e.g., the processor 210, the memory 220, the input/output interface 230, and the communication interface 240);

wherein the processor 210, memory 220, input/output interface 230, and communication interface 240 are communicatively coupled to each other within the device via a bus 250.

The embodiment of the application also provides a storage medium which is a computer-readable storage medium and is used for computer-readable storage, wherein the storage medium stores one or more computer programs, and the one or more computer programs can be executed by one or more processors to realize the federal learning method for sample class imbalance.

The memory, which is a computer-readable storage medium, may be used to store software programs as well as computer-executable programs. Further, the memory may include high speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory optionally includes memory located remotely from the processor, and these remote memories may be connected to the processor through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The embodiments described in the embodiments of the present application are for more clearly illustrating the technical solutions of the embodiments of the present application, and do not constitute a limitation to the technical solutions provided in the embodiments of the present application, and it is obvious to those skilled in the art that the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems with the evolution of technology and the emergence of new application scenarios.

The above-described embodiments of the apparatus are merely illustrative, wherein the units illustrated as separate components may or may not be physically separate, i.e. may be located in one place, or may also be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.

It will be understood by those of ordinary skill in the art that all or some of the steps of the methods, systems, and functional modules/units in the devices disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof.

In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.

The terms "first," "second," "third," "fourth," and the like in the description of the application and the above-described figures, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the application described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

It should be understood that in the present application, "at least one" means one or more, "a plurality" means two or more. "and/or" is used to describe the association relationship of the associated object, indicating that there may be three relationships, for example, "a and/or B" may indicate: only A, only B and both A and B are present, wherein A and B may be singular or plural. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. "at least one of the following" or similar expressions refer to any combination of these items, including any combination of single item(s) or plural items. For example, at least one (one) of a, b, or c, may represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", wherein a, b, c may be single or plural.

In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the above-described units is only one type of logical functional division, and other divisions may be realized in practice, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in each embodiment of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware, and also can be implemented in the form of software functional units.

The integrated units, if implemented in the form of software functional units and sold or used as separate products, may be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, which are essential or part of the technical solutions contributing to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product, which is stored in a storage medium and includes multiple instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method of each embodiment of the present application. And the aforementioned storage medium includes: various media capable of storing programs, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

The preferred embodiments of the present application have been described above with reference to the accompanying drawings, and the scope of the claims of the embodiments of the present application is not limited thereby. Any modifications, equivalents and improvements that may occur to those skilled in the art without departing from the scope and spirit of the embodiments of the present application are intended to be within the scope of the claims of the embodiments of the present application.

Claims

1. A federated learning method for sample class imbalance, the method is applied to a client participating in federated learning, and the client is connected with a server in communication, and the method comprises the following steps:

acquiring a dynamic class balance coefficient;

extracting sample data of the category balance coefficient ratio from the majority of sample sets to obtain a target sample set;

and uploading the trained local model to the server.

2. The federal learning method for sample class imbalance according to claim 1, wherein after the dividing the sample data set according to the sample class distribution to obtain a majority sample set and a minority sample set, before the obtaining the dynamic class balance coefficient, the method further comprises:

the extracting the sample data of the class balance coefficient ratio from the majority of sample sets to obtain a target sample set includes:

3. The federal learning method for sample class imbalances as claimed in claim 2, wherein the entropy is determined by the following formula:

4. The federal learning method for sample class imbalances as in claim 1, wherein prior to said obtaining dynamic class balance coefficients, the method further comprises:

acquiring preset training times;

the obtaining of the dynamic class balance coefficient includes:

acquiring a current training round;

5. The federal learning method for sample class imbalance according to claim 4, wherein the value function of the class balance coefficient is determined by the following formula:

θ _k ＝|sin(kπ/T)|；

6. The federated learning method of sample class imbalance of claim 1, wherein the training of a local model based on the minority sample set and the target sample set comprises:

inputting the minority sample set and the target sample set into a local model so as to obtain a classification predicted value corresponding to each sample data in the minority sample set and the target sample set through the local model;

training the local model based on the loss function.

7. The federal learning method for sample class imbalances as in claim 6, wherein the loss function is determined by the following formula:

Is a weighting factor of class j, said y _b As a focusing factor, said

8. A federal learning apparatus for unbalanced sample classes, the apparatus being applied to a client participating in federal learning, the client being in communication with a server, the apparatus comprising:

9. An electronic device, comprising:

at least one processor; and the number of the first and second groups,

a memory communicatively coupled to the at least one processor; wherein,

the memory stores a computer program for execution by the at least one processor to enable the at least one processor to perform a sample class imbalance federal learning method as claimed in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the sample class imbalance federal learning method as claimed in any one of claims 1 to 7.