WO2022110720A1

WO2022110720A1 - Selective gradient updating-based federated modeling method and related device

Info

Publication number: WO2022110720A1
Application number: PCT/CN2021/096651
Authority: WO
Inventors: 李泽远; 王健宗
Original assignee: 平安科技（深圳）有限公司
Priority date: 2020-11-24
Filing date: 2021-05-28
Publication date: 2022-06-02
Also published as: CN112446040A

Abstract

A selective gradient updating-based federated modeling method and a related device, which relate to the technical field of artificial intelligence, and which can be applied in a smart hospital system. The method comprises: client ends read, from a server end, a global model gradient of a machine learning model (S101); client ends initialize the global model gradient (S102); client ends locally execute model training in parallel according to sample data of each of the client ends, and obtain local model gradients corresponding to each of the client ends, the sample data being medical data (S103); client ends encrypt and upload components of the local model gradients of each of the client ends, or clip the local model gradients of each of the client ends to within a preset range and then performing encryption and uploading, enabling the server end to perform aggregation on the components of the local model gradients uploaded by the client ends or the local model gradients after clipping, and to update the machine learning model according to an aggregated average gradient (S104). The present method implements privacy protection for medical data.

Description

Federated Modeling Method and Related Equipment Based on Selective Gradient Update

This application claims the priority of the Chinese patent application filed on November 24, 2020 with the application number 202011327560.X and the invention titled "Federal Modeling Method and Related Equipment Based on Selective Gradient Update", all of which The contents are incorporated herein by reference.

technical field

This application relates to the field of artificial intelligence technology and digital medicine, specifically to medical informatization, and in particular to a federal modeling method based on selective gradient update and related equipment.

Background technique

Medical data is different from general industry data because of its sensitivity and importance. The law has also formulated a very strict protection mechanism for the privacy of medical data. Medical data includes medical record information, medical insurance information, health logs, genetics, medical experiments, scientific research data, etc. Among them, personal medical record information, medical insurance information and other medical data are related to personal privacy and security, while medical experimental data, scientific research data, etc. It is related to the development of the medical industry and even national security. Therefore, it is not feasible to share data among various hospitals and then centrally train them to improve the accuracy of various disease prediction models.

However, using a deep convolutional neural network to train a model usually requires a large number of different training sample sets. The existing technology has implemented federated learning to break the data barriers between hospitals. The data does not go out of the hospital, and only the encrypted local model gradient needs to be uploaded. Federated model training can be performed to improve model performance. Although federated learning can improve security in terms of privacy, the inventors realized that it still has the potential for training data leakage, such as the ability to reconstruct a single training model through model inversion.

Application content

The purpose of this application is to provide a federated modeling method and related equipment based on selective gradient update, which aims to solve the problem that privacy is easily leaked in the existing federated modeling method based on medical data.

In a first aspect, an embodiment of the present application provides a selective gradient update-based federated modeling method, including:

Each client reads the global model gradient of the machine learning model from the server;

Each of the clients initializes the global model gradient;

Each of the clients performs model training locally in parallel according to their respective sample data, to obtain a local model gradient corresponding to each of the clients; the sample data is medical data;

Each of the clients encrypts and uploads the components of their respective local model gradients, or clips their respective local model gradients to within a preset range and then encrypts and uploads them, so that the server side can compare the local model gradients uploaded by each client. The components or the clipped local model gradients are aggregated, and the machine learning model is updated according to the average gradient obtained by the aggregation.

In a second aspect, an embodiment of the present application provides a selective gradient update-based federated modeling apparatus, including:

The reading unit is used to read the global model gradient of the machine learning model from the server;

an initialization unit for initializing the global model gradient;

a training unit, configured to perform model training locally according to the respective sample data in parallel to obtain the local model gradient corresponding to each of the clients; the sample data is medical data;

The encryption uploading unit is used to encrypt and upload the components of the respective local model gradients, or to clip the respective local model gradients to a preset range before encrypting and uploading, so that the server side uploads the local model gradients from each client. The components or the clipped local model gradients are aggregated, and the machine learning model is updated according to the average gradient obtained by the aggregation.

In a third aspect, an embodiment of the present application provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the computer program When the federated modeling method based on selective gradient update as described in the first aspect is implemented.

In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when executed by a processor, the computer program causes the processor to execute the first The selective gradient update-based federated modeling method described in the aspect.

The embodiments of the present application provide a federated modeling method and related equipment based on selective gradient update. The method includes: each client reads the global model gradient of the machine learning model from the server; Perform initialization; each of the clients performs model training locally according to their respective sample data in parallel to obtain the local model gradient corresponding to each of the clients; each of the clients encrypts and uploads the components of the respective local model gradients, Or clip the respective local model gradients to a preset range, encrypt and upload them, so that the server side aggregates the components of the local model gradients uploaded by each client or the clipped local model gradients, and according to the average gradient obtained by the aggregation Update the machine learning model. The embodiments of the present application apply the selective gradient update technology to protect medical data, and more effectively protect the data security of patients and hospitals.

Description of drawings

In order to explain the technical solutions of the embodiments of the present application more clearly, the following briefly introduces the accompanying drawings used in the description of the embodiments. For those of ordinary skill, other drawings can also be obtained from these drawings without any creative effort.

1 is a schematic flowchart of a federated modeling method based on selective gradient update provided by an embodiment of the present application;

FIG. 2 is a schematic sub-flow diagram of a federated modeling method based on selective gradient update provided by an embodiment of the present application;

3 is a schematic diagram of another sub-flow of the federated modeling method based on selective gradient update provided by an embodiment of the present application;

4 is a schematic diagram of another sub-flow of the federated modeling method based on selective gradient update provided by an embodiment of the present application;

FIG. 5 is a schematic block diagram of a federated modeling apparatus based on selective gradient update provided by an embodiment of the present application;

FIG. 6 is a schematic block diagram of subunits of a federated modeling apparatus based on selective gradient update provided by an embodiment of the present application;

FIG. 7 is a schematic block diagram of another subunit of the federated modeling apparatus based on selective gradient update provided by an embodiment of the present application;

FIG. 8 is a schematic block diagram of another subunit of the federated modeling apparatus based on selective gradient update provided by an embodiment of the present application;

FIG. 9 is a schematic block diagram of a computer device according to an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

It is to be understood that, when used in this specification and the appended claims, the terms "comprising" and "comprising" indicate the presence of the described features, integers, steps, operations, elements and/or components, but do not exclude one or The presence or addition of a number of other features, integers, steps, operations, elements, components, and/or sets thereof.

It should also be understood that the terminology used in the specification of the application herein is for the purpose of describing particular embodiments only and is not intended to limit the application. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural unless the context clearly dictates otherwise.

It should also be further understood that, as used in this specification and the appended claims, the term "and/or" refers to and including any and all possible combinations of one or more of the associated listed items .

Please refer to FIG. 1. FIG. 1 is a schematic flowchart of a federated modeling method based on selective gradient update provided by an embodiment of the present application, which includes steps S101-S104:

S101, each client reads the global model gradient of the machine learning model from the server;

The client in the embodiment of the present application may refer to the local hospital terminal, and joint modeling is performed between the local hospital terminals under the premise of encryption, so as to improve the overall modeling effect.

Each client has a fixed local dataset and suitable computing resources to run mini-batch SGD (stochastic gradient descent) updates. Each client shares the same machine learning model neural network structure and loss function from the server.

During the joint training t rounds of iterations, each client reads the global model gradient W ^(t) of the machine learning model from the server.

S102, each client initializes the global model gradient;

In this step, each client initializes the global model gradient W ^(t) , that is, initializes it to the local model gradient W ^(0,t) .

S103, each of the clients performs model training locally in parallel according to their respective sample data, to obtain a local model gradient corresponding to each of the clients; the sample data is medical data;

In this step, each client performs model training locally according to its own sample data, and updates the local model gradient W ^{(0, t)} to the local model gradient W ^(l, t) by running multiple times of stochastic gradient descent (SGD). ^t) , where l refers to the loss function, and the updated local model gradient can be subsequently expressed as ΔW ^(t) .

The sample data in the embodiment of the present application is a local data set, which includes medical record information, medical insurance information, health logs, genetic inheritance, medical experiments, scientific research data, and the like.

S104: Each of the clients encrypts and uploads the components of their respective local model gradients, or clips their respective local model gradients to within a preset range and then encrypts and uploads them, so that the server side can perform encryption on the local model uploaded by each client. The components of the gradient or the clipped local model gradients are aggregated, and the machine learning model is updated according to the aggregated average gradient.

In this step, after each round of local model training, the local model gradient ΔW ^{(t) is} encrypted and uploaded. Since in the above local model training process, the model reverse attack can extract some patient privacy information from the updated local model gradient ΔW ^(t) or the global model gradient W ^(t) in joint training. Therefore, the embodiment of the present application adopts a selective gradient update method to select and update the gradient, and by limiting the gradient uploaded by the local hospital, it provides strong protection against indirect data leakage during the model training process.

Specifically, the embodiment of the present application can implement selective gradient update in two ways. One is to encrypt and upload the components of the local model gradient, and the other is to clip the local model gradient to a preset range and then encrypt and upload it. . After the server side receives the local model gradient or its components uploaded by each client, aggregation can be performed. polymerization. The machine learning model is then updated based on the aggregated average gradients. The two methods are described in detail below.

In one embodiment, as shown in FIG. 2 , the steps of encrypting and uploading the components of the respective local model gradients include steps S201 to S203:

S201, randomly selecting a component from the local model gradient;

S202, comparing the absolute value of the randomly selected component with a preset threshold;

S203. If the absolute value of the component is greater than the preset threshold, encrypt and upload the component.

In this embodiment, a component W _i is randomly selected from the local model gradient ΔW ^(t) , and then the absolute value abs(W _i ₎ of the randomly selected component Wi is compared with the preset threshold value

For comparison, if the absolute value of the component is greater than the preset threshold, it indicates that the component is sufficiently representative, that is, it can represent the gradient of the local training of the corresponding client in this round, so the component can be encrypted and uploaded.

In an embodiment, before the step S201, it further includes:

The absolute value of the local model gradient is calculated, and the preset threshold is determined according to the percentile of the absolute value of the local model gradient.

In this embodiment, τ ^(t) is determined by the percentile of the absolute value of the local model gradient ΔW ( ^t ), so first calculate the absolute value of the local model gradient ΔW ^(t) abs(ΔW ^(t) ), then obtain the percentile of the absolute value abs(ΔW ^(t) ), and then determine the preset threshold according to the percentile. Percentile is used for descriptive analysis of data, it refers to a location metric, a measure used to measure the location of data, giving information on the distribution of data between minimum and maximum values. For a certain group of data, first sort the group of data from small to large, and calculate the corresponding cumulative percentile, then the value of the data corresponding to a certain percentile is called the percentile of this percentile. In short, a set of data with n values is arranged in ascending order of value, and the value at the p% position is called the pth percentile. Therefore, in this embodiment of the present application, the percentile of the absolute value of the local model gradient of each client in the absolute value of the local model gradients of all clients can be obtained, thereby determining the preset threshold of each client.

In one embodiment, the step S203 includes:

Add noise to the absolute values of the components and upload.

In this step, the encryption method is to add noise to the absolute value of the component, and then upload it. In this way, gradient information is not easily cracked, thereby further protecting medical data from leakage.

In an embodiment, as shown in FIG. 3 , adding noise to the absolute value of the component and then uploading it includes steps S301 to S303:

S301, comparing the absolute value of the component with a noise threshold;

S302. If the absolute value of the component is less than the noise threshold, add noise to the component;

S303 , trim the component after adding the noise to the component threshold range, and upload it.

In this embodiment, the absolute value of the component is first compared with the noise threshold. If the absolute value of the component is smaller than the noise threshold, it means that noise can be added to the component, and then the component after adding the noise is clipped to the component Within the threshold range, upload again.

In one embodiment, the step S303 includes:

The noise-added component Wi is _trimmed into the component threshold range as follows, and uploaded:

Lap(x) represents a random variable sampled from the Laplacian distribution of the gradient x; ε ₂ represents the privacy budget of the noise threshold; clip(x, γ) represents the clipped gradient domain range of the gradient x is [-γ, γ] ; s denotes the gradient sensitivity bounded by γ, and q denotes the number of shared gradients computed.

Wherein, the noise threshold can be

That is, the gradient is added on the basis of the preset threshold

A random variable sampled from the Laplace distribution.

In one embodiment, as shown in FIG. 4 , the clipping of the respective local model gradients to a preset range before encryption and uploading includes steps S401 to S403:

S401, obtaining the part exceeding the upper threshold and the part lower than the lower threshold in the local model gradient;

S402, replacing the part exceeding the upper limit threshold with the upper limit threshold, and replacing the part below the lower limit threshold with the lower limit threshold;

S403. Combine the replaced upper threshold value, lower threshold value and the unreplaced part into a new local model gradient, encrypt and upload it.

In this embodiment, the gradient of the local model is clipped so that it is within a preset range. Specifically, the part of the gradient of the local model that exceeds the upper threshold and the part that is lower than the lower threshold can be obtained first, and then the part exceeding the upper threshold is replaced with the upper threshold, and the part lower than the lower threshold is replaced with the lower threshold, so as to realize the The local model gradient is mapped to a preset range, and the replaced upper threshold, lower threshold and the unreplaced part are combined into a new local model gradient, and then encrypted and uploaded. The part that is not replaced is the part that is originally within the preset range. The combination refers to the combination according to the structure of the original local model gradient, so that the local model gradient can reflect the real situation.

For this cropping method, the aforementioned encryption principle can also be used for encryption, that is, adding noise to the gradient of the new local model, and then encrypting and uploading it.

Please refer to FIG. 5 , which is a schematic block diagram of a federated modeling apparatus based on selective gradient update provided by an embodiment of the present application. The federated modeling apparatus 500 based on selective gradient update includes:

A reading unit 501 is used to read the global model gradient of the machine learning model from the server;

an initialization unit 502, configured to initialize the global model gradient;

A training unit 503, configured to perform model training locally according to the respective sample data in parallel to obtain the local model gradient corresponding to each of the clients; the sample data is medical data;

The encryption uploading unit 504 is used for encrypting and uploading the components of the respective local model gradients, or clipping the respective local model gradients to within a preset range and then encrypting and uploading them, so that the server can upload the local model uploaded by each client. The components of the gradient or the clipped local model gradients are aggregated, and the machine learning model is updated according to the aggregated average gradient.

In one embodiment, as shown in FIG. 6 , the encryption uploading unit 504 includes:

a random selection unit 601, configured to randomly select a component from the local model gradient;

a component comparison unit 602, configured to compare the absolute value of the randomly selected component with a preset threshold;

The uploading unit 603 is configured to encrypt and upload the component if the absolute value of the component is greater than the preset threshold.

In one embodiment, the encryption uploading unit 504 further includes:

A preset threshold determination unit, configured to calculate the absolute value of the gradient of the local model, and determine the preset threshold according to the percentile of the absolute value of the gradient of the local model.

In one embodiment, the uploading unit 603 includes:

The noise adding unit is used for adding noise to the absolute value of the component, and then uploading.

In one embodiment, as shown in FIG. 7 , the noise adding unit includes:

a noise comparison unit 701, configured to compare the absolute value of the component with a noise threshold;

A noise setting unit 702, configured to add noise to the component if the absolute value of the component is less than the noise threshold;

A component clipping unit 703, configured to clip the noise-added component to be within the component threshold range, and upload the component.

In one embodiment, the component cropping unit includes:

The trimming subunit is used to _trim the noise-added component Wi to be within the component threshold range, and upload it as follows:

In one embodiment, as shown in FIG. 8 , the encryption uploading unit 504 further includes:

an obtaining unit 801, configured to obtain the part exceeding the upper threshold and the part lower than the lower threshold in the local model gradient;

a replacement unit 802, configured to replace the part exceeding the upper limit threshold with an upper limit threshold, and replace the part below the lower limit threshold with a lower limit threshold;

The combining unit 803 is configured to combine the replaced upper threshold value, the lower threshold value and the unreplaced part into a new local model gradient, encrypt and upload it.

The device of the embodiment of the present application applies the selective gradient update technology to protect medical data, and more effectively protects the data security of patients and hospitals.

The above-mentioned selective gradient update-based federated modeling apparatus 500 can be implemented in the form of a computer program, and the computer program can be executed on a computer device as shown in FIG. 9 .

Please refer to FIG. 9 , which is a schematic block diagram of a computer device provided by an embodiment of the present application. The computer device 900 is a server, and the server may be an independent server or a server cluster composed of multiple servers.

Referring to FIG. 9 , the computer device 900 includes a processor 902 , a memory and a network interface 905 connected by a system bus 901 , wherein the memory may include a non-volatile storage medium 903 and an internal memory 904 .

The nonvolatile storage medium 903 can store an operating system 9031 and a computer program 9032 . The computer program 9032, when executed, can cause the processor 902 to perform a selective gradient update based federated modeling method.

The processor 902 is used to provide computing and control capabilities to support the operation of the entire computer device 900 .

The internal memory 904 provides an environment for the execution of a computer program 9032 in the non-volatile storage medium 903, the computer program 9032, when executed by the processor 902, can cause the processor 902 to execute the selective gradient update based federated modeling method.

The network interface 905 is used for network communication, such as providing transmission of data information. Those skilled in the art can understand that the structure shown in FIG. 9 is only a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device 900 to which the solution of the present application is applied. The specific computer device 900 may include more or fewer components than shown, or combine certain components, or have a different arrangement of components.

The processor 902 is configured to run the computer program 9032 stored in the memory, so as to realize the following functions: each client reads the global model gradient of the machine learning model from the server; The gradient is initialized; each of the clients performs model training locally according to their respective sample data in parallel to obtain the local model gradient corresponding to each of the clients; each of the clients encrypts and uploads the components of the respective local model gradients , or clip the respective local model gradients to a preset range before encrypting and uploading, so that the server side aggregates the components of the local model gradients uploaded by each client or the clipped local model gradients, and according to the average obtained by the aggregation Gradients update the machine learning model.

In one embodiment, when performing the step of encrypting and uploading the components of the respective local model gradients, the processor 902 performs the following operations: randomly select a component from the local model gradients; The absolute value of the component is compared with a preset threshold; if the absolute value of the component is greater than the preset threshold, the component is encrypted and uploaded.

In one embodiment, before executing the step of randomly selecting a component from the local model gradient, the processor 902 performs the following operations: calculating the absolute value of the local model gradient, and calculating an absolute value of the local model gradient according to the The percentile of absolute values determines the preset threshold.

In one embodiment, when the processor 902 performs the step of encrypting and uploading the component if the absolute value of the component is greater than the preset threshold, the processor 902 performs the following operations: Add noise to the value and upload it.

In one embodiment, when performing the step of adding noise to the absolute value of the component and then uploading, the processor 902 performs the following operations: compare the absolute value of the component with a noise threshold; If the absolute value of the component is less than the noise threshold, then add noise to the component; trim the component after adding the noise to the range of the component threshold, and upload it.

In one embodiment, when the processor 902 performs the step of clipping the noise-added component to the component threshold range and uploading, the processor 902 performs the following operations: the noise-added component W _i Crop to within component thresholds and upload:

In one embodiment, when the processor 902 performs the step of clipping the respective local model gradients to a preset range before encrypting and uploading them, the processor 902 performs the following operations: acquiring the part of the local model gradients that exceeds the upper threshold value; and the part lower than the lower threshold; the part that exceeds the upper threshold is replaced with the upper threshold, and the part lower than the lower threshold is replaced with the lower threshold; the replaced upper threshold, lower threshold and the unreplaced part are combined as New local model gradients, encrypted and uploaded.

Those skilled in the art can understand that the embodiment of the computer device shown in FIG. 9 does not constitute a limitation on the specific structure of the computer device. In other embodiments, the computer device may include more or less components than those shown in the drawings. Either some components are combined, or different component arrangements. For example, in some embodiments, the computer device may only include a memory and a processor. In such an embodiment, the structures and functions of the memory and the processor are the same as those of the embodiment shown in FIG. 9 , and details are not repeated here.

It should be understood that, in this embodiment of the present application, the processor 902 may be a central processing unit (Central Processing Unit, CPU), and the processor 902 may also be other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), Application Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Wherein, the general-purpose processor can be a microprocessor or the processor can also be any conventional processor or the like.

In another embodiment of the present application, a computer-readable storage medium is provided. The computer-readable storage medium may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. The computer-readable storage medium stores a computer program, wherein when the computer program is executed by the processor, the following steps are implemented: each client reads the global model gradient of the machine learning model from the server; Perform initialization; each of the clients performs model training locally according to their respective sample data in parallel to obtain the local model gradient corresponding to each of the clients; each of the clients encrypts and uploads the components of the respective local model gradients, Or clip the respective local model gradients to a preset range, encrypt and upload them, so that the server side aggregates the components of the local model gradients uploaded by each client or the clipped local model gradients, and according to the average gradient obtained by the aggregation Update the machine learning model.

Those skilled in the art can clearly understand that, for the convenience and brevity of description, for the specific working process of the above-described devices, devices and units, reference may be made to the corresponding processes in the foregoing method embodiments, which will not be repeated here. Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. Interchangeability, the above description has generally described the components and steps of each example in terms of function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of this application.

In the several embodiments provided in this application, it should be understood that the disclosed apparatus, apparatus and method may be implemented in other manners. For example, the device embodiments described above are only illustrative. For example, the division of the units is only logical function division. In actual implementation, there may be other division methods, or units with the same function may be grouped into one Units, such as multiple units or components, may be combined or may be integrated into another system, or some features may be omitted, or not implemented. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may also be electrical, mechanical or other forms of connection.

The units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solutions of the embodiments of the present application.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a storage medium. Based on this understanding, the technical solutions of the present application are essentially or part of contributions to the prior art, or all or part of the technical solutions can be embodied in the form of software products, and the computer software products are stored in a storage medium , including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a removable hard disk, a read-only memory (ROM, Read-Only Memory), a magnetic disk or an optical disk and other media that can store program codes.

The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of various equivalents within the technical scope disclosed in the present application. Modifications or substitutions shall be covered by the protection scope of this application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

A federated modeling method based on selective gradient update, which includes:

Each client reads the global model gradient of the machine learning model from the server;

Each of the clients initializes the global model gradient;

Each of the clients performs model training locally in parallel according to their respective sample data, to obtain a local model gradient corresponding to each of the clients; the sample data is medical data;

Each of the clients encrypts and uploads the components of their respective local model gradients, or clips their respective local model gradients to within a preset range and then encrypts and uploads them, so that the server side can compare the local model gradients uploaded by each client. The components or the clipped local model gradients are aggregated, and the machine learning model is updated according to the average gradient obtained by the aggregation.
The federated modeling method based on selective gradient update according to claim 1, wherein said encrypting and uploading the components of respective local model gradients comprises:

randomly select a component from the local model gradient;

comparing the randomly selected absolute value of the component with a preset threshold;

If the absolute value of the component is greater than the preset threshold, the component is encrypted and uploaded.
The federated modeling method based on selective gradient update according to claim 2, wherein before randomly selecting a component from the local model gradient, the method further comprises:

The absolute value of the local model gradient is calculated, and the preset threshold is determined according to the percentile of the absolute value of the local model gradient.
The federated modeling method based on selective gradient update according to claim 2, wherein, if the absolute value of the component is greater than the preset threshold, encrypting and uploading the component, comprising:

Add noise to the absolute values of the components and upload.
The federated modeling method based on selective gradient update according to claim 4, wherein the adding noise to the absolute value of the component, and then uploading, comprises:

comparing the absolute value of the component to a noise threshold;

if the absolute value of the component is less than the noise threshold, adding noise to the component;

Trim the component after adding noise to the component threshold range and upload it.
The federated modeling method based on selective gradient update according to claim 5, wherein the clipping the components after adding noise to the component threshold range and uploading, comprising:

The noise-added component Wi is trimmed into the component threshold range as follows, and uploaded:

Lap(x) represents a random variable sampled from the Laplacian distribution of the gradient x; ε 2 represents the privacy budget of the noise threshold; clip(x, γ) represents the clipped gradient domain range of the gradient x is [-γ, γ] ; s denotes the gradient sensitivity bounded by γ, and q denotes the number of shared gradients computed.
The federated modeling method based on selective gradient update according to claim 1, wherein said clipping the respective local model gradients to a preset range before encrypting and uploading, comprising:

Obtain the part exceeding the upper threshold and the part lower than the lower threshold in the gradient of the local model;

The part exceeding the upper threshold is replaced by the upper threshold, and the part below the lower threshold is replaced by the lower threshold;

Combine the replaced upper threshold, lower threshold and unreplaced parts into a new local model gradient, encrypt and upload.
A federated modeling apparatus based on selective gradient update, comprising:

The reading unit is used to read the global model gradient of the machine learning model from the server;

an initialization unit for initializing the global model gradient;

a training unit, configured to perform model training locally according to the respective sample data in parallel to obtain the local model gradient corresponding to each of the clients; the sample data is medical data;

The encryption uploading unit is used to encrypt and upload the components of the respective local model gradients, or to clip the respective local model gradients to a preset range before encrypting and uploading, so that the server side uploads the local model gradients from each client. The components or the clipped local model gradients are aggregated, and the machine learning model is updated according to the average gradient obtained by the aggregation.
A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the computer program according to claim 1 when executing the computer program Federated Modeling Method Based on Selective Gradient Update.
The computer device according to claim 9, wherein said encrypting and uploading the components of the respective local model gradients comprises:

randomly select a component from the local model gradient;

comparing the randomly selected absolute value of the component with a preset threshold;

If the absolute value of the component is greater than the preset threshold, the component is encrypted and uploaded.
The computer device of claim 10, wherein before randomly selecting a component from the local model gradient, further comprising:

The absolute value of the local model gradient is calculated, and the preset threshold is determined according to the percentile of the absolute value of the local model gradient.
The computer device according to claim 10, wherein, if the absolute value of the component is greater than the preset threshold, encrypting and uploading the component, comprising:

Add noise to the absolute values of the components and upload.
The computer device according to claim 12, wherein the adding noise to the absolute value of the component and then uploading comprises:

comparing the absolute value of the component to a noise threshold;

if the absolute value of the component is less than the noise threshold, adding noise to the component;

Trim the component after adding noise to the component threshold range and upload it.
The computer device according to claim 13, wherein the clipping the noise-added component to be within a component threshold range, and uploading, comprises:

The noise-added component Wi is trimmed into the component threshold range as follows, and uploaded:

Lap(x) represents a random variable sampled from the Laplacian distribution of the gradient x; ε 2 represents the privacy budget of the noise threshold; clip(x, γ) represents the clipped gradient domain range of the gradient x is [-γ, γ] ; s denotes the gradient sensitivity bounded by γ, and q denotes the number of shared gradients computed.
A computer-readable storage medium, wherein the computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to perform the selective gradient-based update of claim 1 federated modeling approach.
The computer-readable storage medium of claim 15, wherein said encrypting and uploading components of respective local model gradients comprises:

randomly select a component from the local model gradient;

comparing the randomly selected absolute value of the component with a preset threshold;

If the absolute value of the component is greater than the preset threshold, the component is encrypted and uploaded.
The computer-readable storage medium of claim 16, wherein before randomly selecting a component from the local model gradient, further comprising:

The absolute value of the local model gradient is calculated, and the preset threshold is determined according to the percentile of the absolute value of the local model gradient.
The computer-readable storage medium according to claim 16, wherein, if the absolute value of the component is greater than the preset threshold, encrypting and uploading the component comprises:

Add noise to the absolute values of the components and upload.
The computer-readable storage medium of claim 18, wherein the adding noise to the absolute value of the component and then uploading comprises:

comparing the absolute value of the component to a noise threshold;

if the absolute value of the component is less than the noise threshold, adding noise to the component;

Trim the component after adding noise to the component threshold range and upload it.
The computer-readable storage medium according to claim 19, wherein the clipping the noise-added component to be within a component threshold range and uploading, comprising:

The noise-added component Wi is trimmed into the component threshold range as follows, and uploaded:

Lap(x) represents a random variable sampled from the Laplacian distribution of the gradient x; ε 2 represents the privacy budget of the noise threshold; clip(x, γ) represents the clipped gradient domain range of the gradient x is [-γ, γ] ; s denotes the gradient sensitivity bounded by γ, and q denotes the number of shared gradients computed.