CN114492827A - Block chain technology-based federated learning model watermark reinforcement method and application - Google Patents
Block chain technology-based federated learning model watermark reinforcement method and application Download PDFInfo
- Publication number
- CN114492827A CN114492827A CN202111488625.3A CN202111488625A CN114492827A CN 114492827 A CN114492827 A CN 114492827A CN 202111488625 A CN202111488625 A CN 202111488625A CN 114492827 A CN114492827 A CN 114492827A
- Authority
- CN
- China
- Prior art keywords
- model
- watermark
- information
- block chain
- learning model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
- G06F21/6245—Protecting personal data, e.g. for financial or medical purposes
Abstract
The invention discloses a block chain technology-based federated learning model watermark reinforcement method and application, comprising the following steps: generating watermark information through convolution kernel information obtained from the model and a randomly generated projection matrix, and adding the watermark information set identity information as a regular item to an original loss function to optimize model parameters, so that the watermark information and the identity information are implanted into the model, and on the basis of ensuring the robustness of the model, the model is reinforced through the watermark information and the identity information to identify the model. On the basis, the block chain accounting book technology is adopted to uniformly record the model gradient information, the projection matrix, the identity information and the convolution kernel information, so that the information is prevented from being omitted and damaged; the extraction and verification of the provided watermark information realize the ownership authentication of the model, and the privacy safety and property right protection of the federal learning model are protected.
Description
Technical Field
The invention belongs to the field of data security, and particularly relates to a block chain technology-based federated learning model watermark reinforcement method and application.
Background
In recent years, artificial intelligence technology is widely applied in many fields such as image recognition, target detection, bio-pharmaceuticals, financial wind control, automatic driving, network security and the like. The machine learning technology based on data driving has stable and accurate effect on tasks such as identification and classification, and in a plurality of specific tasks, the scheme based on the machine learning technology can not only achieve better effect than the traditional technical scheme, but also can complete tasks which are difficult to complete by the traditional technology.
Since training a good model often requires a large amount of high quality data and computational resources, deep learning models obtained by training often have significant value. However, these valuable depth models face a significant risk of infringement. Once an attacker has complete information about a target model, including network structure and weights, the attacker can easily fine-tune the target model over a new data set. Even if an attacker has access to only the output of the target model, another similar surrogate model can still be trained by generating a large-scale input-output training pair. How to protect the intellectual property of the deep learning model is a very important but urgent problem to be researched. Once the model is stolen, the rights of the owner of the model are directly damaged, and meanwhile, how to determine the intellectual property attribution of the deep learning model becomes a great problem.
Federal Learning (fed Learning) is a distributed Learning framework which is gradually developed in recent years, and a plurality of participants share training data on the premise of protecting privacy and safety through a specific model training mode and structure. The federal learning can guarantee the information safety during big data exchange, protect the privacy of terminal data and personal data, and carry out high-efficiency deep learning model training among multiple parties or multiple computing nodes on the premise of guaranteeing legal compliance. Currently, federal learning is classified into horizontal federal learning, vertical federal learning and federal transfer learning according to differences of a data feature space and a sample space. The horizontal federation is suitable for the condition that the feature space of the data is overlapped more but the sample space is overlapped less, the vertical federation is suitable for the condition that the sample space of the data is overlapped more but the feature space is overlapped less, and the federation transfer learning is directed at the condition that the feature space and the sample space are different.
The block chain is a decentralized, data-encrypted and non-falsifiable distributed shared database, and can provide data confidentiality for data exchange of federal learning, and ensure data security among all participants and data consistency of model training. The decentralized technology and the ledger technology of the block chain can also increase the credibility of providing data and updating network model parameters among all the participants.
In the financial and medical fields, although the privacy security of each edge data can be guaranteed by the training mode of the federal learning model, the federal learning model cannot be copied and stolen by a malicious attacker in the training process. Since the edge terminals come from different organizations or organizations and mutual trust relationship is not established between the edge terminals, the edge terminal attacker cannot be determined. Once the federal learning model in federal learning is stolen by a malicious attacker, the rights and interests of all edge end clients can be damaged, and even a huge threat can be caused to the data security of each edge end.
Disclosure of Invention
In view of the foregoing, a first aspect of the present invention is to provide a block chain technology-based federated learning model watermark reinforcing method, which improves the privacy security of a federated learning model by performing watermark reinforcement on the federated learning model, and prevents a malicious attacker from embezzled the federated learning model.
To achieve the first object, an embodiment of the present invention provides a block chain technology-based federated learning model watermark reinforcing method, including the following steps:
initializing a model, and distributing the model to each edge end;
each edge end trains the distributed model by using local sample data, generates watermark information according to convolution kernel information obtained from the model and a randomly generated projection matrix during training, adds the watermark information to an original loss function by using identity information as a regular term, and optimizes model parameters by using a new loss function added with the regular term so as to implant watermark information and identity information in the model;
after the current round of training is finished, uploading model gradient information, a projection matrix, identity information and convolution kernel information to a block chain account book, and storing the block chain account book into each block chain node in a transaction form;
each edge end determines the attribution of the block right according to the workload certification, obtains the edge end of the block right as a temporary server of the current round, aggregates model gradient information uploaded by each edge end to obtain a federal learning model and broadcasts the federal learning model to a block chain account book;
and each edge terminal downloads the aggregation model from the block chain account book for the next round of training.
In one embodiment, when initializing a model, setting a model structure, an initial weight and a bias parameter, and simultaneously setting a training round and the number of edge-end devices participating in federal learning;
and setting training related parameters including optimizer type, learning rate, loss function and model parameter solving algorithm.
In one embodiment, the generating watermark information from convolution kernel information derived from the model and a randomly generated projection matrix includes:
during each round of training, extracting the weight of all convolution kernels of a certain convolution layer from the model, carrying out averaging operation, and taking the average operation result as a convolution kernel vector corresponding to convolution kernel information; randomly generating a projection matrix meeting normal distribution;
and after the projection matrix is multiplied by the convolution kernel vector, the watermark information is activated by an activation function to obtain a watermark implantation vector corresponding to the watermark information. Preferably, the activation function adopts a sigmoid activation function.
In one embodiment, the adding of the watermark information in combination with the identity information as a regularization term to the original loss function comprises:
and (4) performing cross entropy on the identity information vector converted by the identity information and the watermark implantation vector corresponding to the watermark information, and adding the cross entropy serving as a regular term to the original loss function to obtain a new loss function added with the regular term.
In one embodiment, the federated learning model further includes: watermark verification is carried out on the federal learning model by each edge end, and the method comprises the following steps:
extracting all convolution kernels of convolution layers adopted when watermark information is generated from a federal learning model, and averaging the weights of all the convolution kernels to obtain convolution kernel vectors;
and obtaining a projection matrix recorded before from the block chain account book, multiplying the projection matrix by a convolution kernel vector, processing the obtained result by a step function to obtain extraction information, comparing the extraction information with identity information recorded before from the block chain account book, and determining whether the federal learning model is embedded with a watermark or not according to the comparison result so as to eliminate an unreliable edge end and a trained model thereof.
In one embodiment, a step function is used to process the result of multiplying the projection matrix by the convolution kernel vector into a bit sequence consisting of 0 and 1 as the extraction information.
In one embodiment, the randomly generated projection matrix and the identity information for each edge terminal are set to be the same.
In the block chain technology-based federal learning model watermark reinforcement method provided in the above embodiment, the watermark information is generated by convolution kernel information obtained from the model and a randomly generated projection matrix, and the identity information of the watermark information set is added to the original loss function as a regular term to optimize the model parameters, so that the watermark information and the identity information are implanted into the model, and the model is reinforced by the watermark information and the identity information on the basis of ensuring the robustness of the model, so as to identify the model. On the basis, the block chain accounting book technology is adopted to uniformly record the model gradient information, the projection matrix, the identity information and the convolution kernel information, so that the information is prevented from being omitted and damaged; the extraction and verification of the provided watermark information realize the ownership authentication of the model, and the privacy safety and property right protection of the federal learning model are protected.
The second objective of the present invention is to provide a protection method for a demand analysis and prediction model applied in the financial field, which can ensure the accuracy of the demand analysis and prediction model, protect the privacy security of the bank owning the user data and the rights and interests of the demand analysis and prediction model of the bank as the edge end, prevent the demand analysis and prediction model from being stolen, and facilitate the bank to prove the ownership of the demand analysis and prediction model.
In order to achieve the second object, the embodiment provides a protection method for a demand analysis and prediction model applied in the financial field, each bank serves as an edge end, user data used for training the demand analysis and prediction model is used as sample data, the demand analysis and prediction model is used for analyzing and predicting the financial demand of a user according to the user data, and the demand analysis and prediction model is obtained through the block chain technology-based federal learning model watermark reinforcement method provided by the embodiment, so that the demand analysis and prediction model reinforced by watermarks can be prevented from being attacked and stolen.
A third objective of the present invention is to provide a method for protecting a physical status analysis model applied in the medical field, which can ensure the accuracy of the physical status analysis model, protect the privacy and security of doctor-patient data owned by the medical institution and the rights and interests of the physical status analysis model of the medical institution, prevent the physical status analysis model from being stolen, and facilitate the bank to prove the ownership of the physical status analysis model.
In order to achieve the third object, embodiments provide a method for protecting a body state analysis model applied in the medical field, where each medical institution serves as an edge, and possesses doctor-patient data used for training the body state analysis model as sample data, the body state analysis model is used for analyzing and predicting the body state of a patient according to the doctor-patient data, and the body state analysis model is obtained by the block chain technology-based federal learning model watermark reinforcement method provided in the above embodiments, so that the body state analysis model reinforced by the watermark can be prevented from being attacked and stolen.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
Fig. 1 is a flowchart of a block chain technique-based federated learning model watermark reinforcement method provided by an embodiment;
fig. 2 is a flowchart of a watermark information injection model provided by the embodiment;
fig. 3 is a flowchart of model watermark extraction and verification provided by the embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the detailed description and specific examples, while indicating the scope of the invention, are intended for purposes of illustration only and are not intended to limit the scope of the invention.
In the process of carrying out federated learning model training and interaction between the edge end and the server end under the federated learning scene, the federated learning model trained by the federated learning framework is easily stolen, and an attacker realizes the stealing of the federated learning model by acquiring information uploaded to the server end by the edge end and utilizing reverse gradient calculation and loss-based gradient feature extraction. In order to improve the privacy safety of a federal learning model in a federal learning scene and prevent a malicious attacker from embezzlement of the federal learning model, the embodiment provides a block chain technology-based federal learning model watermark reinforcement method.
Fig. 1 is a flowchart of a block chain technique-based federated learning model watermark reinforcement method provided in an embodiment. As shown in fig. 1, an embodiment of the method for reinforcing a watermark based on a block chain technology in a federal learning model includes the following steps:
step 1, data preprocessing, including data set preprocessing and model initialization and distribution.
The participated federated learning comprises a plurality of edge terminals, each edge terminal possesses a local data set, and a trained federated learning model is used for image recognition or image classification by taking an image data set as an example. In an embodiment, the data sets used include a small image handwritten digit MNIST data set and a large image data set ImageNet. The data preprocessing comprises the following steps: the MNIST data set comprises ten types of handwritten Arabic digital images of 0-9, the size of each sample is 28 x 28, 6000 samples of each type are counted, wherein 5000 samples are training sets, and 1000 samples are testing sets; the ImageNet dataset is an image dataset organized in a WordNet hierarchical structure and comprises 1000 classes, each class comprises 1000 samples, the size of each sample is 224 multiplied by 3, in an experiment, 30% of pictures in each class are randomly extracted as a test set, and the rest of pictures are used as a training set.
In an embodiment, the federal learning framework can be horizontal federal learning, vertical federal learning and federal transfer learning, and is selected to be suitable mainly according to sample space and feature space conditions. When the model is initialized, the model structure, the initial weight and the bias parameter are set according to the actual application requirement, and the training round N is setepochThe number M of edge end devices participating in federal learning;
and initializing the same model structure by each edge end for training, and setting unified training related hyper-parameters including optimizer type, learning rate, loss function, model parameter solving algorithm and the like. In the embodiment, the optimizer type is set as adam optimizer, the model parameter solving algorithm is random gradient descent (SGD) algorithm, and the loss function is obtained by adding a regularization parameter λ on the basis of a cross entropy function, and is expressed as:
where p () represents the true label of the sample, q () represents the prediction probability of the model, xiSamples representing inputs, wRepresenting model parameters and lambda a regularization coefficient.
And 2, performing model training at the edge end, and implanting watermark information and identity information into the model at the same time.
During each round of training, each edge end trains the distributed model by using local sample data, during the training, the watermark information of the model is constructed by a projection matrix, and each edge end carries out watermark implantation operation on each model, and the method comprises the following steps: generating watermark information according to convolution kernel information obtained from the model and a randomly generated projection matrix, wherein the watermark information is combined with identity information and serves as a regular term to be added to an original loss function, and optimizing model parameters by using a new loss function added with the regular term so as to implant the watermark information and the identity information into the model, wherein the identity information refers to an identity ID (identity) which is expected to be added into the model. This implant watermark information and identity information can realize the privacy security protection to the nation learning model, in case the model is stolen simultaneously, can demonstrate the ownership of model.
Specifically, the watermark implantation operation includes:
firstly, averaging the weighted values of convolution kernels of a certain convolution layer of the model, in the invention, for convenience of calculation, each edge end performs averaging operation on the weighted values of convolution kernels of a penultimate convolution layer. After the averaging operation is carried out, the average operation result is unfolded to obtain a one-dimensional convolution kernel vector wave。
Secondly, each edge end generates a random projection matrix XPIn the invention, the projection matrix of each edge end is a matrix satisfying normal distribution, and simultaneously, each edge end records the projection matrix generated each time. Projecting the projection matrix XPAnd a convolution kernel vector waveAfter multiplication, obtaining a watermark implantation vector y through a sigmoid activation functionwThe generation process of the watermark embedding vector is represented as:
yi＝σ(∑iXjiwi)
where i denotes the ith element of the vector, XjiValues of elements, w, representing the ith row and the jth column of the projection matrixiRepresents the ith element value of the convolution kernel, σ () is a sigmoid function.
Then, identity information needing to be added into the model is converted into a binary identity information vector B, and the cross entropy of the identity information vector and the watermark implantation vector is used as a regular term:
wherein E (w) represents a cross entropy loss function, w represents a weight value of a convolution kernel, B represents an identity information vector, y represents a watermark implantation vector, and j represents a sequence number.
Finally, adding a regular term E (w) into an original loss function of the edge model training, constraining the training process of the model by using a new loss function added with the regular term, and continuously optimizing and adjusting the weight of a convolution kernel to ensure that a watermark implantation vector y is continuously close to an identity information vector B, as shown in the following formula:
Loss2＝Loss+εE(w)
where ε represents an adjustment factor used to balance normal model training and constrain the degree of the regularization term.
In the embodiment, the projection matrix of each edge end can be set to be consistent with the identity information, and the selected convolution layers are the same, so that the watermark verification of each edge end is facilitated.
And 3, uploading the block chain account book of the relevant information of the model by each edge terminal.
In the embodiment, after the current round of training is completed, each edge end uploads model gradient information, a projection matrix, identity information and convolution kernel information to a block chain account book and stores the model gradient information, the projection matrix, the identity information and the convolution kernel information to each block chain node in a transaction form. Since the blockchain account book is not falsifiable and the time of each upload is recorded, an unalterable blockchain is formed. Thus, once a model theft occurs, ownership of the model can be proven through watermark extraction techniques.
And 4, aggregating the uploading models of each edge terminal.
In the embodiment, each edge determines the attribution of the block right according to the workload certification, obtains the edge of the block right as a temporary server of the current round, and aggregates model gradient information uploaded by each edge to obtain a federal learning model and broadcasts the federal learning model to the block chain account book. Wherein the model aggregation is represented as:
where w represents the model parameters and n represents the number of training data.
And step 5, downloading the aggregation model from the block chain account book by each edge terminal to perform the next round of training.
During next round of training, each edge terminal uses the downloaded aggregation model as the basis to perform model training by using local data, and the steps 2 to 5 are repeated in the training process in a circulating manner until the total round number N is reachedepochUntil now.
And 6, reading recorded information from the block chain account book by each edge end to perform watermark verification on the federated learning model.
In the embodiment, when watermark verification is carried out, all convolution kernels of a convolution layer adopted when watermark information is generated are extracted from a federal learning model, and convolution kernel vectors are obtained through averaging the weights of all the convolution kernels;
the method comprises the steps of obtaining a projection matrix recorded before from a block chain ledger, multiplying the projection matrix by a convolution kernel vector, processing the obtained result through a step function to obtain a bit sequence consisting of 0 and 1 as extraction information, comparing the extraction information with identity information recorded before from the block chain ledger, directly comparing the bit sequences of the extraction information and the identity information due to the fact that the identity information is the bit sequence consisting of 0 and 1, using the similarity of the two bit sequences as a comparison result, and determining whether a watermark is implanted into a federal learning model or not according to the comparison result so as to exclude unreliable edge terminals and models trained by the federal learning model.
The watermark information is implanted into the reliable edge end when the local model is trained, and when the model is attacked, the information comparison is unsuccessful when watermark verification is carried out due to the fact that the watermark information is not implanted into the model in a training mode, and the model and the local data are leaked.
In summary, in the block chain technology-based federal learning model watermark reinforcement method provided in the above embodiment, the watermark information is generated by the convolution kernel information obtained from the model and the randomly generated projection matrix, and the watermark information set identity information is added to the original loss function as a regular term to optimize the model parameters, so that the watermark information and the identity information are implanted into the model, and on the basis of ensuring the robustness of the model, the model is reinforced by the watermark information and the identity information to identify the model. On the basis, the block chain accounting book technology is adopted to uniformly record the model gradient information, the projection matrix, the identity information and the convolution kernel information, so that the information is prevented from being omitted and damaged; the extraction and verification of the provided watermark information realize the ownership authentication of the model, and the privacy safety and property right protection of the federal learning model are protected.
Application scenario one
In the financial field, a bank at a multi-edge end needs to construct a demand analysis and prediction model for analyzing and predicting financial demands of a user in a federal learning mode through local user data, wherein the user data refers to deposit information, credit rating, loan information and the like, and the financial demands refer to loan payment assessment and the like for the user. However, in the actual training process and the application process, some demand analysis and prediction models are found to be attacked and copied and stolen by malicious attackers, and because the edge terminals come from different banks and mutual trust relationship is not established among the edge terminals, the edge terminal attackers cannot be determined. Once the demand analysis and prediction model in federal learning is stolen by a malicious attacker, the rights and interests of all edge end banks can be damaged, and even huge threats can be caused to the user data security of each edge end bank.
In order to solve the problem existing in the application scenario, the embodiment provides a protection method for a demand analysis and prediction model applied to the financial field, the protection method adopts the block chain technology-based federated learning model watermark reinforcement method to construct the demand analysis and prediction model, and the specific construction process comprises the following steps:
step 1, preprocessing user data of each edge bank and initializing and distributing models.
And 2, performing model training at the edge end, and implanting watermark information and identity information into the model at the same time.
And 3, uploading the block chain account book of the relevant information of the model by each edge bank.
And 4, aggregating the uploading models of all the edge end banks.
And 5, downloading the aggregation model from the block chain account book by each edge bank for the next round of training.
The pre-processing of the user data typically includes screening of the user data and partitioning of the training set and the test set. The specific process of the steps 1-5 is basically consistent with the model process of the above-mentioned block chain technology-based federal learning model watermark reinforcement method, except that the adopted local data is different, the local data utilized in the application scenario one is user data, and the constructed federal learning model is a demand analysis and prediction model, which is not described herein again. Therefore, the requirement analysis and prediction model reinforced by the watermark can be prevented from being attacked and embezzled, the privacy requirement of the user data of the bank is protected, and when the requirement analysis and prediction model is required to be verified to be attacked or not, the bank reads the record information from the block chain account book to verify the watermark of the requirement analysis and prediction model by adopting the same mode as the step 6 in the federal learning model watermark reinforcement method based on the block chain technology.
Application scenario two
In the medical field, a medical institution at a multi-margin end needs to construct a physical state analysis model for analyzing and predicting the physical condition of a patient in a federal learning manner through local user data, wherein the doctor and patient data refer to medical history, medication scheme, dosage and the like of a doctor and a patient. However, in the actual training process and the application process, some demand analysis and prediction models are found to be attacked and copied and stolen by malicious attackers, and because the edge terminals come from different medical institutions and mutual trust relationship is not established among the edge terminals, the edge terminal attackers cannot be determined. Once the body state analysis model in federal learning is stolen by a malicious attacker, the rights and interests of all medical structures can be damaged, and even the doctor-patient data security of each medical institution at the edge end can be greatly threatened.
In order to solve the problem of the second application scenario, the embodiment provides a method for protecting a body state analysis model applied in the medical field, the method for protecting the body state analysis model is constructed by adopting the block chain technology-based federal learning model watermark reinforcement method, and the specific construction process comprises the following steps:
step 1, preprocessing doctor-patient data of each edge end medical institution and initializing and distributing models.
And 2, performing model training at the edge end, and implanting watermark information and identity information into the model at the same time.
And 3, uploading the block chain account book of the relevant information of the model by each edge end medical institution.
And 4, aggregating uploaded models of each edge end medical institution.
And 5, downloading the aggregation model from the block chain account book by each edge end medical institution to perform the next round of training.
The preprocessing of the patient data typically includes screening of the user data and partitioning of the training and test sets. The specific process of the steps 1-5 is basically consistent with the model process of the block chain technology-based federated learning model watermark reinforcement method, except that the adopted local data is different, the local data utilized in the application scenario one is doctor-patient data, and the constructed federated learning model is a body state analysis model, which is not described again here. Therefore, the body state analysis model reinforced by the watermark can be prevented from being attacked and stolen, the privacy requirements of medical institution doctor-patient data are protected, and when whether the body state analysis model is attacked needs to be verified, the bank reads the record information from the block chain account book to verify the watermark of the body state analysis model in the same way as the step 6 in the federal learning model watermark reinforcement method based on the block chain technology.
The above-mentioned embodiments are intended to illustrate the technical solutions and advantages of the present invention, and it should be understood that the above-mentioned embodiments are only the most preferred embodiments of the present invention, and are not intended to limit the present invention, and any modifications, additions, equivalents, etc. made within the scope of the principles of the present invention should be included in the scope of the present invention.
Claims (10)
1. A block chain technology-based federated learning model watermark reinforcement method comprises the following steps:
initializing a model, and distributing the model to each edge end;
each edge end trains the distributed model by using local sample data, generates watermark information according to convolution kernel information obtained from the model and a randomly generated projection matrix during training, adds the watermark information to an original loss function by using identity information as a regular term, and optimizes model parameters by using a new loss function added with the regular term so as to implant watermark information and identity information in the model;
after the current round of training is finished, uploading model gradient information, a projection matrix, identity information and convolution kernel information to a block chain account book, and storing the block chain account book into each block chain node in a transaction form;
each edge end determines the attribution of the block right according to the workload certification, obtains the edge end of the block right as a temporary server of the current round, aggregates model gradient information uploaded by each edge end to obtain a federal learning model and broadcasts the federal learning model to a block chain account book;
and each edge terminal downloads the aggregation model from the block chain account book for the next round of training.
2. The block chain technology-based federal learning model watermark reinforcement method of claim 1, wherein when initializing a model, a model structure, an initial weight and a bias parameter are set, and meanwhile, a training round and the number of edge-end devices participating in federal learning are set;
and setting training related parameters including optimizer type, learning rate, loss function and model parameter solving algorithm.
3. The block chain technique-based federal learning model watermark reinforcement method of claim 1, wherein the generating watermark information from convolution kernel information derived from a model and a randomly generated projection matrix comprises:
during each round of training, extracting the weight of all convolution kernels of a certain convolution layer from the model, carrying out averaging operation, and taking the average operation result as a convolution kernel vector corresponding to convolution kernel information; randomly generating a projection matrix meeting normal distribution;
and after the projection matrix is multiplied by the convolution kernel vector, the watermark information is activated by an activation function to obtain a watermark implantation vector corresponding to the watermark information.
4. The block chain technology-based federal learning model watermark reinforcement method of claim 3, wherein the activation function is a sigmoid activation function.
5. The block chain technology-based federated learning model watermark reinforcement method of claim 1, wherein the watermark information is added to the original loss function as a regular term in combination with identity information, including:
and (4) performing cross entropy on the identity information vector converted by the identity information and the watermark implantation vector corresponding to the watermark information, and adding the cross entropy serving as a regular term to the original loss function to obtain a new loss function added with the regular term.
6. The block chain technique-based federated learning model watermark reinforcement method of claim 1, wherein the federated learning model further comprises: watermark verification is carried out on the federal learning model by each edge end, and the method comprises the following steps:
extracting all convolution kernels of convolution layers adopted when watermark information is generated from a federal learning model, and averaging the weights of all the convolution kernels to obtain convolution kernel vectors;
and obtaining a projection matrix recorded before from the block chain account book, multiplying the projection matrix by a convolution kernel vector, processing the obtained result by a step function to obtain extraction information, comparing the extraction information with identity information recorded before from the block chain account book, and determining whether the federal learning model is embedded with a watermark or not according to the comparison result so as to eliminate an unreliable edge end and a trained model thereof.
7. The block chain technique-based federal learning model watermark reinforcement method of claim 6, wherein a step function is used to process the result of multiplying the projection matrix by the convolution kernel vector into a bit sequence consisting of 0 and 1 as the extraction information.
8. The block chain technology-based federal learning model watermark reinforcement method as claimed in claim 1, wherein the randomly generated projection matrix of each edge is set to be the same as the identity information.
9. A protection method of a demand analysis and prediction model applied to the financial field is characterized in that each bank serves as an edge terminal, user data used for training a demand analysis and prediction model is used as sample data, the demand analysis and prediction model is used for analyzing and predicting financial demands of users according to the user data, and the demand analysis and prediction model is obtained through the block chain technology-based federal learning model watermark reinforcement method in any one of claims 1-8, so that the demand analysis and prediction model reinforced by watermarks can be prevented from being attacked and stolen.
10. A protection method of a body state analysis model applied to the medical field is characterized in that each medical institution serves as an edge terminal and possesses doctor-patient data used for training a body state analysis model as sample data, the body state analysis model is used for analyzing and predicting the body state of a patient according to the doctor-patient data, and the body state analysis model is obtained through the block chain technology-based federal learning model watermark reinforcement method of any one of claims 1 to 8, so that the body state analysis model reinforced through the watermark can be prevented from being attacked and stolen.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111488625.3A CN114492827A (en) | 2021-12-08 | 2021-12-08 | Block chain technology-based federated learning model watermark reinforcement method and application |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111488625.3A CN114492827A (en) | 2021-12-08 | 2021-12-08 | Block chain technology-based federated learning model watermark reinforcement method and application |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114492827A true CN114492827A (en) | 2022-05-13 |
Family
ID=81491922
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111488625.3A Pending CN114492827A (en) | 2021-12-08 | 2021-12-08 | Block chain technology-based federated learning model watermark reinforcement method and application |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114492827A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115238250A (en) * | 2022-09-15 | 2022-10-25 | 支付宝(杭州)信息技术有限公司 | Model processing method, device and equipment |
| CN115758402A (en) * | 2022-11-09 | 2023-03-07 | 国网江苏省电力有限公司苏州供电分公司 | Artificial intelligence model federal learning method combining homomorphic encryption and model watermarking |
-
2021
- 2021-12-08 CN CN202111488625.3A patent/CN114492827A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115238250A (en) * | 2022-09-15 | 2022-10-25 | 支付宝(杭州)信息技术有限公司 | Model processing method, device and equipment |
| CN115758402A (en) * | 2022-11-09 | 2023-03-07 | 国网江苏省电力有限公司苏州供电分公司 | Artificial intelligence model federal learning method combining homomorphic encryption and model watermarking |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Nicholls et al. | Financial cybercrime: A comprehensive survey of deep learning approaches to tackle the evolving financial crime landscape | |
| US20190244103A1 (en) | Robust pruned neural networks via adversarial training | |
| Boenisch et al. | When the curious abandon honesty: Federated learning is not private | |
| US11621847B2 (en) | Consensus layer architecture for maintaining security with reduced processing power dependency in untrusted decentralized computing platforms | |
| CN112084917A (en) | Living body detection method and device | |
| CN114492827A (en) | Block chain technology-based federated learning model watermark reinforcement method and application | |
| CN112446310A (en) | Age identification system, method and device based on block chain | |
| CN109361654B (en) | Method and system for managing business secret based on block chain negotiation encryption | |
| Liu et al. | Keep your data locally: Federated-learning-based data privacy preservation in edge computing | |
| AU2019100349A4 (en) | Face - Password Certification Based on Convolutional Neural Network | |
| Nuding et al. | Data poisoning in sequential and parallel federated learning | |
| CN111243698A (en) | Data security sharing method, storage medium and computing device | |
| Fonseca-Bustos et al. | Robust image hashing for content identification through contrastive self-supervised learning | |
| Nguyen et al. | Backdoor attacks and defenses in federated learning: Survey, challenges and future research directions | |
| Manoharan et al. | Implementation of internet of things with blockchain using machine learning algorithm: Enhancement of security with blockchain | |
| Djibrine et al. | Transfer Learning for Animal Species Identification from CCTV Image: Case Study Zakouma National Park | |
| CN116151369A (en) | Bayesian-busy robust federal learning system and method for public audit | |
| CN114863430A (en) | Automatic population information error correction method, device and storage medium thereof | |
| Felix Johannes | Hardened Model Aggregation for Federated Learning backed by Distributed Trust Towards decentralizing Federated Learning using a Blockchain | |
| US20240121084A1 (en) | Cryptographic key generation using machine learning | |
| Hahn et al. | Graffl: Gradient-free federated learning of a bayesian generative model | |
| Camacho | Initialization methods of convolutional neural networks for detection of image manipulations | |
| US20230418921A1 (en) | Intelligent authentication of users in metaverse leveraging non-fungible tokens and behavior analysis | |
| Prem Kumar et al. | Metaheuristics with Optimal Deep Transfer Learning Based Copy-Move Forgery Detection Technique. | |
| Traboulsi | Deepfakes: Analysis of threats and countermeasures |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |