CN113420886A - Training method, device, equipment and storage medium for longitudinal federated learning model - Google Patents
Training method, device, equipment and storage medium for longitudinal federated learning model Download PDFInfo
- Publication number
- CN113420886A CN113420886A CN202110688660.3A CN202110688660A CN113420886A CN 113420886 A CN113420886 A CN 113420886A CN 202110688660 A CN202110688660 A CN 202110688660A CN 113420886 A CN113420886 A CN 113420886A
- Authority
- CN
- China
- Prior art keywords
- encryption
- model
- training
- encryption loss
- loss
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012549 training Methods 0.000 title claims abstract description 136
- 238000000034 method Methods 0.000 title claims abstract description 109
- 238000004364 calculation method Methods 0.000 claims abstract description 82
- 230000008569 process Effects 0.000 claims description 62
- 238000012795 verification Methods 0.000 claims description 28
- 238000004590 computer program Methods 0.000 claims description 15
- 238000004422 calculation algorithm Methods 0.000 claims description 4
- 239000003999 initiator Substances 0.000 description 49
- 238000012360 testing method Methods 0.000 description 9
- 230000007246 mechanism Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 7
- 238000007477 logistic regression Methods 0.000 description 6
- 238000012550 audit Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 230000006870 function Effects 0.000 description 3
- 230000006399 behavior Effects 0.000 description 2
- 230000015572 biosynthetic process Effects 0.000 description 2
- 230000001066 destructive effect Effects 0.000 description 2
- 230000002452 interceptive effect Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 230000000977 initiatory effect Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/602—Providing cryptographic facilities or services
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Medical Informatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Health & Medical Sciences (AREA)
- Bioethics (AREA)
- General Health & Medical Sciences (AREA)
- Computer Hardware Design (AREA)
- Computer Security & Cryptography (AREA)
- Storage Device Security (AREA)
Abstract
The application relates to a training method, a device, computer equipment and a storage medium of a longitudinal federated learning model, wherein the method comprises the following steps: generating an encryption public key for encrypting data and parameters of the model and a decryption private key for decrypting an encryption loss value of the model; and obtaining commitment data of the model to perform zero-knowledge proof, wherein the commitment data is obtained by performing data commitment on the source data set by each second device participating in model training. And obtaining an encryption loss initialization parameter of the model to perform zero knowledge proof, wherein the encryption loss initialization parameter is obtained by calculating each second device based on the encrypted data. And obtaining an encryption loss value of the model to perform zero knowledge proof, wherein the encryption loss value is obtained by calculating each second device based on the encryption loss initialization parameter and the encryption loss calculation weight. And decrypting the encrypted loss value based on the decryption private key to obtain a loss value of the model, and obtaining the trained longitudinal federal learning model when the loss value is smaller than a preset threshold value.
Description
Technical Field
The application relates to the technical field of internet, in particular to a method and a device for training a longitudinal federal learning model, computer equipment and a storage medium.
Background
Existing federal learning mostly assumes that the participants are semi-honest, i.e., model training is performed only according to the protocol flow without any destructive behavior. But this assumption is too ideal in practical applications. In fact, a party to federal learning may have a malicious node, in which case the party needs to prove that its training process is accurate, legal, and consistent, i.e., has the need for integrity for security audit; meanwhile, the data and gradient information of the participants cannot be exposed in the verification process because the initial purpose of federal learning is to protect the privacy of the data used by the participants for training.
Disclosure of Invention
The application provides a training method and device of a longitudinal federated learning model, computer equipment and a storage medium, so that zero knowledge proof is carried out on encryption loss calculation in the training process of the longitudinal federated learning model, the training ending condition of the model is determined according to an encryption loss value proved by the zero knowledge, and overfitting of the model is avoided.
In a first aspect, the present application provides a method for training a longitudinal federated learning model, the method comprising:
generating an encryption public key and a decryption private key; the encryption public key is used for sending to each second device participating in model training to encrypt data and parameters of the model, and the decryption private key is used for decrypting an encryption loss value of the model to obtain a loss value;
obtaining commitment data of the model to perform zero-knowledge certification to obtain a zero-knowledge certification result of the commitment data, wherein the commitment data is obtained by performing data commitment on a source data set by each second device;
acquiring encryption loss initialization parameters of the model to perform zero knowledge certification to obtain a zero knowledge certification result of the encryption loss initialization parameters, wherein the encryption loss initialization parameters are obtained by performing encryption loss initialization calculation on each second device based on the source data set;
obtaining an encryption loss value of the model to perform zero knowledge proof to obtain a zero knowledge proof result of the encryption loss value, wherein the encryption loss value is obtained by calculating each second device based on the encryption loss initialization parameter and the encryption loss calculation weight;
and decrypting the encrypted loss value based on the decryption private key to obtain a loss value in the training process of the longitudinal federated learning model, and obtaining the trained longitudinal federated learning model when the loss value is smaller than a preset threshold value.
In a second aspect, the present application further provides a training apparatus for a longitudinal federal learning model, the apparatus comprising:
the key generation module is used for generating an encryption public key and a decryption private key; the encryption public key is sent to each second device participating in model training and used for encrypting data and parameters of the model, and the decryption private key is used for decrypting an encryption loss value of the model;
the data certification module is used for obtaining committed data of the model and carrying out zero knowledge certification to obtain a zero knowledge certification result of the committed data, wherein the committed data is obtained by carrying out data commitment on the source data set by each second device;
an encryption loss initialization proving module, configured to obtain an encryption loss initialization parameter of the model to perform zero knowledge proving to obtain a zero knowledge proving result of the encryption loss initialization parameter, where the encryption loss initialization parameter is obtained by performing encryption loss initialization calculation on each second device based on the source data set;
the encryption loss calculation proving module is used for obtaining an encryption loss value of the model to perform zero knowledge proving to obtain a zero knowledge proving result of the encryption loss value, wherein the encryption loss value is obtained by calculating each second device based on the encryption loss initialization parameter and the encryption loss calculation weight;
and the model output module is used for decrypting the encrypted loss value based on the decryption private key to obtain a loss value in the model training process, and when the loss value is smaller than a preset threshold value, a well-trained longitudinal federated learning model is obtained.
In a third aspect, the present application further provides a computer device comprising a memory and a processor; the memory is used for storing a computer program; the processor is configured to execute the computer program and, when executing the computer program, implement the training method of the longitudinal federal learning model.
In a fourth aspect, the present application further provides a storage medium storing a computer program, which when executed by a processor causes the processor to implement the method for training a longitudinal federal learning model as described above.
The application discloses a training method and device of a longitudinal federated learning model, computer equipment and a storage medium. Secondly, the loss value of the model is obtained through the encryption loss value in the decryption training process, the loss value of the model is compared with a preset threshold value to realize an early-stop mechanism, overfitting of the model is avoided, calculation and bandwidth expenses of participators in model training are reduced, and the well-trained longitudinal federated learning model is finally obtained.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a schematic flow chart diagram of a method for training a longitudinal federated learning model provided by an embodiment of the present application;
FIG. 2 is a schematic block diagram of a training apparatus for a longitudinal federated learning model provided in an embodiment of the present application;
fig. 3 is a schematic block diagram of a computer device according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some, but not all, embodiments of the present application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The flow diagrams depicted in the figures are merely illustrative and do not necessarily include all of the elements and operations/steps, nor do they necessarily have to be performed in the order depicted. For example, some operations/steps may be decomposed, combined or partially combined, so that the actual execution sequence may be changed according to the actual situation.
It is to be understood that the terminology used in the description of the present application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in the specification of the present application and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should also be understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
Existing federal learning mostly assumes that the participants are semi-honest, i.e., model training is performed only according to the protocol flow without any destructive behavior. But this assumption is too ideal in practical applications.
The participants in federal learning may have the following actions:
1. possibly a lazy node, transmits encrypted gradient information obtained from others, obtained in previous training rounds, or even false meaningless to other participants.
2. The wrong encrypted gradient information is transmitted to interfere with the training of the model, which may be a malicious node or a node attacked by a malicious attacker.
In this case, the participants need to prove that their training process is accurate, legal, and consistent, i.e., have the need for integrity for security auditing; meanwhile, the data and gradient information of the participants cannot be exposed in the verification process because the initial purpose of federal learning is to protect the privacy of the data used by the participants for training. Zero Knowledge Proof (ZKP) is therefore most suitable as a privacy preserving technique that does not need to rely on any hardware. In this technique, the prover can convince the verifier that a certain argument is correct without providing the verifier with any useful information, and a zero-knowledge proof can construct an integrity proof while preserving the privacy of the data entered.
On the other hand, the security audit method of federal learning also needs to calculate a loss function to realize an early-stop mechanism, so that the following problems caused by the fact that training is stopped after only specified iteration times are run are avoided:
1. overfitting the model, overfitting the model to the training data, can be less effective on the test data.
2. The model training time is too long, extra calculation and bandwidth overhead are brought, and for federal learning, the overhead needs to be borne by each participant.
Therefore, the application provides a training method of a longitudinal federated learning model, and a security auditing method of federated learning combined with an early-stopping mechanism is realized, so that the model training failure caused by the participation of malicious nodes is ensured.
The embodiment of the application provides a method and a device for training a longitudinal federated learning model, computer equipment and a storage medium. The method for training the longitudinal federated learning model can be applied to a server, malicious nodes in the training process of the longitudinal federated learning model are identified by carrying out zero knowledge proof on a source data set, an encryption loss initialization parameter and an encryption loss value of the longitudinal federated learning model, the training effect of the longitudinal federated learning model is ensured, and an early-stopping mechanism is realized by comparing the decrypted loss value of the model with a preset threshold value, so that overfitting of the model is avoided. The server may be an independent server or a server cluster.
Some embodiments of the present application will be described in detail below with reference to the accompanying drawings. The embodiments described below and the features of the embodiments can be combined with each other without conflict.
Referring to fig. 1, fig. 1 is a schematic flow chart of a method for training a longitudinal federated learning model provided in an embodiment of the present application. The method for training the longitudinal federated learning model can be applied to a server, and is used for identifying the malicious participants in the training process of the longitudinal federated learning model by zero-knowledge proof, avoiding the malicious participants from influencing the training effect of the longitudinal federated learning model, realizing an early-stop mechanism by comparing the loss value of the model after decryption with a preset threshold value, avoiding overfitting of the model, and finally obtaining the trained longitudinal federated learning model.
Fig. 1 shows a training method of a longitudinal federal learning model provided in an embodiment of the present application, where the training method is applied to a first device participating in training of the longitudinal federal learning model, and the method specifically includes steps S101 to S105.
S101, generating an encryption public key and a decryption private key; the encryption public key is used for sending to each second device participating in model training to encrypt data and parameters of the model, and the decryption private key is used for decrypting an encryption loss value of the model.
First, suppose two data-owner enterprises a and B want to jointly train a machine learning model, and their business systems own the relevant data of their respective users. In addition, enterprise a also has label data that the model needs to predict. Data exchange between a and B cannot be performed directly for data privacy and security, and at this time, a federal learning system can be used to build a model. In order to ensure the confidentiality of data in the training process, integration and audit are required by means of a third-party central node C, and the federal learning is also suitable for the condition that a plurality of data owners participate in modeling.
It should be noted that the first device represents a central node C that integrates and audits the training, and the second devices represent two or more data owners (e.g., a and B) that participate in the training.
In the embodiment of the application, a training method of the longitudinal federal learning model of the application is described by taking a logistic regression model of longitudinal federal learning in a three-party scene as an example. The training participants of the logistic regression model include an initiator, a receiver, and a central node. The initiator and the receiver commit own data and use the own data to carry out initialization, federal training and loss calculation of loss. The central node is responsible for verifying the data and the calculation process of the initiator and the receiver, integrating the encrypted gradient of the splicing two parties and calculating the total loss of the model, and judging whether to end the current training process according to the total loss. The data owned by the initiator and the recipient contains the same client but only the initiator has an annotation of the data.
Illustratively, in this case, the first device is a central node, and the second devices represent an initiator and a receiver of the logistic regression model.
First, a central node for model training first generates a pair of an encryption public key and a decryption private key based on an encryption algorithm. The encryption public key is used for being sent to each second device participating in training of the longitudinal federated learning model, and the participator encrypts data, parameters and calculation tasks in the model training process based on the encryption public key. And the decryption private key is used for decrypting the encryption loss value in the model training process to obtain the loss value of the model training.
In some embodiments, the central node generates the homomorphic cryptographic public key pk based on a federally learned homomorphic cryptographic algorithmcAnd a decryption key SkcSelecting a mask m and using pkcEncrypted to obtain [ m]And applying the encrypted mask [ m ]]And encrypting the public key pkcAnd sending the data to the participants of the longitudinal federated learning model training for encrypting the data and parameters of the model training.
Illustratively, we use [ · to]Indicating homomorphic encryption, e.g. m is plaintext, [ m ]]For the ciphertext after homomorphic encryption, the homomorphic encryption for multiplication homomorphic is as follows: [ m ] of1+m2]＝[m1]+[m2]，[cm]＝c[m]Wherein c is a positive integer.
And S102, obtaining commitment data of the model to perform zero-knowledge certification to obtain a zero-knowledge certification result of the commitment data, wherein the commitment data is obtained by performing data commitment on the source data set by each second device.
In the embodiment provided by the application, a simplified non-interactive computing reliable zero-knowledge proof (zk-SNARK) is adopted as an implementation mode of the zero-knowledge proof. The zero knowledge proof method allows a user to provide proof in a non-interactive manner and can be verified in a short amount of time. zk-SNARK generally comprises three processes:
(1) generating a zero knowledge proof key pair from the computation task: pk, vk, this process is only run once during the system initialization process, and the formula is characterized as:
KeyGen(1λ,C)→(pk,vk)
in the formula, λ is a positive integer security parameter, C is a calculation task that we express through a circuit and needs to be certified, vk is a certification key used for generating certification, and vk is a verification key used for verifying the certification of others.
(2) The participator of the model training generates a proof pi based on the proof secret key, and the formula is characterized in that:
prove(pk,x,a)→π
in the formula, pi is a proof generated by a participant and used for verifying a central node; a is a public input that does not require privacy protection; x is input data requiring privacy protection and pk is a certification key used to generate a certification.
(3) And the central node verifies the proof generated by the participants of the model training, and the process is passed, and whether the verification process is correct is calculated. The formula is characterized as:
prove(vk,a,π)→b
in the formula, b is a Boolean value variable and represents whether the verification result is correct or not; vk is a verification key for verifying the proof, a is a public input, and pi is the proof generated by the participants of the model training;
specifically, a corresponding zero knowledge proof circuit is generated according to a calculation task in a model training process needing verification, a proof key and a verification key for zero knowledge proof of the calculation task are generated according to the zero knowledge proof circuit, and the calculation task comprises: data commitment, encryption loss initialization and encryption loss calculation.
Firstly, the committed data input by the initiator and the receiver is verified to determine whether false data exists, and a source data set corresponding to the committed data comprises a training set and a test set data owned by the initiator and the receiver. Specifically, the central node first generates a zero knowledge proof gate circuit based on a calculation process of the data commitment.
In some embodiments, the data of the model training process is committed based on a Pedersen commitment (Pedersen commitment) that can ensure that the message itself is hidden from others while the message is not modified by the committer; in the opening stage, the prompter can disclose the message, and the receiving end verifies whether the message is consistent with the prompter, so as to meet the hiding property and the binding property.
Exemplary, C1`For a zero-knowledge proof circuit generated based on data commitments of an initiator and a receiver, the calculation process is expressed as follows:
wherein, X is data used by the test set, r is a random number, and COMM is a commitment function.
The central node passes through keyGen (1)λ,C1) Generating an attestation key and an authentication key (pk) for zero knowledge attestation of initiator and recipient data commitments1,vk1). The central node will then be used to generate the attestation key pk for the attestation1And sending the data to the initiator and the receiver.
Illustratively, to promise data for initiators in a training processThe process of performing zero knowledge proof is specifically as follows: first, the initiator sends all the input data XAGenerating committed data
All tags yAGenerating post-commitment tags
The formula is characterized as:
second, the initiator sets X ═ X (X)A,rA) A () and by save (pk)1X, a) generating proof for validating commitment dataA,x(ii) a Setting x ═ yA) And pass through pro (pk)1X, a) generating proof for validating a commitment labelA,y. Wherein a is public input, namely input without privacy protection; x is input data requiring privacy protection.
The initiator transmits the committed data
Promised promise label
Proof pi for validating commitment dataA,xProof pi for verifying commitment labelA,yAnd sending the data to the central node for zero knowledge proof. The central node sets a () to pass through Verify (vk)1,a,πA,x) And Verify (vk)1,a,πA,y) Verifying whether the promised data and promised label of the initiator are correct, if the result is false, indicating the promised data or promised labelFalse data exists in the source data set corresponding to the label, and the initiator is a malicious node.
In some embodiments, if it is determined that spurious data is present, the second device providing the spurious data is removed from the participants in the model training, and the first device resumes training of the model with other benevolent nodes.
It should be noted that the process of zero-knowledge proof of data commitment of the receiving party is similar to that of the initiating party, except that the commitment data at this time is a commitment data set after the receiving party passes the data commitment.
When the commitment data set passes the zero-knowledge proof, the data set provided by the initiator and the receiver of the model training is proved to have no false data, and then the source data set corresponding to the commitment data can be used for the model training and the encryption loss initialization calculation to determine the encryption loss initialization parameter.
S103, obtaining encryption loss initialization parameters of the model to perform zero knowledge certification to obtain a zero knowledge certification result of the encryption loss initialization parameters, wherein the encryption loss initialization parameters are obtained by performing encryption loss initialization calculation on each second device based on the source data set.
Specifically, a corresponding zero knowledge proof circuit is generated according to a calculation process of encryption loss initialization, and then an proof key and a verification key for zero knowledge proof are generated according to the zero knowledge proof circuit, wherein the proof key is used for sending corresponding proofs generated by each second device, and the verification key is used for verifying the proofs by the first device.
Illustratively, the steps of performing zero-knowledge proof on the encryption loss initialization parameter of the longitudinal federated learned logistic regression model in the three-party scenario are as follows:
since the calculation processes of each participant in the model training are different in the encryption loss initialization, it is necessary to generate corresponding zero-knowledge proof circuits respectively for the encryption loss initialization calculation of the initiator and the encryption loss initialization calculation of the receiver.
Exemplary, C2Is a pair ofThe zero knowledge proof circuit generated by initialization calculation according to the encryption loss of the initiator expresses the calculation process as follows:
[o]＝[m]·yAH
h[p]＝[o]TXAH
wherein [ m ]]For homomorphic masks for encryption, rARandom number selected for initiator, XAHInput data of test sets requiring secrecy for the initiator, YAHIs XAHCorresponding data label, h is the size of the test set.
The central node passes through keyGen (1)λ,C2) Generating an attestation key and a verification key (pk) for zero knowledge attestation of an encryption loss initialization parameter of an initiator2,vk2)。
Exemplary, C3The zero knowledge proof circuit generated for initializing calculation according to the encryption loss of the receiving party expresses the calculation process as follows:
h[q]＝[o]TXBH
[μ]＝concatenate([p],[q])
in the formula, XBHInput data of the test set, r, which needs to be kept secret for the receiverBA random number selected for the receiving party; the concatenate is a connection function, i.e. all inputs are connected in series.
The central node passes through keyGen (1)λ,C3) Generating an attestation key and a verification key (pk) for zero knowledge attestation of encryption loss initialization parameters of a receiving party3,vk3)。
The central node will then be used to generate the attestation key pk for the attestation2And pk3Sending to the initiator and the receiverThe initiator and the receiver respectively generate a certificate according to the certificate secret key and the encryption loss initialization parameter, and the central node verifies the certificate according to the verification secret key.
First, the central node divides the data set of the initiator and the receiver into a training set and a test set according to the previous commitment data, wherein the test set of the initiator
The test set of the receiving party is
The central node will
And
respectively sent to the initiator and the receiver. The initiator calculates variables required for encryption loss calculation, and the formula is characterized as follows:
[o]＝[m]·yAH
initiator sets X ═ X (X)AH,yAH)、
And by cave (pk)2X, a) generating π2And encrypt the parameter [ o ]]、[p]And prove pi2And sending the data to the central node. Central node arrangement
And passing through Verify (vk)2,a,π2) And verifying whether the encryption loss initialization process of the initiator is correct or not, if the result is false, the encryption loss initialization parameter of the initiator is a false parameter, and the initiator is indicated to be a malicious node. If the result is true, the encryption loss initialization parameter of the initiator is indicated to passAfter zero knowledge proves, the central node initializes the encryption loss parameter o obtained by the initiator]、[p]And sending to the receiving party for further training of the model. Variable required for the receiver to calculate the encryption loss calculation q]、[μ]The formula is characterized as:
[μ]＝concatenate([p],[q])
receiver sets X ═ XBH)，
By Prove (pk)3X, a) generating π3. And initializing the encryption loss initialization parameter q obtained by the receiving party]、[μ]、π3And sending the data to the central node for zero knowledge proof. Central node arrangement
And passing through Verify (vk)3,a,π3) And verifying whether the encryption loss initialization process of the receiver is correct or not, if the result is false, the encryption loss initialization parameter of the receiver is a false parameter, and the receiver is indicated to be a malicious node. And if the result is true, calculating the loss of the model according to the encryption loss initialization parameter.
In some embodiments, if it is determined that a malicious node exists, the first device notifies the second devices and stops training of the model.
When the encryption loss initialization parameters of the initiator and the receiver are proved by zero knowledge, the encryption loss initialization calculation process is proved to have no malicious node, and the obtained encryption loss initialization parameters are not false parameters and can be used for the next encryption loss calculation.
S104, obtaining an encryption loss value of the model, and performing zero knowledge proof to obtain a zero knowledge proof result of the encryption loss value, wherein the encryption loss value is obtained by calculating each second device based on the encryption loss initialization parameter and the encryption loss calculation weight.
Specifically, a corresponding zero knowledge proof circuit is generated according to encryption loss calculation, and then an proof key and a verification key for zero knowledge proof are generated according to the zero knowledge proof circuit, wherein the proof key is used for sending each second device to generate a corresponding proof, and the verification key is used for verifying the proof by the first device.
Illustratively, the step of zero knowledge proof of the encryption loss value of the longitudinal federally learned logistic regression model in the three-party scenario is as follows:
since the calculation processes of the encryption loss calculation of each participant in the model training are different, corresponding zero-knowledge proof circuits need to be generated respectively for the encryption loss calculation of the initiator and the encryption loss calculation of the receiver.
Exemplary, C4The zero knowledge proof circuit generated for the encryption loss calculation according to the initiator expresses the calculation process as follows:
[u]＝[m]·XAHθA
8h[u']＝(XAHθA·XAHθA)T[m]
the central node passes through keyGen (1)λ,C4) Generating an attestation key and a verification key (pk) for zero knowledge attestation of a first encryption loss value of an initiator4,vk4) Wherein the first encryption loss value is an intermediate result of the encryption loss values of the model.
C5The zero knowledge proof circuit generated for the calculation according to the encryption loss of the receiving party expresses the calculation process as follows:
8h[l(θ)]＝8h[u']+(XBHθB·XBHθB)T[m]+2XBHθB[u]-2θT[μ]
the central node passes through keyGen (1)λ,C5) Generating an attestation key and an authentication key (pk) for zero knowledge attestation of a second encryption loss value of the receiving party5,vk5) And the second encryption loss value is the final encryption loss value of the model. The central node will then be used to generate the attestation key pk for the attestation4And pk5And sending the information to the initiator and the receiver, wherein the initiator and the receiver generate certificates based on the certificate secret keys and the respective encryption loss values, and the central node performs verification based on the corresponding verification secret keys.
After generating the attestation key and the verification key for zero knowledge attestation of the encryption loss value, zero knowledge attestation of the encryption loss value of the model in each loss calculation may begin.
First, the central node sends the updated weight θ for each training to the initiator and the receiver.
The initiator splits theta from thetaAAnd calculate [ u ]]、[u']The formula is characterized as:
[u]＝[m]·XAHθA
initiator sets X ═ X (X)AH)、
And by cave (pk)4X, a) proof of formation pi4And will [ u ]]、[u']、π4And sending the data to the central node for verification. Central node arrangement
And passing through Verify (vk)4,a,π4) And verifying whether the first encryption loss value of the initiator is correct or not, and if the result is false, indicating that the participant of the encryption loss calculation is a malicious node.
If the result is true, the encryption loss calculation of the initiator passes the zero knowledge proof, and the central node compares the first encryption loss value [ u ] calculated by the initiator]、[u']And sending the encryption loss value to a receiving party, and calculating a second encryption loss value of the model by the receiving party on the basis of the encryption loss value. Specifically, the receiving party splits theta from thetaBAnd calculating the loss l (theta)]The formula is characterized as:
receiver sets X ═ XBH)，
And pass through pro (pk)5X, a) proof of formation pi5The receiver compares the encryption loss value l (theta) of the model training]And proof pi for verifying the encryption loss value5And sending the data to the central node. If the result is true, the encryption loss value calculated by the receiving party passes the zero knowledge proof. If the result is false, the malicious node exists.
In some embodiments, when it is determined that the encryption loss calculation has a malicious node, the second devices are notified and training of the model is stopped.
When the encryption loss value of each second device passes the zero knowledge proof of the first device, which indicates that no malicious node exists in the encryption loss calculation process, the encryption loss value can be sent to the central node for decryption to obtain the loss value of the model.
S105, decrypting the encrypted loss value based on the decryption private key to obtain a loss value in the model training process, and obtaining a well-trained longitudinal federated learning model when the loss value is smaller than a preset threshold value.
When the encryption loss calculation of each second device passes the zero knowledge proof, the first device may decrypt the encryption loss value based on the decryption private key to obtain a loss value of the model, and compare the loss value with a preset threshold value to implement an early-stop mechanism.
The early-stop mechanism is a widely used method and in many cases is better than the regularization method. The basic meaning is that the expression of the model on the verification set is calculated in the training process, and when the expression of the model on the verification set begins to decline, the training is stopped, so that the problem of overfitting caused by continuous training can be avoided. Specifically, when the loss value of the model training is lower than the preset threshold value, the participants of the model training are informed to finish the training.
Illustratively, the central node decrypts [ l (θ)]Obtaining a loss value l (theta) of the longitudinal federal learning model, and when l (theta) is smaller than a preset loss threshold value lTWhen the model is trained, the central node informs the initiator and the receiver of finishing the training of the model; and obtaining a well-trained longitudinal federated learning logistic regression model. And when l (theta) does not reach the condition of the preset threshold value, updating the weight of encryption loss calculation, and sending the updated weight to the initiator and the receiver to continue the iterative training of the model. It should be noted that the preset threshold is set according to a specific scenario, and the present application does not limit this.
According to the training method of the longitudinal federated learning model, the malicious participants in the training process of the longitudinal federated learning model are identified through zero knowledge proof of data, encryption loss initialization and encryption loss calculation of the longitudinal federated learning model, and the fact that the malicious participants influence the training effect of the longitudinal federated learning model is avoided. Secondly, a loss value of model training is obtained through an encryption loss value in the decryption training process, the loss value is compared with a preset threshold value to realize an early-stop mechanism, model overfitting is avoided, the calculation and bandwidth expenditure of a participant of model training is reduced, and a well-trained longitudinal federal learning model is finally obtained.
Referring to fig. 2, fig. 2 is a schematic block diagram of a training apparatus of a longitudinal federal learning model according to an embodiment of the present application, which is used for executing the aforementioned training method of the longitudinal federal learning model. Wherein, the training device of the longitudinal federal learning model can be configured in the server.
As shown in fig. 2, the training apparatus 400 of the longitudinal federal learning model includes:
a key generation module 401, configured to generate an encrypted public key and a decrypted private key; the encryption public key is used for sending to each second device participating in model training to encrypt data and parameters of the model, and the decryption private key is used for decrypting an encryption loss value of the model to obtain a loss value;
the data certification module 402 is configured to obtain committed data of the model for zero-knowledge certification, and obtain a zero-knowledge certification result of the committed data, where the committed data is obtained by performing data commitment on the source data set by each second device;
an encryption loss initialization certification module 403, configured to obtain an encryption loss initialization parameter of the model for zero knowledge certification, so as to obtain a zero knowledge certification result of the encryption loss initialization parameter, where the encryption loss initialization parameter is obtained by performing encryption loss initialization calculation on each second device based on the source data set;
an encryption loss calculation certification module 404, configured to obtain an encryption loss value of the model for zero knowledge certification, so as to obtain a zero knowledge certification result of the encryption loss value, where the encryption loss value is calculated by each second device based on the encryption loss initialization parameter and the encryption loss calculation weight;
and the model output module 405 is configured to decrypt the encrypted loss value based on the decryption private key to obtain a loss value in the model training process, and when the loss value is smaller than a preset threshold value, obtain a trained longitudinal federated learning model.
It should be noted that, as will be clearly understood by those skilled in the art, for convenience and brevity of description, the specific working processes of the apparatus and the modules described above may refer to the corresponding processes in the foregoing embodiment of the training method of the longitudinal federal learning model, and are not described herein again.
The apparatus described above may be implemented in the form of a computer program which is executable on a computer device as shown in fig. 3.
Referring to fig. 3, fig. 3 is a schematic block diagram of a computer device according to an embodiment of the present application. The computer device may be a server.
Referring to fig. 3, the computer device includes a processor, a memory, and a network interface connected through a system bus, wherein the memory may include a storage medium and an internal memory.
The storage medium may store an operating system and a computer program. The computer program includes program instructions that, when executed, cause a processor to perform any one of the methods of training for a longitudinal federated learning model.
The processor is used for providing calculation and control capability and supporting the operation of the whole computer equipment.
The internal memory provides an environment for the execution of a computer program on a storage medium, which when executed by the processor causes the processor to perform any one of the methods of training the longitudinal federated learning model.
The network interface is used for network communication, such as sending assigned tasks and the like. Those skilled in the art will appreciate that the architecture shown in fig. 3 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
It should be understood that the Processor may be a Central Processing Unit (CPU), and the Processor may be other general purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components, etc. Wherein a general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
Wherein, in one embodiment, the processor is configured to execute a computer program stored in the memory to implement the steps of:
generating an encryption public key and a decryption private key; the encryption public key is used for sending to each second device participating in model training to encrypt data and parameters of the model, and the decryption private key is used for decrypting an encryption loss value of the model;
obtaining commitment data of the model to perform zero-knowledge certification to obtain a zero-knowledge certification result of the commitment data, wherein the commitment data is obtained by performing data commitment on a source data set by each second device;
acquiring encryption loss initialization parameters of the model to perform zero knowledge certification to obtain a zero knowledge certification result of the encryption loss initialization parameters, wherein the encryption loss initialization parameters are obtained by performing encryption loss initialization calculation on each second device based on the source data set;
obtaining an encryption loss value of the model to perform zero knowledge proof to obtain a zero knowledge proof result of the encryption loss value, wherein the encryption loss value is obtained by calculating each second device based on the encryption loss initialization parameter and the encryption loss calculation weight;
and decrypting the encrypted loss value based on the decryption private key to obtain a loss value in the model training process, and obtaining a well-trained longitudinal federated learning model when the loss value is smaller than a preset threshold value.
In one embodiment, the processor, when implementing the training of the longitudinal federated learning model, is configured to implement:
and when the loss value is greater than or equal to the preset threshold value, updating the encryption loss calculation weight according to the loss value to obtain an updated encryption loss calculation weight, and sending the updated encryption loss calculation weight to each second device for iterative training of the model.
In one embodiment, the processor, when implementing the training of the longitudinal federated learning model, is configured to implement:
generating a zero knowledge proof gate circuit based on a computational task, the computational task comprising: data commitment, encryption loss initialization and encryption loss calculation; generating a certification key and a verification key according to the zero knowledge certification gate circuit; the certification key is used for generating a certification by each second device, and the verification key is used for verifying the certification; and verifying the certification based on the verification secret key to obtain a zero-knowledge certification result of the computing task.
In one embodiment, the processor, when implementing the training of the longitudinal federated learning model, is configured to implement:
determining whether false data exists according to a zero-knowledge proof result of the committed data; when determining that no false data exists, using the source data set for encryption loss initialization calculation of the model; and when determining that the false data exists, determining a second device providing the false data as a malicious node, removing the malicious node, and restarting the training of the model.
In one embodiment, the processor, when implementing the training of the longitudinal federated learning model, is configured to implement:
determining whether a malicious node exists in the encryption loss initialization process according to a zero knowledge proof result of the encryption loss initialization parameter; when it is determined that no malicious node exists in the encryption loss initialization process, using the encryption loss initialization parameter for encryption loss calculation of the model; and when determining that the encryption loss initialization process has the malicious node, notifying each second device, and stopping the training of the model.
In one embodiment, the processor, when implementing the training of the longitudinal federated learning model, is configured to implement:
determining whether a malicious node exists in the encryption loss calculation process according to a zero knowledge proof result of the encryption loss value; when it is determined that no malicious node exists in the encryption loss calculation process, the encryption loss value is used for decryption to obtain a loss value in the model training process; and when determining that the encryption loss calculation process has the malicious node, notifying each second device, and stopping the training of the model.
In one embodiment, the processor, when effecting generating the encrypted public key and the decrypted private key, is operable to effect: and generating the encryption public key and the decryption private key based on a homomorphic encryption algorithm.
The embodiment of the application further provides a storage medium, wherein a computer program is stored in the storage medium, the computer program comprises program instructions, and the processor executes the program instructions to implement any one of the longitudinal federal learning model training methods provided by the embodiments of the application.
The storage medium may be an internal storage unit of the computer device described in the foregoing embodiment, for example, a hard disk or a memory of the computer device. The storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, provided on the computer device.
While the invention has been described with reference to specific embodiments, the scope of the invention is not limited thereto, and those skilled in the art can easily conceive various equivalent modifications or substitutions within the technical scope of the invention. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (10)
1. A method for training a longitudinal federated learning model is applied to a first device in a longitudinal federated learning model training process, and comprises the following steps:
generating an encryption public key and a decryption private key; the encryption public key is used for sending to each second device participating in model training to encrypt data and parameters of the model, and the decryption private key is used for decrypting an encryption loss value of the model;
obtaining commitment data of the model to perform zero-knowledge certification to obtain a zero-knowledge certification result of the commitment data, wherein the commitment data is obtained by performing data commitment on a source data set by each second device;
acquiring encryption loss initialization parameters of the model to perform zero knowledge certification to obtain a zero knowledge certification result of the encryption loss initialization parameters, wherein the encryption loss initialization parameters are obtained by performing encryption loss initialization calculation on each second device based on the source data set;
obtaining an encryption loss value of the model to perform zero knowledge proof to obtain a zero knowledge proof result of the encryption loss value, wherein the encryption loss value is obtained by calculating each second device based on the encryption loss initialization parameter and the encryption loss calculation weight;
and decrypting the encrypted loss value based on the decryption private key to obtain a loss value in the model training process, and obtaining a well-trained longitudinal federated learning model when the loss value is smaller than a preset threshold value.
2. Training method according to claim 1, characterized in that the method further comprises:
and when the loss value is greater than or equal to the preset threshold value, updating the encryption loss calculation weight according to the loss value to obtain an updated encryption loss calculation weight, and sending the updated encryption loss calculation weight to each second device for iterative training of the model.
3. Training method according to claim 1, characterized in that it comprises:
generating a zero knowledge proof gate circuit based on a computational task, the computational task comprising: data commitment, encryption loss initialization and encryption loss calculation;
generating an attestation key and a verification key according to the zero knowledge attestation gate circuit, wherein the attestation key is used for generating an attestation by each second device, and the verification key is used for verifying the attestation;
and verifying the certification based on the verification secret key to obtain a zero-knowledge certification result of the calculation task result.
4. Training method according to claim 1, characterized in that the method further comprises:
determining whether false data exists according to a zero-knowledge proof result of the committed data;
when determining that no false data exists, using the source data set for encryption loss initialization calculation of the model;
and when determining that the false data exists, determining a second device providing the false data as a malicious node, removing the malicious node, and restarting the training of the model.
5. Training method according to claim 1, characterized in that the method further comprises:
determining whether a malicious node exists in the encryption loss initialization process according to a zero knowledge proof result of the encryption loss initialization parameter;
when it is determined that no malicious node exists in the encryption loss initialization process, using the encryption loss initialization parameter for encryption loss calculation of the model;
and when determining that the encryption loss initialization process has the malicious node, notifying each second device, and stopping the training of the model.
6. Training method according to claim 1, characterized in that the method further comprises:
determining whether a malicious node exists in the encryption loss calculation process according to a zero knowledge proof result of the encryption loss value;
when it is determined that no malicious node exists in the encryption loss calculation process, the encryption loss value is used for decryption to obtain a loss value in the model training process;
and when determining that the encryption loss calculation process has the malicious node, notifying each second device, and stopping the training of the model.
7. Training method according to any of claims 1 to 6, wherein said generating a public encryption key and a private decryption key comprises:
and generating the encryption public key and the decryption private key based on a homomorphic encryption algorithm.
8. A training device for a longitudinal federated learning model, comprising:
the key generation module is used for generating an encryption public key and a decryption private key; the encryption public key is used for sending to each second device participating in model training to encrypt data and parameters of the model, and the decryption private key is used for decrypting an encryption loss value of the model to obtain a loss value;
the data certification module is used for obtaining committed data of the model and carrying out zero knowledge certification to obtain a zero knowledge certification result of the committed data, wherein the committed data is obtained by carrying out data commitment on the source data set by each second device;
an encryption loss initialization proving module, configured to obtain an encryption loss initialization parameter of the model to perform zero knowledge proving to obtain a zero knowledge proving result of the encryption loss initialization parameter, where the encryption loss initialization parameter is obtained by performing encryption loss initialization calculation on each second device based on the source data set;
the encryption loss calculation proving module is used for obtaining an encryption loss value of the model to perform zero knowledge proving to obtain a zero knowledge proving result of the encryption loss value, wherein the encryption loss value is obtained by calculating each second device based on the encryption loss initialization parameter and the encryption loss calculation weight;
and the model output module is used for decrypting the encrypted loss value based on the decryption private key to obtain a loss value in the model training process, and when the loss value is smaller than a preset threshold value, a well-trained longitudinal federated learning model is obtained.
9. A computer device, wherein the computer device comprises a memory and a processor;
the memory is used for storing a computer program;
the processor, configured to execute the computer program and when executing the computer program, implement the method for training a longitudinal federated learning model as recited in any of claims 1 to 7.
10. A storage medium storing a computer program which, when executed by a processor, causes the processor to implement a method of training a longitudinal federal learning model as claimed in any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110688660.3A CN113420886B (en) | 2021-06-21 | 2021-06-21 | Training method, device, equipment and storage medium for longitudinal federal learning model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110688660.3A CN113420886B (en) | 2021-06-21 | 2021-06-21 | Training method, device, equipment and storage medium for longitudinal federal learning model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113420886A true CN113420886A (en) | 2021-09-21 |
| CN113420886B CN113420886B (en) | 2024-05-10 |
Family
ID=77789708
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110688660.3A Active CN113420886B (en) | 2021-06-21 | 2021-06-21 | Training method, device, equipment and storage medium for longitudinal federal learning model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113420886B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113989036A (en) * | 2021-12-30 | 2022-01-28 | 百融至信(北京)征信有限公司 | Federal learning prediction method and system without exposure of model-entering variable |
| CN114841363A (en) * | 2022-04-11 | 2022-08-02 | 北京理工大学 | Privacy protection and verifiable federal learning method based on zero-knowledge proof |
| CN116702922A (en) * | 2023-06-05 | 2023-09-05 | 京信数据科技有限公司 | Method, device, terminal equipment and storage medium for federal learning training |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111539731A (en) * | 2020-06-19 | 2020-08-14 | 支付宝(杭州)信息技术有限公司 | Block chain-based federal learning method and device and electronic equipment |
| CN111598254A (en) * | 2020-05-22 | 2020-08-28 | 深圳前海微众银行股份有限公司 | Federal learning modeling method, device and readable storage medium |
-
2021
- 2021-06-21 CN CN202110688660.3A patent/CN113420886B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111598254A (en) * | 2020-05-22 | 2020-08-28 | 深圳前海微众银行股份有限公司 | Federal learning modeling method, device and readable storage medium |
| CN111539731A (en) * | 2020-06-19 | 2020-08-14 | 支付宝(杭州)信息技术有限公司 | Block chain-based federal learning method and device and electronic equipment |
Non-Patent Citations (2)
| Title |
|---|
| 周俊 等: "联邦学习安全与隐私保护研究综述", 西华大学学报(自然科学版), no. 04, 10 July 2020 (2020-07-10) * |
| 周俊 等: "联邦学习安全与隐私保护研究综述", 西华大学学报（自然科学版）, vol. 39, no. 4, pages 9 - 17 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113989036A (en) * | 2021-12-30 | 2022-01-28 | 百融至信(北京)征信有限公司 | Federal learning prediction method and system without exposure of model-entering variable |
| CN113989036B (en) * | 2021-12-30 | 2022-03-18 | 百融至信(北京)征信有限公司 | Federal learning prediction method and system without exposure of model-entering variable |
| CN114841363A (en) * | 2022-04-11 | 2022-08-02 | 北京理工大学 | Privacy protection and verifiable federal learning method based on zero-knowledge proof |
| CN114841363B (en) * | 2022-04-11 | 2024-07-23 | 北京理工大学 | Zero knowledge proof-based privacy protection and verifiable federal learning method |
| CN116702922A (en) * | 2023-06-05 | 2023-09-05 | 京信数据科技有限公司 | Method, device, terminal equipment and storage medium for federal learning training |
| CN116702922B (en) * | 2023-06-05 | 2024-06-07 | 京信数据科技有限公司 | Training method, training device, terminal equipment and training medium based on malicious behavior detection |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113420886B (en) | 2024-05-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7295927B2 (en) | Methods and systems implemented by blockchain | |
| CN110971405B (en) | SM2 signing and decrypting method and system with cooperation of multiple parties | |
| CN110073633B (en) | Block chain data protection using homomorphic encryption | |
| Wang et al. | Authenticated garbling and efficient maliciously secure two-party computation | |
| Choudhuri et al. | Fairness in an unfair world: Fair multiparty computation from public bulletin boards | |
| CN112232527B (en) | Safe distributed federal deep learning method | |
| CN110419053B (en) | System and method for information protection | |
| WO2021120861A1 (en) | Method and apparatus for multi-party joint model data processing | |
| US9860058B2 (en) | Secret computation system, arithmetic unit, secret computation method and program | |
| CN113420886B (en) | Training method, device, equipment and storage medium for longitudinal federal learning model | |
| US10846372B1 (en) | Systems and methods for trustless proof of possession and transmission of secured data | |
| CN112733163B (en) | Monitorable zero-knowledge proof method and device based on discrete logarithm equality proof | |
| CN111162912B (en) | Verification method and device suitable for block chain and storage medium | |
| Xu et al. | Proof-carrying cloud computation: The case of convex optimization | |
| CN111861467A (en) | Supply chain financial transaction privacy protection method and system | |
| CN113193948A (en) | Multi-party united privacy data statistical analysis method and information data processing terminal | |
| Tran et al. | An efficient privacy-enhancing cross-silo federated learning and applications for false data injection attack detection in smart grids | |
| US11856095B2 (en) | Apparatus and methods for validating user data by using cryptography | |
| CN114337994A (en) | Data processing method, device and system | |
| Keshavarzkalhori et al. | Federify: a verifiable federated learning scheme based on zksnarks and blockchain | |
| CN108259180B (en) | Method for quantum specifying verifier signature | |
| US20220345312A1 (en) | Zero-knowledge contingent payments protocol for granting access to encrypted assets | |
| EP3364397B1 (en) | Secret authentication code adding device, secret authentification code adding method, and program | |
| Jiang | Timed encryption with application to deniable key exchange | |
| Pavithra et al. | Blockchain-based criminal smart contract for symmetric key selling using ZK-SNARKs |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |