CN111695674A - Federal learning method and device, computer equipment and readable storage medium - Google Patents
Federal learning method and device, computer equipment and readable storage medium Download PDFInfo
- Publication number
- CN111695674A CN111695674A CN202010408557.4A CN202010408557A CN111695674A CN 111695674 A CN111695674 A CN 111695674A CN 202010408557 A CN202010408557 A CN 202010408557A CN 111695674 A CN111695674 A CN 111695674A
- Authority
- CN
- China
- Prior art keywords
- data
- sample data
- vector
- federal learning
- sample
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 59
- 238000003860 storage Methods 0.000 title claims abstract description 16
- 239000013598 vector Substances 0.000 claims abstract description 146
- 238000012549 training Methods 0.000 claims abstract description 26
- 238000004590 computer program Methods 0.000 claims description 23
- 238000012545 processing Methods 0.000 claims description 19
- 238000006243 chemical reaction Methods 0.000 claims description 16
- 230000002776 aggregation Effects 0.000 claims description 14
- 238000004220 aggregation Methods 0.000 claims description 14
- 238000004422 calculation algorithm Methods 0.000 claims description 8
- 230000005540 biological transmission Effects 0.000 claims description 6
- 238000013473 artificial intelligence Methods 0.000 abstract description 9
- 230000008569 process Effects 0.000 abstract description 4
- 230000006399 behavior Effects 0.000 description 24
- 230000006870 function Effects 0.000 description 18
- 210000002569 neuron Anatomy 0.000 description 8
- 230000010354 integration Effects 0.000 description 7
- 238000010586 diagram Methods 0.000 description 5
- 230000004913 activation Effects 0.000 description 4
- 238000013136 deep learning model Methods 0.000 description 4
- 238000000605 extraction Methods 0.000 description 4
- 238000010606 normalization Methods 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 238000010801 machine learning Methods 0.000 description 3
- 238000005457 optimization Methods 0.000 description 3
- 230000003213 activating effect Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000013145 classification model Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 210000005036 nerve Anatomy 0.000 description 2
- 230000001537 neural effect Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000005304 joining Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/602—Providing cryptographic facilities or services
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
- G06Q40/08—Insurance
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Mathematical Physics (AREA)
- Computing Systems (AREA)
- Business, Economics & Management (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biophysics (AREA)
- Finance (AREA)
- Computational Linguistics (AREA)
- Accounting & Taxation (AREA)
- Computer Hardware Design (AREA)
- Computer Security & Cryptography (AREA)
- Bioethics (AREA)
- Development Economics (AREA)
- Economics (AREA)
- Marketing (AREA)
- Strategic Management (AREA)
- Technology Law (AREA)
- General Business, Economics & Management (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Medical Informatics (AREA)
- Image Analysis (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
Abstract
The invention relates to artificial intelligence and provides a method and a device for federated learning, computer equipment and a computer readable storage medium. The federal learning method acquires sample data and an identification code of the sample data; converting the sample data into a vector to obtain vector representation of the sample data; coding the vector representation to obtain a characteristic vector of the sample data; and transmitting the characteristic vector and the identification code to a data request end, enabling the data request end to search a label of the sample data according to the identification code, and performing federated learning model training according to the characteristic vector and the label. The invention improves the safety of data in the federal learning process.
Description
Technical Field
The invention relates to the technical field of artificial intelligence, in particular to a method and a device for bang learning, computer equipment and a computer readable storage medium.
Background
With the development of artificial intelligence technology, machine learning modeling by joining different participants (or party, also called data owner, or client) becomes a trend of development, namely federal learning.
In federal learning, how to ensure the safety of data while avoiding data distortion becomes a problem to be solved.
Disclosure of Invention
In view of the foregoing, there is a need for a federated learning method, apparatus, computer device, and computer-readable storage medium that can generate a federated learning model through federated learning.
A first aspect of the present application provides a federated learning method, which includes:
acquiring sample data and an identification code of the sample data;
converting the sample data into a vector to obtain vector representation of the sample data;
coding the vector representation to obtain a characteristic vector of the sample data;
and transmitting the characteristic vector and the identification code to a data request end, enabling the data request end to search a label of the sample data according to the identification code, and performing federated learning model training according to the characteristic vector and the label.
In another possible implementation manner, the converting the sample data into a vector includes:
acquiring the data type of the sample data;
judging whether the sample data needs to be converted into a vector according to the data type of the sample data;
and if the sample data is judged to be required to be converted into the vector according to the data type of the sample data, converting the sample data into the vector according to a preset conversion method corresponding to the data type of the sample data.
In another possible implementation, the encoding the vector representation includes:
obtaining a sample vector;
training a coding model through a back propagation algorithm according to the sample vector, wherein the coding model is composed of an encoder and a decoder;
the vector representation is encoded with a trained encoder.
In another possible implementation manner, the transmitting the feature vector and the identification code to the data request side includes:
and transmitting the feature vector and the identification code to the data request terminal through an encryption algorithm.
In another possible implementation manner, the performing federal learning model training according to the feature vector and the label includes:
the data request terminal acquires initial parameters of the federal learning model from a preset server terminal;
the data request terminal initializes the federal learning model by using the initial parameters;
the data request terminal carries out local training on the initialized federated learning model according to the feature vector and the label, and updates the parameters of the initialized federated learning model to obtain updated parameters;
the data request end uploads the updated parameters to the preset service end, the preset service end carries out aggregation processing on the parameters uploaded by each request end to obtain aggregation parameters, and when the fact that the federal learning model updated by the aggregation parameters is in a convergence state is detected, the updated federal learning model is issued to the data request end;
and the data request end receives the federal learning model issued by the preset server end.
In another possible implementation manner, the federal learning method further includes:
and optimizing the trained coding model by a sparse coding method.
In another possible implementation manner, the federal learning method further includes:
acquiring a processing result of the data to be processed;
acquiring a preset result of the data to be processed;
and adjusting parameters and/or hyper-parameters of the coding model and/or the federal learning model according to the processing result of the data to be processed and the preset result of the data to be processed.
A second aspect of the present application provides a federal learning device, which includes:
the acquisition module is used for acquiring sample data and the identification code of the sample data;
the conversion module is used for converting the sample data into a vector to obtain vector representation of the sample data;
the encoding module is used for encoding the vector representation to obtain a characteristic vector of the sample data;
and the transmission module is used for transmitting the characteristic vector and the identification code to a data request end, so that the data request end searches for the label of the sample data according to the identification code and performs federated learning model training according to the characteristic vector and the label.
A third aspect of the application provides a computer device comprising a processor for implementing the federal learning method when executing a computer program stored in a memory.
A fourth aspect of the present application provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the federal learning method.
In the invention, the coding model is used for carrying out feature learning and feature integration on the vector representation. Under the condition that the data request end does not have a decoder, the data request end cannot interpret and obtain the sample data corresponding to the vector representation, and the safety of the data is guaranteed. The data request end does not directly obtain the data of the data providing end, and the safety of the data in the federal learning process is improved
Drawings
Fig. 1 is a flowchart of a federal learning method provided in an embodiment of the present invention.
Fig. 2 is a structural diagram of a federal learning device according to an embodiment of the present invention.
Fig. 3 is a schematic diagram of a computer device provided by an embodiment of the present invention.
Detailed Description
In order that the above objects, features and advantages of the present invention can be more clearly understood, a detailed description of the present invention will be given below with reference to the accompanying drawings and specific embodiments. It should be noted that the embodiments and features of the embodiments of the present application may be combined with each other without conflict.
In the following description, numerous specific details are set forth to provide a thorough understanding of the present invention, and the described embodiments are merely a subset of the embodiments of the present invention, rather than a complete embodiment. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.
Preferably, the federated learning method of the present invention is implemented in one or more computer devices. The computer device is a device capable of automatically performing numerical calculation and/or information processing according to a preset or stored instruction, and the hardware includes, but is not limited to, a microprocessor, an Application Specific Integrated Circuit (ASIC), a Programmable Gate Array (FPGA), a Digital Signal Processor (DSP), an embedded device, and the like.
The computer device can be a desktop computer, a notebook, a palm computer, a cloud server and other computing devices. The computer equipment can carry out man-machine interaction with a user through a keyboard, a mouse, a remote controller, a touch panel or voice control equipment and the like.
Example one
Fig. 1 is a flowchart of a federal learning method according to an embodiment of the present invention. The federal learning method is applied to a data providing terminal, and the data providing terminal is computer equipment and is used for generating a federal learning model through federal learning.
As shown in fig. 1, the federal learning method includes:
101, obtaining sample data and an identification code of the sample data.
The identification code is the only identification information of the sample data and is used for identifying the sample data between the data providing end and a third party requesting the data.
The sample data can comprise different data types such as text data, one-hot data, numerical data, embedding data and the like.
For example, the data provider may be a financial company, the third party requesting data may be an insurance company, the sample data may be user behavior data of the financial company, and the data type of the sample data may be text data. The identification code can be a mobile phone number or an identity card number of the user corresponding to the user behavior data. The insurance company needs to obtain the coded user behavior data of the financial company, and the reliability scoring is carried out on the user according to the obtained coded user behavior data through the insurance reliability scoring model. Namely, the insurance company does not need to directly acquire the user behavior data of the financial company, and the data security of the financial company is protected. The insurance reliability scoring model may be a specific federal learning model, and the local federal learning model of the insurance company may be a deep learning model.
For another example, the data providing end may be an e-commerce company, the third party requesting data may be an advertisement company, the sample data may be commodity click behavior data of the e-commerce company, and the data type of the sample data may be numerical data. The identification code can be a mobile phone number or an identity card number of the user corresponding to the commodity click behavior data. The advertisement company needs to obtain the coded commodity click behavior data of the commercial company, and recommends commodities to the user according to the obtained coded commodity click behavior data through the commodity recommendation model. Namely, the advertisement company does not directly obtain the commodity click behavior data of the e-commerce company, and the data security of the e-commerce company is protected. The product recommendation model may be a specific federal learning model and the local federal learning model of the advertising company may be a deep learning model.
And 102, converting the sample data into a vector to obtain vector representation of the sample data.
In a specific embodiment, the converting the sample data into a vector includes:
acquiring the data type of the sample data;
judging whether the sample data needs to be converted into a vector according to the data type of the sample data;
and if the sample data is judged to be required to be converted into the vector according to the data type of the sample data, converting the sample data into the vector according to a preset conversion method corresponding to the data type of the sample data.
For example, if the data type of the sample data is text data (a preset conversion method corresponding to the text data is a word2vec method), the sample data is converted into a vector according to the word2vec method. For another example, if the data type of the sample data is numerical data (the preset conversion method corresponding to the numerical data is a normalization method), the sample data is converted into a vector according to the normalization method.
Further, the determining whether the sample data needs to be converted into a vector according to the data type of the sample data includes:
acquiring a data type table to be converted;
if the data type of the sample data exists in the data type table to be converted, the sample data needs to be converted into a vector;
and if the data type of the sample data does not exist in the data type table to be converted, the sample data does not need to be converted into a vector. At this time, the data to be converted is a vector and conversion is not needed.
103, encoding the vector representation to obtain a feature vector of the sample data.
In a specific embodiment, said encoding said vector representation comprises:
obtaining a sample vector;
training a coding model through a back propagation algorithm according to the sample vector, wherein the coding model is composed of an encoder and a decoder;
the vector representation is encoded with a trained encoder.
And the data providing end cannot directly request the sample data for a third party of the data in order to ensure the security of the sample data.
In a specific embodiment, the trained coding model is optimized by Deep auto-encoder (depth coding) or sparse auto-encoder (sparse coding) method.
The trained coding model may be optimized by a sparse coding method according to a difference between an output and an input of the trained coding model. The sparse coding method is mainly used for optimizing the trained coding model by adding a sparsity limiting condition to a nerve unit in the trained coding model.
Specifically, sparsity constraints may include activating neurons when their output is close to 1 (e.g., greater than 0.9); when the output value of the neuron approaches 0 (e.g., less than or equal to 0.9), the neuron is not activated.
The coding model is used for carrying out feature learning and feature integration on the vector representation. Under the condition that the data request end does not have a decoder, the data request end cannot interpret and obtain the sample data corresponding to the vector representation, and the safety of the data is guaranteed.
And 104, transmitting the feature vector and the identification code to a data request end, enabling the data request end to search a label of the sample data according to the identification code, and performing federated learning model training according to the feature vector and the label.
The federated learning model includes: LR, XGB, DNN, etc. And the models such as LR, XGB, DNN and the like are used for machine learning training to achieve the purpose of service use. The federated learning model may be a specific artificial intelligence model, such as an artificial intelligence classification model, an artificial intelligence recognition model, and the like.
In a specific embodiment, the transmitting the feature vector and the identification code to the data request end includes:
and transmitting the feature vector and the identification code to the data request terminal through an encryption algorithm.
For example, encrypting the feature vector and the identification code by a private key of the data provider; and transmitting the encrypted characteristic vector and the identification code to the data request end, so that the data request end decrypts the encrypted characteristic vector and the encrypted identification code through the public key of the data providing end.
When the sample data is user behavior data, the labels of the sample data are 'risk user' and 'normal user'. And when the sample data is commodity click behavior data, the labels of the sample data are 'recommend commodity one', 'recommend commodity two', and the like.
In a specific embodiment, the performing federal learning model training according to the feature vector and the tag includes:
the data request terminal acquires initial parameters of the federal learning model from a preset server terminal;
the data request terminal initializes the federal learning model by using the initial parameters;
the data request terminal carries out local training on the initialized federated learning model according to the feature vector and the label, and updates the parameters of the initialized federated learning model to obtain updated parameters;
the data request end uploads the updated parameters to the preset service end, the preset service end carries out aggregation processing on the parameters uploaded by each request end to obtain aggregation parameters, and when the fact that the federal learning model updated by the aggregation parameters is in a convergence state is detected, the updated federal learning model is issued to the data request end;
and the data request end receives the federal learning model issued by the preset server end.
In another embodiment, before the updated federated learning model is sent to the data request end, when the preset service end detects that the federated learning model updated with the aggregation parameters is in a non-convergence state, the preset service end returns the aggregation parameters to the data request end, so that the data request end continues iterative training.
Embodiment one a federated learning model is generated through federated learning. The coding model is used for carrying out feature learning and feature integration on the vector representation, and the data request end cannot interpret and obtain sample data corresponding to the vector representation under the condition that a decoder is not arranged, so that the safety of data is guaranteed, and data leakage is prevented. The coding model does not add noise to the vector representation, and avoids generating additional interference information due to the addition of the noise. The establishment of the federal model has direct feedback on the coding result, thereby being beneficial to optimization and adjustment. The coding model can adjust the information loss degree and the information safety degree for performing feature learning and feature integration on the vector representation, find the benefit break-over point of information safety and information loss, and obtain more optimized parameters of the whole federal learning model. The data request end does not directly obtain the data of the data providing end, and the safety of the data in the federal learning process is improved.
In a specific embodiment, the federal learning method further includes:
adjusting hyper-parameters of the encoder and/or the federated learning model. The hyper-parameters include network structure, number of neural layers, number of neurons per layer, activation functions, learning rate, regularization and penalty coefficients, loss functions, and the like. In particular, when the loss function floats without convergence, the loss function, learning rate, and/or network structure, etc. may be adjusted. When the gradient disappears or the gradient explodes, the activation function is adjusted.
In a specific embodiment, the federal learning method further includes:
acquiring parameters of the trained federated learning model from the data request terminal;
acquiring data to be processed;
updating a local federal learning model by using the parameters of the trained federal learning model;
and processing the data to be processed by using the updated local federal learning model.
In a specific embodiment, the federal learning method further includes:
acquiring a processing result of the data to be processed;
acquiring a preset result of the data to be processed;
and adjusting parameters and/or hyper-parameters of the coding model and/or the federal learning model according to the processing result of the data to be processed and the preset result of the data to be processed.
Specifically, whether the coding model excessively encodes data can be judged according to the processing result of the data to be processed and the preset result of the data to be processed, and the excessively encoded data can cause the coding model to lose the capability of extracting effective features; and adjusting the coding model according to the judgment result so as to improve the feature extraction capability of the coding model and balance the feature extraction capability with the data security achieved by coding.
Example two
Fig. 2 is a structural diagram of a federal learning device according to a second embodiment of the present invention. The federal learning device 20 is applied to a data providing terminal, which is a computer device. The federal learning device 20 is used for generating a federal learning model through federal learning.
As shown in fig. 2, the federal learning device 20 may include an acquisition module 201, a conversion module 202, an encoding module 203, and a transmission module 204.
The obtaining module 201 is configured to obtain sample data and an identification code of the sample data.
The identification code is the only identification information of the sample data and is used for identifying the sample data between the data providing end and a third party requesting the data.
The sample data can comprise different data types such as text data, one-hot data, numerical data, embedding data and the like.
For example, the data provider may be a financial company, the third party requesting data may be an insurance company, the sample data may be user behavior data of the financial company, and the data type of the sample data may be text data. The identification code can be a mobile phone number or an identity card number of the user corresponding to the user behavior data. The insurance company needs to obtain the coded user behavior data of the financial company, and the reliability scoring is carried out on the user according to the obtained coded user behavior data through the insurance reliability scoring model. Namely, the insurance company does not need to directly acquire the user behavior data of the financial company, and the data security of the financial company is protected. The insurance reliability scoring model may be a specific federal learning model, and the local federal learning model of the insurance company may be a deep learning model.
For another example, the data providing end may be an e-commerce company, the third party requesting data may be an advertisement company, the sample data may be commodity click behavior data of the e-commerce company, and the data type of the sample data may be numerical data. The identification code can be a mobile phone number or an identity card number of the user corresponding to the commodity click behavior data. The advertisement company needs to obtain the coded commodity click behavior data of the commercial company, and recommends commodities to the user according to the obtained coded commodity click behavior data through the commodity recommendation model. Namely, the advertisement company does not directly obtain the commodity click behavior data of the e-commerce company, and the data security of the e-commerce company is protected. The product recommendation model may be a specific federal learning model and the local federal learning model of the advertising company may be a deep learning model.
A conversion module 202, configured to convert the sample data into a vector, so as to obtain a vector representation of the sample data.
In a specific embodiment, the converting the sample data into a vector includes:
acquiring the data type of the sample data;
judging whether the sample data needs to be converted into a vector according to the data type of the sample data;
and if the sample data is judged to be required to be converted into the vector according to the data type of the sample data, converting the sample data into the vector according to a preset conversion method corresponding to the data type of the sample data.
For example, if the data type of the sample data is text data (a preset conversion method corresponding to the text data is a word2vec method), the sample data is converted into a vector according to the word2vec method. For another example, if the data type of the sample data is numerical data (the preset conversion method corresponding to the numerical data is a normalization method), the sample data is converted into a vector according to the normalization method.
Further, the determining whether the sample data needs to be converted into a vector according to the data type of the sample data includes:
acquiring a data type table to be converted;
if the data type of the sample data exists in the data type table to be converted, the sample data needs to be converted into a vector;
and if the data type of the sample data does not exist in the data type table to be converted, the sample data does not need to be converted into a vector. At this time, the data to be converted is a vector and conversion is not needed.
And the encoding module 203 is configured to encode the vector representation to obtain a feature vector of the sample data.
In a specific embodiment, said encoding said vector representation comprises:
obtaining a sample vector;
training a coding model through a back propagation algorithm according to the sample vector, wherein the coding model is composed of an encoder and a decoder;
the vector representation is encoded with a trained encoder.
And the data providing end cannot directly request the sample data for a third party of the data in order to ensure the security of the sample data.
In another embodiment, the federal learning device 20 further includes an optimization module for optimizing the trained coding model by Deep auto-encoder (Deep coding) or sparse auto-encoder (sparse coding).
The trained coding model may be optimized by a sparse coding method according to a difference between an output and an input of the trained coding model. The sparse coding method is mainly used for optimizing the trained coding model by adding a sparsity limiting condition to a nerve unit in the trained coding model.
Specifically, sparsity constraints may include activating neurons when their output is close to 1 (e.g., greater than 0.9); when the output value of the neuron approaches 0 (e.g., less than or equal to 0.9), the neuron is not activated.
The coding model is used for carrying out feature learning and feature integration on the vector representation. Under the condition that the data request end does not have a decoder, the data request end cannot interpret and obtain the sample data corresponding to the vector representation, and the safety of the data is guaranteed.
And the transmission module 204 is configured to transmit the feature vector and the identification code to a data request end, so that the data request end searches for a tag of the sample data according to the identification code, and performs federated learning model training according to the feature vector and the tag.
The federated learning model includes: LR, XGB, DNN, etc. And the models such as LR, XGB, DNN and the like are used for machine learning training to achieve the purpose of service use. The federated learning model may be a specific artificial intelligence model, such as an artificial intelligence classification model, an artificial intelligence recognition model, and the like.
In a specific embodiment, the transmitting the feature vector and the identification code to the data request end includes:
and transmitting the feature vector and the identification code to the data request terminal through an encryption algorithm.
For example, encrypting the feature vector and the identification code by a private key of the data provider; and transmitting the encrypted characteristic vector and the identification code to the data request end, so that the data request end decrypts the encrypted characteristic vector and the encrypted identification code through the public key of the data providing end.
When the sample data is user behavior data, the labels of the sample data are 'risk user' and 'normal user'. And when the sample data is commodity click behavior data, the labels of the sample data are 'recommend commodity one', 'recommend commodity two', and the like.
In a specific embodiment, the performing federal learning model training according to the feature vector and the tag includes:
the data request terminal acquires initial parameters of the federal learning model from a preset server terminal;
the data request terminal initializes the federal learning model by using the initial parameters;
the data request terminal carries out local training on the initialized federated learning model according to the feature vector and the label, and updates the parameters of the initialized federated learning model to obtain updated parameters;
the data request end uploads the updated parameters to the preset service end, the preset service end carries out aggregation processing on the parameters uploaded by each request end to obtain aggregation parameters, and when the fact that the federal learning model updated by the aggregation parameters is in a convergence state is detected, the updated federal learning model is issued to the data request end;
and the data request end receives the federal learning model issued by the preset server end.
The federal learning device 20 of the second embodiment generates a federal learning model through federal learning. The coding model is used for carrying out feature learning and feature integration on the vector representation, and the data request end cannot interpret and obtain sample data corresponding to the vector representation under the condition that a decoder is not arranged, so that the safety of data is guaranteed, and data leakage is prevented. The coding model does not add noise to the vector representation, and avoids generating additional interference information due to the addition of the noise. The establishment of the federal model has direct feedback on the coding result, thereby being beneficial to optimization and adjustment. The coding model can adjust the information loss degree and the information safety degree for performing feature learning and feature integration on the vector representation, find the benefit break-over point of information safety and information loss, and obtain more optimized parameters of the whole federal learning model. The data request end does not directly obtain the data of the data providing end, and the safety of the data in the federal learning process is improved.
In a specific embodiment, the federal learning device 20 further includes an adjustment module for adjusting the hyper-parameters of the encoder and/or the federal learning model. The hyper-parameters include network structure, number of neural layers, number of neurons per layer, activation functions, learning rate, regularization and penalty coefficients, loss functions, and the like. In particular, when the loss function floats without convergence, the loss function, learning rate, and/or network structure, etc. may be adjusted. When the gradient disappears or the gradient explodes, the activation function is adjusted.
In a specific embodiment, the federal learning device 20 further includes a processing module, configured to obtain parameters of the trained federal learning model from the data request end; acquiring data to be processed; updating a local federal learning model by using the parameters of the trained federal learning model; and processing the data to be processed by using the updated local federal learning model.
In a specific embodiment, the adjusting module is further configured to obtain a processing result of the data to be processed; acquiring a preset result of the data to be processed; and adjusting parameters and/or hyper-parameters of the coding model and/or the federal learning model according to the processing result of the data to be processed and the preset result of the data to be processed.
Specifically, whether the coding model excessively encodes data can be judged according to the processing result of the data to be processed and the preset result of the data to be processed, and the excessively encoded data can cause the coding model to lose the capability of extracting effective features; and adjusting the coding model according to the judgment result so as to improve the feature extraction capability of the coding model and balance the feature extraction capability with the data security achieved by coding.
EXAMPLE III
The present embodiment provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the computer program implements the steps in the above-mentioned federal learning method embodiment, for example, the steps 101 and 104 shown in fig. 1:
101, acquiring sample data and an identification code of the sample data;
102, converting the sample data into a vector to obtain a vector representation of the sample data;
103, encoding the vector representation to obtain a feature vector of the sample data;
and 104, transmitting the feature vector and the identification code to a data request end, enabling the data request end to search a label of the sample data according to the identification code, and performing federated learning model training according to the feature vector and the label.
Alternatively, the computer program, when executed by the processor, implements the functions of the modules in the above device embodiments, such as the module 201 and 204 in fig. 2:
an obtaining module 201, configured to obtain sample data and an identification code of the sample data;
a conversion module 202, configured to convert the sample data into a vector, so as to obtain a vector representation of the sample data;
the encoding module 203 is configured to encode the vector representation to obtain a feature vector of the sample data;
and the transmission module 204 is configured to transmit the feature vector and the identification code to a data request end, so that the data request end searches for a tag of the sample data according to the identification code, and performs federated learning model training according to the feature vector and the tag.
Example four
Fig. 3 is a schematic diagram of a computer device according to a third embodiment of the present invention. The computer device 30 comprises a memory 301, a processor 302 and a computer program 303, such as a federal learning program, stored in the memory 301 and executable on the processor 302. The processor 302, when executing the computer program 303, implements the steps in the above-described federal learning method embodiment, such as 101-104 shown in fig. 1:
101, acquiring sample data and an identification code of the sample data;
102, converting the sample data into a vector to obtain a vector representation of the sample data;
103, encoding the vector representation to obtain a feature vector of the sample data;
and 104, transmitting the feature vector and the identification code to a data request end, enabling the data request end to search a label of the sample data according to the identification code, and performing federated learning model training according to the feature vector and the label.
Alternatively, the computer program, when executed by the processor, implements the functions of the modules in the above device embodiments, such as the module 201 and 204 in fig. 2:
an obtaining module 201, configured to obtain sample data and an identification code of the sample data;
a conversion module 202, configured to convert the sample data into a vector, so as to obtain a vector representation of the sample data;
the encoding module 203 is configured to encode the vector representation to obtain a feature vector of the sample data;
and the transmission module 204 is configured to transmit the feature vector and the identification code to a data request end, so that the data request end searches for a tag of the sample data according to the identification code, and performs federated learning model training according to the feature vector and the tag.
Illustratively, the computer program 303 may be partitioned into one or more modules that are stored in the memory 301 and executed by the processor 302 to perform the present method. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, which are used to describe the execution of the computer program 303 in the computer device 30. For example, the computer program 303 may be divided into the obtaining module 201, the transforming module 202, the encoding module 203, and the transmitting module 204 in fig. 2, and the specific functions of each module are described in the second embodiment.
Those skilled in the art will appreciate that the schematic diagram 3 is merely an example of the computer device 30 and does not constitute a limitation of the computer device 30, and may include more or less components than those shown, or combine certain components, or different components, for example, the computer device 30 may also include input and output devices, network access devices, buses, etc.
The Processor 302 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor 302 may be any conventional processor or the like, the processor 302 being the control center for the computer device 30 and connecting the various parts of the overall computer device 30 using various interfaces and lines.
The memory 301 may be used to store the computer program 303, and the processor 302 may implement various functions of the computer device 30 by running or executing the computer program or module stored in the memory 301 and calling data stored in the memory 301. The memory 301 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data created according to the use of the computer device 30, and the like. Further, the memory 301 may include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), at least one magnetic disk storage device, a Flash memory device, or other non-volatile solid state storage device.
The modules integrated by the computer device 30 may be stored in a computer-readable storage medium if they are implemented in the form of software functional modules and sold or used as separate products. Based on such understanding, all or part of the flow of the method according to the embodiments of the present invention may also be implemented by a computer program, which may be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method embodiments may be implemented. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying said computer program code, recording medium, U-disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM).
In the embodiments provided in the present invention, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is only one logical functional division, and other divisions may be realized in practice.
The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical modules, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
In addition, functional modules in the embodiments of the present invention may be integrated into one processing module, or each of the modules may exist alone physically, or two or more modules are integrated into one module. The integrated module can be realized in a hardware form, and can also be realized in a form of hardware and a software functional module.
The integrated module implemented in the form of a software functional module may be stored in a computer-readable storage medium. The software functional module is stored in a storage medium and includes several instructions to enable a computer device (which may be a personal computer, a server, or a network device) or a processor (processor) to execute some steps of the federal learning method according to various embodiments of the present invention.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference signs in the claims shall not be construed as limiting the claim concerned. Furthermore, it is to be understood that the word "comprising" does not exclude other modules or steps, and the singular does not exclude the plural. A plurality of modules or means recited in the system claims may also be implemented by one module or means in software or hardware. The terms first, second, etc. are used to denote names, but not any particular order.
Finally, it should be noted that the above embodiments are only for illustrating the technical solutions of the present invention and not for limiting, and although the present invention is described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions may be made on the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.
Claims (10)
1. A federated learning method is characterized in that the federated learning method comprises the following steps:
acquiring sample data and an identification code of the sample data;
converting the sample data into a vector to obtain vector representation of the sample data;
coding the vector representation to obtain a characteristic vector of the sample data;
and transmitting the characteristic vector and the identification code to a data request end, enabling the data request end to search a label of the sample data according to the identification code, and performing federated learning model training according to the characteristic vector and the label.
2. The federal learning method of claim 1, wherein said converting the sample data into vectors comprises:
acquiring the data type of the sample data;
judging whether the sample data needs to be converted into a vector according to the data type of the sample data;
and if the sample data is judged to be required to be converted into the vector according to the data type of the sample data, converting the sample data into the vector according to a preset conversion method corresponding to the data type of the sample data.
3. The federal learning method as claimed in claim 1, wherein said encoding the vector representation comprises:
obtaining a sample vector;
training a coding model through a back propagation algorithm according to the sample vector, wherein the coding model is composed of an encoder and a decoder;
the vector representation is encoded with a trained encoder.
4. The federal learning method as claimed in claim 1, wherein said transmitting the feature vector and the identification code to a data requestor comprises:
and transmitting the feature vector and the identification code to the data request terminal through an encryption algorithm.
5. The federal learning method of claim 1, wherein the federal learning model training based on the feature vectors and the tags comprises:
the data request terminal acquires initial parameters of the federal learning model from a preset server terminal;
the data request terminal initializes the federal learning model by using the initial parameters;
the data request terminal carries out local training on the initialized federated learning model according to the feature vector and the label, and updates the parameters of the initialized federated learning model to obtain updated parameters;
the data request end uploads the updated parameters to the preset service end, the preset service end carries out aggregation processing on the parameters uploaded by each request end to obtain aggregation parameters, and when the fact that the federal learning model updated by the aggregation parameters is in a convergence state is detected, the updated federal learning model is issued to the data request end;
and the data request end receives the federal learning model issued by the preset server end.
6. The federal learning method of any of claims 1-5, further comprising:
and optimizing the trained coding model by a sparse coding method.
7. The federal learning method of any of claims 1-5, further comprising:
acquiring a processing result of the data to be processed;
acquiring a preset result of the data to be processed;
and adjusting parameters and/or hyper-parameters of the coding model and/or the federal learning model according to the processing result of the data to be processed and the preset result of the data to be processed.
8. The utility model provides a federal learning device, its characterized in that, federal learning device includes:
the acquisition module is used for acquiring sample data and the identification code of the sample data;
the conversion module is used for converting the sample data into a vector to obtain vector representation of the sample data;
the encoding module is used for encoding the vector representation to obtain a characteristic vector of the sample data;
and the transmission module is used for transmitting the characteristic vector and the identification code to a data request end, so that the data request end searches for the label of the sample data according to the identification code and performs federated learning model training according to the characteristic vector and the label.
9. A computer device comprising a processor configured to execute a computer program stored in a memory to implement the federal learning method as claimed in any of claims 1-7.
10. A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the federal learning method as claimed in any of claims 1-7.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010408557.4A CN111695674B (en) | 2020-05-14 | 2020-05-14 | Federal learning method, federal learning device, federal learning computer device, and federal learning computer readable storage medium |
| PCT/CN2020/098890 WO2021114618A1 (en) | 2020-05-14 | 2020-06-29 | Federated learning method and apparatus, computer device, and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010408557.4A CN111695674B (en) | 2020-05-14 | 2020-05-14 | Federal learning method, federal learning device, federal learning computer device, and federal learning computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111695674A true CN111695674A (en) | 2020-09-22 |
| CN111695674B CN111695674B (en) | 2024-04-09 |
Family
ID=72477764
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010408557.4A Active CN111695674B (en) | 2020-05-14 | 2020-05-14 | Federal learning method, federal learning device, federal learning computer device, and federal learning computer readable storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN111695674B (en) |
| WO (1) | WO2021114618A1 (en) |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112364919A (en) * | 2020-11-11 | 2021-02-12 | 深圳前海微众银行股份有限公司 | Data processing method, device, equipment and storage medium |
| CN112396189A (en) * | 2020-11-27 | 2021-02-23 | 中国银联股份有限公司 | Method and device for multi-party construction of federal learning model |
| CN112463161A (en) * | 2020-11-27 | 2021-03-09 | 平安科技（深圳）有限公司 | Code annotation generation method, system and device based on federal learning |
| CN113239351A (en) * | 2020-12-08 | 2021-08-10 | 武汉大学 | Novel data pollution attack defense method for Internet of things system |
| CN113283185A (en) * | 2021-07-23 | 2021-08-20 | 平安科技（深圳）有限公司 | Federal model training and client imaging method, device, equipment and medium |
| CN113722987A (en) * | 2021-08-16 | 2021-11-30 | 京东科技控股股份有限公司 | Federal learning model training method and device, electronic equipment and storage medium |
| CN114710415A (en) * | 2022-05-23 | 2022-07-05 | 北京理工大学 | Redundant coded passive message reliable transmission and processing system |
| CN115034816A (en) * | 2022-06-07 | 2022-09-09 | 青岛文达通科技股份有限公司 | Demand prediction method and system based on unsupervised and federal learning |
| CN115495541A (en) * | 2022-11-18 | 2022-12-20 | 深译信息科技(珠海)有限公司 | Corpus database, corpus database maintenance method, apparatus, device and medium |
| WO2023134076A1 (en) * | 2022-01-12 | 2023-07-20 | 平安科技（深圳）有限公司 | Data protection method and system, and storage medium |
| WO2024155026A1 (en) * | 2023-01-18 | 2024-07-25 | 삼성전자 주식회사 | Asynchronous federated learning method for protecting user privacy |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113781397B (en) * | 2021-08-11 | 2023-11-21 | 中国科学院信息工程研究所 | Medical image focus detection modeling method, device and system based on federal learning |
| CN114154392A (en) * | 2021-10-15 | 2022-03-08 | 海南火链科技有限公司 | Model co-construction method, device and equipment based on block chain and federal learning |
| CN114648130B (en) * | 2022-02-07 | 2024-04-16 | 北京航空航天大学 | Longitudinal federal learning method, device, electronic equipment and storage medium |
| CN114996317B (en) * | 2022-07-05 | 2024-02-23 | 中国电信股份有限公司 | Asynchronous optimization method and device based on longitudinal federal learning and storage medium |
| CN117424765B (en) * | 2023-12-19 | 2024-03-22 | 天津医康互联科技有限公司 | Distributed single-heat encoding method, device, electronic equipment and computer storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140013452A1 (en) * | 2012-07-03 | 2014-01-09 | Selim Aissi | Data protection hub |
| CN110738323A (en) * | 2018-07-03 | 2020-01-31 | 百度在线网络技术（北京）有限公司 | Method and device for establishing machine learning model based on data sharing |
| CN111027715A (en) * | 2019-12-11 | 2020-04-17 | 支付宝(杭州)信息技术有限公司 | Monte Carlo-based federated learning model training method and device |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110297848B (en) * | 2019-07-09 | 2024-02-23 | 深圳前海微众银行股份有限公司 | Recommendation model training method, terminal and storage medium based on federal learning |
| CN110807207B (en) * | 2019-10-30 | 2021-10-08 | 腾讯科技（深圳）有限公司 | Data processing method and device, electronic equipment and storage medium |
-
2020
- 2020-05-14 CN CN202010408557.4A patent/CN111695674B/en active Active
- 2020-06-29 WO PCT/CN2020/098890 patent/WO2021114618A1/en active Application Filing
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140013452A1 (en) * | 2012-07-03 | 2014-01-09 | Selim Aissi | Data protection hub |
| CN110738323A (en) * | 2018-07-03 | 2020-01-31 | 百度在线网络技术（北京）有限公司 | Method and device for establishing machine learning model based on data sharing |
| CN111027715A (en) * | 2019-12-11 | 2020-04-17 | 支付宝(杭州)信息技术有限公司 | Monte Carlo-based federated learning model training method and device |
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112364919A (en) * | 2020-11-11 | 2021-02-12 | 深圳前海微众银行股份有限公司 | Data processing method, device, equipment and storage medium |
| CN112463161B (en) * | 2020-11-27 | 2023-07-28 | 平安科技（深圳）有限公司 | Code annotation generation method, system and device based on federal learning |
| CN112396189A (en) * | 2020-11-27 | 2021-02-23 | 中国银联股份有限公司 | Method and device for multi-party construction of federal learning model |
| CN112463161A (en) * | 2020-11-27 | 2021-03-09 | 平安科技（深圳）有限公司 | Code annotation generation method, system and device based on federal learning |
| CN112396189B (en) * | 2020-11-27 | 2023-09-01 | 中国银联股份有限公司 | Method and device for constructing federal learning model by multiple parties |
| CN113239351A (en) * | 2020-12-08 | 2021-08-10 | 武汉大学 | Novel data pollution attack defense method for Internet of things system |
| CN113239351B (en) * | 2020-12-08 | 2022-05-13 | 武汉大学 | Novel data pollution attack defense method for Internet of things system |
| CN113283185A (en) * | 2021-07-23 | 2021-08-20 | 平安科技（深圳）有限公司 | Federal model training and client imaging method, device, equipment and medium |
| CN113722987A (en) * | 2021-08-16 | 2021-11-30 | 京东科技控股股份有限公司 | Federal learning model training method and device, electronic equipment and storage medium |
| CN113722987B (en) * | 2021-08-16 | 2023-11-03 | 京东科技控股股份有限公司 | Training method and device of federal learning model, electronic equipment and storage medium |
| WO2023134076A1 (en) * | 2022-01-12 | 2023-07-20 | 平安科技（深圳）有限公司 | Data protection method and system, and storage medium |
| CN114710415B (en) * | 2022-05-23 | 2022-08-12 | 北京理工大学 | Redundant coded passive message reliable transmission and processing system |
| CN114710415A (en) * | 2022-05-23 | 2022-07-05 | 北京理工大学 | Redundant coded passive message reliable transmission and processing system |
| CN115034816A (en) * | 2022-06-07 | 2022-09-09 | 青岛文达通科技股份有限公司 | Demand prediction method and system based on unsupervised and federal learning |
| CN115495541A (en) * | 2022-11-18 | 2022-12-20 | 深译信息科技(珠海)有限公司 | Corpus database, corpus database maintenance method, apparatus, device and medium |
| WO2024155026A1 (en) * | 2023-01-18 | 2024-07-25 | 삼성전자 주식회사 | Asynchronous federated learning method for protecting user privacy |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111695674B (en) | 2024-04-09 |
| WO2021114618A1 (en) | 2021-06-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111695674B (en) | Federal learning method, federal learning device, federal learning computer device, and federal learning computer readable storage medium | |
| CN112732911B (en) | Semantic recognition-based speaking recommendation method, device, equipment and storage medium | |
| CN111680217B (en) | Content recommendation method, device, equipment and storage medium | |
| Younus et al. | Video steganography using knight tour algorithm and LSB method for encrypted data | |
| WO2022105117A1 (en) | Method and device for image quality assessment, computer device, and storage medium | |
| CN112035743B (en) | Data recommendation method and device, computer equipment and storage medium | |
| CN113011646B (en) | Data processing method, device and readable storage medium | |
| CA3043809A1 (en) | System and method for machine learning architecture with adversarial attack defence | |
| CN111949886B (en) | Sample data generation method and related device for information recommendation | |
| US11886955B2 (en) | Self-supervised data obfuscation in foundation models | |
| US20240144005A1 (en) | Interpretable Tabular Data Learning Using Sequential Sparse Attention | |
| CN112529101A (en) | Method and device for training classification model, electronic equipment and storage medium | |
| CN113128196A (en) | Text information processing method and device, storage medium | |
| CN111552865A (en) | User interest portrait method and related equipment | |
| CN113065027A (en) | Video recommendation method and device, electronic equipment and storage medium | |
| CN117312535A (en) | Method, device, equipment and medium for processing problem data based on artificial intelligence | |
| JP2023517518A (en) | Vector embedding model for relational tables with null or equivalent values | |
| CN116127049A (en) | Model training method, text generation method, terminal device and computer medium | |
| CN115309905A (en) | Advertisement text generation method, device, equipment and medium | |
| CN115204299A (en) | Data processing method and device, storage medium and electronic equipment | |
| Lin et al. | A novel non-negative matrix factorization technique for decomposition of Chinese characters with application to secret sharing | |
| CN113705071A (en) | Equipment identification method, device, equipment and storage medium | |
| CN117591770B (en) | Policy pushing method and device and computer equipment | |
| CN116595978B (en) | Object category identification method, device, storage medium and computer equipment | |
| CN114676312A (en) | Data processing method, device, storage medium and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |