CN111046857A - Face recognition method, device, equipment, medium and system based on knowledge federation - Google Patents
Face recognition method, device, equipment, medium and system based on knowledge federation Download PDFInfo
- Publication number
- CN111046857A CN111046857A CN202010174510.6A CN202010174510A CN111046857A CN 111046857 A CN111046857 A CN 111046857A CN 202010174510 A CN202010174510 A CN 202010174510A CN 111046857 A CN111046857 A CN 111046857A
- Authority
- CN
- China
- Prior art keywords
- face recognition
- model
- federation
- gradient
- ciphertext
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Collating Specific Patterns (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a face recognition method based on knowledge federation, which relates to the technical field of computers and comprises the following steps: training a face recognition model based on the acquired face data set, generating a face recognition gradient ciphertext through encrypting the obtained face recognition model gradient, and sending the face recognition gradient ciphertext to a third party terminal to enable the face recognition gradient ciphertext to obtain a federal model parameter ciphertext through decryption, knowledge federation and encryption; and continuously training the face recognition model based on the decrypted federal model parameters, and performing face recognition according to the trained face recognition model. The face recognition method based on the Federal training ensures that original data does not leave the local, can effectively protect data privacy, has strong universality of a face recognition model obtained through Federal training, and is suitable for business scenes of terminals of organizations in different fields. The invention also discloses a face recognition device, electronic equipment, a storage medium and a system based on the knowledge federation. The scale of the mechanism terminal in the system is not limited, and the mechanism terminal can be flexibly added or withdrawn.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a face recognition method, a face recognition device, face recognition equipment, face recognition media and a face recognition system based on knowledge federation.
Background
The face recognition technology is widely applied to the fields of financial insurance, retail, security protection, public security and the like, has cross-industry requirements, and for example, when a user opens an account in the financial industry, the face photo acquired on site at a terminal in real time needs to be compared with a certificate photo called from a public security system; or when the security industry is controlled, the face photo shot by the monitoring camera needs to be compared with the certificate photo in the public security system.
However, because the distribution of face data in each field is inconsistent, for example, the data distribution of the identification photo of a public security system, the face photo in a complex scene in the financial industry, and the face photo in a natural scene in the security field is very different, the face recognition model obtained by training the face data set in each field is poor in generalization capability when applied to different application scenes. For example, a face recognition model trained from a security field data set cannot be used in the financial insurance industry.
The collected face data of each industry need to comply with the relevant safety standard of personal information, so that the collected face data can not be intercommunicated and used, and therefore, how to break a data island and integrate the face data of each field to train a universal face recognition model is urgent for solving the problem of cross-industry face recognition.
Disclosure of Invention
In order to overcome the defects of the prior art, one of the purposes of the invention is to provide a face recognition method based on the knowledge federation, which obtains a federated model parameter ciphertext obtained through the knowledge federation by sending a face recognition gradient ciphertext to a third-party terminal, further trains a face recognition model, and realizes face recognition according to the trained face recognition model.
One of the purposes of the invention is realized by adopting the following technical scheme:
a face recognition method based on knowledge federation comprises the following steps:
acquiring a face data set, and initializing a face recognition model according to preset parameters;
training the face recognition model based on the face data set to obtain a face recognition model gradient;
the face recognition gradient is encrypted to generate a face recognition gradient ciphertext;
sending the face recognition gradient ciphertext to a third-party terminal, and enabling the face recognition gradient ciphertext to obtain a federal model parameter ciphertext through decryption, knowledge federation and encryption;
receiving the federal model parameter ciphertext;
decrypting the ciphertext of the federal model parameter to obtain a federal model parameter;
and continuously training the face recognition model based on the federal model parameters, and recognizing the face according to the trained face recognition model.
Further, the gradient of the face recognition model is homomorphic encrypted by adopting a Parllier algorithm.
Further, continuing to train the face recognition model based on the federal model parameters includes:
updating the face recognition model parameters of the face recognition model based on the federal model parameters;
iteratively training the face recognition model according to the face data set and the updated face recognition model parameters;
and when the iterative training times of the face recognition model are equal to a preset iterative time threshold value, obtaining the trained face recognition model.
The second purpose of the invention is to provide a face recognition device based on the knowledge federation, which obtains a federation model parameter ciphertext obtained through the knowledge federation by sending a face recognition gradient ciphertext to a third-party terminal, further trains a face recognition model, and realizes face recognition according to the trained face recognition model.
The second purpose of the invention is realized by adopting the following technical scheme:
a knowledge federation-based face recognition device, comprising:
the data acquisition module is used for acquiring a face data set and initializing a face recognition model according to preset parameters;
the training encryption module is used for training the face recognition model based on the face data set to obtain a face recognition model gradient; the face recognition gradient is encrypted to generate a face recognition gradient ciphertext;
the federation module is used for sending the face recognition gradient ciphertext to a third-party terminal so that the face recognition gradient ciphertext is decrypted, subjected to knowledge federation and encrypted to obtain a federation model parameter ciphertext;
the face recognition module is used for receiving the federal model parameter ciphertext; decrypting the ciphertext of the federal model parameter to obtain a federal model parameter; and continuously training the face recognition model based on the federal model parameters, and recognizing the face according to the trained face recognition model.
It is a further object of the present invention to provide an electronic device for performing one of the objects of the invention, comprising a processor, a storage medium and a computer program, the computer program being stored in the storage medium, the computer program being such that when executed by the processor, the method for face recognition based on the knowledge federation is one of the objects of the invention.
It is a fourth object of the present invention to provide a computer-readable storage medium storing one of the objects of the invention, having a computer program stored thereon, which when executed by a processor, implements a knowledge federation-based face recognition method that is one of the objects of the invention.
The fifth purpose of the invention is to provide a face recognition system based on the knowledge federation, which decrypts, knowledge federation and encrypts a face recognition gradient ciphertext sent by each mechanism terminal through a third party terminal to obtain a federation model parameter ciphertext and sends the federation model parameter ciphertext to the mechanism terminals, each mechanism terminal updates local face recognition model parameters based on the federation model parameter ciphertext so as to train a local face recognition model, and face recognition is realized according to the trained face recognition model.
The fifth purpose of the invention is realized by adopting the following technical scheme:
a knowledge federation-based face recognition system, comprising: at least two agency terminals and at least one third party terminal;
the mechanism terminal is electronic equipment provided by the third object of the invention; the third-party terminal is used for initializing a federal model; receiving a face recognition gradient ciphertext, decrypting the face recognition gradient ciphertext and carrying out knowledge federation to obtain a federation-behind gradient, updating a federation model parameter based on the federation-behind gradient, and encrypting the updated federation model parameter to generate a federation model parameter ciphertext and sending the federation model parameter ciphertext to the institution terminal.
Further, the face recognition model on the mechanism terminal and the federal model adopt the same model, and the face recognition model and the federal model are initialized based on the same preset parameters.
Further, when the mechanism terminal completes initialization of the face recognition model, a deployment success instruction is sent to the third-party terminal; and the third-party terminal sends a training starting instruction to each mechanism terminal, so that each mechanism terminal simultaneously starts to train the face recognition model.
Further, when the number of the mechanism terminals is N, the third-party terminal decrypts and federally decrypts the face recognition gradient ciphertext to obtain a federal gradient, and updates federal model parameters based on the federal gradient, including:
the face recognition gradient ciphertext is decrypted to obtain face recognition model gradients of the N mechanism terminals;
calculating the average value or the median value of the gradients of the face recognition models of the N mechanism terminals to obtain the gradient after the federation;
and the current federal model parameter and the federal gradient are calculated by addition to obtain an updated federal model parameter.
Compared with the prior art, the invention has the beneficial effects that:
according to the invention, the original data of the user is ensured not to leave the local mechanism terminal through encryption, and the data privacy can be effectively protected; the face recognition model obtained based on knowledge federation training has stronger universality and is suitable for service scenes of terminals of mechanisms in various fields; the scale of the mechanism terminal in the system is not limited, and the mechanism terminal can flexibly select to join or quit the federal training of the face recognition model.
Drawings
FIG. 1 is a flow chart of a face recognition method based on knowledge federation according to a first embodiment of the present invention;
fig. 2 is a block diagram of a structure of a face recognition apparatus based on knowledge federation according to a second embodiment of the present invention;
fig. 3 is a block diagram of an electronic device according to a third embodiment of the present invention;
fig. 4 is a schematic view of a face recognition system based on the knowledge federation according to a fifth embodiment of the present invention.
Detailed Description
The present invention will now be described in more detail with reference to the accompanying drawings, in which the description of the invention is given by way of illustration and not of limitation. The various embodiments may be combined with each other to form other embodiments not shown in the following description.
Example one
The embodiment I provides a face recognition method based on knowledge federation, and aims to obtain a federation model parameter ciphertext obtained through the knowledge federation by sending a face recognition gradient ciphertext to a third-party terminal, further train a face recognition model, and realize face recognition according to the trained face recognition model. The method ensures that the original data of the user does not leave the local mechanism terminal through encryption, can effectively protect the data privacy, has strong universality of the trained face recognition model, and is suitable for face recognition in various fields of service scenes.
Referring to fig. 1, a face recognition method based on knowledge federation includes the following steps:
and S110, acquiring a face data set, and initializing a face recognition model according to preset parameters.
The face data set adopted by the local institution terminal can be a data set obtained from an existing face database in an industry, or a face data set acquired in real time in the industry, and the industry is not limited to one of the industries of financial insurance, retail, security and public security.
The face recognition model adopted by the mechanism terminal is a common deep learning neural network and is not limited to one of MLP, CNN and RNN. The federal model on the third-party terminal and the face recognition model deployed on each organization terminal adopt the same model.
The initialization of the face recognition model comprises the initialization of model parameters and the setting of some common parameters in the training process, and the preset parameters comprise but are not limited to optimizer parameters, training step size, batch-size, initialized learning rate and learning rate attenuation strategy. In order to ensure that the models on all the mechanism terminals can be stably converged when the local models based on the knowledge federation on the third-party terminal are trained in the subsequent steps, all the initialized parameters are consistent at each mechanism terminal.
And S120, training a face recognition model based on the face data set to obtain the gradient of the face recognition model.
Training a face recognition model on a local mechanism terminal by a common model training method, for example, calculating to obtain a face recognition model gradient based on a plurality of tasks training face recognition models, for example, chinese patent application No. CN201711290768.7 discloses a face recognition method, apparatus, computer device and readable storage medium; for example, chinese patent application No. CN201811550968.6 discloses a method and an apparatus for face recognition based on deep learning, and training of the face recognition model is not limited to the above-mentioned training method.
In this embodiment, the face sample data required for one training is selected from the face data set according to the preset batch-size parameter. Inputting the face sample data of a batch into a face recognition model, and training the face recognition model. The face recognition model is trained through inner-layer learning and outer-layer learning. In the inner-layer learning, aiming at a single task in a plurality of divided tasks, training a face recognition model based on a preset loss function and a gradient descent algorithm to obtain a loss value of each single task; in the outer learning, global optimization is carried out on the parameters of the face recognition model based on the loss values corresponding to the tasks, and then the gradient of the face recognition model is obtained.
When the local mechanism terminal finishes training the face sample data of one batch, the obtained face recognition model gradient can accurately contain the data characteristics of the local face sample data of one batch.
And S130, generating a face recognition gradient ciphertext by encrypting the face recognition model gradient.
After the local mechanism terminal finishes training the face sample data of one batch, the obtained face recognition model gradient is encrypted, and the encrypted face recognition model gradient is sent to the third party terminal, so that the original data of the user does not leave the local mechanism terminal, and the data privacy of the user can be effectively protected.
In this embodiment, a homomorphic encryption algorithm is used to encrypt the face recognition model gradient. Homomorphic encryption is a cryptographic technique based on the theory of computational complexity of mathematical problems. The homomorphic encrypted data is processed to produce an output, which is decrypted, the result being the same as the output obtained by processing the unencrypted original data in the same way. Homomorphic encryption can ensure that the encrypted ciphertext data continuously keeps the basic properties of plaintext data during calculation.
The homomorphic encryption algorithm includes but is not limited to one of the Gentry algorithm and the Parllier algorithm. In this embodiment, a Parllier algorithm is adopted to homomorphically encrypt the face recognition model gradient, and a face recognition gradient ciphertext is obtained.
And S140, sending the face recognition gradient ciphertext to a third-party terminal, and enabling the face recognition gradient ciphertext to obtain a federal model parameter ciphertext through decryption, knowledge federation and encryption.
The face recognition gradient ciphertext is sent to the third-party terminal, so that the original data of the user can not leave the local mechanism terminal, and the data privacy of the local mechanism terminal user can be effectively protected.
And the face recognition gradient ciphertext and the face recognition gradient ciphertexts of other mechanism terminals are decrypted to obtain the face recognition model gradients of the N mechanism terminals. The mechanism terminal may be a newly added mechanism terminal, or may be an existing mechanism terminal, which is not limited herein.
And the N face recognition model gradients obtained by decryption pass through the knowledge federation, and the gradients after the federation are obtained. The knowledge federation can directly calculate the average value or the median value of the gradients of the N face recognition models, or can intercept the gradients of each face recognition model, the intercepted weights are all in a preset weight interval, and the average value or the median value of the intercepted gradients of the face recognition models is calculated. In this embodiment, the average value of the gradients of the N face recognition models is directly calculated to obtain the federate gradient.
And multiplying the federate gradient and the preset learning rate to obtain a calculation result, and adding the calculation result to the current federate model parameter of the third-party terminal to obtain an updated federate model parameter. And the updated federal model parameters are encrypted to obtain a federal model parameter ciphertext and are fed back to the organization terminal, so that the data privacy safety is ensured. Because each mechanism terminal belongs to different industry fields, the face recognition model trained based on the federal model parameter ciphertext can be suitable for the service scenes of various mechanism terminals, and the universality is strong.
S150, receiving the federal model parameter ciphertext, and decrypting the federal model parameter ciphertext to obtain the federal model parameter.
And decrypting the received federal model parameter ciphertext to obtain the federal model parameter, wherein the federal model parameter is the same as the updated federal model parameter before encryption.
And S160, continuously training the face recognition model based on the federal model parameters, and recognizing the face according to the trained face recognition model.
And updating the model parameters of the face recognition model according to the federal model parameters. And (4) performing iterative training on the updated face recognition model according to face sample data of one batch of the local mechanism terminal, and repeating the steps from S120 to S140 until the iterative training times of the face recognition model are equal to a preset iterative time threshold value, so as to obtain the trained face recognition model. The model parameters of the trained face recognition model are obtained by the gradient of each face recognition model through the knowledge federation on the third-party terminal, and the model parameters accord with the characteristics of face data of a terminal user of a local organization, so that the local face data is recognized, and the trained face recognition model is strong in universality and convenient to apply.
Preferably, when the mechanism terminal or the mechanism terminal is newly added with face data, the face recognition model can be further trained by using the newly added face data on the basis of the face recognition model which is trained at present, and the face recognition model is sent to a third-party terminal so as to pass through the knowledge federation, so that a new face recognition model is obtained, and local face data are effectively recognized.
Example two
The second embodiment discloses a face recognition device based on the knowledge federation corresponding to the first embodiment, which is a virtual device structure of the first embodiment, and please refer to fig. 2, including:
a data obtaining module 210, configured to obtain a face data set, and initialize a face recognition model according to preset parameters;
a training encryption module 220, configured to train the face recognition model based on the face data set, so as to obtain a face recognition model gradient; the face recognition gradient is encrypted to generate a face recognition gradient ciphertext;
the federation module 230 is configured to send the face recognition gradient ciphertext to a third-party terminal, so that the face recognition gradient ciphertext is decrypted, subjected to knowledge federation, and encrypted to obtain a federation model parameter ciphertext;
a face recognition module 240, configured to receive the federal model parameter ciphertext; decrypting the ciphertext of the federal model parameter to obtain a federal model parameter; and continuously training the face recognition model based on the federal model parameters, and recognizing the face according to the trained face recognition model.
EXAMPLE III
Fig. 3 is a schematic structural diagram of an electronic device according to a third embodiment of the present invention, as shown in fig. 3, the electronic device includes a processor 310, a memory 320, an input device 330, and an output device 340; the number of the processors 310 in the computer device may be one or more, and one processor 310 is taken as an example in fig. 3; the processor 310, the memory 320, the input device 330 and the output device 340 in the electronic apparatus may be connected by a bus or other means, and the connection by the bus is exemplified in fig. 3.
The memory 320 may be used as a computer-readable storage medium for storing software programs, computer-executable programs, and modules, such as program instructions/modules corresponding to the knowledge-federation-based face recognition method in the embodiment of the present invention (for example, the data acquisition module 210, the training encryption module 220, the federation module 230, and the face recognition module 240 in the knowledge-federation-based face recognition apparatus). The processor 310 executes various functional applications and data processing of the electronic device by executing the software programs, instructions and modules stored in the memory 320, so as to implement the knowledge federation-based face recognition method according to the first embodiment.
The memory 320 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the terminal, and the like. Further, the memory 320 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some examples, the memory 320 may further include memory located remotely from the processor 310, which may be connected to the electronic device through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The input device 330 may be used to receive face data sets, federal model parameter ciphertexts, and the like. The output device 340 may include a display device such as a display screen.
Example four
An embodiment of the present invention further provides a storage medium containing computer-executable instructions, where the computer-executable instructions are executed by a computer processor to perform a knowledge federation-based face recognition method, and the method includes:
acquiring a face data set, and initializing a face recognition model according to preset parameters;
training the face recognition model based on the face data set to obtain a face recognition model gradient;
the face recognition gradient is encrypted to generate a face recognition gradient ciphertext;
sending the face recognition gradient ciphertext to a third-party terminal, and enabling the face recognition gradient ciphertext to obtain a federal model parameter ciphertext through decryption, knowledge federation and encryption;
receiving the federal model parameter ciphertext;
decrypting the ciphertext of the federal model parameter to obtain a federal model parameter;
and continuously training the face recognition model based on the federal model parameters, and recognizing the face according to the trained face recognition model.
Of course, the storage medium provided by the embodiment of the present invention contains computer-executable instructions, and the computer-executable instructions are not limited to the operations of the method described above, and may also perform related operations in the knowledge federation-based face recognition method provided by any embodiment of the present invention.
EXAMPLE five
The fifth embodiment of the present invention further provides a face recognition system based on the knowledge federation, including: at least two agency terminals and at least one third party terminal.
Wherein, the mechanism terminal is the electronic device provided by the third embodiment, such as a tablet computer; the third-party terminal is used for initializing the federal model; receiving the face recognition gradient ciphertext, decrypting the face recognition gradient ciphertext and carrying out knowledge federation to obtain a federation-behind gradient, updating a federation model parameter based on the federation-behind gradient, encrypting the updated federation model parameter to generate a federation model parameter ciphertext and sending the federation model parameter ciphertext to an organization terminal. The institution to which the institution terminal belongs is not limited to security institutions, financial institutions, public security institutions, and retail institutions, and the third-party terminal belongs to a neutral institution.
Each organization terminal acquires a respective face data set of a local organization, and deploys a face recognition model on each organization terminal and a federal model on a third party terminal at the same time. All face recognition models and federal models adopt the same common deep learning neural network, and are not limited to one of MLP, CNN and RNN. In order to ensure that the model can be converged stably in the federal training process, the face recognition model and the federal model are initialized based on the same preset parameters. The preset parameters include some common parameters in the model parameter initialization and training process, and in this embodiment, the preset parameters include an optimizer parameter, a training step size, a batch-size, an initialized learning rate, and a learning rate decay strategy.
Preferably, when each mechanism terminal finishes initializing the local face recognition model, a deployment success instruction is sent to the third party terminal; after receiving the deployment success instruction from each mechanism terminal, the third-party terminal sends a training starting instruction to each mechanism terminal, so that each mechanism terminal simultaneously starts training the face recognition model.
According to the preset batch-size parameter, each mechanism terminal selects the face sample data required by one training from the local face data set, inputs the face sample data of one batch into the face recognition model, and trains the face recognition model.
Each mechanism terminal trains a face recognition model through the same common model training method, for example, the face recognition model gradient can be calculated based on a plurality of tasks, for example, the chinese patent with application number CN201711290768.7 discloses a face recognition method, a device, a computer device and a readable storage medium; for example, chinese patent application No. CN201811550968.6 discloses a method and an apparatus for face recognition based on deep learning, and training of the face recognition model is not limited to the above-mentioned training method.
In this embodiment, the face recognition model is trained by inner-layer learning and outer-layer learning. In the inner-layer learning, aiming at a single task in a plurality of divided tasks, training a face recognition model based on a preset loss function and a gradient descent algorithm to obtain a loss value of each single task; in the outer learning, global optimization is carried out on the parameters of the face recognition model based on the loss values corresponding to the tasks, and then the gradient of the face recognition model is obtained. The face recognition model gradient of each mechanism terminal can accurately contain the data characteristics of local face data.
After each mechanism terminal finishes training the face sample data of one batch, the obtained face recognition model gradient is encrypted to generate a face recognition gradient ciphertext. And the homomorphic encryption algorithm is adopted for encryption, so that the basic property of plaintext data can be continuously maintained during calculation of the encrypted ciphertext data. The homomorphic encryption algorithm includes but is not limited to one of the Gentry algorithm and the Parllier algorithm. In this embodiment, a Parllier algorithm is adopted to homomorphically encrypt the face recognition model gradient of each mechanism terminal.
Referring to fig. 4, when N mechanism terminals are provided, the N mechanism terminals all send respective face recognition gradient ciphertexts to the third-party terminal, so that it is ensured that the original data of the user does not leave the local mechanism terminal, and the data privacy of the local user can be effectively protected. And the third-party terminal decrypts the received face recognition gradient ciphertexts to obtain face recognition model gradients of the N mechanism terminals, and further calculates the average value or the median value of the face recognition model gradients of the N mechanism terminals through the knowledge federation to obtain the gradient after the federation. In this embodiment, the average value of the gradients of the face recognition models of the N mechanism terminals is directly calculated. The current federal model parameters and the gradient after the federal are calculated through addition to obtain updated federal model parameters, and the updated federal model parameters are encrypted to obtain a federal model parameter ciphertext which is fed back to each organization terminal, so that the data privacy safety is ensured.
And each mechanism terminal decrypts the received federal model parameter ciphertext to obtain the federal model parameter. And updating the model parameters of the face recognition model on the local organization terminal according to the federal model parameters, and performing iterative training on the updated face recognition model according to the local face data set. And when the iterative training times of the face recognition model are equal to a preset iterative time threshold value, obtaining the trained face recognition model, wherein the face recognition model accords with the data characteristics of local face data. And each mechanism terminal identifies local face data according to the trained face identification model.
The system ensures that the original data of the user does not leave the local terminal through encryption, can effectively protect the data privacy, is not limited by the scale of the mechanism terminal participating in the federal training, can flexibly select to join or withdraw from the federal training of the face recognition model, is strong in universality of the face recognition model obtained through the federal training, and is suitable for the service scene of face recognition of the mechanism terminals in different fields.
EXAMPLE six
The sixth embodiment is an application description of the fifth embodiment, and the system includes a security institution terminal a, a financial institution terminal B, a public security institution terminal C, and a third party terminal D. The security institution terminal a, the financial institution terminal B and the public security institution terminal C respectively have 10000 face data sets of face data.
The face recognition model deployed on the mechanism terminal A, the mechanism terminal B and the mechanism terminal C and the federal model of the third-party terminal D all adopt resnet50 as a basic network. The preset parameters include a loss function of arcfacce, an optimizer of SGD, an initial learning rate set to 0.01, a training epoch of 100, and a learning rate decay pattern of every 25 epochs divided by 10. And initializing the face recognition model and the federal model on each mechanism terminal according to the preset parameters.
Preferably, after the initialization of each mechanism terminal is completed, a deployment success instruction is sent to the third-party terminal, the third-party terminal sends a training start instruction to each mechanism terminal, and each mechanism terminal simultaneously starts training the face recognition model.
And the mechanism terminal A, the mechanism terminal B and the mechanism terminal C calculate respective face recognition model gradients, and send the face recognition model gradients to the third party terminal D after encryption. The third-party terminal D decrypts the received face recognition gradient ciphertexts of the three organization terminals, the face recognition model gradient obtained through decryption passes through the knowledge federation to obtain a federation-behind gradient, the federation model parameter of the third-party terminal D is updated based on the federation-behind gradient, and the updated federation model parameter is encrypted and then sent to each organization terminal. And the mechanism terminal A, the mechanism terminal B and the mechanism terminal C decrypt the received Federal model parameter ciphertext, update the local model according to the decrypted Federal model parameter, and continue training the face recognition model according to the same flow until the iterative training times of the model are equal to a preset iterative time threshold value, so that the model training is completed.
And the security mechanism terminal A, the financial mechanism terminal B and the public security mechanism terminal C respectively identify local face data according to the trained face identification model.
From the above description of the embodiments, it is obvious for those skilled in the art that the present invention can be implemented by software and necessary general hardware, and certainly, can also be implemented by hardware, but the former is a better embodiment in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which may be stored in a computer-readable storage medium, such as a floppy disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a FLASH Memory (FLASH), a hard disk or an optical disk of a computer, and includes instructions for enabling an electronic device (which may be a mobile phone, a personal computer, a server, or a network device) to execute the methods according to the embodiments of the present invention.
It should be noted that, in the embodiment of the face recognition device based on the knowledge federation, each included unit and each module are only divided according to functional logic, but are not limited to the above division, as long as the corresponding function can be realized; in addition, specific names of the functional units are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present invention.
Various other modifications and changes may be made by those skilled in the art based on the above-described technical solutions and concepts, and all such modifications and changes should fall within the scope of the claims of the present invention.
Claims (10)
1. A face recognition method based on knowledge federation is characterized in that: the method comprises the following steps:
acquiring a face data set, and initializing a face recognition model according to preset parameters;
training the face recognition model based on the face data set to obtain a face recognition model gradient;
the face recognition gradient is encrypted to generate a face recognition gradient ciphertext;
sending the face recognition gradient ciphertext to a third-party terminal, and enabling the face recognition gradient ciphertext to obtain a federal model parameter ciphertext through decryption, knowledge federation and encryption;
receiving the federal model parameter ciphertext;
decrypting the ciphertext of the federal model parameter to obtain a federal model parameter;
and continuously training the face recognition model based on the federal model parameters, and recognizing the face according to the trained face recognition model.
2. The knowledge federation-based face recognition method of claim 1, wherein: and carrying out homomorphic encryption on the gradient of the face recognition model by adopting a Parllier algorithm.
3. A knowledge federation-based face recognition method as claimed in claim 1 or 2, wherein: continuing to train the face recognition model based on the federal model parameters, including:
updating the face recognition model parameters of the face recognition model based on the federal model parameters;
iteratively training the face recognition model according to the face data set and the updated face recognition model parameters;
and when the iterative training times of the face recognition model are equal to a preset iterative time threshold value, obtaining the trained face recognition model.
4. A face recognition device based on knowledge federation is characterized in that: it includes:
the data acquisition module is used for acquiring a face data set and initializing a face recognition model according to preset parameters;
the training encryption module is used for training the face recognition model based on the face data set to obtain a face recognition model gradient; the face recognition gradient is encrypted to generate a face recognition gradient ciphertext;
the federation module is used for sending the face recognition gradient ciphertext to a third-party terminal so that the face recognition gradient ciphertext is decrypted, subjected to knowledge federation and encrypted to obtain a federation model parameter ciphertext;
the face recognition module is used for receiving the federal model parameter ciphertext; decrypting the ciphertext of the federal model parameter to obtain a federal model parameter; and continuously training the face recognition model based on the federal model parameters, and recognizing the face according to the trained face recognition model.
5. An electronic device comprising a processor, a storage medium, and a computer program stored in the storage medium, wherein the computer program when executed by the processor performs the federal knowledge face recognition method in any of claims 1 to 3.
6. A computer storage medium having a computer program stored thereon, characterized in that: the computer program when executed by a processor implements the knowledge-federation-based face recognition method of any one of claims 1 to 3.
7. A knowledge federation-based face recognition system, comprising: at least two agency terminals and at least one third party terminal;
wherein the organization terminal is the electronic device of claim 5; the third-party terminal is used for initializing a federal model; receiving a face recognition gradient ciphertext, decrypting the face recognition gradient ciphertext and carrying out knowledge federation to obtain a federation-behind gradient, updating a federation model parameter based on the federation-behind gradient, and encrypting the updated federation model parameter to generate a federation model parameter ciphertext and sending the federation model parameter ciphertext to the institution terminal.
8. A knowledge federation-based face recognition system as claimed in claim 7 wherein: the face recognition model on the mechanism terminal and the federal model adopt the same model, and the face recognition model and the federal model are initialized based on the same preset parameters.
9. A knowledge federation-based face recognition system as claimed in claim 8 wherein: when the mechanism terminal finishes initializing the face recognition model, a deployment success instruction is sent to the third party terminal; and the third-party terminal sends a training starting instruction to each mechanism terminal, so that each mechanism terminal simultaneously starts to train the face recognition model.
10. A knowledge federation-based face recognition system as claimed in any one of claims 7 to 9 wherein: when the number of the mechanism terminals is N, the third-party terminal decrypts the face recognition gradient ciphertext and carries out knowledge federation to obtain a federation-behind gradient, and the federation model parameters are updated based on the federation-behind gradient, wherein the process comprises the following steps:
the face recognition gradient ciphertext is decrypted to obtain face recognition model gradients of the N mechanism terminals;
calculating the average value or the median value of the gradients of the face recognition models of the N mechanism terminals to obtain the gradient after the federation;
and the current federal model parameter and the federal gradient are calculated by addition to obtain an updated federal model parameter.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010174510.6A CN111046857A (en) | 2020-03-13 | 2020-03-13 | Face recognition method, device, equipment, medium and system based on knowledge federation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010174510.6A CN111046857A (en) | 2020-03-13 | 2020-03-13 | Face recognition method, device, equipment, medium and system based on knowledge federation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111046857A true CN111046857A (en) | 2020-04-21 |
Family
ID=70231069
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010174510.6A Pending CN111046857A (en) | 2020-03-13 | 2020-03-13 | Face recognition method, device, equipment, medium and system based on knowledge federation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111046857A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112348192A (en) * | 2020-09-18 | 2021-02-09 | 同盾控股有限公司 | Knowledge reasoning method, system, device and medium based on knowledge federation |
| CN112668037A (en) * | 2020-06-02 | 2021-04-16 | 华控清交信息科技(北京)有限公司 | Model training method and device and electronic equipment |
| CN112906139A (en) * | 2021-04-08 | 2021-06-04 | 平安科技（深圳）有限公司 | Vehicle fault risk assessment method and device, electronic equipment and storage medium |
| CN113887741A (en) * | 2021-11-05 | 2022-01-04 | 深圳市电子商务安全证书管理有限公司 | Data generation method, device, equipment and storage medium based on federal learning |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109165515A (en) * | 2018-08-10 | 2019-01-08 | 深圳前海微众银行股份有限公司 | Model parameter acquisition methods, system and readable storage medium storing program for executing based on federation's study |
| CN109167695A (en) * | 2018-10-26 | 2019-01-08 | 深圳前海微众银行股份有限公司 | Alliance Network construction method, equipment and readable storage medium storing program for executing based on federation's study |
| CN109189825A (en) * | 2018-08-10 | 2019-01-11 | 深圳前海微众银行股份有限公司 | Lateral data cutting federation learning model building method, server and medium |
| CN109684855A (en) * | 2018-12-17 | 2019-04-26 | 电子科技大学 | A kind of combined depth learning training method based on secret protection technology |
| US10484530B2 (en) * | 2017-11-07 | 2019-11-19 | Google Llc | Sensor based component activation |
| CN110719158A (en) * | 2019-09-11 | 2020-01-21 | 南京航空航天大学 | Edge calculation privacy protection system and method based on joint learning |
| US10554931B1 (en) * | 2018-10-01 | 2020-02-04 | At&T Intellectual Property I, L.P. | Method and apparatus for contextual inclusion of objects in a conference |
-
2020
- 2020-03-13 CN CN202010174510.6A patent/CN111046857A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10484530B2 (en) * | 2017-11-07 | 2019-11-19 | Google Llc | Sensor based component activation |
| CN109165515A (en) * | 2018-08-10 | 2019-01-08 | 深圳前海微众银行股份有限公司 | Model parameter acquisition methods, system and readable storage medium storing program for executing based on federation's study |
| CN109189825A (en) * | 2018-08-10 | 2019-01-11 | 深圳前海微众银行股份有限公司 | Lateral data cutting federation learning model building method, server and medium |
| US10554931B1 (en) * | 2018-10-01 | 2020-02-04 | At&T Intellectual Property I, L.P. | Method and apparatus for contextual inclusion of objects in a conference |
| CN109167695A (en) * | 2018-10-26 | 2019-01-08 | 深圳前海微众银行股份有限公司 | Alliance Network construction method, equipment and readable storage medium storing program for executing based on federation's study |
| CN109684855A (en) * | 2018-12-17 | 2019-04-26 | 电子科技大学 | A kind of combined depth learning training method based on secret protection technology |
| CN110719158A (en) * | 2019-09-11 | 2020-01-21 | 南京航空航天大学 | Edge calculation privacy protection system and method based on joint learning |
Non-Patent Citations (3)
| Title |
|---|
| HONGYU LI 等: "An End-to-End Encrypted Neural Network for Gradient Updates Transmission in Federated Learning", 《2019 DATA COMPRESSION CONFERENCE (DCC)》 * |
| QIANG YANG 等: "Federated Machine Learning: Concept and Applications", 《ACM TRANS. INTELL. SYST. TECHNOL》 * |
| TIAN LI 等: "Federated Learning: Challenges, Methods, and Future Directions", 《AFFILIATIONS. CSD CMU》 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112668037A (en) * | 2020-06-02 | 2021-04-16 | 华控清交信息科技(北京)有限公司 | Model training method and device and electronic equipment |
| CN112668037B (en) * | 2020-06-02 | 2024-04-05 | 华控清交信息科技(北京)有限公司 | Model training method and device and electronic equipment |
| CN112348192A (en) * | 2020-09-18 | 2021-02-09 | 同盾控股有限公司 | Knowledge reasoning method, system, device and medium based on knowledge federation |
| CN112906139A (en) * | 2021-04-08 | 2021-06-04 | 平安科技（深圳）有限公司 | Vehicle fault risk assessment method and device, electronic equipment and storage medium |
| CN113887741A (en) * | 2021-11-05 | 2022-01-04 | 深圳市电子商务安全证书管理有限公司 | Data generation method, device, equipment and storage medium based on federal learning |
| CN113887741B (en) * | 2021-11-05 | 2022-09-30 | 深圳市电子商务安全证书管理有限公司 | Data generation method, device, equipment and storage medium based on federal learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111046857A (en) | Face recognition method, device, equipment, medium and system based on knowledge federation | |
| US11256900B1 (en) | Facial recognition methods and apparatuses | |
| CN113239404B (en) | Federal learning method based on differential privacy and chaotic encryption | |
| CN111723404B (en) | Method and device for jointly training business model | |
| EP3220573B1 (en) | Method and system for controlling encryption of information and analyzing information as well as terminal | |
| CN111680676B (en) | Training face recognition model, image registration and face recognition method and device | |
| CN111753318A (en) | Multi-party security calculation method, device and system for private data | |
| CN111242611A (en) | Method and system for recovering digital wallet key | |
| CN107154916A (en) | A kind of authentication information acquisition methods, offer method and device | |
| CN111191267A (en) | Model data processing method, device and equipment | |
| CN112381000A (en) | Face recognition method, device, equipment and storage medium based on federal learning | |
| CN112818369A (en) | Combined modeling method and device | |
| CN113472805B (en) | Model training method and device, storage medium and electronic equipment | |
| CN113436008A (en) | Loan purpose monitoring method and device, storage medium and electronic equipment | |
| CN105279443A (en) | Picture encrypting and decrypting methods and devices | |
| CN112948883A (en) | Multi-party combined modeling method, device and system for protecting private data | |
| US11133926B2 (en) | Attribute-based key management system | |
| CN112380404A (en) | Data filtering method, device and system | |
| CN116865938A (en) | Multi-server federation learning method based on secret sharing and homomorphic encryption | |
| CN110853042A (en) | Automatic secret-involved vector graph segmentation system and method based on image recognition | |
| CN113761570B (en) | Data interaction method for privacy intersection | |
| TWI611316B (en) | Text processing method for safe input method, text processing device and text processing system | |
| CN114547684A (en) | Method and device for protecting multi-party joint training tree model of private data | |
| CN109409111A (en) | It is a kind of to search for method generally towards encrypted image | |
| CN114866312A (en) | Common data determination method and device for protecting data privacy |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200421 |
|
| RJ01 | Rejection of invention patent application after publication |