WO2023040335A1 - 一种人脸识别方法、设备及系统 - Google Patents

一种人脸识别方法、设备及系统 Download PDF

Info

Publication number
WO2023040335A1
WO2023040335A1 PCT/CN2022/095037 CN2022095037W WO2023040335A1 WO 2023040335 A1 WO2023040335 A1 WO 2023040335A1 CN 2022095037 W CN2022095037 W CN 2022095037W WO 2023040335 A1 WO2023040335 A1 WO 2023040335A1
Authority
WO
WIPO (PCT)
Prior art keywords
information
face
ciphertext
encrypted
feature information
Prior art date
Application number
PCT/CN2022/095037
Other languages
English (en)
French (fr)
Inventor
丁杰
张瑞
Original Assignee
超聚变数字技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 超聚变数字技术有限公司 filed Critical 超聚变数字技术有限公司
Publication of WO2023040335A1 publication Critical patent/WO2023040335A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60—Protecting data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60—Protecting data
    • G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions

Definitions

  • the present application relates to the field of electronic technology, and in particular to a face recognition method, device and system.
  • face information belongs to the user's unique biological information, compared with fingerprint recognition technology, it can make users feel indifferent and the authentication process is more friendly. Therefore, authentication systems based on face information are becoming more and more popular in practical applications. For example, login authentication based on face recognition, payment based on face recognition, access control based on face recognition, etc.
  • face information is sensitive information of biological individuals, once leaked, the authentication system may be attacked, and user privacy will be leaked at the same time. At present, face information has been collected and used at will, and the need for privacy protection of face information is becoming more and more urgent.
  • face information is extracted, stored and used in plain text, which is obviously very insecure.
  • the face feature vector extracted by the face feature extraction module is stored for calculation of face recognition and face authentication.
  • the current solution adopts homomorphic encryption technology, and homomorphic encryption technology can make data available and invisible, that is, after data is encrypted, it can still be processed in the same way as plaintext, and the processing result is the same as that of plaintext. Only the processing result remains in the ciphertext state, and only the user who has the private key decrypts it through the private key to get the plaintext result. It is true that no information will be leaked before, during or after the processing.
  • the disadvantage of homomorphic encryption is performance issues, especially operations such as ciphertext multiplication and ciphertext shifting. Compared with plaintext operations, there is a performance gap of 4 orders of magnitude. Therefore, there are no practical application scenarios at present, especially for real-time For scenes with higher requirements, performance still needs to be improved for practical use.
  • the embodiment of the present application provides a face recognition method, which encrypts face feature information by adopting a fully homomorphic encryption (FHE) method, so that ciphertext is used throughout the calculation on the server side, ensuring data security. safety.
  • FHE fully homomorphic encryption
  • the dimension information of the feature is combined on the server side, which greatly reduces the calculation amount of the server side and significantly improves the calculation performance.
  • a face recognition method which is applied to a terminal device, and the method includes: acquiring login information input by a user, where the login information includes user identification information; collecting the first face image of the user; The first face feature information is obtained from the face image; the first face feature information is encrypted by using a fully homomorphic encryption algorithm through the pre-stored public key; the encrypted first face feature information and the pre-stored evaluation key, The user identity information is sent to the server, wherein the evaluation key and public key are determined by the terminal device according to the user identity information; the identification and authentication information sent by the server is received, wherein the identification and authentication information is ciphertext; according to the pre-stored private key
  • the fully homomorphic encryption algorithm is used to decrypt the identification and authentication information to obtain the user's face recognition authentication result.
  • This application uses a fully homomorphic encryption method to encrypt face feature information, so that ciphertext is used throughout the calculation on the server side, ensuring data security. At the same time, combining the dimensions of features on the server side greatly reduces the amount of calculation and significantly improves the calculation performance.
  • the method before collecting the face image of the user, the method further includes: collecting a second face image of the user, where the second face image is used for registering face information; obtaining the second face image according to the second face image
  • the method before collecting the user's face image, the method further includes: determining a public key, an evaluation key, and a private key according to user identification information, security strength information, performance information, and scene information, wherein, The public key and the private key are used as a pair of asymmetric encryption keys, and the security strength information, performance information and scene information are pre-configured or obtained by receiving user input.
  • the fully homomorphic encryption algorithm is used to decrypt the identification and authentication information according to the pre-stored private key to obtain the user's face recognition authentication result, including: using the fully homomorphic encryption algorithm to decrypt the identification information according to the private key
  • the authentication information is decrypted to determine the face similarity information; when the face similarity information is greater than or equal to the preset similarity threshold, it is determined that the user's face recognition authentication has passed; when the face similarity information is less than the similarity threshold, then Determine that the user's face recognition authentication has failed.
  • a face recognition method is provided, the method is applied to a server, the server includes a face feature database, and the face feature database includes at least one registered face feature information, the method includes: receiving the encrypted first sent by the terminal device A face feature information and user identity information, wherein the encrypted first face feature information is encrypted using a fully homomorphic encryption algorithm; determine the corresponding registrant from the face feature database according to the user identity information Facial features: use the encrypted first facial feature information and the registered facial feature information corresponding to the user identity information to perform similarity calculations to determine the identification authentication information, which is used to represent the encrypted first facial feature information The similarity between the registered face feature information corresponding to the user identity information; sending the identification authentication information to the terminal device.
  • the similarity calculation is performed in combination with the feature dimension information, which greatly reduces the calculation amount and significantly improves the calculation performance.
  • the method further includes: receiving the encrypted second face feature information, user identity information, feature dimension and evaluation key sent by the terminal device; and receiving the encrypted second face feature information, User identity information, feature dimensions and evaluation keys are stored in the face feature database as registered face feature information.
  • the registered face feature information includes encrypted second face feature information, user identity information, feature dimension and evaluation key; the encrypted first face feature information and user identity
  • the similarity calculation is performed on the registered face feature information corresponding to the identification information to determine the identification and authentication information, including: multiplying the encrypted first face feature information and the encrypted second face feature information by using the evaluation key to perform ciphertext multiplication ; Use the evaluation key to shift the ciphertext after multiplying the ciphertext.
  • the number of ciphertext shifts is 2 ⁇ i bits, where i is the current cycle number, i is a natural number, and i is determined according to the feature dimension ; Add the information after the ciphertext shift and the information before the ciphertext shift to obtain the ciphertext of the shifted addition result; perform the ciphertext shift and ciphertext addition again on the ciphertext of the shifted and added result , until the loop is executed i times, the ciphertext of the shifted and added result obtained for the ith time is used as the identification authentication information.
  • the registered facial feature information includes encrypted second facial feature information; the method further includes: extending the encrypted second facial feature information, and determining to expand the second facial feature information; Using the encrypted first face feature information and the registered face feature information corresponding to the user identity information to perform similarity calculation, including: using the encrypted first face feature information and the extended second face feature information to perform similarity calculation calculate.
  • a terminal device for face recognition includes: a collection module, configured to obtain login information input by a user, where the login information includes user identity information; the collection module is also used to collect the user's The first human face image; the human face detection and feature extraction module is used to obtain the first human face feature information according to the first human face image; the encryption and decryption module is used to use the fully homomorphic encryption algorithm for the public key stored in advance.
  • the first face feature information is encrypted; the sending module is used to send the encrypted first face feature information and the pre-stored evaluation key and user identity information to the server, wherein the evaluation key and the public key are encrypted
  • the key generation module is determined according to the user identity information; the receiving module is used to receive the identification and authentication information sent by the server, wherein the identification and authentication information is ciphertext; the encryption and decryption module is also used to use the identical key according to the pre-stored private key
  • the state encryption algorithm decrypts the identification and authentication information to obtain the user's face recognition authentication result.
  • This application uses a fully homomorphic encryption method to encrypt face feature information, so that ciphertext is used throughout the calculation on the server side, ensuring data security. At the same time, combining the dimensions of features on the server side greatly reduces the amount of calculation and significantly improves the calculation performance.
  • the collection module is also used to collect a second face image of the user, and the second face image is used to register face information; the face detection and feature extraction module is also used to The face image obtains the second face feature information; the face detection and feature extraction module is also used to determine the feature dimension according to the second face feature information; the encryption and decryption module is also used to use a fully homomorphic encryption algorithm according to the public key.
  • the second facial feature information is encrypted; the sending module is also used to send the encrypted second facial feature information, user identification information, feature dimension and evaluation key to the server.
  • the encryption and decryption module is also used to: determine the public key, evaluation key and private key according to user identity information, security strength information, performance information and scene information, wherein the public key and private key As a pair of two keys for asymmetric encryption, the security strength information, performance information and scene information are pre-configured or obtained by the acquisition module receiving user input.
  • the encryption and decryption module is also used to decrypt the identification and authentication information by using a fully homomorphic encryption algorithm according to the private key to determine the face similarity information; the terminal device also includes: a determination module for when If the face similarity information is greater than or equal to the preset similarity threshold, it is determined that the user's face recognition authentication has passed; and when the face similarity information is smaller than the similarity threshold, it is determined that the user's face recognition authentication has failed.
  • a server for face recognition includes a face feature database, and the face feature database includes at least one registered face feature information, and the server includes: a receiving module for receiving encrypted information sent by a terminal device The first face feature information and user identity information after encryption, wherein the encrypted first face feature information is obtained by encrypting using a fully homomorphic encryption algorithm; the face similarity verification module is used to The information determines the corresponding registered face features from the face feature database; the face similarity verification module is also used to perform similarity by using the encrypted first face feature information and the registered face feature information corresponding to the user identity information.
  • the identification and authentication information is used to represent the similarity between the encrypted first facial feature information and the registered facial feature information corresponding to the user identity information; the sending module is used to send the identification and authentication information to the terminal device.
  • the similarity calculation is performed in combination with the feature dimension information, which greatly reduces the calculation amount and significantly improves the calculation performance.
  • the receiving module is also used to: receive the encrypted second face feature information, user identity information, feature dimension and evaluation key sent by the terminal device; Information, user identification information, feature dimensions and evaluation keys are stored in the face feature database as registered face feature information.
  • the registered face feature information includes encrypted second face feature information, user identity information, feature dimensions and evaluation keys; the face similarity verification module is also used to:
  • the first face feature information and the encrypted second face feature information use the evaluation key to perform ciphertext multiplication;
  • the information after ciphertext multiplication uses the evaluation key to carry out ciphertext shift, and the ciphertext shift
  • the number of digits is 2 ⁇ i bits, where i is the current number of cycles, i is a natural number, and i is determined according to the feature dimension;
  • the information after the ciphertext shift is added to the information before the ciphertext shift to obtain the shift
  • the ciphertext shift and ciphertext addition are performed on the ciphertext of the shift addition result again until the loop is executed i times, and the ciphertext of the shift addition result obtained for the ith time is used as the identification authentication information .
  • the registered face feature information includes encrypted second face feature information; the face similarity verification module is also used to: extend the encrypted second face feature information, and determine the extended The facial feature information of the two persons; the similarity calculation is performed by using the encrypted first facial feature information and the extended second facial feature information.
  • a system for face recognition includes: a terminal device and a server.
  • the terminal device and the server are connected in a wired or wireless manner, so that the system executes any one method in the first aspect and the second aspect above.
  • a terminal device for face recognition includes: a processor for coupling with a memory, and reading and executing instructions stored in the memory; when the processor is running, the instructions are executed, so that the processing A device is used to perform the method of any one of the first aspect.
  • a server for face recognition includes: a processor for coupling with a memory, and reading and executing instructions stored in the memory; when the processor is running, the instructions are executed, so that the processor is used for A method of performing any item of the second aspect.
  • a computer-readable storage medium is provided. Instructions are stored in the computer-readable storage medium. When the instructions are run on a terminal, the terminal is made to execute the method according to any one of the first aspect.
  • a computer-readable storage medium is provided. Instructions are stored in the computer-readable storage medium. When the instructions are run on a server, the server is made to execute the method according to any one of the second aspect.
  • a computer device containing instructions, which, when running on a terminal, causes the terminal to execute any one of the methods in the first aspect.
  • a computer device containing instructions, which, when running on a server, causes the server to perform any one of the methods in the second aspect.
  • a computer program product containing instructions, which, when run on a computer, cause the computer to execute any one of the methods in the first aspect.
  • a computer program product containing instructions, which, when run on a computer, cause the computer to execute any one of the methods in the second aspect.
  • the present application discloses a face recognition method, which encrypts face feature information by using a fully homomorphic encryption method, so that ciphertext is used throughout the calculation at the server end, thereby ensuring data security.
  • the dimension information of the feature is combined on the server side, which greatly reduces the calculation amount of the server side and significantly improves the calculation performance.
  • FIG. 1 is a schematic diagram of an application scenario provided by an embodiment of the present application
  • FIG. 2 is a schematic diagram of a face recognition process
  • Fig. 3 is a schematic diagram of a hierarchical fully homomorphic encryption process
  • Fig. 4 is a schematic diagram of a ciphertext calculation process after fully homomorphic encryption
  • FIG. 5 is a schematic diagram of a face recognition system framework provided by an embodiment of the present application.
  • FIG. 6 is a schematic diagram of a face recognition system device at the registration stage provided by an embodiment of the present application.
  • FIG. 7 is a schematic diagram of a face recognition system device in the verification stage provided by the embodiment of the present application.
  • FIG. 8 is a flow chart of a face recognition method provided by an embodiment of the present application.
  • FIG. 9 is a schematic diagram of a ciphertext inner product calculation provided in an embodiment of the present application.
  • FIG. 10 is a schematic diagram of another ciphertext inner product calculation provided in the embodiment of the present application.
  • FIG. 11 is a schematic diagram of a terminal device provided in an embodiment of the present application.
  • FIG. 12 is a schematic diagram of a server provided by an embodiment of the present application.
  • This application is mainly applied to face recognition scenarios, such as shown in FIG. 1 , where a user performs face recognition or face authentication through a terminal device 100 .
  • the terminal device 100 includes a camera, and the terminal device collects a user's face image through the camera, so as to perform face recognition or face authentication.
  • Face recognition A face recognition system extracts features from input face images and compares them with images in a face database. If the similarity between it and the closest photo in the library is greater than a certain threshold, we judge that this photo is a photo of the person in the face library, otherwise we consider it an unknown face. It is usually used for face-based attendance check-in, personnel search, etc.
  • Face verification is a technology to judge whether two face pictures are the same person. This is a two-category problem, which is usually used for face-based login control, access control, identity confirmation, etc. For example, face-based mobile phone unlocking and operating system login.
  • Some of the current schemes can be shown in Figure 2, through face detection, face alignment and face normalization, feature extraction, and finally obtain the feature value represented by the n-dimensional vector.
  • the authentication scenario by calculating the similarity between the current feature value and the feature value at the time of registration, compare the obtained similarity with the optimal threshold after training using the data set, if it is less than the threshold, it can be considered as the same person, and the authentication is passed; otherwise Fail.
  • the recognition scene by calculating the similarity of the current face feature value and different registration feature values one by one, and taking the result with the lowest similarity, it can identify which registered person the current face is.
  • the specific face similarity calculation algorithms include Euclidean distance and cosine distance. For the normalized eigenvalues, the calculation results of the two algorithms are the same. Here we introduce the cosine similarity calculation process.
  • i' is any integer between 1 and n'
  • d represents the cosine distance
  • n' is a positive integer
  • the cosine of the angle between two vectors is the cosine similarity between the two vectors.
  • the cosine similarity between two vectors can be obtained by introducing the vector computation procedure into the equation:
  • the numerical range of the cosine similarity is the range of the cosine value, ie (-1,1). The higher the value, the greater the similarity. This confirms the numerical significance of cosine similarity. It can be seen from the calculation formula that when using the normalized vector features for similarity calculation, the essence is to calculate the inner product of two vectors.
  • the face feature vector is not the original image of the face, only the face feature vector information is stored, but existing studies have shown that the face can be restored based on the face feature vector information, thus causing the leakage of face privacy information . Even if it is encrypted and saved with a conventional encryption algorithm, it still needs to be decrypted for calculation during use. Therefore, the current security technology cannot perfectly solve the privacy protection problem of face storage and use.
  • LFHE leveled full homomorphic encryption
  • CKKS cheon-kim-kim-song, CKKS
  • BFV hierarchical fully homomorphic BFV algorithm
  • the encryption parameters are determined in advance according to the complexity of the calculation business, so as to avoid the time-consuming bootstrap scheme.
  • operations such as ciphertext state multiplication and ciphertext state shifting still have at least 4 orders of magnitude gap with plaintext calculations.
  • the computing mode such as avoiding or reducing similar time-consuming operations to improve performance, makes the real-time performance of the business based on homomorphic operations unaffected compared with plaintext, which is of great significance to whether homomorphic encryption can be used in actual business.
  • FIG. 3 A typical hierarchical fully homomorphic encryption process can be shown in Figure 3, in which, through canonical embedding inverse mapping, the component elements of n/2 complex number fields can be mapped to a circular polynomial
  • the number of component elements is also called slot_count, that is, how many plaintext component elements can be packed by a ciphertext circular polynomial.
  • slot_count that is, how many plaintext component elements can be packed by a ciphertext circular polynomial.
  • the addition and multiplication of circular polynomials are equivalent to the addition and multiplication of each component element.
  • ⁇ m is an m-th primitive unit root
  • slot1 (value 35) is exactly the inner product of ciphertext feature vectors ciphertext1 and ciphertext2, and the value of slot1 can be obtained after decryption to obtain the inner product value of plaintext feature vectors.
  • the number of cycles that need to be shifted and added is log2(slot_count)-1 times, and in the i-th cycle, the shift step is 2 i steps.
  • the pseudocode of the algorithm is as follows:
  • ct1 represents ciphertext 1
  • ct2 represents ciphertext 2
  • Multiply represents the multiplication of ciphertext 1 and ciphertext 2, including the necessary relinearization steps in homomorphic operations
  • Rotate(ct3,2 i ) represents the left Shift 2 i bits.
  • Ciphertext shifting is a time-consuming operation in homomorphic operations. Compared with the plaintext, there is a gap of 4 orders of magnitude. When the slot_count is large, it becomes the bottleneck of the entire operation. In the practical application of the hierarchical fully homomorphic algorithm, in order to ensure that the security strength reaches the minimum level of 128 bits, the highest order of the BFV or CKKS ciphertext polynomial is generally 4096 or above.
  • the present application provides a face recognition method, which encrypts face feature information by using a fully homomorphic encryption method, so that ciphertext is used throughout the calculation at the server side, ensuring data security.
  • the dimension information of the feature is combined on the server side, which greatly reduces the calculation amount of the server side and significantly improves the calculation performance.
  • This application can solve the two scenarios of face verification and recognition.
  • the face features saved in plain text can be restored by the attack program, thereby leaking private information, and using the restored face to attack the authentication system.
  • the application can also solve the performance bottleneck problem in the process of face verification and recognition ciphertext calculation.
  • this application can use homomorphic encryption algorithms (CKKS, BFV) to implement the scheme in single-mode face verification, multi-mode face verification, and face recognition scenarios, aiming at the existing plaintext face feature similarity calculation technology Transformation, encrypt the face feature vector and save the ciphertext vector, and the whole process of similarity calculation and the calculation result will not disclose any information, only the user who has the private key can get the calculation result.
  • CKKS, BFV homomorphic encryption algorithms
  • the present application can also modify the existing ciphertext face feature similarity calculation method in the ciphertext similarity calculation process of the single-mode face verification scene, by extracting the face feature vector dimension information and introducing the calculation process, reducing Small calculation rounds are required without affecting the accuracy of calculation results and improving performance.
  • the application can also extract and encrypt the current face information to be verified by selecting appropriate homomorphic encryption parameters, and repeatedly expand and package the same face information into a ciphertext,
  • the original registration information of different faces of the same person is packaged into a ciphertext, and the face feature vector dimension information is extracted and introduced into the calculation process to achieve parallel effects, improve the success rate of verification and improve the efficiency of ciphertext verification.
  • this application can also extract and encrypt the current face information to be verified by selecting appropriate homomorphic encryption parameters, and repeatedly expand and package the same face feature information into a ciphertext, such as At the same time, the original face registration information of different individuals is packaged into a ciphertext, and the face feature vector dimension information is extracted at the same time and introduced into the calculation process to achieve a parallel effect and improve the throughput of face ciphertext recognition.
  • FIG. 5 is a schematic diagram of a face recognition system provided by an embodiment of the present application.
  • the face recognition system is composed of a terminal device 501 and a server 502 controlled by the user himself.
  • the server may also be referred to as cloud, server end, service end, etc.
  • the terminal device may also be referred to as terminal.
  • the terminal collects face information and performs authentication on the cloud. Because the terminal is controlled by the user himself, it is a trusted domain, while the server (such as the cloud) is an untrusted domain for the user, that is, the user does not trust the cloud side that he does not control.
  • the system can only store encrypted face data information in the cloud, and also maintain the ciphertext state during the face comparison calculation process, including the ciphertext results .
  • the whole system is divided into two processes: the face registration process and the face verification (recognition) process.
  • the terminal During face registration, the terminal generates the private key and public key used by the homomorphic operation through the key generation module according to the specific parameters of the full homomorphism and the security strength to be achieved, and saves them in the medium that meets the requirements on the terminal side, and collects them locally.
  • the face image is extracted by the face detection and feature extraction module, and the face feature vector information is extracted, and then encrypted with the public key and transmitted to the cloud side, and stored in the registration template database through the face registration and verification module, and the public key is also Transfer to cloud storage.
  • the face registration and verification module uses the ciphertext face information registered in the registration stage to perform similarity calculation with the ciphertext face information uploaded now, and the calculation result is returned to the terminal in ciphertext, and the terminal decryption module uses the previously stored private key to decrypt, Obtain the verification (identification) plaintext result.
  • FIG. 6 is a schematic diagram of a face recognition system device at the registration stage provided by an embodiment of the present application.
  • the terminal device 501 may include a face detection and feature extraction module 601 , an encryption and decryption module 602 and a key generation module 603 .
  • the encryption and decryption module 602 may correspond to the encryption module and the decryption module in FIG. 5 .
  • the terminal device 501 may also include a collection module (not shown in the figure), a sending module (not shown in the figure) and a receiving module (not shown in the figure).
  • the server 502 may include a face information registration module 604 and a face database 605 .
  • the key generation module 603 When registering, the user enters the ID, and according to the security strength, performance requirements and application scenarios, the key generation module 603 generates the homomorphic public key (public key, PK), private key (secret key, SK) and evaluation key used for encryption. Key (evaluation key, EK).
  • the public key PK and private key SK are stored locally.
  • the encryption and decryption module 602 After the output from the face detection and feature extraction module, the encryption and decryption module 602 encrypts the extracted face feature vector using the public key PK, and extracts the dimension of the feature vector according to the vector information. Then upload the ID, encrypted face information, evaluation key EK, and dimension information n to the cloud side, and store them in the face database 605 by the face information registration module. At this time, the face information database stores encrypted data, so users don't have to worry The problem of cloud leakage.
  • the encryption and decryption module 602 not only extracts dimension information according to the feature vector input by the face detection and feature extraction module 601, but also adopts different encoding and encryption methods based on different verification scenarios.
  • a single-face single-mode verification scenario only a single face extraction feature is packaged and encrypted, and the insufficient positions are filled with zeros, and then encrypted into a ciphertext, which is then transmitted to the cloud registration module for registration along with ID, EK, and dimension n.
  • a single face multi-mode verification scenario collect the feature vectors of multiple users in different modes (illumination, angle and other conditions), pack and encrypt them into a ciphertext, and transmit it to the cloud registration module together with the user ID, EK and dimension n register.
  • Fig. 7 is a schematic diagram of a face recognition system device in the verification stage provided by the embodiment of the present application.
  • the face similarity verification module 701 stores and extracts the previously registered face ciphertext, corresponding EK, and face feature dimension n in the face database 605 according to the user ID, and performs similarity calculation.
  • the calculated ciphertext result is returned to the user, and the user uses the previously generated private key to decrypt through the encryption and decryption module 602, and finally obtains the plaintext verification result.
  • the face similarity verification module involves ciphertext inner product calculation, which is different from the existing ciphertext inner product calculation technology.
  • the ciphertext calculation performance and throughput are greatly improved.
  • the module logic is shown in Figure 8.
  • the received face ciphertext information and ID retrieve the previously registered ciphertext face information, evaluation key EK and face dimension information.
  • Use homomorphic multiplication to multiply the two face ciphertexts, and calculate the number of times shiftcount that requires cyclic ciphertext shifting and addition based on the face dimension information read from the registration information, and then perform ciphertext cyclic shifting and addition.
  • the result after cyclic calculation is the ciphertext result of face similarity, and the plaintext verification result can be obtained after decryption.
  • FIG. 8 is a flow chart of a face recognition method provided by an embodiment of the present application.
  • the present application provides a face recognition method.
  • This method can be applied to the above-mentioned face recognition system. Can include the following steps:
  • Embodiment 1 of the present application is aimed at a single-face single-mode verification scenario.
  • each ciphertext only saves one face feature vector information
  • the input for each verification is also a ciphertext of a face feature vector.
  • the face feature vector dimension feature_vector_dimention 64, slot_count to 2048
  • Ciphertext1' represents the registered original face feature vector ciphertext
  • Ciphertext2' represents the face feature vector ciphertext that needs to be verified
  • encrypts the face feature When it is ciphertext, more than 64 slots are filled with zeros, so a ciphertext only contains a single face information.
  • Ciphertext0 represents the ciphertext product of Ciphertext1' and Ciphertext2', which already contains information about the pairwise multiplication of each slot in the plaintext.
  • Hierarchical fully homomorphic algorithms such as CKKS, BFV
  • Rotate (ciphertext, step) operation that is, the ciphertext ciphertext can be shifted by step slot operations, step is a positive left shift, step Negative shift right.
  • the face feature vector dimension information feature_vector_dimention is introduced to participate in the shift calculation.
  • the algorithm 2 of Embodiment 1 only needs log2(feature_vector_dimention) -1 ciphertext shift and addition calculation.
  • the feature_vector_dimention is 64, only 6 ciphertext shifts and additions are required, that is, the first time shifts 20 slots to the left and adds to the original ciphertext; the second time is the first shift On the result of the addition, shift 21 slots to the left and add to the previous result; for the third time, on the result of the second shift and addition, shift to the left 22 slots and add to the last result ;For the fourth time, on the result of the third shift and addition, shift left by 23 slots and add to the result of the previous time; for the fifth time, on the result of the fourth shift and addition, shift left by 24 slots slot, and add it to the previous result; the sixth time, on the result of the fifth shift and addition, move 25 slots to the left, and add it to
  • feature_vector1 represents the plaintext face feature vector 1
  • feature_vector2 represents the plaintext face feature vector 2
  • ExtendVector(feature_vector1, slot_count, 0) means that after the feature vector 1 is put into the front part of the vector composed of slot_count slots, the rest will be supplemented zero.
  • Encrypt() uses the public key to encrypt the vector, and Multiply multiplies the ciphertext, including the necessary relinearization measures for multiplication of homomorphic ciphertext.
  • this embodiment 1 needs log2(slot_count)-1 times of shifting and adding, and the embodiment 1 of this application now only needs log2(feature_vector_dimention)-1 times of ciphertext cyclic shifting and phase
  • the slot_count is large, the calculation time is greatly saved, and the higher the polynomial degree and the smaller the vector dimension, the more time is saved, thereby improving the performance of ciphertext face verification.
  • Embodiment 2 of the present application can be used in two scenarios, that is, a single face multi-mode (referring to the face information collected by the same person under different lighting, angle, etc.) verification scenarios and face recognition scenarios.
  • each ciphertext saves the face feature vector information of the same person in different dimensions, such as the face information of the same person wearing glasses, not wearing glasses, and having a certain left tilt angle, that is, the face information of different dimensions
  • the same face feature vector is packed into a ciphertext.
  • the same input face feature vector is repeatedly expanded and encoded into a plaintext vector with a slot_count length, that is, slot_count/feature_vector_dimention times are repeated.
  • Ciphertext1' represents the registered original face feature vector ciphertext, which contains 32 different dimensions of the same face information. If the dimension is not enough to 32, the method of embodiment 1 can be adopted to fill with zeros.
  • Ciphertext2' represents the face feature vector ciphertext that needs to be verified, which is the ciphertext after expanding the same face feature vector 32 times.
  • Ciphertext0 represents the ciphertext product of Ciphertext1' and Ciphertext2', which already contains information about the pairwise multiplication of each slot in the plaintext.
  • Hierarchical fully homomorphic algorithms (such as CKKS, BFV) provide the ciphertext shift operation Rotate (ciphertext, step) operation, that is, the ciphertext ciphertext can be shifted by step slot operations, step is a positive left shift, step Negative shift right.
  • the algorithm 3 of the second embodiment only needs log2(feature_vector_dimention)-1 ciphertext shift and addition calculation, and the calculation amount is the same as that of the first embodiment.
  • the verification results of different dimensions of the same face can be obtained in the same time, and the value with the largest similarity is selected for comparison with the threshold, and the ciphertext result of the comparison can be obtained.
  • the specific calculation process is the same as in Embodiment 1.
  • the feature_vector_dimention is 64, only 6 ciphertext shifts and additions are required, that is, the first left shift 20 slots, and add it to the original ciphertext; the second time, on the result of the first shift and addition, shift left by 21 slots, and add it to the result of the previous time; the third time, on the result of the second shift and addition On the result of bit addition, shift left by 22 slots and add to the previous result; for the fourth time, on the result of the third shift and addition, shift left by 23 slots and add to the previous result Add; for the fifth time, on the result of the fourth shift and addition, shift left by 24 slots, and add to the result of the previous time; for the sixth time, on the result of the fifth shift and addition, shift left by 25 slot, and add it to the previous result; the final result is in the ciphertext Ciphertext5. different from
  • vector1' is obtained by extending feature_vector1, which is the plaintext extension of the face feature vector that needs to be verified at present, and vector2' is the plaintext extension filled with different face feature vectors of the same person registered during registration.
  • Encrypt and Multiply have the same function Embodiment one.
  • vector2' is the plaintext extension of different face vector features, which is different from the single face multi-verification mode scenario.
  • the ciphertext inner product result ciphertext3 is the face to be verified and different.
  • N face information can be compared at the same time, which greatly improves the throughput of face ciphertext recognition.
  • N slot_count/feature_vector_dimention ciphertext comparison results can be obtained within the same time (i.e. one ciphertext comparison operation time).
  • the second embodiment improves the single Accuracy of face verification.
  • the second embodiment greatly improves the throughput, and the throughput increases linearly with N.
  • a pre-configured neural network can be used for feature extraction, and the specific network model can be selected according to the actual situation, which is not limited in this application.
  • This application mainly uses homomorphic encryption technology to perform homomorphic encryption and processing of face information, especially for the process of calculating the similarity of face ciphertexts, and introduces the dimension information of face feature vectors, so that the original slotcount-1 time-consuming
  • the ciphertext cycle calculation process is compressed to log2(feature_vector_dimention)-1 cycle, and feature_vector_dimention ⁇ slotcount, (for example, the general feature_vector_dimention is 64, 128, and the slotcount is 2048, 4096, 8192, 16384), which greatly reduces the amount of calculation .
  • this application is aimed at the multi-mode verification scenario of a single face.
  • the original information of different faces of the same person is packaged into one ciphertext storage during registration;
  • face information the same face information is repeatedly expanded and packaged into a ciphertext, and the face feature vector dimension information is extracted at the same time as the comparison and introduced into the calculation process to improve performance and increase the success rate of verification.
  • slot_count 4096
  • the success rate of verification is improved.
  • this application also lies in that, for the face recognition scene, by packing and encrypting different facial features into a ciphertext for storage, the verification needs to verify the facial features through expansion and pack them into a ciphertext for comparison with the registration ciphertext.
  • the introduction of face dimension information can improve the verification efficiency, and compare N face information at the same time, which greatly improves the throughput of face ciphertext recognition.
  • the method can also be implemented using hardware.
  • a field programmable gate array field programmable gate array, FPGA
  • FPGA field programmable gate array
  • this application extracts and encrypts the current face information to be verified by selecting appropriate homomorphic encryption parameters, and expands and packages different face feature information into one ciphertext.
  • the original face registration information of different individuals is packed into a ciphertext, and the method of the present invention is applied at the same time to extract the dimension information of the face feature vector and introduce the calculation process to achieve a parallel effect and improve the face ciphertext recognition. throughput.
  • this application can also be used in other biometric scenarios, such as fingerprint recognition scenarios, to protect the privacy of biological information.
  • biometric scenarios such as fingerprint recognition scenarios
  • terminal devices in this application may include, but are not limited to, mobile phones, smart TVs, smart speakers, wearable devices, tablet computers, desktop computers, all-in-one computers, handheld computers, notebook computers, super mobile personal computers ( ultra-mobile personal computer (UMPC), netbook, personal digital assistant (personal digital assistant, PDA), laptop computer (laptop), mobile computer, augmented reality (augmented reality, AR) device, virtual reality (virtual reality, VR) Any terminal equipment or portable terminal equipment such as equipment, artificial intelligence (AI) equipment, and/or vehicle-mounted equipment.
  • UMPC ultra-mobile personal computer
  • PDA personal digital assistant
  • laptop computer laptop
  • mobile computer augmented reality (augmented reality, AR) device
  • virtual reality virtual reality
  • Any terminal equipment or portable terminal equipment such as equipment, artificial intelligence (AI) equipment, and/or vehicle-mounted equipment.
  • the terminal device and the server can be connected in a wired or wireless way, wherein the wireless way can include 2G/3G/4G/5G/6G and other wireless communication solutions.
  • the wireless way can include 2G/3G/4G/5G/6G and other wireless communication solutions.
  • wireless local area networks wireless local area networks, WLAN
  • Bluetooth bluetooth, BT
  • global navigation satellite system global navigation satellite system
  • GNSS global navigation satellite system
  • frequency modulation frequency modulation, FM
  • short-range wireless communication technology near field communication, NFC
  • Zigbee (zigbee) and infrared technology infrared, IR
  • IR infrared technology
  • the WLAN may be, for example, a wireless fidelity (wireless fidelity, Wi-Fi) network.
  • FIG. 11 is a schematic diagram of a terminal device provided in an embodiment of the present application.
  • the present application also provides a terminal device 1100 .
  • the device terminal device 1100 may be the terminal device 100 and the terminal device 501 described above in FIGS. 1 to 10 .
  • the terminal device 1100 may include: a processor 1110, an external memory interface 1120, an internal memory 1121, a universal serial bus (universal serial bus, USB) interface 1130, a charging management module 1140, a power management module 1141, a battery 1142, an antenna 1, Antenna 2, mobile communication module 1150, wireless communication module 1160, camera 1170, etc.
  • the terminal device 1100 may include more or fewer components than shown, or combine some components, or separate some components, or arrange different components.
  • the illustrated components can be realized in hardware, software or a combination of software and hardware.
  • the processor 1110 may be a processor with architectures such as advanced reduced instruction set computing machines (ARM), X86, and microprocessor without interlocked piped stages (MIPS).
  • Processor 1110 may include one or more processing units, for example: application processor (application processor, AP), modem processor, GPU, image signal processor (image signal processor, ISP), controller, video codec device, digital signal processor (digital signal processor, DSP), baseband processor and/or neural network processor (neural-network processing unit, NPU), etc.
  • application processor application processor, AP
  • modem processor GPU
  • image signal processor image signal processor
  • ISP image signal processor
  • controller video codec device
  • digital signal processor digital signal processor
  • DSP digital signal processor
  • NPU neural-network processing unit
  • different processing units may be independent devices, or may be integrated in one or more processors.
  • the controller can generate an operation control signal according to the instruction opcode and timing signal, and complete the control of fetching and executing the instruction.
  • a memory may also be provided in the processor 1110 for storing instructions and data.
  • the memory in processor 1110 is a cache memory.
  • the memory may hold instructions or data that the processor 1110 has just used or recycled. If the processor 1110 needs to use the instruction or data again, it can be called directly from the memory. Repeated access is avoided, and the waiting time of the processor 1110 is reduced, thereby improving the efficiency of the system.
  • processor 1110 may include one or more interfaces.
  • the interface may include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous transmitter (universal asynchronous receiver/transmitter, UART) interface, mobile industry processor interface (mobile industry processor interface, MIPI), general-purpose input and output (general-purpose input/output, GPIO) interface, subscriber identity module (subscriber identity module, SIM) interface and / Or a universal serial bus (universal serial bus, USB) interface, etc.
  • I2C integrated circuit
  • I2S integrated circuit built-in audio
  • PCM pulse code modulation
  • PCM pulse code modulation
  • UART universal asynchronous transmitter
  • MIPI mobile industry processor interface
  • GPIO general-purpose input and output
  • subscriber identity module subscriber identity module
  • SIM subscriber identity module
  • USB universal serial bus
  • the charging management module 1140 is used for receiving charging input from the charger.
  • the charger may be a wireless charger or a wired charger.
  • the charging management module 1140 can receive charging input from the wired charger through the USB interface 1130 . In some wireless charging embodiments, the charging management module 1140 may receive wireless charging input through the wireless charging coil of the terminal device 1100 . While the charging management module 1140 is charging the battery 1142 , it can also supply power to the terminal device 1100 through the power management module 1141 .
  • the wireless communication function of the terminal device 1100 may be implemented by the antenna 1, the antenna 2, the mobile communication module 1150, the wireless communication module 1160, the modem processor, the baseband processor, and the like.
  • the mobile communication module 1150 can provide wireless communication solutions including 2G/3G/4G/5G/6G applied on the terminal device 1100 .
  • the wireless communication module 1160 can provide wireless communication solutions including WLAN, BT, GNSS, FM, NFC, zigbee and IR applied on the terminal device 1100 .
  • the WLAN may be, for example, a Wi-Fi network.
  • the terminal device 1100 may be connected to the external display 200 through the foregoing wireless manner. Of course, it can also be connected in a wired way.
  • the terminal device 1100 implements a display function through a GPU, a display screen 1170, an application processor, and the like.
  • the GPU is a microprocessor for image processing, and is connected to the display screen 1170 and the application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering.
  • Processor 1110 may include one or more GPUs that execute program instructions to generate or change display information.
  • the external memory interface 1120 may be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the terminal device 1100.
  • the external memory card communicates with the processor 1110 through the external memory interface 1120 to implement a data storage function. For example saving files such as images in an external memory card.
  • the internal memory 1121 may be used to store computer-executable program codes including instructions.
  • the internal memory 1121 may include an area for storing programs and an area for storing data. Wherein, the stored program area can store an operating system, an application program required by at least one function, and the like.
  • the storage data area may store data created during use of the terminal device 1100 and the like.
  • the internal memory 1121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, flash memory device, universal flash storage (universal flash storage, UFS) and the like.
  • the processor 1110 executes various functional applications and data processing of the terminal device 1100 by executing instructions stored in the internal memory 1121 and/or instructions stored in a memory provided in the processor.
  • Camera 1170 is used to capture still images or video.
  • the object generates an optical image through the lens and projects it to the photosensitive element.
  • the photosensitive element may be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor.
  • CMOS complementary metal-oxide-semiconductor
  • the photosensitive element converts the light signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal.
  • the ISP outputs the digital image signal to the DSP for processing.
  • DSP converts digital image signals into standard RGB, YUV and other image signals.
  • the terminal device 1100 may include one or more cameras 1170 .
  • the terminal device 1100 provided in this application can implement any one of the methods described above in FIG. 1 to FIG. 10 , and the specific implementation manner can refer to the corresponding descriptions in FIG. 1 to FIG. 10 , which will not be repeated here.
  • FIG. 12 is a schematic diagram of a server provided by an embodiment of the present application.
  • the present application also provides a server 1200 .
  • the device server 1200 may be the server 100 and the server 501 described above in FIG. 1 to FIG. 10 .
  • the server 1200 may include: a processor 1210, an external memory interface 1220, an internal memory 1221, a universal serial bus (universal serial bus, USB) interface 1230, a charging management module 1240, a power management module 1241, a battery 1242, an antenna 1, and an antenna 2.
  • Mobile communication module 1250 and wireless communication module 1260 etc.
  • the server 1200 may include more or fewer components than shown, or some components may be combined, or some components may be split, or a different arrangement of components.
  • the illustrated components can be realized in hardware, software or a combination of software and hardware.
  • the processor 1210 may be a processor with architectures such as advanced reduced instruction set computing machines (ARM), X86, and microprocessor without interlocked piped stages (MIPS).
  • Processor 1210 may include one or more processing units, such as: application processor (application processor, AP), modem processor, GPU, image signal processor (image signal processor, ISP), controller, video codec device, digital signal processor (digital signal processor, DSP), baseband processor and/or neural network processor (neural-network processing unit, NPU), etc. Wherein, different processing units may be independent devices, or may be integrated in one or more processors.
  • the controller can generate an operation control signal according to the instruction opcode and timing signal, and complete the control of fetching and executing the instruction.
  • a memory may also be provided in the processor 1210 for storing instructions and data.
  • the memory in processor 1210 is a cache memory.
  • the memory may hold instructions or data that the processor 1210 has just used or recycled. If the processor 1210 needs to use the instruction or data again, it can be directly recalled from the memory. Repeated access is avoided, and the waiting time of the processor 1210 is reduced, thereby improving the efficiency of the system.
  • processor 1210 may include one or more interfaces.
  • the interface may include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous transmitter (universal asynchronous receiver/transmitter, UART) interface, mobile industry processor interface (mobile industry processor interface, MIPI), general-purpose input and output (general-purpose input/output, GPIO) interface, subscriber identity module (subscriber identity module, SIM) interface and / Or a universal serial bus (universal serial bus, USB) interface, etc.
  • I2C integrated circuit
  • I2S integrated circuit built-in audio
  • PCM pulse code modulation
  • PCM pulse code modulation
  • UART universal asynchronous transmitter
  • MIPI mobile industry processor interface
  • GPIO general-purpose input and output
  • subscriber identity module subscriber identity module
  • SIM subscriber identity module
  • USB universal serial bus
  • the charging management module 1240 is used for receiving charging input from the charger.
  • the charger may be a wireless charger or a wired charger.
  • the charging management module 1240 can receive charging input from a wired charger through the USB interface 1230 . In some wireless charging embodiments, the charging management module 1240 may receive wireless charging input through the wireless charging coil of the server 1200 . While the charging management module 1240 is charging the battery 1242 , it can also supply power to the server 1200 through the power management module 1241 .
  • the wireless communication function of the server 1200 may be realized by the antenna 1, the antenna 2, the mobile communication module 1250, the wireless communication module 1260, the modem processor and the baseband processor.
  • the mobile communication module 1250 can provide wireless communication solutions including 2G/3G/4G/5G/6G applied on the server 1200 .
  • the wireless communication module 1260 can provide wireless communication solutions including WLAN, BT, GNSS, FM, NFC, zigbee and IR applied on the server 1200 .
  • the WLAN may be, for example, a Wi-Fi network.
  • the server 1200 can be connected to the external display 200 through the above wireless manner. Of course, it can also be connected in a wired way.
  • the server 1200 implements a display function through a GPU, a display screen 1270, and an application processor.
  • the GPU is a microprocessor for image processing, and is connected to the display screen 1270 and the application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering.
  • Processor 1210 may include one or more GPUs that execute program instructions to generate or change display information.
  • the external memory interface 1220 may be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the server 1200.
  • the external memory card communicates with the processor 1210 through the external memory interface 1220 to implement a data storage function. For example saving files such as images in an external memory card.
  • the internal memory 1221 can be used to store computer-executable program codes, which include instructions.
  • the internal memory 1221 may include an area for storing programs and an area for storing data. Wherein, the stored program area can store an operating system, an application program required by at least one function, and the like.
  • the storage data area may store data created during use of the server 1200 and the like.
  • the internal memory 1221 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, flash memory device, universal flash storage (universal flash storage, UFS) and the like.
  • the processor 1210 executes various functional applications and data processing of the server 1200 by executing instructions stored in the internal memory 1221 and/or instructions stored in a memory provided in the processor.
  • the server 1200 provided in this application can implement any one of the methods described above in FIG. 1 to FIG. 10 , and the specific implementation manner can refer to the corresponding description in FIG. 1 to FIG. 10 , which will not be repeated here.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computing Systems (AREA)
  • Computer Hardware Design (AREA)
  • Evolutionary Computation (AREA)
  • Computer Security & Cryptography (AREA)
  • Multimedia (AREA)
  • Bioethics (AREA)
  • Artificial Intelligence (AREA)
  • Databases & Information Systems (AREA)
  • Human Computer Interaction (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Medical Informatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • Mathematical Physics (AREA)
  • Collating Specific Patterns (AREA)

Abstract

提供了一种人脸识别方法，方法包括：获取用户输入的包括用户身份标识信息的登录信息；采集用户的第一人脸图像；根据第一人脸图像获取第一人脸特征信息；通过公钥采用全同态加密算法对第一人脸特征信息进行加密；将加密后的第一人脸特征信息以及评估密钥、用户身份标识信息发送至服务器；接收服务器发送的识别认证信息，其中，识别认证信息为密文；根据私钥采用全同态加密算法对识别认证信息进行解密，以得到用户的人脸识别认证结果。

Description

一种人脸识别方法、设备及系统
本申请要求于2021年09月14日提交中国专利局、申请号为202111076938.8、申请名称为“一种人脸识别方法、设备及系统”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及电子技术领域，尤其涉及一种人脸识别方法、设备及系统。
背景技术
随着人脸识别技术的成熟，因为人脸信息属于用户独有生物信息，相比指纹识别技术，可以做到让用户无感，认证过程更友好。所以基于人脸信息来进行认证的系统在实际应用中也越来越普及。例如，基于人脸识别的登录认证、基于人脸识别的支付、基于人脸识别门禁等。但人脸信息作为生物个体的敏感信息，一旦泄露将会有可能使认证系统受到攻击，同时还会泄露用户隐私。目前，人脸信息已被无感地采集并随意使用，人脸信息的隐私保护需求越来越紧迫。
目前的人脸识别系统中，人脸信息提取、保存与使用时都是明文，显然明文是非常不安全的。虽然系统里不存储人脸照片原信息，而是存储经过人脸特征提取模块提取的人脸特征向量，以用于人脸识别、人脸认证时的计算。但现有方式存在可以通过仅仅使用提取的人脸特征向量信息对人脸进行还原，还原的人脸图像不但会泄露个人隐私信息，甚至存在使用还原的人脸图像去欺骗验证系统，从而绕过安全检查的情况。
因此，当前方案采用同态加密技术，而同态加密技术可以做到数据可用不可见，即数据被加密后，仍然可以进行和明文一样的计算处理，其处理结果和对明文处理是一样的，只是处理结果也保持在密文状态，只有拥有私钥的用户通过私钥解密后，才能得到明文结果。真正做到处理过程前、处理过程中与处理过程后都不会泄露任何信息。但是，同态加密的硬伤是性能问题，特别是密文乘法、密文移位等操作，相比明文操作存在4个数量级的性能差距，因此目前还没有实际应用场景，特别是对于实时性要求较高的场景，仍然需要提高性能以实用化。
发明内容
本申请实施例提供了一种人脸识别方法，通过采用全同态加密(full homomorphic encryption，FHE)方式对人脸特征信息进行加密，使得在服务器端计算时全程采用密文，保障了数据的安全性。同时在服务器端结合特征的维度信息，大大降低了服务端的计算量，并显著提升计算性能。
第一方面，提供了一种人脸识别方法，方法应用于终端设备，方法包括：获取用户输入的登录信息，登录信息包括用户身份标识信息；采集用户的第一人脸图像；根据第一人脸图像获取第一人脸特征信息；通过预先存储的公钥采用全同态加密算法对第一人脸特征信息进行加密；将加密后的第一人脸特征信息以及预先存储的评估密钥、用户身份标识信息发送至服务器，其中，评估密钥与公钥为终端设备根据用户身份标识信息确定的；接收服务器发送的识别认证信息，其中，识别认证信息为密文；根据预先存储的私钥采用全同 态加密算法对识别认证信息进行解密，以得到用户的人脸识别认证结果。本申请采用全同态加密方式对人脸特征信息进行加密，使得在服务器端计算时全程采用密文，保障了数据的安全性。同时在服务端结合特征的维度从而极大的减少计算量，并显著提升了计算性能。
在一个可能的实施方式中，在采集用户的人脸图像之前，方法还包括：采集用户的第二人脸图像，第二人脸图像用于注册人脸信息；根据第二人脸图像获取第二人脸特征信息；根据第二人脸特性信息确定特征维度；根据公钥采用全同态加密算法对第二人脸特征信息进行加密；将加密后的第二人脸特征信息、用户身份标识信息、特征维度和评估密钥发送至服务器。
在一个可能的实施方式中，在采集用户的人脸图像之前，方法还包括：根据用户身份标识信息、安全强度信息、性能信息和场景信息，确定公钥、评估密钥和私钥，其中，公钥和私钥作为一对非对称加密的两个密钥，安全强度信息、性能信息和场景信息为预先配置或接收用户输入得到的。
在一个可能的实施方式中，根据预先存储的私钥采用全同态加密算法对识别认证信息进行解密，以得到用户的人脸识别认证结果，包括：根据私钥采用全同态加密算法对识别认证信息进行解密，确定人脸相似度信息；当人脸相似度信息大于或等于预设的相似度阈值，则确定用户的人脸识别认证通过；当人脸相似度信息小于相似度阈值，则确定用户的人脸识别认证未通过。
第二方面，提供了一种人脸识别方法，方法应用于服务器，服务器包含人脸特征数据库，人脸特征数据库包括至少一个注册人脸特征信息，方法包括：接收终端设备发送的加密后的第一人脸特征信息和用户身份标识信息，其中，加密后的第一人脸特征信息为采用全同态加密算法进行加密得到的；根据用户身份标识信息从人脸特征数据库中确定对应的注册人脸特征；采用加密后的第一人脸特征信息和用户身份标识信息对应的注册人脸特征信息进行相似度计算，确定识别认证信息，识别认证信息用于表示加密后的第一人脸特征信息和用户身份标识信息对应的注册人脸特征信息之间的相似度；将识别认证信息发送至终端设备。本申请在服务器端，结合特征的维度信息进行相似度计算，大大降低了计算量，并显著提升计算性能。
在一个可能的实施方式中，方法还包括：接收终端设备发送的加密后的第二人脸特征信息、用户身份标识信息、特征维度和评估密钥；将加密后的第二人脸特征信息、用户身份标识信息、特征维度和评估密钥作为注册人脸特征信息存储至人脸特征数据库中。
在一个可能的实施方式中，注册人脸特征信息包括有加密后的第二人脸特征信息、用户身份标识信息、特征维度和评估密钥；采用加密后的第一人脸特征信息和用户身份标识信息对应的注册人脸特征信息进行相似度计算，确定识别认证信息，包括：对加密后的第一人脸特征信息和加密后的第二人脸特征信息采用评估密钥进行密文相乘；对密文相乘后的信息采用评估密钥进行密文移位，密文移位的位数为2^i位，其中i为当前循环次数，i为自然数，i是根据特征维度确定的；将密文移位后的信息与密文移位前的信息进行相加，以得到移位相加结果密文；对移位相加结果密文再次进行密文移位和密文相加，直至循环执行i次后，将第i次得到移位相加结果密文作为识别认证信息。
在一个可能的实施方式中，注册人脸特征信息包括加密后的第二人脸特征信息；方法还包括：对加密后的第二人脸特征信息进行扩展，确定扩展第二人脸特征信息；采用加密 后的第一人脸特征信息和用户身份标识信息对应的注册人脸特征信息进行相似度计算，包括：采用加密后的第一人脸特征信息和扩展第二人脸特征信息进行相似度计算。
第三方面，提供了一种用于人脸识别的终端设备，终端设备包括：采集模块，用于获取用户输入的登录信息，登录信息包括用户身份标识信息；采集模块还用于，采集用户的第一人脸图像；人脸检测与特征提取模块，用于根据第一人脸图像获取第一人脸特征信息；加密与解密模块，用于通过预先存储的公钥采用全同态加密算法对第一人脸特征信息进行加密；发送模块，用于将加密后的第一人脸特征信息以及预先存储的评估密钥、用户身份标识信息发送至服务器，其中，评估密钥与公钥为密钥产生模块根据用户身份标识信息确定的；接收模块，用于接收服务器发送的识别认证信息，其中，识别认证信息为密文；加密与解密模块还用于，根据预先存储的私钥采用全同态加密算法对识别认证信息进行解密，以得到用户的人脸识别认证结果。本申请采用全同态加密方式对人脸特征信息进行加密，使得在服务器端计算时全程采用密文，保障了数据的安全性。同时在服务端结合特征的维度从而极大的减少计算量，并显著提升了计算性能。
在一个可能的实施方式中，采集模块还用于，采集用户的第二人脸图像，第二人脸图像用于注册人脸信息；人脸检测与特征提取模块还用于，根据第二人脸图像获取第二人脸特征信息；人脸检测与特征提取模块还用于，根据第二人脸特性信息确定特征维度；加密与解密模块还用于，根据公钥采用全同态加密算法对第二人脸特征信息进行加密；发送模块还用于，将加密后的第二人脸特征信息、用户身份标识信息、特征维度和评估密钥发送至服务器。
在一个可能的实施方式中，加密与解密模块还用于：根据用户身份标识信息、安全强度信息、性能信息和场景信息，确定公钥、评估密钥和私钥，其中，公钥和私钥作为一对非对称加密的两个密钥，安全强度信息、性能信息和场景信息为预先配置或采集模块接收用户输入得到的。
在一个可能的实施方式中，加密与解密模块还用于，根据私钥采用全同态加密算法对识别认证信息进行解密，确定人脸相似度信息；终端设备还包括：确定模块，用于当人脸相似度信息大于或等于预设的相似度阈值，则确定用户的人脸识别认证通过；以及，当人脸相似度信息小于相似度阈值，则确定用户的人脸识别认证未通过。
第四方面，提供了一种用于人脸识别的服务器，服务器包含人脸特征数据库，人脸特征数据库包括至少一个注册人脸特征信息，服务器包括：接收模块，用于接收终端设备发送的加密后的第一人脸特征信息和用户身份标识信息，其中，加密后的第一人脸特征信息为采用全同态加密算法进行加密得到的；人脸相似度验证模块，用于根据用户身份标识信息从人脸特征数据库中确定对应的注册人脸特征；人脸相似度验证模块还用于，采用加密后的第一人脸特征信息和用户身份标识信息对应的注册人脸特征信息进行相似度计算，确定识别认证信息，识别认证信息用于表示加密后的第一人脸特征信息和用户身份标识信息对应的注册人脸特征信息之间的相似度；发送模块，用于将识别认证信息发送至终端设备。本申请在服务器端，结合特征的维度信息进行相似度计算，大大降低了计算量，并显著提升计算性能。
在一个可能的实施方式中，接收模块还用于：接收终端设备发送的加密后的第二人脸特征信息、用户身份标识信息、特征维度和评估密钥；将加密后的第二人脸特征信息、用 户身份标识信息、特征维度和评估密钥作为注册人脸特征信息存储至人脸特征数据库中。
在一个可能的实施方式中，注册人脸特征信息包括有加密后的第二人脸特征信息、用户身份标识信息、特征维度和评估密钥；人脸相似度验证模块还用于：对加密后的第一人脸特征信息和加密后的第二人脸特征信息采用评估密钥进行密文相乘；对密文相乘后的信息采用评估密钥进行密文移位，密文移位的位数为2^i位，其中i为当前循环次数，i为自然数，i是根据特征维度确定的；将密文移位后的信息与密文移位前的信息进行相加，以得到移位相加结果密文；对移位相加结果密文再次进行密文移位和密文相加，直至循环执行i次后，将第i次得到移位相加结果密文作为识别认证信息。
在一个可能的实施方式中，注册人脸特征信息包括加密后的第二人脸特征信息；人脸相似度验证模块还用于：对加密后的第二人脸特征信息进行扩展，确定扩展第二人脸特征信息；采用加密后的第一人脸特征信息和扩展第二人脸特征信息进行相似度计算。
第五方面，提供了一种用于人脸识别的系统，该系统包括：终端设备和服务器。终端设备和服务器通过有线或无线方式连接，以便该系统执行上述第一方面以及第二方面中的任意一项方法。
第六方面，提供了一种用于人脸识别的终端设备，终端设备包括：处理器用于与存储器耦合，以及读取并执行存储在存储器中的指令；当处理器运行时执行指令，使得处理器用于执行第一方面任意一项的方法。
第七方面，提供了一种用于人脸识别的服务器，服务器包括：处理器用于与存储器耦合，以及读取并执行存储在存储器中的指令；当处理器运行时执行指令，使得处理器用于执行第二方面任意一项的方法。
第八方面，提供了一种计算机可读存储介质，计算机可读存储介质中存储有指令，当指令在终端上运行时，使得终端执行如第一方面任意一项的方法。
第九方面，提供了一种计算机可读存储介质，计算机可读存储介质中存储有指令，当指令在服务器上运行时，使得服务器执行如第二方面任意一项的方法。
第十方面，提供了一种包含指令的计算机设备，当其在终端上运行时，使得终端执行如第一方面中的任意一项的方法。
第十一方面，提供了一种包含指令的计算机设备，当其在服务器上运行时，使得服务器执行如第二方面中的任意一项的方法。
第十二方面，提供了一种包含指令的计算机程序产品，当其在计算机上运行时，使得计算机执行第一方面中任意一项的方法。
第十三方面，提供了一种包含指令的计算机程序产品，当其在计算机上运行时，使得计算机执行第二方面中任意一项的方法。
本申请公开了一种人脸识别方法，通过采用全同态加密方式对人脸特征信息进行加密，使得在服务器端计算时全程采用密文，保障了数据的安全性。同时在服务器端结合特征的维度信息，大大降低了服务端的计算量，并显著提升计算性能。
附图说明
图1为本申请实施例提供的一种应用场景示意图；
图2为一种人脸识别流程示意图；
图3为一种层次型全同态加密过程示意图；
图4为一种全同态加密后的密文计算过程示意图；
图5为本申请实施例提供的一种人脸识别系统框架示意图；
图6为本申请实施例提供的一种注册阶段人脸识别系统装置示意图；
图7为本申请实施例提供的一种验证阶段人脸识别系统装置示意图；
图8为本申请实施例提供的一种人脸识别方法流程图；
图9为本申请实施例提供的一种密文内积计算示意图；
图10为本申请实施例提供的另一种密文内积计算示意图；
图11为本申请实施例提供的一种终端设备示意图；
图12为本申请实施例提供的一种服务器示意图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行描述。
本申请主要应用于人脸识别场景，例如图1所示出的，用户通过终端设备100进行人脸识别或者人脸认证。其中，终端设备100包含摄像头，终端设备通过摄像头采集用户的人脸图像，以便进行人脸识别或者人脸认证。
人脸识别(face recognition)：一个人脸识别系统从输入人脸图片中提取特征然后和人脸库中的图片进行比较。如果它和库里最接近照片的相似度大于一定阈值，我们就判断这张照片是人脸库里此人的照片，否则就认为这是一个未知的人脸。通常用于基于人脸的考勤打卡、人员搜索等。
人脸验证(face verification)：人脸验证是判断两张人脸图片是否是同一个人的技术，这是一个两分类问题，通常用于基于人脸的登录控制、访问控制、身份确认等方面，比如基于人脸的手机解锁，操作系统登录。
当先的一些方案可以如图2所示出的，通过人脸检测、人脸对齐与人脸规范化、特征提取，最后得到n维向量表示的特征值。在认证场景，通过对当前特征值与注册时候的特征值得进行相似度计算，把所得相似度和使用数据集训练最优后的阈值进行比较，小于阈值则可认为是同一个人，认证通过；反之不通过。在识别场景，通过对当前人脸特征值和不同的注册特征值进行一一的相似度计算，取相似度最低的结果，可识别当前人脸是哪一个已注册人员。
具体人脸相似度计算算法有欧氏距离和余弦距离，针对归一化后的特征值，两种算法计算结果相同，在此介绍余弦相似度计算过程。余弦距离是指向量空间中两个向量之间角度的余弦值，也称为余弦相似性。对于两个n’维空间点a＝(x 1、x 2、...、x n)和b＝(y 1、y 2、...、y n)，它们的余弦距离定义如下：
Figure PCTCN2022095037-appb-000001
其中，i’为1到n’之间的任意一个整数，d表示余弦距离，n’为正整数。
两个向量之间角度的余弦值是两个向量之间的余弦相似性。通过将矢量计算过程引入方程，可以获得两个矢量之间的余弦相似性：
Figure PCTCN2022095037-appb-000002
余弦相似度的数值范围是余弦值的范围，即(-1,1)。值越高，相似性越大。这证实了余弦相似性的数值意义。从计算公式可知，使用归一化后的向量特征进行相似度计算时，实质是计算两个向量的内积。
但是该方案中，人脸特征向量虽然不是人脸原始图像，只存储人脸特征向量信息，但已有研究表明，可以依据人脸特征向量信息对人脸进行恢复，因此造成人脸隐私信息泄露。即使使用常规加密算法加密保存，在使用中仍然需要解密后进行运算，因此当前安全技术并不能完美解决人脸存储与使用的隐私保护问题。
目前还存在一些方案利用了层次型全同态加密(leveled full homomorphic encryption，LFHE)方案。例如采用了层次型全同态CKKS算法(cheon-kim-kim-song，CKKS)、层次型全同态BFV算法(brakerski-fan and vercauteren，BFV)。提前根据计算业务复杂度来确定加密参数，从而可以避免耗时的自举方案，但密文态乘法与密文态移位这样的操作仍然与明文计算有至少4个数量级的差距，针对实际业务的计算模式，如避免或减少类似的耗时运算以提升性能，使基于同态运算的业务实时性相比明文不受影响，对同态加密是否能在实际业务中具有非常重要的意义。
典型的层次型全同态加密过程可以如图3所示，其中，通过规范嵌入逆映射，可以把n/2个复数域的分量元素映射到一个分圆多项式
Figure PCTCN2022095037-appb-000003
上，其中分量元素的个数又称为slot_count，即一个密文分圆多项式可以打包多少个明文分量元素。而分圆多项式的相加、相乘等价于每个分量元素和相加、相乘。其中，规范嵌入定义如下：
定义δ i:
Figure PCTCN2022095037-appb-000004
Figure PCTCN2022095037-appb-000005
ζ m是一个m次本原单位根；
Figure PCTCN2022095037-appb-000006
是一个
Figure PCTCN2022095037-appb-000007
维向量函数；
结合人脸识别业务的计算类型，主要是涉及人脸相似度的计算，而人脸相似度计算时，针对归一化的特征向量，其实质为特征向量的内积计算。在密文状态下，其计算过程可以如图4所示。
通过2次移位相加，slot1(值35)正好是密文特征向量ciphertext1与ciphertext2的内积，解密后取slot1的值可得明文特征向量内积值。推广到一般情况下，发现需要移位相加的循环次数为log2(slot_count)-1次，第i次循环中，移位step为2 i步。算法伪码如下：
算法伪码如下：
其中ct1表示密文1，ct2表示密文2，Multiply表示密文1与密文2相乘，包含同态运算中必要的重线性化步骤，Rotate(ct3,2 i)表示对密文3左移2 i位。
Figure PCTCN2022095037-appb-000008
上述方式总共需要log2(slot_count)-1次循环，每次循环又需要对当前密文向左移2 i步之后再加上原有密文，密文移位是同态运算里相当耗时操作，和明文相比，有4个数量级的差距，当slot_count很大时，成为整个运算的瓶颈点。在层次型全同态算法的实际应用中，为保证安全强度达到最低的128bits级别，BFV或CKKS密文多项式的最高次数一般取4096及以上。
目前，人脸认证技术已经广泛用于各种商业活动中，但因人脸信息保护不当而发生的人脸信息泄露、滥用事件也层出不穷，对用户隐私造成了严重影响。本申请采用同态人脸识别技术结合人脸识别与同态加密技术，基于密码学方法保证人脸信息使用前、使用中、使用后不泄露用户隐私。
因此，本申请提供了一种人脸识别方法，通过采用全同态加密方式对人脸特征信息进行加密，使得在服务器端计算时全程采用密文，保障了数据的安全性。同时在服务器端结合特征的维度信息，大大降低了服务端的计算量，并显著提升计算性能。
本申请可解决人脸验证与识别两种场景下，明文保存的人脸特征能被攻击程序还原，从而泄露隐私信息，并使用还原的人脸攻击认证系统。同时本申请还可以解决人脸验证与识别密文计算过程中的性能瓶颈问题。
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行详细描述。
一方面，本申请可以在单模式人脸验证、多模式人脸验证、人脸识别场景下，针对现有明文人脸特征相似度计算技术，使用同态加密算法(CKKS、BFV)对方案进行改造，加密人脸特征向量并保存密文向量，且相似度计算整个过程及计算结果不泄露任何信息，只有掌握私钥的用户能得到计算结果。
另一方面，本申请还可以在单模式人脸验证场景的密文相似度计算过程中，修改现有密文人脸特征相似性计算方法，通过提取人脸特征向量维度信息并引入计算过程，减小所需要的计算轮次，同时不影响计算结果准确性，提升性能。
又一方面，本申请还可以在多模式人脸验证场景下，通过选取合适的同态加密参数，提取并加密当前要验证人脸信息时，将同一人脸信息重复扩展打包到一个密文中，比对时，将同一人不同人脸原始注册信息打包到一个密文中，同时提取人脸特征向量维度信息并引入计算过程，达到并行效果，提高验证成功率并提高密文验证效率。
再一方面，本申请还可以在人脸识别场景下，通过选取合适的同态加密参数，提取并加密当前要验证人脸信息时，将同一人脸特征信息重复扩展打包到一个密文中，比对时，将不同个体的人脸原始注中册信息打包到一个密文，同时提取人脸特征向量维度信息并引入计算过程，达到并行效果，可提高人脸密文识别吞吐量。
图5为本申请实施例提供的一种人脸识别系统示意图。
如图5所示人脸识别系统由用户自己控制的终端设备501和服务器502组成。其中，服务器也可以称为云端、服务器端、服务端等，终端设备也可以称为终端。终端进行人脸信息采集并在云端进行认证。终端因为是用户自己掌握，为可信域，而服务端(如云端)对用户来说为不可信域，即用户不信任自己不掌控的云侧。但是又希望借助云端提供的人脸识别服务(face recognition service，FRS)或人脸验证服务(face verification service，FVS)，同时不泄露有关人脸隐私的任何信息给云端。因此，系统通过结合人脸识别与同态加密的能力，达到云端仅仅存储加密的人脸数据信息，并且在人脸比对计算过程中也保持密文状 态，包括得到的结果的是密文结果。具体来说，整个系统分为两个过程：人脸注册过程和人脸验证(识别)过程。
人脸注册时，终端依据全同态的具体参数和要达到的安全强度，通过密钥生成模块，生成同态运算使用的私钥和公钥并保存在端侧符合要求的介质内，本地采集的人脸图像经过人脸检测与特征提取模块，提取人脸特征向量信息，然后使用公钥加密后传输到云侧，通过人脸登记与验证模块存储到注册模板数据库中，同时公钥也一同传输到云侧存储。
人脸验证(识别)时，终端侧采集人脸图像后，经过人脸检测与特征提取模块，提取出人脸特征，并使用之前保存的公钥进行加密后传输到云端人脸登记与验证模块，人脸登记与验证模块使用注册阶段注册的密文人脸信息与现在上传的密文人脸信息进行相似度计算，计算结果以密文返回给终端，终端解密模块使用之前存储的私钥进行解密，得到验证(识别)明文结果。
上述整个过程，人脸信息不以明文在云侧存储及应用，云端除了提供算力外，获取不到任何隐私信息。
图6为本申请实施例提供的一种注册阶段人脸识别系统装置示意图。
其中，终端设备501可以包括人脸检测与特征提取模块601、加密与解密模块602和密钥产生模块603。其中，加密与解密模块602可以对应于图5中的加密模块和解密模块。可以理解的是，终端设备501还可以包括采集模块(图中未示出)、发送模块(图中未示出)和接收模块(图中未示出)。服务器502中可以包括人脸信息注册模块604和人脸数据库605。
在注册时，用户输入ID，并根据安全强度、性能要求及应用场景，由密钥产生模块603生成加密使用的同态公钥(public key，PK)、私钥(secrete key，SK)和评估密钥(evaluation key，EK)。公钥PK与私钥SK本地留存。根据人脸检测与特征提取模块输出后，加密与解密模块602使用公钥PK对提取的人脸特征向量进行加密，并根据向量信息提取出特征向量的维度。然后把ID、加密人脸信息、评估密钥EK、维度信息n上传云侧，由人脸信息注册模块存储到人脸数据库605，此时，人脸信息数据库存储的是加密数据，用户不用担心云侧泄露的问题。
其中加密与解密模块602除了根据人脸检测与特征提取模块601输入的特征向量提取维度信息，还基于验证场景不同采用不同的编码加密方式。当使用单个人脸单模式验证场景，只针对单一人脸提取特征进行打包加密，不足位置补零，然后加密为一个密文后与ID、EK及维度n一并传输给云端注册模块注册。当使用单个人脸多模式验证场景，收集多个用户不同模式(光照、角度等条件)的特征向量进行打包加密为一个密文后，与用户ID、EK及维度n一并传输给云端注册模块注册。
图7为本申请实施例提供的一种验证阶段人脸识别系统装置示意图。
验证时，依据模式不同进行不同处理。当使用单模式时，当接收到人脸检测与特征提取模块601输出的人脸特征时，不作扩展，单个特征进行打包并加密为一个密文，随用户ID传输到人脸验证模块进行验证；当使用多模式时，对接收的单个人脸特征进行重复扩展打包并加密为一个密文后，随用户ID传输到人脸相似度验证模块701进行验证。人脸相似度验证模块701收到验证数据后，依据用户ID到人脸数据库605存提取之前注册的人脸密文、相应的EK、人脸特征维度n，并进行相似度计算。计算完成的密文结果返回给用户， 用户通过加密与解密模块602，使用之前产生的私钥解密，最终得到明文验证结果。
其中人脸相似度验证模块涉及到密文内积计算，区别于已有的密文内积计算技术，通过引入人脸特征维度n，极大提升密文计算性能与吞吐量。模块逻辑如图8所示，根据接收到的人脸密文信息与ID，检索之前注册的密文人脸信息、评估密钥EK及人脸维度信息。使用同态乘法对两个人脸密文相乘，并根据从注册信息读取的人脸维度信息计算需要循环密文移位与相加的次数shiftcount，然后进行密文循环移位相加。循环计算后的结果，即为人脸相似度密文结果，解密后可得到明文验证结果。
图8为本申请实施例提供的一种人脸识别方法流程图。
如图8所示，本申请提供了一种人脸识别方法。该方法可以应用于上述人脸识别系统中。可以包括以下步骤：
S801，开始。
S802，接收已注册人脸密文信息；读取要验证人脸信息、EK与人脸维度(即维度信息)。
S803，认证人脸密文与注册人脸密文相乘。
S804，循环变量i是否小于shiftcount。若是则执行S805，否则执行S807。
S805，使用EK进行密文左移2 i步运算，i＝i+1。
S806，移位前与移位后密文相加。
S807，返回认证密文结果。
实施例一。
本申请实施例一针对单个人脸单模式验证场景，在该场景下，每个密文只保存一个人脸特征向量信息，每次验证时的输入也是一个人脸特征向量的密文。具体设定人脸特征向量维度feature_vector_dimention设置为64，slot_count设为2048，Ciphertext1’表示已注册的原始人脸特征向量密文，Ciphertext2’表示需要验证的人脸特征向量密文，在人脸特征加密为密文时，超过64的slot填充补零，因此一个密文只包含单个人脸信息。Ciphertext0表示Ciphertext1’和Ciphertext2’的密文乘积，已包含明文各个slot两两对应相乘的信息，为了得到密文内积，需要对密文态下的各个slot相加。层次型全同态算法(如CKKS，BFV)都提供了密文移位操作Rotate(ciphertext,step)运算,即可以对密文ciphertext进行移位step个slot操作，step为正向左移，step为负向右移。在本场景实施过程中，通过引入人脸特征向量维度信息feature_vector_dimention参与移位计算，相比现有技术的slot_count-1次密文移位与加法计算，实施例一算法2仅需要log2(feature_vector_dimention)-1次密文移位与加法计算。如图所示，当feature_vector_dimention为64时，只需要6次密文移位与相加，即第一次左移20个slot，并和原密文相加；第二次在第一次移位相加的结果上，左移21个slot，并和上一次的结果相加；第三次在第二次移位相加的结果上，左移22个slot，并和上一次的结果相加；第四次在第三次移位相加的结果上，左移23个slot，并和上一次的结果相加；第五次在第四次移位相加的结果上，左移24个slot，并和上一次的结果相加；第六次在第五次移位相加的结果上，左移25个slot，并和上一次的结果相加；最后的结果密文Ciphertext5中，slot1的结果即是需要的最终密文内积结果，它包含了密文人脸特征向量Ciphertext1’和Ciphertext2’相乘密文结果的slot1到slot64的和的明文信息。
具体过程可以如图9所示。
具体算法Algorithm2如下所示：
其中feature_vector1代表明文人脸特征向量1，feature_vector2代表明文人脸特征向量2，ExtendVector(feature_vector1,slot_count,0)表示把特征向量1放入由slot_count个slot组成的向量前部后，其余部分进行进行补零。Encrypt()使用公钥对向量进行加密，Multiply对密文进行乘法操作，包括了同态密文相乘必要的重线性化措施。
Figure PCTCN2022095037-appb-000009
该实施例一相比现有方式中计算密文内积需要log2(slot_count)-1次移位相加，本申请实施例一现在只需要log2(feature_vector_dimention)-1次密文循环移位与相加，当slot_count很大时，极大节省运算时间，且多项式次数越高、向量维度越小，越节省时间，从而提高密文态人脸验证的性能。
实施例二。
本申请实施例二可用于两个场景，即单个人脸多模式(指同一人在不同光照、角度等条件下采集的人脸信息)验证场景和人脸识别场景。
在单一人脸多模式验证场景下，每个密文保存同一个人的不同维度人脸特征向量信息，如同一人戴眼镜、不戴眼镜、有一定左倾斜角度的人脸信息等，即不同维度的同一人脸特征向量打包到一个密文中。针对这一实施例场景，有两个前提条件：1、feature_vector_dimention<<slot_count且feature_vector_dimention与slot_count都为2的幂；2、slot_count是feature_vector_dimention的整数倍。验证时把输入的同一个人脸特征向量重复扩展编码到一个slot_count长的明文向量中，即重复slot_count/feature_vector_dimention次。
具体实施时设定人脸特征向量维度feature_vector_dimention为64，slot_count为2048，此时一个密文可以打包32个人脸特征向量，可以在注册阶段提取不同光照、角度等条件下的人脸特征向量，并打包到同一个密文中，每个向量占据64个slot。如图所示，1到64为第一个同一人脸的特征向量，1985到2048为第32个同一人脸的特征向量。Ciphertext1’表示已注册的原始人脸特征向量密文，包含32个不同维度的同一人脸信息，如果维度不够32，可采取实施例一的方式，补零填充。Ciphertext2’表示需要验证的人脸特征向量密文，是扩 展同一人脸特征向量32次后的密文。Ciphertext0表示Ciphertext1’和Ciphertext2’的密文乘积，已包含明文各个slot两两对应相乘的信息，为了得到密文内积，需要对密文态下的各个slot相加。层次型全同态算法(如CKKS，BFV)都提供了密文移位操作Rotate(ciphertext,step)运算,即可以对密文ciphertext进行移位step个slot操作，step为正向左移，step为负向右移。在本场景实施过程中，通过引入人脸特征向量维度信息feature_vector_dimention参与移位计算，实施例二算法3仅需要log2(feature_vector_dimention)-1次密文移位与加法计算，计算量同实施例一，但可以在相同时间内得到同一人脸不同维度的验证结果，选取相似度最大的值与阈值进行比较，可以得到比较的密文结果。
具体过程可以如图10所示。
如图10所示，具体计算过程同实施例一，通过在计算中引入人脸向量的维度信息，当feature_vector_dimention为64时，只需要6次密文移位与相加，即第一次左移20个slot，并和原密文相加；第二次在第一次移位相加的结果上，左移21个slot，并和上一次的结果相加；第三次在第二次移位相加的结果上，左移22个slot，并和上一次的结果相加；第四次在第三次移位相加的结果上，左移23个slot，并和上一次的结果相加；第五次在第四次移位相加的结果上，左移24个slot，并和上一次的结果相加；第六次在第五次移位相加的结果上，左移25个slot，并和上一次的结果相加；最后的结果密文Ciphertext5中。不同于
实施例一，Ciphertext5中包含32个密文人脸向量内积的结果，分别为slot(i*64+1)，i＝0,…,31。解密后只需要取相似度最大的slot的值和阈值比较即得到人脸认证结果。
根据以上实施例二描述，具体算法Algorithm3如下所示：
其中vector1’由feature_vector1作扩展得到，是目前需要进行验证的人脸特征向量明文扩展，vector2’是使用注册时候注册的同一人的不同人脸特征向量进行填充后的明文扩展，Encrypt与Multiply功能同实施例一。
Figure PCTCN2022095037-appb-000010
在人脸识别场景下，使用同一算法Algorithm3，vector2’为不同人脸向量特征的明文 扩展，区别于单一人脸多验证模式场景，此时密文内积结果ciphertext3为需要验证的人脸和不同人脸相比较的结果，即在同实施例一相同时间内，可以同时比对N个人脸信息，极大提高了人脸密文识别吞吐量。
实施例二相对于现有方案及实施例一，可以在相同时间内(即一次密文比较运算时间)，得到N＝slot_count/feature_vector_dimention个密文比较结果，针对实施例二的场景，因为增加了针对同一人脸的比较维度，即在人脸注册时，多收集不同人脸信息并存储，比较时选取相似度最高的即可，相比现有技术和实施例一，实施例二提高了单一人脸验证的准确率。同时，针对人脸识别场景，相比现有技术，实施例二极大提高了吞吐量，吞吐量提高随N线性增长。
本申请中，在进行人脸特征提取时，可以采用预先配置好的神经网络进行特征提取，具体网络模型可以根据实际情况进行选择，本申请不做限定。
本申请主要在于使用同态加密技术对人脸信息进行同态加密与处理，特别是针对人脸密文计算相似性的过程，引入人脸特征向量维度信息，让原本需要slotcount-1次耗时密文循环计算过程压缩到log2(feature_vector_dimention)-1次循环，而feature_vector_dimention<<slotcount，(例如，一般feature_vector_dimention取64，128，而slotcount取2048，4096，8192，16384)，极大减小运算量。当feature_vector_dimention＝64,slotcount＝2048时，减少55％运算量；当feature_vector_dimention＝64,slotcount＝4096时，减少58％运算量；当feature_vector_dimention＝64,slotcount＝16384时，减少64％运算量；总体性能提升约2倍。
其次，本申请在于针对单一人脸多模式验证场景，通过选取合适的同态加密参数，在注册时将同一人不同人脸原始信息打包到一个密文存储；验证时，提取并加密当前要验证人脸信息时，将同一人脸信息重复扩展打包到一个密文中，比对时同时提取人脸特征向量维度信息并引入计算过程，提升性能，同时可提高验证成功率。例如slot_count＝4096,feature_vector_dimention＝64,使用Algorithm2时间归一化为1，则在规一化时间1内，可以得到64个人脸识别结果，吞吐量提高64倍；当slot_count＝16384时，吞吐量提高256倍。另外，相比只通过一个原始信息进行验证，因为增加了个人多模式信息(光照、角度)，因此提高验证成功率。
同时，本申请还在于，针对人脸识别场景，通过将不同人脸特征打包加密到一个密文中存储，验证时通过扩展需要验证人脸特征并打包到一个密文中和注册密文进行比对，同时引入人脸维度信息，可以提升验证效率，并同时比对N个人脸信息，极大提高了人脸密文识别吞吐量。
可以理解的是，本申请除了使用软件实施外，该方法还可以使用硬件实现。如现场可编程逻辑门阵列(field programmable gate array，FPGA)用于计算加速时，兼有数据并行与流水线并行的优点，能极大提高吞吐量。针对人脸密文识别与密文批量验证场景，采用FPGA实现计算加速，可进一步提高方案的吞吐量。
同时，还应当理解，本申请在批量人脸验证场景下，通过选取合适的同态加密参数，提取并加密当前要验证人脸信息时，将不同人脸特征信息扩展打包到一个密文中，比对时，将不同个体的人脸原始注中册信息打包到一个密文中，同时应用本发明的方法，提取人脸特征向量维度信息并引入计算过程，达到并行效果，可提高人脸密文识别吞吐量。
以及，本申请除了可用于人脸识别场景外，还可用于其它生物识别场景，如指纹识别场景，保护生物信息隐私。本申请在此不作限定。
可以理解的是，本申请中的终端设备可以包括但不限于手机、智能电视、智能音响、可穿戴设备、平板电脑、桌面型计算机、电脑一体机、手持计算机、笔记本电脑、超级移动个人计算机(ultra-mobile personal computer，UMPC)、上网本、个人数字助理(personal digitalassistant，PDA)、膝上型计算机(laptop)、移动电脑、增强现实(augmented reality，AR)设备、虚拟现实(virtual reality，VR)设备、人工智能(artificial intelligence，AI)设备和/或车载设备等任意终端设备或便携式终端设备。
终端设备与服务器可以通过有线或无线的方式相连接，其中，无线方式可以包括2G/3G/4G/5G/6G等无线通信的解决方案。或是包括无线局域网(wireless local area networks，WLAN)、蓝牙(bluetooth，BT)、全球导航卫星系统(global navigation satellite system，GNSS)、调频(frequency modulation，FM)、近距离无线通信技术(near field communication，NFC)、紫蜂(zigbee)和红外技术(infrared，IR)等无线通信的解决方案。其中，WLAN例如可以是无线保真(wireless fidelity，Wi-Fi)网络。
图11为本申请实施例提供的一种终端设备示意图。
如图11所示，本申请还提供了一种终端设备1100。该装置终端设备1100可以是上述图1至图10中所描述的终端设备100、终端设备501。该终端设备1100可以包括：处理器1110、外部存储器接口1120、内部存储器1121、通用串行总线(universal serial bus，USB)接口1130、充电管理模块1140、电源管理模块1141、电池1142、天线1、天线2、移动通信模块1150、无线通信模块1160和摄像头1170等。
可以理解的是，本申请实施例示意的结构并不构成对终端设备1100的具体限定。终端设备1100可以包括比图示更多或更少的部件，或者组合某些部件，或者拆分某些部件，或者不同的部件布置。图示的部件可以以硬件，软件或软件和硬件的组合实现。
处理器1110可以是高级精简指令集处理器(advanced reduced instruction set computing machines，ARM)、X86、无内部互锁流水级的微处理器(microprocessor without interlocked piped stages，MIPS)等架构的处理器。处理器1110可以包括一个或多个处理单元，例如：应用处理器(application processor，AP)，调制解调处理器，GPU，图像信号处理器(image signal processor，ISP)，控制器，视频编解码器，数字信号处理器(digital signal processor，DSP)，基带处理器和/或神经网络处理器(neural-network processing unit，NPU)等。其中，不同的处理单元可以是独立的器件，也可以集成在一个或多个处理器中。
控制器可以根据指令操作码和时序信号，产生操作控制信号，完成取指令和执行指令的控制。
处理器1110中还可以设置存储器，用于存储指令和数据。在一些实施例中，处理器1110中的存储器为高速缓冲存储器。该存储器可以保存处理器1110刚用过或循环使用的指令或数据。如果处理器1110需要再次使用该指令或数据，可从存储器中直接调用。避免了重复存取，减少了处理器1110的等待时间，因而提高了系统的效率。
在一些实施例中，处理器1110可以包括一个或多个接口。接口可以包括集成电路(inter-integrated circuit，I2C)接口、集成电路内置音频(inter-integrated circuit sound，I2S)接口、脉冲编码调制(pulse code modulation，PCM)接口、通用异步收发传输器(universal  asynchronous receiver/transmitter，UART)接口、移动产业处理器接口(mobile industry processor interface，MIPI)，通用输入输出(general-purpose input/output，GPIO)接口、用户标识模块(subscriber identity module，SIM)接口和/或通用串行总线(universal serial bus，USB)接口等。
充电管理模块1140用于从充电器接收充电输入。其中，充电器可以是无线充电器，也可以是有线充电器。
在一些有线充电的实施例中，充电管理模块1140可以通过USB接口1130接收有线充电器的充电输入。在一些无线充电的实施例中，充电管理模块1140可以通过终端设备1100的无线充电线圈接收无线充电输入。充电管理模块1140为电池1142充电的同时，还可以通过电源管理模块1141为终端设备1100供电。
终端设备1100的无线通信功能可以通过天线1，天线2，移动通信模块1150，无线通信模块1160，调制解调处理器以及基带处理器等实现。
移动通信模块1150可以提供应用在终端设备1100上的包括2G/3G/4G/5G/6G等无线通信的解决方案。无线通信模块1160可以提供应用在终端设备1100上的包括WLAN、BT、GNSS、FM、NFC、zigbee和IR等无线通信的解决方案。其中，WLAN例如可以是Wi-Fi网络。
可以理解的是，通过上述无线方式，终端设备1100可以和外接显示器200相连接。当然还可以采用有线的方式相连接。
终端设备1100通过GPU，显示屏1170，以及应用处理器等实现显示功能。GPU为图像处理的微处理器，连接显示屏1170和应用处理器。GPU用于执行数学和几何计算，用于图形渲染。处理器1110可包括一个或多个GPU，其执行程序指令以生成或改变显示信息。
外部存储器接口1120可以用于连接外部存储卡，例如Micro SD卡，实现扩展终端设备1100的存储能力。外部存储卡通过外部存储器接口1120与处理器1110通信，实现数据存储功能。例如将图像等文件保存在外部存储卡中。
内部存储器1121可以用于存储计算机可执行程序代码，可执行程序代码包括指令。内部存储器1121可以包括存储程序区和存储数据区。其中，存储程序区可存储操作系统，至少一个功能所需的应用程序等。存储数据区可存储终端设备1100使用过程中所创建的数据等。此外，内部存储器1121可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件，闪存器件，通用闪存存储器(universal flash storage，UFS)等。处理器1110通过运行存储在内部存储器1121的指令，和/或存储在设置于处理器中的存储器的指令，执行终端设备1100的各种功能应用以及数据处理。
摄像头1170用于捕获静态图像或视频。物体通过镜头生成光学图像投射到感光元件。感光元件可以是电荷耦合器件(charge coupled device，CCD)或互补金属氧化物半导体(complementary metal-oxide-semiconductor，CMOS)光电晶体管。感光元件把光信号转换成电信号，之后将电信号传递给ISP转换成数字图像信号。ISP将数字图像信号输出到DSP加工处理。DSP将数字图像信号转换成标准的RGB，YUV等格式的图像信号。在一些实施例中，终端设备1100可以包括1个或多个摄像头1170。
本申请所提供的终端设备1100可以实现上述图1至图10中描述的任意一种方法，具体实现方式可以参考述图1至图10的相应描述，在此不再赘述。
图12为本申请实施例提供的一种服务器示意图。
如图12所示，本申请还提供了一种服务器1200。该装置服务器1200可以是上述图1至图10中所描述的服务器100、服务器501。该服务器1200可以包括：处理器1210、外部存储器接口1220、内部存储器1221、通用串行总线(universal serial bus，USB)接口1230、充电管理模块1240、电源管理模块1241、电池1242、天线1、天线2、移动通信模块1250和无线通信模块1260等。
可以理解的是，本申请实施例示意的结构并不构成对服务器1200的具体限定。服务器1200可以包括比图示更多或更少的部件，或者组合某些部件，或者拆分某些部件，或者不同的部件布置。图示的部件可以以硬件，软件或软件和硬件的组合实现。
处理器1210可以是高级精简指令集处理器(advanced reduced instruction set computing machines，ARM)、X86、无内部互锁流水级的微处理器(microprocessor without interlocked piped stages，MIPS)等架构的处理器。处理器1210可以包括一个或多个处理单元，例如：应用处理器(application processor，AP)，调制解调处理器，GPU，图像信号处理器(image signal processor，ISP)，控制器，视频编解码器，数字信号处理器(digital signal processor，DSP)，基带处理器和/或神经网络处理器(neural-network processing unit，NPU)等。其中，不同的处理单元可以是独立的器件，也可以集成在一个或多个处理器中。
控制器可以根据指令操作码和时序信号，产生操作控制信号，完成取指令和执行指令的控制。
处理器1210中还可以设置存储器，用于存储指令和数据。在一些实施例中，处理器1210中的存储器为高速缓冲存储器。该存储器可以保存处理器1210刚用过或循环使用的指令或数据。如果处理器1210需要再次使用该指令或数据，可从存储器中直接调用。避免了重复存取，减少了处理器1210的等待时间，因而提高了系统的效率。
在一些实施例中，处理器1210可以包括一个或多个接口。接口可以包括集成电路(inter-integrated circuit，I2C)接口、集成电路内置音频(inter-integrated circuit sound，I2S)接口、脉冲编码调制(pulse code modulation，PCM)接口、通用异步收发传输器(universal asynchronous receiver/transmitter，UART)接口、移动产业处理器接口(mobile industry processor interface，MIPI)，通用输入输出(general-purpose input/output，GPIO)接口、用户标识模块(subscriber identity module，SIM)接口和/或通用串行总线(universal serial bus，USB)接口等。
充电管理模块1240用于从充电器接收充电输入。其中，充电器可以是无线充电器，也可以是有线充电器。
在一些有线充电的实施例中，充电管理模块1240可以通过USB接口1230接收有线充电器的充电输入。在一些无线充电的实施例中，充电管理模块1240可以通过服务器1200的无线充电线圈接收无线充电输入。充电管理模块1240为电池1242充电的同时，还可以通过电源管理模块1241为服务器1200供电。
服务器1200的无线通信功能可以通过天线1，天线2，移动通信模块1250，无线通信模块1260，调制解调处理器以及基带处理器等实现。
移动通信模块1250可以提供应用在服务器1200上的包括2G/3G/4G/5G/6G等无线通信的解决方案。无线通信模块1260可以提供应用在服务器1200上的包括WLAN、BT、GNSS、 FM、NFC、zigbee和IR等无线通信的解决方案。其中，WLAN例如可以是Wi-Fi网络。
可以理解的是，通过上述无线方式，服务器1200可以和外接显示器200相连接。当然还可以采用有线的方式相连接。
服务器1200通过GPU，显示屏1270，以及应用处理器等实现显示功能。GPU为图像处理的微处理器，连接显示屏1270和应用处理器。GPU用于执行数学和几何计算，用于图形渲染。处理器1210可包括一个或多个GPU，其执行程序指令以生成或改变显示信息。
外部存储器接口1220可以用于连接外部存储卡，例如Micro SD卡，实现扩展服务器1200的存储能力。外部存储卡通过外部存储器接口1220与处理器1210通信，实现数据存储功能。例如将图像等文件保存在外部存储卡中。
内部存储器1221可以用于存储计算机可执行程序代码，可执行程序代码包括指令。内部存储器1221可以包括存储程序区和存储数据区。其中，存储程序区可存储操作系统，至少一个功能所需的应用程序等。存储数据区可存储服务器1200使用过程中所创建的数据等。此外，内部存储器1221可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件，闪存器件，通用闪存存储器(universal flash storage，UFS)等。处理器1210通过运行存储在内部存储器1221的指令，和/或存储在设置于处理器中的存储器的指令，执行服务器1200的各种功能应用以及数据处理。
本申请所提供的服务器1200可以实现上述图1至图10中描述的任意一种方法，具体实现方式可以参考述图1至图10的相应描述，在此不再赘述。
本领域普通技术人员应该还可以进一步意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、计算机软件或者二者的结合来实现，为了清楚地说明硬件和软件的可互换性，在上述说明中已经按照功能一般性地描述了各示例的组成及步骤。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分步骤是可以通过程序来指令处理器完成，所述的程序可以存储于计算机可读存储介质中，所述存储介质是非短暂性(英文：non-transitory)介质，例如随机存取存储器，只读存储器，快闪存储器，硬盘，固态硬盘，磁带(英文：magnetic tape)，软盘(英文：floppy disk)，光盘(英文：optical disc)及其任意组合。
以上所述，仅为本申请较佳的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到的变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应该以权利要求的保护范围为准。

Claims (22)

  1. 一种人脸识别方法，其特征在于，所述方法应用于终端设备，所述方法包括：
    获取用户输入的登录信息，所述登录信息包括用户身份标识信息；
    采集所述用户的第一人脸图像；
    根据所述第一人脸图像获取第一人脸特征信息；
    通过预先存储的公钥采用全同态加密算法对所述第一人脸特征信息进行加密；
    将加密后的第一人脸特征信息以及预先存储的评估密钥、所述用户身份标识信息发送至服务器，其中，所述评估密钥与所述公钥为所述终端设备根据所述用户身份标识信息确定的；
    接收服务器发送的识别认证信息，其中，所述识别认证信息为密文；
    根据预先存储的私钥采用所述全同态加密算法对所述识别认证信息进行解密，以得到所述用户的人脸识别认证结果。
  2. 如权利要求1所述的方法，其特征在于，在所述采集所述用户的人脸图像之前，所述方法还包括：
    采集所述用户的第二人脸图像，所述第二人脸图像用于注册人脸信息；
    根据所述第二人脸图像获取第二人脸特征信息；
    根据所述第二人脸特性信息确定特征维度；
    根据所述公钥采用所述全同态加密算法对所述第二人脸特征信息进行加密；
    将加密后的第二人脸特征信息、所述用户身份标识信息、所述特征维度和所述评估密钥发送至服务器。
  3. 如权利要求1或2所述的方法，其特征在于，在所述采集所述用户的人脸图像之前，所述方法还包括：
    根据所述用户身份标识信息、安全强度信息、性能信息和场景信息，确定所述公钥、所述评估密钥和私钥，其中，所述公钥和所述私钥作为一对非对称加密的两个密钥，所述安全强度信息、所述性能信息和所述场景信息为预先配置或接收用户输入得到的。
  4. 如权利要求1-3任意一项所述的方法，其特征在于，所述根据预先存储的私钥采用所述全同态加密算法对所述识别认证信息进行解密，以得到所述用户的人脸识别认证结果，包括：
    根据所述私钥采用所述全同态加密算法对所述识别认证信息进行解密，确定人脸相似度信息；
    当所述人脸相似度信息大于或等于预设的相似度阈值，则确定所述用户的人脸识别认证通过；
    当所述人脸相似度信息小于所述相似度阈值，则确定所述用户的人脸识别认证未通过。
  5. 一种人脸识别方法，其特征在于，所述方法应用于服务器，所述服务器包含人脸特征数据库，所述人脸特征数据库包括至少一个注册人脸特征信息，所述方法包括：
    接收终端设备发送的加密后的第一人脸特征信息和用户身份标识信息，其中，所述加密后的第一人脸特征信息为采用全同态加密算法进行加密得到的；
    根据所述用户身份标识信息从所述人脸特征数据库中确定对应的注册人脸特征；
    采用所述加密后的第一人脸特征信息和所述用户身份标识信息对应的注册人脸特征信息进行相似度计算，确定识别认证信息，所述识别认证信息用于表示所述加密后的第一人脸特征信息和所述用户身份标识信息对应的注册人脸特征信息之间的相似度；
    将所述识别认证信息发送至所述终端设备。
  6. 如权利要求5所述的方法，其特征在于，所述方法还包括：
    接收所述终端设备发送的加密后的第二人脸特征信息、用户身份标识信息、特征维度和评估密钥；
    将所述加密后的第二人脸特征信息、所述用户身份标识信息、所述特征维度和所述评估密钥作为所述注册人脸特征信息存储至所述人脸特征数据库中。
  7. 如权利要求5或6所述的方法，其特征在于，所述注册人脸特征信息包括有加密后的第二人脸特征信息、用户身份标识信息、特征维度和评估密钥；
    所述采用所述加密后的第一人脸特征信息和所述用户身份标识信息对应的注册人脸特征信息进行相似度计算，确定识别认证信息，包括：
    对所述加密后的第一人脸特征信息和所述加密后的第二人脸特征信息采用所述评估密钥进行密文相乘；
    对密文相乘后的信息采用所述评估密钥进行密文移位，所述密文移位的位数为2^i位，其中i为当前循环次数，i为自然数，i是根据所述特征维度确定的；
    将密文移位后的信息与密文移位前的信息进行相加，以得到移位相加结果密文；
    对所述移位相加结果密文再次进行密文移位和密文相加，直至循环执行i次后，将第i次得到移位相加结果密文作为所述识别认证信息。
  8. 如权利要求5-7任意一项所述的方法，其特征在于，所述注册人脸特征信息包括加密后的第二人脸特征信息；
    所述方法还包括：
    对所述加密后的第二人脸特征信息进行扩展，确定扩展第二人脸特征信息；
    所述采用所述加密后的第一人脸特征信息和所述用户身份标识信息对应的注册人脸特征信息进行相似度计算，包括：
    采用所述加密后的第一人脸特征信息和所述扩展第二人脸特征信息进行相似度计算。
  9. 一种用于人脸识别的终端设备，其特征在于，所述终端设备包括：
    采集模块，用于获取用户输入的登录信息，所述登录信息包括用户身份标识信息；
    所述采集模块还用于，采集所述用户的第一人脸图像；
    人脸检测与特征提取模块，用于根据所述第一人脸图像获取第一人脸特征信息；
    加密与解密模块，用于通过预先存储的公钥采用全同态加密算法对所述第一人脸特征信息进行加密；
    发送模块，用于将加密后的第一人脸特征信息以及预先存储的评估密钥、所述用户身份标识信息发送至服务器，其中，所述评估密钥与所述公钥为密钥产生模块根据所述用户身份标识信息确定的；
    接收模块，用于接收服务器发送的识别认证信息，其中，所述识别认证信息为密文；
    所述加密与解密模块还用于，根据预先存储的私钥采用所述全同态加密算法对所述识 别认证信息进行解密，以得到所述用户的人脸识别认证结果。
  10. 如权利要求9所述的终端设备，其特征在于，所述采集模块还用于，采集所述用户的第二人脸图像，所述第二人脸图像用于注册人脸信息；
    所述人脸检测与特征提取模块还用于，根据所述第二人脸图像获取第二人脸特征信息；
    所述人脸检测与特征提取模块还用于，根据所述第二人脸特性信息确定特征维度；
    所述加密与解密模块还用于，根据所述公钥采用所述全同态加密算法对所述第二人脸特征信息进行加密；
    所述发送模块还用于，将加密后的第二人脸特征信息、所述用户身份标识信息、所述特征维度和所述评估密钥发送至服务器。
  11. 如权利要求9或10所述的终端设备，其特征在于，所述加密与解密模块还用于：
    根据所述用户身份标识信息、安全强度信息、性能信息和场景信息，确定所述公钥、所述评估密钥和私钥，其中，所述公钥和所述私钥作为一对非对称加密的两个密钥，所述安全强度信息、所述性能信息和所述场景信息为预先配置或所述采集模块接收用户输入得到的。
  12. 如权利要求9-11任意一项所述的终端设备，其特征在于，所述加密与解密模块还用于，根据所述私钥采用所述全同态加密算法对所述识别认证信息进行解密，确定人脸相似度信息；
    所述终端设备还包括：
    确定模块，用于当所述人脸相似度信息大于或等于预设的相似度阈值，则确定所述用户的人脸识别认证通过；以及，当所述人脸相似度信息小于所述相似度阈值，则确定所述用户的人脸识别认证未通过。
  13. 一种用于人脸识别的服务器，其特征在于，所述服务器包含人脸特征数据库，所述人脸特征数据库包括至少一个注册人脸特征信息，所述服务器包括：
    接收模块，用于接收终端设备发送的加密后的第一人脸特征信息和用户身份标识信息，其中，所述加密后的第一人脸特征信息为采用全同态加密算法进行加密得到的；
    人脸相似度验证模块，用于根据所述用户身份标识信息从所述人脸特征数据库中确定对应的注册人脸特征；
    所述人脸相似度验证模块还用于，采用所述加密后的第一人脸特征信息和所述用户身份标识信息对应的注册人脸特征信息进行相似度计算，确定识别认证信息，所述识别认证信息用于表示所述加密后的第一人脸特征信息和所述用户身份标识信息对应的注册人脸特征信息之间的相似度；
    发送模块，用于将所述识别认证信息发送至所述终端设备。
  14. 如权利要求13所述的服务器，其特征在于，所述接收模块还用于：
    接收所述终端设备发送的加密后的第二人脸特征信息、用户身份标识信息、特征维度和评估密钥；
    将所述加密后的第二人脸特征信息、所述用户身份标识信息、所述特征维度和所述评估密钥作为所述注册人脸特征信息存储至所述人脸特征数据库中。
  15. 如权利要求13或14所述的服务器，其特征在于，所述注册人脸特征信息包括有加 密后的第二人脸特征信息、用户身份标识信息、特征维度和评估密钥；
    所述人脸相似度验证模块还用于：
    对所述加密后的第一人脸特征信息和所述加密后的第二人脸特征信息采用所述评估密钥进行密文相乘；
    对密文相乘后的信息采用所述评估密钥进行密文移位，所述密文移位的位数为2^i位，其中i为当前循环次数，i为自然数，i是根据所述特征维度确定的；
    将密文移位后的信息与密文移位前的信息进行相加，以得到移位相加结果密文；
    对所述移位相加结果密文再次进行密文移位和密文相加，直至循环执行i次后，将第i次得到移位相加结果密文作为所述识别认证信息。
  16. 如权利要求13-15任意一项所述的服务器，其特征在于，所述注册人脸特征信息包括加密后的第二人脸特征信息；
    所述人脸相似度验证模块还用于：
    对所述加密后的第二人脸特征信息进行扩展，确定扩展第二人脸特征信息；
    采用所述加密后的第一人脸特征信息和所述扩展第二人脸特征信息进行相似度计算。
  17. 一种用于人脸识别的终端设备，其特征在于，所述终端设备包括：
    处理器用于与存储器耦合，以及读取并执行存储在所述存储器中的指令；
    当所述处理器运行时执行所述指令，使得所述处理器用于执行权利要求1-4任意一项所述的方法。
  18. 一种用于人脸识别的服务器，其特征在于，所述终端设备包括：
    处理器用于与存储器耦合，以及读取并执行存储在所述存储器中的指令；
    当所述处理器运行时执行所述指令，使得所述处理器用于执行权利要求5-8任意一项所述的方法。
  19. 一种计算机可读存储介质，所述计算机可读存储介质中存储有指令，其特征在于，当所述指令在终端上运行时，使得所述终端执行如权利要求1-4任意一项所述的方法。
  20. 一种计算机可读存储介质，所述计算机可读存储介质中存储有指令，其特征在于，当所述指令在服务器上运行时，使得所述服务器执行如权利要求5-8任意一项所述的方法。
  21. 一种包含指令的计算机设备，当其在终端上运行时，使得所述终端执行如权利要求1-4中的任意一项所述的方法。
  22. 一种包含指令的计算机设备，当其在服务器上运行时，使得所述服务器执行如权利要求5-8中的任意一项所述的方法。
PCT/CN2022/095037 2021-09-14 2022-05-25 一种人脸识别方法、设备及系统 WO2023040335A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111076938.8A CN115810208A (zh) 2021-09-14 2021-09-14 一种人脸识别方法、设备及系统
CN202111076938.8 2021-09-14

Publications (1)

Publication Number Publication Date
WO2023040335A1 true WO2023040335A1 (zh) 2023-03-23

Family

ID=85481619

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/095037 WO2023040335A1 (zh) 2021-09-14 2022-05-25 一种人脸识别方法、设备及系统

Country Status (2)

Country Link
CN (1) CN115810208A (zh)
WO (1) WO2023040335A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112287322A (zh) * 2020-11-23 2021-01-29 上海同态信息科技有限责任公司 一种加密人脸识别模型预加载模式
CN116383793A (zh) * 2023-04-23 2023-07-04 上海万雍科技股份有限公司 人脸数据处理方法、装置、电子设备和计算机可读介质
CN116882945A (zh) * 2023-09-05 2023-10-13 圣奥科技股份有限公司 基于办公区域工位信息的协同办公方法、设备及介质

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116305281B (zh) * 2023-03-24 2024-01-23 江苏洋井公用管廊有限公司 一种基于感官认知的人脸识别系统及人脸识别方法
CN116110159B (zh) * 2023-04-13 2023-06-23 新兴际华集团财务有限公司 基于cfca认证标准的用户认证方法、设备和介质
CN117809348A (zh) * 2023-12-26 2024-04-02 浙江汉邦瑞商信息技术有限公司 一种安防人脸比对搜索系统
CN117576763A (zh) * 2024-01-11 2024-02-20 杭州世平信息科技有限公司 云环境下基于声纹信息和人脸信息的身份识别方法及系统

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH07129085A (ja) * 1993-10-13 1995-05-19 Jonan Denki Kogyosho:Kk Ｒｓａ暗号方式の暗号化器及び復号器に使用する高速ベキ乗剰余演算法
US20160119119A1 (en) * 2014-05-15 2016-04-28 Xeror Corporation Compact fuzzy private matching using a fully-homomorphic encryption scheme
US20160182226A1 (en) * 2014-12-22 2016-06-23 Fujitsu Limited Information processing method, recording medium, and information processing apparatus
CN107819587A (zh) * 2017-12-13 2018-03-20 陈智罡 基于全同态加密的认证方法和用户设备以及认证服务器
CN109165581A (zh) * 2018-08-09 2019-01-08 广州洪荒智能科技有限公司 一种基于同态加密的隐私保护人脸识别方法
CN112200133A (zh) * 2020-10-28 2021-01-08 支付宝(杭州)信息技术有限公司 保护隐私的人脸识别方法及装置
US20210243005A1 (en) * 2018-07-04 2021-08-05 Shenzhen University Fully homomorphic encryption method and device and computer readable storage medium
CN114093001A (zh) * 2021-11-16 2022-02-25 中国电子科技集团公司第三十研究所 一种保护隐私安全的人脸识别方法

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH07129085A (ja) * 1993-10-13 1995-05-19 Jonan Denki Kogyosho:Kk Ｒｓａ暗号方式の暗号化器及び復号器に使用する高速ベキ乗剰余演算法
US20160119119A1 (en) * 2014-05-15 2016-04-28 Xeror Corporation Compact fuzzy private matching using a fully-homomorphic encryption scheme
US20160182226A1 (en) * 2014-12-22 2016-06-23 Fujitsu Limited Information processing method, recording medium, and information processing apparatus
CN107819587A (zh) * 2017-12-13 2018-03-20 陈智罡 基于全同态加密的认证方法和用户设备以及认证服务器
US20210243005A1 (en) * 2018-07-04 2021-08-05 Shenzhen University Fully homomorphic encryption method and device and computer readable storage medium
CN109165581A (zh) * 2018-08-09 2019-01-08 广州洪荒智能科技有限公司 一种基于同态加密的隐私保护人脸识别方法
CN112200133A (zh) * 2020-10-28 2021-01-08 支付宝(杭州)信息技术有限公司 保护隐私的人脸识别方法及装置
CN114093001A (zh) * 2021-11-16 2022-02-25 中国电子科技集团公司第三十研究所 一种保护隐私安全的人脸识别方法

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112287322A (zh) * 2020-11-23 2021-01-29 上海同态信息科技有限责任公司 一种加密人脸识别模型预加载模式
CN116383793A (zh) * 2023-04-23 2023-07-04 上海万雍科技股份有限公司 人脸数据处理方法、装置、电子设备和计算机可读介质
CN116383793B (zh) * 2023-04-23 2023-09-19 上海万雍科技股份有限公司 人脸数据处理方法、装置、电子设备和计算机可读介质
CN116882945A (zh) * 2023-09-05 2023-10-13 圣奥科技股份有限公司 基于办公区域工位信息的协同办公方法、设备及介质
CN116882945B (zh) * 2023-09-05 2023-12-26 圣奥科技股份有限公司 基于办公区域工位信息的协同办公方法、设备及介质

Also Published As

Publication number Publication date
CN115810208A (zh) 2023-03-17

Similar Documents

Publication Publication Date Title
WO2023040335A1 (zh) 一种人脸识别方法、设备及系统
AU2018266602B2 (en) System and method for biometric identification
CN111046365B (zh) 人脸图像传输方法、数值转移方法、装置及电子设备
CN102572314B (zh) 图像传感器以及支付认证方法
CN115336223A (zh) 经优化私人生物特征匹配
EP3079374A1 (en) Contents security method and electronic apparatus for providing contents security function
US11323254B2 (en) Device, system, and method of generating and handling cryptographic parameters
Li et al. Privacy-preserving edge-assisted image retrieval and classification in IoT
US20200272748A1 (en) Methods and apparatus for validating media content
CN105337742B (zh) 基于人脸图像特征和gps信息的lfsr文件加密及解密方法
Zhou et al. Implementation of cryptographic algorithm in dynamic QR code payment system and its performance
CN111294482B (zh) 一种图像处理方法及系统
Yan et al. SSIR: Secure similarity image retrieval in IoT
Venkatesan et al. Secure online payment through facial recognition and proxy detection with the help of TripleDES encryption
CN106161947A (zh) 用于拍摄终端的照片加密方法及系统
CN114357418A (zh) 加密认证方法、系统、终端设备、服务器及存储介质
Wang et al. Face detection for privacy protected images
WO2018090685A1 (zh) 图像数据加密方法及装置、网络摄像机
Bai et al. CryptoMask: Privacy-preserving Face Recognition
CN110190947B (zh) 信息加密、解密方法、终端及计算机可读存储介质
CN111581622A (zh) 信息处理方法、装置及电子设备
CN116311389A (zh) 指纹识别的方法和装置
CN115701017A (zh) 一种图像处理方法及装置
US20220345302A1 (en) Information matching system and information matching method
CN117544430B (zh) 智能化数据加密方法及系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22868712

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22868712

Country of ref document: EP

Kind code of ref document: A1