CN114170658B - Face recognition encryption authentication method and system combining watermarking and deep learning - Google Patents

Face recognition encryption authentication method and system combining watermarking and deep learning Download PDF

Info

Publication number
CN114170658B
CN114170658B CN202111441926.0A CN202111441926A CN114170658B CN 114170658 B CN114170658 B CN 114170658B CN 202111441926 A CN202111441926 A CN 202111441926A CN 114170658 B CN114170658 B CN 114170658B
Authority
CN
China
Prior art keywords
face
hamming distance
watermark
face feature
algorithm
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202111441926.0A
Other languages
Chinese (zh)
Other versions
CN114170658A (en
Inventor
彭长根
吴俊青
谭伟杰
蒋合领
丁红发
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guizhou University
Original Assignee
Guizhou University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guizhou University filed Critical Guizhou University
Priority to CN202111441926.0A priority Critical patent/CN114170658B/en
Publication of CN114170658A publication Critical patent/CN114170658A/en
Application granted granted Critical
Publication of CN114170658B publication Critical patent/CN114170658B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/22Matching criteria, e.g. proximity measures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00General purpose image data processing
    • G06T1/0021Image watermarking
    • G06T1/0042Fragile watermarking, e.g. so as to detect tampering
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00General purpose image data processing
    • G06T1/0021Image watermarking
    • G06T1/005Robust watermarking, e.g. average attack or collusion attack resistant
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2201/00General purpose image data processing
    • G06T2201/005Image watermarking
    • G06T2201/0065Extraction of an embedded watermark; Reliable detection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2201/00General purpose image data processing
    • G06T2201/005Image watermarking
    • G06T2201/0083Image watermarking whereby only watermarked image required at decoder, e.g. source-based, blind, oblivious

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Collating Specific Patterns (AREA)
  • Image Analysis (AREA)

Abstract

The invention relates to a face recognition encryption authentication method and a face recognition encryption authentication system combining watermarking and deep learning, which belong to the field of face recognition authentication, wherein a camera collects face images and embeds the watermarking to obtain watermarked face images; the client extracts watermark information and performs comparison authentication with the watermark image; the client side adopts a deep learning algorithm based on faceNet to extract the face characteristics of the face image which is successfully authenticated, and adopts a CKS homomorphic encryption algorithm and a national encryption SM4 algorithm to encrypt so as to obtain face characteristic ciphertext data; the database server calculates the hamming distance between the encrypted face feature template and the pre-stored face feature template, and encrypts to obtain hamming distance ciphertext data; the identity verification server decrypts the Hamming distance ciphertext data, compares the Hamming distance plaintext value with a preset Hamming distance threshold value, and obtains a face recognition authentication result, so that the safety of the source of face image data can be ensured, and the recognition accuracy and efficiency are improved.

Description

Face recognition encryption authentication method and system combining watermarking and deep learning
Technical Field
The invention relates to the field of face recognition authentication, in particular to a face recognition encryption authentication method and system combining watermarking and deep learning.
Background
The main idea of the existing face recognition encryption authentication method is to encrypt the extracted face features, and finish face recognition authentication in a ciphertext domain, wherein the authentication is successful. However, the following problems are common in the conventional face recognition: (1) safety problems. The malicious attacker can bypass the camera to directly provide the facial image with the problem for the feature extraction module or tamper the facial image in the process of sending the facial image to the feature extraction module, so that the reliability and the safety of the source of the facial image data can not be ensured; (2) problems with performance. The whole process of face feature extraction, encryption and identification authentication is long in time consumption, low in efficiency and low in identification accuracy. Therefore, a new face recognition encryption and authentication method is needed at present to solve the problem that the safety of the source of the face image data cannot be ensured and the recognition accuracy is low in the prior art.
Disclosure of Invention
The invention aims to provide a face recognition encryption authentication method and a face recognition encryption authentication system combining watermarking and deep learning, which can ensure the safety of a face image data source, improve the recognition accuracy and solve the problems that the safety of the face image data source cannot be ensured and the recognition accuracy is low in the prior art.
In order to achieve the above object, the present invention provides the following solutions:
on one hand, the invention provides a face recognition encryption authentication method combining watermarking and deep learning, which comprises the following steps:
the method comprises the steps that a camera collects face images, watermark images are embedded into the face images, watermark face images are obtained, and the watermark face images are sent to a client;
the client extracts watermark information in the watermark face image, and compares and authenticates the watermark information with the watermark image to obtain a comparison and authentication result;
the client side extracts face feature data of the face image which is successfully authenticated by adopting a deep learning algorithm based on the faceNet, encrypts the extracted face feature data by adopting a CKS (full homomorphic encryption) algorithm and a national secret SM4 algorithm to obtain face feature ciphertext data, and sends the face feature ciphertext data to a database server;
the database server generates an encrypted face feature template corresponding to the face feature ciphertext data, calculates the hamming distance between the encrypted face feature template and the face feature template of the same user prestored in the database server, encrypts the hamming distance to obtain hamming distance ciphertext data, and sends the hamming distance ciphertext data to the identity verification server;
the identity verification server decrypts the hamming distance ciphertext data to obtain a hamming distance plaintext value;
and the identity verification server compares the hamming distance plaintext value with a preset hamming distance threshold value to obtain a face recognition authentication result.
Optionally, the step of collecting the face image and embedding the watermark image into the face image to obtain the watermark face image specifically includes:
collecting a face image;
and embedding a preset watermark image into the face image by adopting an LSB blind watermark algorithm to obtain the watermark face image.
Optionally, the extracting watermark information in the watermark face image, and comparing and authenticating the watermark information with the watermark image to obtain a comparison and authentication result specifically includes:
extracting watermark information in the watermark face image by adopting an LSB watermark extraction algorithm to obtain extracted watermark information;
performing comparison authentication on the extracted watermark information and the watermark image by adopting a PSNR value comparison method, and calculating to obtain a PSNR value;
determining a comparison authentication result according to the PSNR value; when the PSNR value is equal to a preset PSNR threshold value, comparing the authentication result to be successful authentication; when the PSNR value is larger or smaller than a preset PSNR threshold value, the authentication result is authentication failure.
Optionally, the deep learning algorithm based on FaceNet is used for extracting face feature data of the face image successfully authenticated, and the CKKS homomorphic encryption algorithm and the national encryption SM4 algorithm are used for encrypting the extracted face feature data to obtain face feature ciphertext data, which specifically comprises:
extracting face feature data of the face image successfully authenticated by adopting a deep learning algorithm based on the FaceNet to obtain extracted face feature data;
encrypting the extracted face feature data by adopting a CKS full homomorphic encryption algorithm to obtain a face feature ciphertext ct;
encrypting the face characteristic ciphertext ct by adopting a national encryption SM4 algorithm to obtain the face characteristic ciphertext ct s
Optionally, the generating an encrypted face feature template corresponding to the face feature ciphertext data, calculating a hamming distance between the encrypted face feature template and a face feature template of the same user prestored in a database server, and encrypting the hamming distance to obtain hamming distance ciphertext data specifically includes:
generating a corresponding encrypted face feature template according to the face feature ciphertext data;
performing multiplication operation on the face feature ciphertext ct in the face feature ciphertext data, and calculating the hamming distance between the encrypted face feature template and the face feature template of the same user prestored in a database server;
and encrypting the Hamming distance by adopting a SM4 cryptographic algorithm to obtain Hamming distance ciphertext data.
Optionally, after the database server receives the face feature ciphertext data sent by the client and before generating the encrypted face feature template corresponding to the face feature ciphertext data, the method further includes:
the database server judges that the control instruction of the client is an execution storage function instruction or an execution identification comparison function instruction;
when the control instruction is an instruction for executing a storage function, the database server only stores the encrypted data;
when the control instruction is an instruction for executing the identification comparison function, the database server adopts an SM4 decryption algorithm to perform the facial feature ciphertext ct corresponding to the same user ID s Decrypting to obtain a face characteristic ciphertext ct so as to authorize a temporary ciphertext domain comparison function; the hamming distance calculation and the hamming distance encryption can be performed only in a state of being authorized to temporarily perform the ciphertext domain comparison function.
Optionally, the decrypting the hamming distance ciphertext data to obtain a hamming distance plaintext value specifically includes:
and decrypting the hamming distance ciphertext data by adopting a CKS full homomorphic decryption algorithm and an SM4 decryption algorithm to obtain a hamming distance plaintext value.
Optionally, the comparing the hamming distance plaintext value with a preset hamming distance threshold to obtain a face recognition authentication result specifically includes:
comparing the hamming distance plaintext value with a preset hamming distance threshold value to obtain a face recognition authentication result;
when the hamming distance plaintext value is smaller than a preset hamming distance threshold value, the face recognition authentication result is that the face recognition authentication is successful;
and when the hamming distance plaintext value is larger than or equal to a preset hamming distance threshold value, the face recognition authentication result is that the face recognition authentication fails.
Optionally, the preset hamming distance threshold is 1.242.
On the other hand, the invention also provides a human face identification encryption authentication system combining watermark and deep learning, which comprises the following steps:
the camera is used for collecting the face image, embedding the watermark image into the face image to obtain the watermark face image, and sending the watermark face image to the client;
the client is connected with the output end of the camera and used for extracting watermark information in the watermark face image, comparing and authenticating the watermark information with the watermark image to obtain a comparison and authentication result; extracting face feature data of a face image which is successfully authenticated by adopting a deep learning algorithm based on FaceNet, encrypting the extracted face feature data by adopting a CKS full homomorphic encryption algorithm and a national secret SM4 algorithm to obtain face feature ciphertext data, and transmitting the face feature ciphertext data to a database server;
the database server is connected with the output end of the client and is used for generating an encrypted face feature template corresponding to the face feature ciphertext data, calculating the Hamming distance between the encrypted face feature template and the face feature template of the same user prestored in the database server, encrypting the Hamming distance to obtain Hamming distance ciphertext data, and transmitting the Hamming distance ciphertext data to the identity verification server;
the identity verification server is respectively connected with the output end of the database server and the input end of the client and used for decrypting the hamming distance ciphertext data to obtain a hamming distance plaintext value, comparing the hamming distance plaintext value with a preset hamming distance threshold value to obtain a face recognition authentication result, and sending the face recognition authentication result to the client for display.
According to the specific embodiment provided by the invention, the invention discloses the following technical effects:
the invention provides a face recognition encryption authentication method and a face recognition encryption authentication system combining watermarking and deep learning, which are used for embedding a watermark image into a face image to generate the watermark face image, extracting watermark information on a client, comparing and authenticating with the original watermark image, judging whether the watermark face image received by the client is tampered in the transmission process according to a comparison authentication result, thereby determining whether the source of the face image data is safe and reliable, and further ensuring the reliability and safety of the source of the face image data. Watermark information is embedded and hidden in the face image and transmitted between the camera and the client, and then the watermark information is utilized to authenticate and verify the source of the face image, and only the face image which is successfully authenticated is subjected to subsequent operation, so that the security of the whole face identification encryption authentication process is ensured. The method comprises the steps of extracting and encrypting the face features in the face image successfully authenticated by a client, calculating the hamming distance between an encrypted face feature template and a face feature template of the same pre-stored user according to face feature ciphertext data in a database server, encrypting again, and ensuring the safety of the face image in the transmission process from the client to the database server to the identity verification server through the two encryption processes. Finally, decrypting the Hamming distance ciphertext data by using the identity verification server to obtain a Hamming distance plaintext value, and comparing the Hamming distance plaintext value with a preset Hamming distance threshold value to obtain a final face recognition authentication result.
The invention utilizes watermark hiding and embedding technology and multiple encryption and decryption processes, enhances the capability of resisting malicious tampering and various illegal attacks, can finish safe face recognition in a ciphertext domain, ensures the safety in face recognition encryption authentication, and avoids the occurrence of malicious tampering and illegal attacks, thereby effectively improving the recognition accuracy and solving the problems that the safety of the source of face image data cannot be ensured and the recognition accuracy is low in the prior art. Meanwhile, in the final stage of face recognition authentication, after the hamming distance is obtained through efficient calculation, the hamming distance plaintext value is obtained through encryption and decryption, the hamming distance plaintext value is compared with a preset hamming distance threshold value in value, whether the face recognition authentication passes or not can be determined by means of the fact that the two values are relatively large and small, the face recognition authentication is simpler and more convenient, and the face recognition encryption authentication efficiency can be improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art. The following drawings are not intended to be drawn to scale, emphasis instead being placed upon illustrating the principles of the invention.
Fig. 1 is a flowchart of a face recognition encryption authentication method combining watermark and deep learning provided in embodiment 1 of the present invention;
fig. 2 is a schematic diagram of a face recognition encryption authentication method combining watermarking and deep learning provided in embodiment 1 of the present invention;
fig. 3 is a schematic diagram of the LSB blind watermarking algorithm embedded watermark image according to embodiment 1 of the present invention;
fig. 4 is a schematic diagram of extracting face feature data by using the deep learning algorithm based on FaceNet provided in embodiment 1 of the present invention;
fig. 5 is a schematic diagram of the national cipher SM4 algorithm provided in embodiment 1 of the present invention;
fig. 6 is a block diagram of a face recognition encryption authentication system combining watermarking and deep learning according to embodiment 2 of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
As used in the specification and in the claims, the terms "a," "an," "the," and/or "the" are not specific to a singular, but may include a plurality, unless the context clearly dictates otherwise. In general, the terms "comprises" and "comprising" merely indicate that the steps and elements are explicitly identified, and they do not constitute an exclusive list, as other steps or elements may be included in a method or apparatus.
Although the present invention makes various references to certain modules in a system according to embodiments of the present invention, any number of different modules may be used and run on a user terminal and/or server. The modules are merely illustrative, and different aspects of the systems and methods may use different modules.
A flowchart is used in the present invention to describe the operations performed by a system according to embodiments of the present invention. It should be understood that the preceding or following operations are not necessarily performed in order precisely. Rather, the various steps may be processed in reverse order or simultaneously, as desired. Also, other operations may be added to or removed from these processes.
In the process of face feature data transmission, storage and identification comparison, the privacy of a user is at risk of being revealed, maliciously tampered and attacked, and the face feature information is prevented from being stolen and abused. The invention aims to provide a face recognition encryption authentication method and a face recognition encryption authentication system combining watermark and deep learning, which adopt watermark embedding authentication technology, deep learning algorithm, homomorphic encryption and national encryption technology to ensure the reliability of data sources and enable the data sources to have the capability of resisting various attacks, improve the safety without affecting the face recognition performance almost, and can solve the problems that the safety of the face image data sources cannot be ensured and the recognition accuracy is low in the prior art.
In order that the above-recited objects, features and advantages of the present invention will become more readily apparent, a more particular description of the invention will be rendered by reference to the appended drawings and appended detailed description.
Example 1
As shown in fig. 1, the embodiment provides a face recognition encryption authentication method combining watermarking and deep learning, which is based on hardware devices such as a camera, a client, a database server and an identity verification server for face recognition authentication, wherein the camera, the client, the database server and the identity verification server are electrically connected in sequence, the client is also electrically connected with the identity verification server directly, the identity verification server can send a face recognition authentication result to the client, the client refers to the face recognition authentication client with a display screen, and the face recognition authentication result is displayed through the display screen.
As shown in fig. 1 and 2, the method specifically includes the following steps:
s1, acquiring a face image by using a camera, embedding a watermark image into the face image to obtain a watermark face image, and sending the watermark face image to a client. The method specifically comprises the following steps:
s1.1, shooting a face by using a camera, and collecting a face image;
s1.2, embedding a preset watermark image into the face image by adopting an LSB (Least Significant Bit ) blind watermark algorithm to obtain the watermark face image;
as shown in fig. 3, the LSB blind watermarking algorithm changes the internal pixels of the face image by the bits of watermark information, and although the number of the embedded grid takes up 8 bytes, it needs to change 1 to 4 minimum bits according to the embedded information. The watermark information is typically hidden by modifying only 50% of the number of bits in the face image. Because the watermark image quality is low and is smaller than the least significant bit of 4 bits, changing the least significant bit of the pixels causes little change of the color intensity of the image and cannot be perceived by a human visual system, so the LSB blind watermark authentication algorithm has good invisibility. Therefore, the invention can effectively embed and hide watermark information into the face image by utilizing the LSB blind watermark algorithm.
It should be noted that, in this embodiment, the LSB blind watermarking algorithm is preferred, and a DCT (Discrete Cosine Transform ) algorithm or a DWT (Discrete Wavelet Transform, discrete wavelet transform) algorithm may be used instead of the LSB blind watermarking algorithm to hide the watermark into the face image, so that the purpose of hiding the watermark into the face image can be achieved as well.
And S1.3, sending the watermark face image to the client through the camera.
It should be noted that, the camera in this embodiment actually refers to an image capturing device, and is not limited to a photographing camera on mobile devices such as a mobile phone, a camera, and a video camera, but also includes an independent camera, for example, an independent camera used in airport entrance security inspection, and may also be any type of image capturing sensor. Therefore, any kind and type of image acquisition device should be included and should fall within the scope of the present invention.
And S2, extracting watermark information in the watermark face image by using the client, and comparing and authenticating the watermark information with the watermark image to obtain a comparison and authentication result. The method specifically comprises the following steps:
step S2.1, after the client receives the watermark face image sent by the camera, extracting watermark information in the watermark face image by adopting an LSB watermark extraction algorithm to obtain extracted watermark information;
and S2.2, comparing and authenticating the extracted watermark information with the watermark image by adopting a PSNR (Peak Signal to Noise Ratio ) value comparison method, and calculating to obtain a PSNR value. The calculation formula of the PSNR value is expressed as:
wherein W, H represents the width and height of the watermark image, i is equal to or less than 1 and equal to or less than W, j is equal to or less than 1 and equal to or less than H, X (i, j) represents the pixel value of the (i, j) point in the extracted watermark image, Y (i, j) represents the pixel value of the (i, j) point of the preset watermark image, and n represents the number of pixel bits.
S2.3, determining a comparison authentication result according to the PSNR value, wherein the comparison authentication result comprises authentication success and authentication failure, and is specific:
when the PSNR value is equal to a preset PSNR threshold value, comparing the authentication result to be successful authentication, and indicating that the source of the face image data is safe;
when the PSNR value is not equal to the preset PSNR threshold value, the comparison authentication result is authentication failure, and the fact that the source of the face image data is unsafe is indicated.
In the invention, a PSNR value, namely a peak signal-to-noise ratio value, is taken as a comparison method, a PSNR threshold value is preset to be 100 in the embodiment, and authentication success is indicated when the PSNR value is 100, so that the source of the face image data is safe; otherwise, the source of the face image data is doubtful, and authentication fails.
The invention embeds the watermark image into the face image to generate the watermark face image, introduces LSB blind watermark authentication technology on the client to ensure the reliability and safety of the source of the face image, extracts watermark information by using LSB blind watermark authentication algorithm, performs comparison authentication with the original watermark image, and judges whether the watermark face image received by the client is tampered in the transmission process according to the comparison authentication result, thereby determining whether the source of the face image data is safe and reliable. In addition, the adopted LSB blind watermark authentication algorithm has good invisibility, the peak signal-to-noise ratio values of the face images before and after embedding the watermark are above 48dB and are far greater than the limit of 30dB visible to human eyes, so that data tampering and 'replay' attack are effectively prevented, and the reliability and the safety of the face image data source can be ensured.
And S3, the client side extracts face feature data of the face image which is successfully authenticated by adopting a deep learning algorithm based on the faceNet, encrypts the extracted face feature data to obtain face feature ciphertext data, and sends the face feature ciphertext data to a database server.
As shown in fig. 4, the architecture initial part of the deep learning algorithm based on FaceNet is a batch input layer and deep CNN (Convolutional Neural Network ) architecture, then the unit vector is normalized by using an L2 paradigm to obtain an embedding face feature vector, training is performed by using a triplet loss function, all face feature vectors are mapped to an hypersphere through the triplet loss function, the feature distance between the same identity is as small as possible, and the feature distance between different identities is as large as possible, so that the distinction between different identities is completed, the comparison between face features extracted from different face images in the follow-up process is facilitated, and the face recognition authentication is realized.
The step S3 of the invention specifically comprises the following steps:
s3.1, the client side extracts face feature data of the face image successfully authenticated by adopting a deep learning algorithm based on the FaceNet to obtain the extracted face feature data;
step S3.2, a public key pk, a private key sk and a calculation key evk are generated by adopting a CKS full homomorphic encryption algorithm, and the extracted face characteristic data is encrypted by utilizing the public key pk to obtain a face characteristic ciphertext ct;
step S3.3, setting a key MK by adopting a national encryption SM4 algorithm, and further encrypting the face characteristic ciphertext ct by using the key MK to obtain the face characteristic ciphertext ct s
The encryption algorithm adopted in the embodiment is a combined framework of a CKS full homomorphic encryption algorithm and a national encryption SM4 algorithm, the corresponding decryption algorithm is a CKS full homomorphic decryption algorithm and an SM4 decryption algorithm, and the security and the anti-attack capability in the face recognition authentication process are effectively improved by utilizing the ciphertext domain operation property of the CKS full homomorphic encryption algorithm and combining the anti-attack capability of the national encryption SM4 algorithm.
The architecture diagram of the SM4 cryptographic algorithm is shown in fig. 5, wherein X is the input face characteristic ciphertext ct; y is the face characteristic ciphertext ct of the output s ;rk i A round key representing an encryption algorithm; τ represents the nonlinear transformation of parallel S-boxes, and the input of a certain round of nonlinear transformation τOutput->The transformation expression is:
(b 0 ,b 1 ,b 2 ,b 3 )=τ(A)=(Sbox(a 0 ),Sbox(a 1 ),Sbox(a 2 ),Sbox(a 3 ))
wherein Sbox represents S box replacement operation in SM4 cryptographic algorithm, and input a of each S box i Is an 8-bit byte, the 16-ary number corresponding to the first four bits of the 8-bit byte is used as a row number, the 16-ary number corresponding to the last four bits is used as a column number, and then the byte in the corresponding position is used to replace the input byte.
The input of the linear transformation L is the output of the nonlinear transformation tauThe output of the linear transformation L isWhere Z represents a set of 32-bit bytes, and the transform expression of the linear transform L is:
where C represents the output of the linear transformation L, B represents the input of the linear transformation L (also the output of the nonlinear transformation τ),for concatenating symbols, "<" means binary unsigned left shift, e.g., "< 2" means binary unsigned left shift by 2 bits.
The application of the Chinese secret SM4 algorithm overcomes the defects of the identical-state CKS encryption algorithm, further enhances the capability of resisting malicious tampering and various attacks, such as key recovery attacks, fake attacks, side channel attacks, guessing determination attacks, linear attacks, multidimensional linear attacks, differential attacks, impossible differential attacks and the like, and ensures the authentication performance while greatly improving the security.
It should be further noted that, in the present embodiment, the cryptographic SM4 algorithm may be a cryptographic algorithm of the ancestral process instead of the cryptographic SM4 algorithm, that is, a combination frame of "CKKS homomorphic cryptographic algorithm+cryptographic algorithm of the ancestral process" may also be used, and then the cryptographic algorithm of the ancestral process is used correspondingly during decryption. It should be noted that, the above various encryption algorithms and decryption algorithms are all in the prior art, and specific processes are not described herein.
Step S3.4, the face characteristic ciphertext ct s The calculation key evk and the user ID are transmitted to the database server as face feature ciphertext data.
And S4, the database server generates an encrypted face feature template corresponding to the face feature ciphertext data, calculates the Hamming distance between the encrypted face feature template and the face feature template of the same user prestored in the database server, encrypts the Hamming distance to obtain Hamming distance ciphertext data, and sends the Hamming distance ciphertext data to the identity verification server. The method specifically comprises the following steps:
step S4.1, receiving a ciphertext ct which is sent by a client and contains the face characteristics at a database server s After calculating the key evk and the face feature ciphertext data of the user ID, the database server determines, according to the type of the control instruction of the client, whether the database server should execute the storage function or the identification comparison function, and specifically includes the following two cases:
(1) When the control instruction is an instruction for executing a storage function, the database server only stores the encrypted data and does not perform identification comparison operation;
(2) When the control instruction is an instruction for executing the identification comparison function, the database server adopts an SM4 decryption algorithm to perform the facial feature ciphertext ct corresponding to the same user ID s And decrypting to obtain a face characteristic ciphertext ct so as to authorize a temporary ciphertext domain comparison function. In a state where the temporary ciphertext domain comparison function is authorized, the hamming distance calculation, the hamming distance encryption, and the like can be performed. The ciphertext domain comparison function is derived from the property of allowing addition and multiplication operations to be performed in the ciphertext domain in the CKKS isomorphic encryption algorithm, and the principle is that for any operation f and plaintext x, the formula is satisfied: f (Enc (x))=enc (f (x)).
Because of the fully homomorphic encryption characteristic of the CKS fully homomorphic encryption algorithm, after encrypting the face features, the database server can perform multiplication operation on the face feature ciphertext ct in the face feature ciphertext data under the state of authorizing the temporary ciphertext domain comparison function, and the hamming distance between the encrypted face feature template and the face feature template of the same user prestored in the database server is calculated.
According to the invention, the ciphertext domain comparison function is subjected to authority grant and release, and the subsequent steps of Hamming distance calculation, encryption and decryption, size comparison between the Hamming distance plaintext value and the preset Hamming distance threshold value and the like can be continued only after the ciphertext domain comparison function is subjected to temporary authorization to obtain the use permission of the ciphertext domain comparison function, so that the safety in the face recognition authentication process is effectively ensured, illegal program invasion and unauthorized recognition authentication operation are prevented, and the reliability and accuracy of the face recognition authentication result are further improved.
S4.2, generating a corresponding encrypted face feature template according to the face feature ciphertext data; because the face feature ciphertext data contains a plurality of face features, the face features can form a complete face, and the face formed by the face features is an encrypted face feature template.
And S4.3, performing multiplication operation on the face feature ciphertext ct in the face feature ciphertext data according to the calculation key evk, and calculating to obtain the hamming distance between the encrypted face feature template and the face feature template of the same user prestored in the database server, so as to complete recognition and comparison of the face features.
A large number of face feature templates, namely a large number of face images containing face features, are prestored in the database server, and because the face feature ciphertext data sent to the database server by the client contains user ID information, the Hamming distance between the encrypted face feature templates of the user in real time and the face feature templates of the same user prestored in the database server is calculated and compared, so that whether the current user is the same person as the face feature templates stored in the database server is identified.
Besides, the method can compare the real-time encrypted face feature template of the user with each face feature template pre-stored in the database server, calculate the hamming distance between the encrypted face feature template and each face feature template pre-stored in the database server, so as to obtain a plurality of hamming distances, and then compare the hamming distances with preset hamming distance thresholds after encryption and decryption, so as to determine the similarity between the face image of the current user and each face image, and further identify the identity of the current user.
And S4.4, encrypting the Hamming distance by adopting a SM4 cryptographic algorithm to obtain Hamming distance ciphertext data, and transmitting the Hamming distance ciphertext data to an identity authentication server. The encrypted hamming distance ciphertext data is sent to an identity verification server, and ciphertext data in a database server is converted into new ciphertext data due to the key evolution characteristic of the SM4 cryptographic algorithm.
And S5, decrypting the hamming distance ciphertext data by using the identity verification server to obtain a hamming distance plaintext value. The method specifically comprises the following steps:
after the identity verification server receives the hamming distance ciphertext data sent by the database server, the hamming distance ciphertext data is decrypted by adopting a CKS homomorphic decryption algorithm and an SM4 decryption algorithm, and a hamming distance plaintext value is obtained.
Because the cryptographic SM4 algorithm in this embodiment may be replaced by the cryptographic algorithm of the ancestor, the Hamming distance ciphertext data may be decrypted by adopting the "CKS homomorphic decryption algorithm and the SM4 decryption algorithm" or the "CKS homomorphic decryption algorithm and the" ancestor decryption algorithm "corresponding to the cryptographic algorithm according to the cryptographic algorithm of the cryptographic SM4 algorithm or the cryptographic algorithm of the ancestor actually adopted, so as to obtain the Hamming distance plaintext value. When the CKKS homomorphic decryption algorithm is used for decryption, the private key sk that is generated in advance in step S3.2 by the CKKS homomorphic encryption algorithm and corresponds to the public key pk is used for decrypting the hamming distance ciphertext data.
And S6, comparing the hamming distance plaintext value with a preset hamming distance threshold value through the identity verification server to obtain a face recognition authentication result. The method specifically comprises the following steps:
step S6.1, the identity verification server compares the hamming distance plaintext value with a preset hamming distance threshold value to obtain the following two face recognition authentication results:
(1) When the hamming distance plaintext value is smaller than a preset hamming distance threshold value, the face recognition authentication result is that the face recognition authentication is successful;
(2) When the hamming distance plaintext value is greater than or equal to a preset hamming distance threshold, the face recognition authentication result is that the face recognition authentication fails;
and S6.2, the identity verification server sends the face recognition authentication result to the client, and the client is utilized to display the face recognition authentication result.
In this embodiment, when the preset hamming distance threshold is 1.242 and the hamming distance plaintext is smaller than 1.242, the face recognition authentication is successful, otherwise the authentication fails. It is easy to understand that the preset hamming distance threshold is only a preferred value, not fixed and unique, but can also be other values, and can be set by itself according to practical situations.
The invention provides a face recognition encryption authentication method and a face recognition encryption authentication system combining watermarking and deep learning, which ensure the reliability and the safety of a face image source by introducing an LSB blind watermarking authentication technology, efficiently extract face feature data by adopting a deep learning algorithm based on FaceNet, and effectively coordinate biological feature ambiguity and the accuracy of a password system; the combination framework of the CKS homomorphic encryption algorithm and the national secret SM4 algorithm is adopted, the safe face recognition is completed in the ciphertext domain by utilizing the ciphertext domain operation property of the CKS homomorphic encryption algorithm and the anti-attack capability of the national secret SM4 algorithm, the national secret SM4 algorithm can make up for the defect of the CKS homomorphic encryption algorithm in the aspect of attack resistance, thereby further enhancing the capability of resisting malicious tampering and various attacks, effectively improving the security, simultaneously guaranteeing the authentication performance, taking only 2.4185s in the whole process, enabling the size of a single encrypted face feature template to be only 65KB, enabling the recognition accuracy to reach 99.64%, and effectively guaranteeing the security and recognition accuracy in face recognition authentication.
Example 2
As shown in fig. 6, the present embodiment provides a face recognition encryption authentication system combining watermarking and deep learning, which specifically includes:
the camera M1 is used for collecting a face image, embedding a watermark image into the face image to obtain a watermark face image, and sending the watermark face image to the client;
the client M2 is connected with the output end of the camera and is used for extracting watermark information in the watermark face image, and comparing and authenticating the watermark information with the watermark image to obtain a comparison and authentication result; extracting face feature data of a face image which is successfully authenticated by adopting a deep learning algorithm based on FaceNet, encrypting the extracted face feature data by adopting a CKS full homomorphic encryption algorithm and a national secret SM4 algorithm to obtain face feature ciphertext data, and transmitting the face feature ciphertext data to a database server;
the database server M3 is connected with the output end of the client and is used for generating an encrypted face feature template corresponding to the face feature ciphertext data, calculating the Hamming distance between the encrypted face feature template and the face feature template of the same user prestored in the database server, encrypting the Hamming distance to obtain Hamming distance ciphertext data, and transmitting the Hamming distance ciphertext data to the identity verification server;
the identity verification server M4 is respectively connected with the output end of the database server and the input end of the client and used for decrypting the Hamming distance ciphertext data to obtain a Hamming distance plaintext value, comparing the Hamming distance plaintext value with a preset Hamming distance threshold value to obtain a face recognition authentication result, and sending the face recognition authentication result to the client for display.
According to the face recognition encryption authentication method and system combining watermarking and deep learning, the LSB blind watermarking authentication algorithm is introduced to ensure the reliability and the safety of the source of face image data in consideration of the safety problem, and the national encryption SM4 algorithm is adopted to combine with the CKS homomorphic encryption algorithm to enhance the anti-attack capability in the transmitting, storing and comparing processes. The method has the advantages that the method greatly ensures the performance of the method, and can effectively improve the face recognition accuracy and the recognition authentication efficiency.
Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
The foregoing is illustrative of the present invention and is not to be construed as limiting thereof. Although a few exemplary embodiments of this invention have been described, those skilled in the art will readily appreciate that many modifications are possible in the exemplary embodiments without materially departing from the novel teachings and advantages of this invention. Accordingly, all such modifications are intended to be included within the scope of this invention as defined in the following claims. It is to be understood that the foregoing is illustrative of the present invention and is not to be construed as limited to the specific embodiments disclosed, and that modifications to the disclosed embodiments, as well as other embodiments, are intended to be included within the scope of the appended claims. The invention is defined by the claims and their equivalents.

Claims (10)

1. The face recognition encryption authentication method combining the watermarking and the deep learning is characterized by comprising the following steps of:
the method comprises the steps that a camera collects face images, watermark images are embedded into the face images, watermark face images are obtained, and the watermark face images are sent to a client;
the client extracts watermark information in the watermark face image, and compares and authenticates the watermark information with the watermark image to obtain a comparison and authentication result;
the client side extracts face feature data of the face image which is successfully authenticated by adopting a deep learning algorithm based on the faceNet, encrypts the extracted face feature data by adopting a CKS (full homomorphic encryption) algorithm and a national secret SM4 algorithm to obtain face feature ciphertext data, and sends the face feature ciphertext data to a database server;
the database server generates an encrypted face feature template corresponding to the face feature ciphertext data, calculates the hamming distance between the encrypted face feature template and the face feature template of the same user prestored in the database server, encrypts the hamming distance to obtain hamming distance ciphertext data, and sends the hamming distance ciphertext data to the identity verification server;
the identity verification server decrypts the hamming distance ciphertext data to obtain a hamming distance plaintext value;
and the identity verification server compares the hamming distance plaintext value with a preset hamming distance threshold value to obtain a face recognition authentication result.
2. The face recognition encryption authentication method combining watermarking and deep learning according to claim 1, wherein the steps of collecting a face image, and embedding the watermark image into the face image to obtain the watermark face image comprise:
collecting a face image;
and embedding a preset watermark image into the face image by adopting an LSB blind watermark algorithm to obtain the watermark face image.
3. The method for encrypting and authenticating face recognition by combining watermarking and deep learning according to claim 1, wherein the steps of extracting watermark information from a watermark face image, comparing and authenticating the watermark information with the watermark image to obtain a comparison and authentication result comprise:
extracting watermark information in the watermark face image by adopting an LSB watermark extraction algorithm to obtain extracted watermark information;
performing comparison authentication on the extracted watermark information and the watermark image by adopting a PSNR value comparison method, and calculating to obtain a PSNR value;
determining a comparison authentication result according to the PSNR value; when the PSNR value is equal to a preset PSNR threshold value, comparing the authentication result to be successful authentication; when the PSNR value is larger or smaller than a preset PSNR threshold value, the authentication result is authentication failure.
4. The method for encrypting and authenticating human face by combining watermark and deep learning according to claim 1, wherein the deep learning algorithm based on FaceNet is adopted to extract the human face feature data of the successfully authenticated human face image, and the CKS homomorphic encryption algorithm and the national secret SM4 algorithm are adopted to encrypt the extracted human face feature data to obtain human face feature ciphertext data, and the method specifically comprises the following steps:
extracting face feature data of the face image successfully authenticated by adopting a deep learning algorithm based on the FaceNet to obtain extracted face feature data;
encrypting the extracted face feature data by adopting a CKS full homomorphic encryption algorithm to obtain a face feature ciphertext ct;
encrypting the face characteristic ciphertext ct by adopting a national encryption SM4 algorithm to obtain the face characteristic ciphertext ct s
5. The method for encrypting and authenticating face recognition by combining watermark and deep learning according to claim 4, wherein generating an encrypted face feature template corresponding to the face feature ciphertext data, calculating a hamming distance between the encrypted face feature template and a face feature template of the same user prestored in a database server, and encrypting the hamming distance to obtain hamming distance ciphertext data, comprises:
generating a corresponding encrypted face feature template according to the face feature ciphertext data;
performing multiplication operation on the face feature ciphertext ct in the face feature ciphertext data, and calculating the hamming distance between the encrypted face feature template and the face feature template of the same user prestored in a database server;
and encrypting the Hamming distance by adopting a SM4 cryptographic algorithm to obtain Hamming distance ciphertext data.
6. The method for encrypting authentication for face recognition by combining watermarking and deep learning according to claim 5, wherein after the database server receives the face feature ciphertext data sent by the client and before generating the encrypted face feature template corresponding to the face feature ciphertext data, further comprising:
the database server judges that the control instruction of the client is an execution storage function instruction or an execution identification comparison function instruction;
when the control instruction is an instruction for executing a storage function, the database server only stores the encrypted data;
when the control instruction is an instruction for executing the identification comparison function, the database server adopts an SM4 decryption algorithm to perform the facial feature ciphertext ct corresponding to the same user ID s Decrypting to obtain a face characteristic ciphertext ct so as to authorize a temporary ciphertext domain comparison function; the hamming distance calculation and the hamming distance encryption can be performed only in a state of being authorized to temporarily perform the ciphertext domain comparison function.
7. The face recognition encryption authentication method combining watermarking and deep learning according to claim 1, wherein the decrypting the hamming distance ciphertext data to obtain a hamming distance plaintext value specifically comprises:
and decrypting the hamming distance ciphertext data by adopting a CKS full homomorphic decryption algorithm and an SM4 decryption algorithm to obtain a hamming distance plaintext value.
8. The method for encrypting and authenticating human face recognition by combining watermarking and deep learning according to claim 1, wherein the comparing the hamming distance plaintext value with a preset hamming distance threshold value to obtain the human face recognition authentication result specifically comprises:
comparing the hamming distance plaintext value with a preset hamming distance threshold value to obtain a face recognition authentication result;
when the hamming distance plaintext value is smaller than a preset hamming distance threshold value, the face recognition authentication result is that the face recognition authentication is successful;
and when the hamming distance plaintext value is larger than or equal to a preset hamming distance threshold value, the face recognition authentication result is that the face recognition authentication fails.
9. The face recognition encryption authentication method combining watermarking and deep learning according to claim 1, wherein the preset hamming distance threshold is 1.242.
10. A face recognition encryption authentication system combining watermarking and deep learning, comprising:
the camera is used for collecting the face image, embedding the watermark image into the face image to obtain the watermark face image, and sending the watermark face image to the client;
the client is connected with the output end of the camera and used for extracting watermark information in the watermark face image, comparing and authenticating the watermark information with the watermark image to obtain a comparison and authentication result; extracting face feature data of a face image which is successfully authenticated by adopting a deep learning algorithm based on FaceNet, encrypting the extracted face feature data by adopting a CKS full homomorphic encryption algorithm and a national secret SM4 algorithm to obtain face feature ciphertext data, and transmitting the face feature ciphertext data to a database server;
the database server is connected with the output end of the client and is used for generating an encrypted face feature template corresponding to the face feature ciphertext data, calculating the Hamming distance between the encrypted face feature template and a face feature template prestored in the database server, encrypting the Hamming distance to obtain Hamming distance ciphertext data, and transmitting the Hamming distance ciphertext data to the identity verification server;
the identity verification server is respectively connected with the output end of the database server and the input end of the client and used for decrypting the hamming distance ciphertext data to obtain a hamming distance plaintext value, comparing the hamming distance plaintext value with a preset hamming distance threshold value to obtain a face recognition authentication result, and sending the face recognition authentication result to the client for display.
CN202111441926.0A 2021-11-30 2021-11-30 Face recognition encryption authentication method and system combining watermarking and deep learning Active CN114170658B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111441926.0A CN114170658B (en) 2021-11-30 2021-11-30 Face recognition encryption authentication method and system combining watermarking and deep learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111441926.0A CN114170658B (en) 2021-11-30 2021-11-30 Face recognition encryption authentication method and system combining watermarking and deep learning

Publications (2)

Publication Number Publication Date
CN114170658A CN114170658A (en) 2022-03-11
CN114170658B true CN114170658B (en) 2024-02-27

Family

ID=80481651

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111441926.0A Active CN114170658B (en) 2021-11-30 2021-11-30 Face recognition encryption authentication method and system combining watermarking and deep learning

Country Status (1)

Country Link
CN (1) CN114170658B (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114630130B (en) * 2022-03-17 2024-05-03 北京影数科技有限公司 Face-changing video tracing method and system based on deep learning
CN114785597A (en) * 2022-04-22 2022-07-22 贵州爱信诺航天信息有限公司 Domestic password-based personal authentication method, system and storage medium
CN115277972B (en) * 2022-07-29 2023-09-26 西安电子科技大学 Image information hiding method based on cryptographic algorithm
CN116383793B (en) * 2023-04-23 2023-09-19 上海万雍科技股份有限公司 Face data processing method, device, electronic equipment and computer readable medium
CN116582281B (en) * 2023-07-10 2023-09-22 中国人民解放军国防科技大学 Safe face recognition method, system and equipment based on password technology

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105323209A (en) * 2014-06-05 2016-02-10 江苏博智软件科技有限公司 Cloud data security protection method adopting fully homomorphic encryption technology and multiple digital watermarking technology
CN105631296A (en) * 2015-12-30 2016-06-01 北京工业大学 Design method of safety face verification system based on CNN (convolutional neural network) feature extractor
WO2021182683A1 (en) * 2020-03-09 2021-09-16 주식회사 퍼즐에이아이 Voice authentication system into which watermark is inserted, and method therefor

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105323209A (en) * 2014-06-05 2016-02-10 江苏博智软件科技有限公司 Cloud data security protection method adopting fully homomorphic encryption technology and multiple digital watermarking technology
CN105631296A (en) * 2015-12-30 2016-06-01 北京工业大学 Design method of safety face verification system based on CNN (convolutional neural network) feature extractor
WO2021182683A1 (en) * 2020-03-09 2021-09-16 주식회사 퍼즐에이아이 Voice authentication system into which watermark is inserted, and method therefor

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
基于高效隐秘汉明距离计算的安全人脸识别;刘妍;金鑫;赵耿;李晓东;陈迎亚;郭魁;;计算机工程与设计;20160916(第09期);全文 *

Also Published As

Publication number Publication date
CN114170658A (en) 2022-03-11

Similar Documents

Publication Publication Date Title
CN114170658B (en) Face recognition encryption authentication method and system combining watermarking and deep learning
CN101057448B (en) Securely computing a similarity measure
Barman et al. Fingerprint-based crypto-biometric system for network security
Thanh et al. An image zero-watermarking algorithm based on the encryption of visual map feature with watermark information
US20020056043A1 (en) Method and apparatus for securely transmitting and authenticating biometric data over a network
KR20080002244A (en) Watermarking method, watermark validation method, watermarking device and watermark validation device
Najih et al. An improved secure image hiding technique using PN-sequence based on DCT-OTP
Komninos et al. Protecting biometric templates with image watermarking techniques
Mehndiratta Data hiding system using cryptography & steganography: a comprehensive modern investigation
Selvaraju et al. A method to improve the security level of ATM banking systems using AES algorithm
CN114090994A (en) Face recognition authentication method and system based on block chain
Al-Assam et al. Combining steganography and biometric cryptosystems for secure mutual authentication and key exchange
KR101010218B1 (en) Biometric authentication method
CN110580405A (en) Picture copyright protection system and method based on social network
CN112651007B (en) Threshold predicate encryption biological characteristic authentication method based on digital watermark
Abiega-L’Eglisse et al. A new fuzzy vault based biometric system robust to brute-force attack
CN114782238A (en) Image self-authentication method based on hash function and information hiding
Khan Transmission of Secure Biometric Data for Network-based User Authentication
Barbier et al. Image watermarking with biometric data for copyright protection
Panchal Bio-Crypto System
Hoang et al. Bit priority-based biometric watermarking
Joshi et al. An Improved Commutative Reversible Watermarking and Encryption for Fingerprint Image
Halvi et al. A robust and secured cloud based distributed biometric system using symmetric key cryptography and microsoft cognitive API
Nagm et al. A novel watermarking approach for protecting image integrity based on a hybrid security technique
Ingle et al. Using Advanced Encryption Standard (AES) Algorithm Upgrade the Security Level of ATM Banking Systems

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant