CN117473543A

CN117473543A - Privacy protection method and system in face recognition authentication

Info

Publication number: CN117473543A
Application number: CN202311139790.7A
Authority: CN
Inventors: 张明扬; 吴若凡; 郝秀全; 刘腾飞; 张天翼; 王维强
Original assignee: Alipay Hangzhou Information Technology Co Ltd
Current assignee: Alipay Hangzhou Information Technology Co Ltd
Priority date: 2023-09-05
Filing date: 2023-09-05
Publication date: 2024-01-30

Abstract

The embodiment of the specification discloses a privacy protection method in face recognition authentication, which is applied to a server side and comprises the following steps: responding to a face identification authentication request of a client, and acquiring an uploaded random vector of a face to be identified and first encryption information; the face random vector to be identified is obtained after the face feature to be identified is transformed through a disposable first random integer vector, and the first encryption information is obtained by encrypting the first random integer vector through a preset encryption algorithm; retrieving a target face random vector and second encryption information which are stored in advance; the target face random vector is obtained by transforming the target face features through a disposable second random integer vector, and the second random integer vector is encrypted by second encryption information; and determining the similarity between the face features to be recognized and the target face features based on the data obtained in the steps, and determining the face recognition authentication result. Correspondingly, the invention discloses a face recognition authentication system.

Description

Privacy protection method and system in face recognition authentication

Technical Field

The invention relates to the technical field of computers, in particular to a privacy protection method and system in face recognition authentication.

Background

The basic process of face image retrieval is divided into two stages of registration and retrieval. In the registration stage, the user sends the face representation vector to a server for storage. In the retrieval stage, the user acquires the face characterization vector through the same process and sends the face characterization vector to the server, the server compares the received characterization vector with the stored database, and finally, the subsequent processes such as identity confirmation and the like are completed according to the comparison result. However, the face representation vector contains important privacy information of the user, and the malicious server can recover the face image of the user or important identity characteristics of the user from the face representation vector through methods such as data reconstruction.

Disclosure of Invention

One or more embodiments of the present disclosure provide a privacy protection method and system in face recognition authentication, which can implement accurate and efficient face recognition authentication on the premise of protecting face information security.

According to a first aspect, a privacy protection method in face recognition authentication is provided, applied to a server side, and includes:

responding to a face identification authentication request of a client, and acquiring a face random vector to be identified and first encryption information uploaded by the client; the random vector of the face to be identified is obtained by transforming the feature of the face to be identified through a disposable first random integer vector; the first encryption information is obtained by encrypting the first random integer vector through a preset encryption algorithm;

Retrieving a target face random vector and second encryption information which are stored in advance; the target face random vector is obtained by transforming the target face features through a disposable second random integer vector; the second encryption information is obtained by encrypting the second random integer vector through the encryption algorithm;

determining the similarity between the face feature to be identified and the target face feature based on the face random vector to be identified, the first encryption information, the target face random vector and the second encryption information;

and determining a face recognition authentication result based on the similarity.

The privacy protection method in face identification authentication provided by the embodiment of the specification firstly utilizes the first random integer vector to transform the face characteristics to be identified to obtain the random vector of the face to be identified, then encrypts the first random integer vector, and the target face characteristics are also processed according to a similar method, so that the face characteristics to be identified and the target face characteristics are indirectly encrypted, and the face privacy information is protected. In addition, the face feature to be recognized and the target face feature encrypted by the steps can be directly subjected to face similarity calculation, so that the accuracy and the efficiency of face recognition authentication are ensured.

As an optional implementation manner of the method of the first aspect, the first random integer vector includes a first random integer sub-vector and a second random integer sub-vector that are equal to the number of feature bits of the face to be identified, and a first normalization factor; transforming the face features to be identified through the first random integer vector specifically comprises the following steps:

carrying out random transformation on the face features to be identified through the first random integer sub-vector and the second random integer sub-vector;

and carrying out normalization processing on the face features to be identified after random transformation through the first normalization factor to obtain the face random vector to be identified.

As an optional implementation manner of the method of the first aspect, the second random integer vector includes a third random integer sub-vector and a fourth random integer sub-vector that are equal to the number of feature bits of the target face, and a second normalization factor; transforming the target face feature through the second random integer vector specifically includes:

performing random transformation on the target face features through the third random integer sub-vector and the fourth random integer sub-vector;

And normalizing the target face features subjected to random transformation through the second normalization factor to obtain the target face random vector.

As an optional implementation manner of the method of the first aspect, encrypting, by the encryption algorithm, the second random integer vector specifically includes:

the client acquires a public key, a private key and a reversible encryption matrix issued by a trusted third party;

the client generates a second random sub-vector for each random integer in the second random integer vector;

the client encodes the second random integer vector by using the second random sub-vector to obtain a second random matrix;

the client encrypts the second random matrix by using an inverse matrix of the encryption matrix to obtain second encrypted data;

and the client signs the second encrypted data by using the private key to obtain the second encrypted information.

Specifically, encrypting the first random integer vector by the encryption algorithm specifically includes:

the client generates a first random sub-vector for each random integer in the first random integer vector;

The client encodes the first random integer vector by using the first random sub-vector to obtain a first random matrix;

the client encrypts the first random matrix by using the encryption matrix to obtain first encrypted data;

and the client encrypts the first encrypted data by using the public key to obtain the first encrypted information.

As an optional implementation manner of the method of the first aspect, determining the similarity between the face feature to be recognized and the target face feature based on the face random vector to be recognized, the first encryption information, the target face random vector and the second encryption information specifically includes:

obtaining a sum vector of the first random integer vector and the second random integer vector based on the first encryption information and the second encryption information;

and obtaining the similarity between the face feature to be recognized and the target face feature based on the sum vector, the face random vector to be recognized and the target face random vector.

As an optional implementation manner of the method of the first aspect, determining a face recognition authentication result based on the similarity specifically includes:

Responding to the similarity being greater than or equal to a preset similarity threshold value, and determining that the face recognition authentication result is authentication passing;

and determining that the face recognition authentication result is that authentication is not passed according to the fact that the similarity is smaller than the similarity threshold.

According to a second aspect, there is provided a privacy protection method in face recognition authentication, applied to a client, including:

acquiring face characteristics to be identified;

generating a third random integer vector, and carrying out random transformation on the face features to be identified through the third random integer vector to obtain a face random vector to be identified;

encrypting the third random integer vector through a preset encryption algorithm to obtain third encryption information;

a face recognition authentication request is initiated to the server, and the face random vector to be recognized and the third encryption information are sent to the server; the server determines the similarity between the face feature to be identified and the target face feature based on the face random vector to be identified, the third encryption information, the pre-stored target face random vector and the fourth encryption information, and determines a face identification authentication result based on the similarity;

And obtaining the face recognition authentication result fed back by the server side.

As an optional implementation manner of the method of the second aspect, the acquiring the face feature to be identified specifically includes:

acquiring a face image to be identified;

and inputting the face image to be identified into a feature extraction network to obtain the face feature to be identified.

As an optional implementation manner of the method of the second aspect, the third random integer vector includes a fifth random integer sub-vector and a sixth random integer sub-vector that are equal to the number of feature bits of the face to be identified, and a third normalization factor; the method for carrying out random transformation on the face features to be identified through the third random integer vector specifically comprises the following steps:

carrying out random transformation on the face features to be identified through the fifth random integer sub-vector and the sixth random integer sub-vector;

and carrying out normalization processing on the face features to be identified after random transformation through the third normalization factor to obtain the face random vector to be identified.

As an optional implementation manner of the method of the second aspect, before initiating a face recognition authentication request to the server side, the method further includes:

Acquiring target face characteristics;

generating a fourth random integer vector, and carrying out random transformation on the target face features through the fourth random integer vector to obtain a target face random vector;

encrypting the fourth random integer vector through a preset encryption algorithm to obtain fourth encryption information;

and initiating a registration request to the server, and sending the target face random vector and the fourth encryption information to the server for storage.

Specifically, the method for acquiring the target face features specifically includes:

acquiring a target face image;

and inputting the target face image into a feature extraction network to obtain the target face features.

Specifically, the fourth random integer vector includes a seventh random integer sub-vector and an eighth random integer sub-vector that are equal to the number of feature bits of the target face, and a fourth normalization factor; the random transformation is carried out on the target face features through the fourth random integer vector, and the method specifically comprises the following steps:

randomly transforming the target face feature through the seventh random integer sub-vector and the eighth random integer sub-vector;

and normalizing the target face features subjected to random transformation by the fourth normalization factor to obtain the target face random vector.

Specifically, encrypting the fourth random integer vector by the encryption algorithm to obtain the fourth encrypted information, which specifically includes:

obtaining a public key, a private key and a reversible secret matrix issued by a trusted third party;

generating a fourth random sub-vector for each random integer in said fourth random integer vector;

encoding the fourth random integer vector by using the fourth random sub-vector to obtain a fourth random matrix;

encrypting the fourth random matrix by using the inverse matrix of the encryption matrix to obtain fourth encrypted data;

and signing the fourth encrypted data by using the private key to obtain the fourth encrypted information.

More specifically, the third random integer vector is encrypted by the encryption algorithm to obtain the third encrypted information, which specifically includes

Generating a third random sub-vector for each random integer in said third random integer vector;

encoding the third random integer vector by using the third random sub-vector to obtain a third random matrix;

encrypting the third random matrix by using the encryption matrix to obtain third encrypted data;

And encrypting the third encrypted data by using the public key to obtain the third encrypted information.

According to a third aspect, there is provided a face recognition authentication system comprising: a client and a server;

the server side is configured to implement the privacy protection method applied to the face recognition authentication of the server side in any one of the above in the face recognition authentication process;

the client is configured to implement the privacy protection method applied to the face recognition authentication of the client in any one of the above in the face recognition authentication process.

According to a fourth aspect, there is provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a privacy protection method as defined in any of the above in face recognition authentication applied to a server side.

According to a fifth aspect, there is provided an electronic device comprising

One or more processors; and

a memory associated with the one or more processors, the memory configured to store program instructions that, when read for execution by the one or more processors, perform the steps of the privacy preserving method in face recognition authentication applied to a server side as claimed in any one of the preceding claims.

According to a sixth aspect, there is provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a privacy protection method as defined in any one of the above in face recognition authentication applied to a client.

According to a seventh aspect, there is provided an electronic device comprising

One or more processors; and

a memory associated with the one or more processors for storing program instructions that, when read for execution by the one or more processors, perform the steps of the privacy preserving method in face recognition authentication applied to a client as claimed in any one of the preceding claims.

The privacy protection method in face identification authentication according to one or more embodiments of the present disclosure has the advantages that the first random integer vector is utilized to perform random transformation on the face feature to be identified to obtain the random vector of the face to be identified, then the first random integer vector is encrypted, and the target face feature is also processed according to a similar method, so that the face feature to be identified and the target face feature are indirectly encrypted, the security of the face privacy information is enhanced, and the calculation efficiency can be improved. In addition, the face features to be identified and the target face features encrypted by the steps can be directly subjected to face similarity calculation under the condition that the server cannot decrypt the face features, so that the accuracy and the efficiency of face identification authentication are ensured.

The face recognition authentication system described in the embodiment of the present specification also has the above-described advantageous effects.

Drawings

In order to more clearly illustrate the embodiments of the present description or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, and it is obvious that the drawings in the following description are some embodiments of the present description, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 schematically illustrates a schematic diagram of a face recognition authentication system provided in one or more embodiments of the present disclosure.

Fig. 2 schematically illustrates a flow chart of a face recognition authentication method according to one or more embodiments of the present disclosure in an implementation manner.

Fig. 3 schematically illustrates a flowchart of a privacy protection method in face recognition authentication according to one or more embodiments of the present disclosure applied to a server side in one implementation manner.

Fig. 4 schematically illustrates a flowchart of a privacy protection method in face recognition authentication according to one or more embodiments of the present disclosure, where the privacy protection method is applied to a client in one implementation manner.

Fig. 5 exemplarily shows a block diagram of an electronic device provided in an embodiment of the present specification.

Detailed Description

First, it will be understood by those skilled in the art that the terminology used in the embodiments of the present invention is for the purpose of describing particular embodiments only, and is not intended to be limiting of the invention. As used in this application and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

The face image retrieval has wide application in various service scenes such as face payment, electronic entrance guard and the like, and the basic flow is divided into two stages of registration and retrieval. In the registration stage, a user acquires a face image through a terminal, maps the face image into a face representation vector by using models such as a deep neural network and the like, and sends the face representation vector to a server side for storage. In the retrieval stage, the user acquires the face representation vector through the same process and sends the face representation vector to the server, the server compares the face representation vector received currently with face data stored in the database, and finally, the subsequent processes such as identity confirmation and the like are completed according to the comparison result.

However, the face representation vector contains important privacy information of the user, and an attacker (such as a malicious server) can recover the face image of the user or important identity characteristics of the user from the face representation vector through a data reconstruction method and the like, so that the privacy of the user is threatened. In order to protect the personal privacy of users, many privacy-preserving face retrieval methods are being created. According to the method, the face representation vector is encrypted or subjected to special transformation (like state encryption, hash and the like), so that an attacker cannot easily infer the face information of the user through the transformed face representation vector, and the server can still perform effective face retrieval based on the transformed face representation vector. Existing solutions tend to be difficult to balance in three dimensions of safety, efficiency and accuracy.

In view of the above, one or more embodiments of the present disclosure provide a privacy protection method in face recognition authentication, which achieves higher security, and meanwhile, has face retrieval efficiency and lossless retrieval accuracy suitable for large-scale data scenarios.

In order to make the technical solutions in the present specification better understood by those skilled in the art, the technical solutions in the embodiments of the present specification will be clearly and completely described below with reference to the drawings in the embodiments of the present specification, and it is obvious that the described embodiments are only some embodiments of the present specification, not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are intended to be within the scope of the present disclosure.

It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the method may include more or fewer steps than described in this specification. Furthermore, individual steps described in this specification, in other embodiments, may be described as being split into multiple steps; while various steps described in this specification may be combined into a single step in other embodiments.

One or more embodiments of the present specification provide a privacy preserving method in face recognition authentication. Referring to fig. 1, fig. 1 schematically illustrates a schematic diagram of a face recognition authentication system provided in one or more embodiments of the present disclosure, which may be used to implement a privacy preserving method in the face recognition authentication. It should be noted that, the privacy protection method in face recognition authentication according to one or more embodiments of the present disclosure may be implemented by the face recognition authentication system shown in fig. 1, but is not limited to the face recognition authentication system.

As shown in fig. 1, the face recognition authentication system includes a server side and a user terminal, where the server side is connected to the user terminal through a communication link, and the communication link may be a wired network or a wireless network. For example, the server side may establish communication connection with the user terminal by adopting a WIFI, bluetooth, infrared, and other communication modes. Or, the communication connection may be established with the user terminal through a mobile network, where the network system of the mobile network may be any one of 2G (GSM), 2.5G (GPRS), 3G (WCDMA, TD-SCDMA, CDMA2000, UTMS), 4G (LTE), 4g+ (lte+), wiMax, and the like.

The user terminal can be a device with an image acquisition function and a data processing function such as a mobile phone, an intelligent watch and a tablet personal computer, is provided with a client APP, and is used for acquiring face images of a user and mapping the face images into face feature vectors to be identified, encrypting the face feature vectors to be identified, and storing the face feature vectors to a server for face registration or transmitting the face feature vectors to the server for face retrieval.

The server side is used for receiving the face features to be identified, retrieving target face features from the local database, and performing similarity matching with the face features to be identified to obtain face identification authentication results. The server side may be any apparatus, device, platform, cluster of devices with computing, processing capabilities. In this embodiment, the implementation form of the server side is not limited, for example, the server side may be disposed in a single server, or may be disposed in a server cluster formed by a plurality of servers, and the server side may also be disposed in a cloud server, which is also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system.

In some embodiments, the present disclosure provides a face recognition authentication method, which applies a privacy protection method, please refer to fig. 2, and the method includes the following steps:

S100: and acquiring the face characteristics to be identified.

S102: generating a first random integer vector, and carrying out random transformation on the face features to be identified through the first random integer vector to obtain the face random vector to be identified.

S104: and encrypting the first random integer vector through a preset encryption algorithm to obtain first encryption information.

S106: retrieving a pre-stored target face random vector and corresponding encryption information; the target face random vector is obtained by transforming the target face features through a disposable second random integer vector; the second encryption information is obtained by encrypting the second random integer vector by an encryption algorithm.

S108: and determining the similarity between the face features to be recognized and the target face features based on the face random vector to be recognized, the first encryption information, the target face random vector and the second encryption information.

S110: and determining a face recognition authentication result based on the similarity.

Specifically, in order to enable the server side to calculate the similarity of the face features and not decrypt the uploaded face features to be identified, so that the malicious server is prevented from stealing the privacy of the face of the user, and the security of the face features to be identified and the security of the target face features can be enhanced by using a function encryption method. The function encryption is a generalized public key encryption technology, and aiming at a specific function, the function encryption method can directly obtain the result of the function applied to a plaintext by executing a series of operations on a ciphertext, so that the result of executing the specific operation on the face feature is obtained on the premise of not leaking the face privacy.

More specifically, when calculating the similarity between the face feature to be recognized and the target face feature, an inner product or cosine similarity may be employed. Therefore, it is required to ensure that the server cannot decrypt the first random integer vector and the second random integer vector, so that the face feature to be identified and the target face feature before transformation cannot be restored, but the similarity between the face feature to be identified and the target face feature can still be calculated according to the face random vector to be identified, the first encryption information, the target face random vector and the second encryption information, so that the server can be used for face identification authentication even if the server is not trusted.

In some embodiments of the present disclosure, a privacy protection method in face recognition authentication is provided, which is applied to a server, as shown in fig. 3, and includes:

s200: responding to a face identification authentication request of a client, and acquiring a face random vector to be identified and uploaded by the client and first encryption information; the random vector of the face to be identified is obtained by transforming the feature of the face to be identified through a disposable first random integer vector; the first encryption information is obtained by encrypting the first random integer vector through a preset encryption algorithm.

The face recognition authentication request of the client can comprise information such as user identity information initiating the request, an application platform requesting face recognition authentication service, a service type requested by face recognition authentication, and the like, for example, the service type requested by face recognition authentication can be login account number, fund transaction, information authorization, and the like. Before the client initiates the face recognition authentication request, the face image to be recognized of the user is acquired, and the corresponding face feature to be recognized is extracted by utilizing the feature extraction network.

The first random integer vector performs random transformation such as random scaling, symbol inversion and the like on the face features to be identified to obtain the face random vector to be identified, wherein the first random integer vector is composed of a plurality of groups of random integers. Performing a random transformation of the face features to be identified is advantageous for enhancing the security of the face information.

In some embodiments, the first random integer vector includes first and second random integer sub-vectors equal to the number of face features to be identified, and a first normalization factor; transforming the face features to be identified through a first random integer vector, specifically including:

Randomly transforming the face features to be identified through the first random integer sub-vector and the second random integer sub-vector;

and carrying out normalization processing on the face features to be identified after the random transformation by using a first normalization factor to obtain a face random vector to be identified.

Firstly, determining the bit number of the face feature to be identified, generating two groups of random integers equal to the bit number by a client to form a first random integer sub-vector and a second random integer sub-vector, carrying out random transformation on the face feature to be identified according to the first random integer sub-vector and the second random integer sub-vector, and setting a normalization factor; and then carrying out normalization processing on the face features to be identified after random transformation by using normalization factors so as to speed up the operation speed, thereby obtaining the random vectors of the faces to be identified. And the first random integer vector is encoded by the first random integer sub-vector, the second random integer sub-vector, and the normalization factor.

After random transformation is carried out on the face features to be identified, a first random integer vector which carries out random transformation is encrypted by using a preset encryption algorithm, so that the face privacy is protected by adopting an indirect encryption mode, the server side can also carry out similarity calculation and face identification authentication under the condition that the face data cannot be decrypted, and the calculation efficiency can be improved.

S202: retrieving a target face random vector and second encryption information which are stored in advance; the target face random vector is obtained by transforming the target face features through a disposable second random integer vector; the second encryption information is obtained by encrypting the second random integer vector by an encryption algorithm.

In the registration stage of face recognition authentication, the server side registers and stores a plurality of sets of target faces for retrieval in the face recognition authentication process, wherein the target face features are processed by the client side according to steps similar to those of the step of processing the face features to be recognized in S200, and therefore, the description thereof will not be provided in detail herein.

In some embodiments, the second random integer vector includes a third random integer sub-vector and a fourth random integer sub-vector equal to the number of bits of the target face feature, and a second normalization factor; transforming the target face features through a second random integer vector, specifically including:

randomly transforming the target face features through the third random integer sub-vector and the fourth random integer sub-vector;

and carrying out normalization processing on the target face features after random transformation through a second normalization factor to obtain target face random vectors.

In some embodiments, encrypting the second random integer vector by an encryption algorithm specifically includes:

the client encodes a second random integer vector by using a second random sub-vector to obtain a second random matrix;

and the client signs the second encrypted data by using the private key to obtain second encrypted information.

The trusted third party may be an authoritative certification authority, such as a trusted certification authority (Certifcation Authority, CA), which may be implemented in the form of a server. The trusted third party is configured to issue a public-private key pair for use by the client and issue a certificate to prove the authenticity of the client's identity.

The reversible encryption matrix is obtained by random sampling after the size and dimension of the matrix are preset by the client.

And each random integer in the second random integer vector is randomly encoded, then the inverse matrix of the encryption matrix is used for encryption, and then the private key signature of the client is used for greatly enhancing the confidentiality of the second random integer vector, so that the difficulty of an attacker in restoring the target face features according to the second random integer vector is increased.

Accordingly, encrypting the first random integer vector by an encryption algorithm specifically includes:

the client encodes a first random integer vector by using a first random sub-vector to obtain a first random matrix;

the client encrypts the first encrypted data by using the public key to obtain first encrypted information.

In the retrieval stage of face recognition authentication, a first random integer vector used for carrying out random transformation on face features to be recognized is encrypted according to a step similar to a second random integer vector, and the difference is that the retrieval stage directly uses an encryption matrix to encrypt the encoded first random integer vector, so that the addition of the encryption matrix does not influence the calculation of the similarity; and the client further encrypts with the public key during the retrieval phase.

S204: and determining the similarity between the face features to be recognized and the target face features based on the face random vector to be recognized, the first encryption information, the target face random vector and the second encryption information.

The face random vector to be recognized and the target face random vector respectively contain face feature information to be recognized and target face feature information, and the first encryption information and the second encryption information respectively contain random integer vector information for executing random transformation on the face feature to be recognized and the target face feature, so that a result of executing specific operation on the face feature to be recognized and the target face feature can be obtained based on the face random vector to be recognized, the first encryption information, the target face random vector and the second encryption information, and the similarity between the face feature to be recognized and the target face feature can be determined.

In some embodiments, determining the similarity between the face feature to be identified and the target face feature based on the face random vector to be identified, the first encryption information, the target face random vector, and the second encryption information specifically includes:

and obtaining the similarity between the face features to be recognized and the target face features based on the sum vector, the face random vector to be recognized and the target face random vector.

Specifically, the similarity of the face features to be recognized and the target face features can be determined by calculating the inner product or cosine similarity between the two.

S206: and determining a face recognition authentication result based on the similarity.

In some embodiments, based on the similarity, determining the face recognition authentication result specifically includes:

and determining that the face recognition authentication result is that the authentication is not passed in response to the similarity being smaller than the similarity threshold.

After receiving a face recognition authentication request from a client, a server acquires a face random vector to be recognized which is transformed at the client and encrypted first encryption information; then searching a prestored target face random vector and second encryption information which are also encrypted from the database; the similarity between the face features to be recognized and the target face features can be calculated through the data, so that the face recognition authentication result is determined, and the balance among face privacy safety, face recognition authentication efficiency and accuracy is achieved.

In some embodiments of the present disclosure, a privacy protection method in face recognition authentication is further provided, which is applied to a client, as shown in fig. 4, and includes:

S300: and acquiring the face characteristics to be identified.

The client can be arranged on a mobile phone, a smart watch, a tablet personal computer and other equipment with an image acquisition function and a data processing function, and face recognition authentication service is provided for users.

In some embodiments, obtaining the face feature to be identified specifically includes:

acquiring a face image to be identified;

and inputting the face image to be identified into a feature extraction network to obtain the face features to be identified.

When a user performs face recognition authentication, firstly, acquiring a face image to be recognized through image acquisition equipment; and then inputting the facial features to the feature extraction network in the client to extract corresponding facial features to be identified. Specifically, a feature extraction network may be constructed based on a network structure such as ResNet, transformer, so as to extract the face features to be identified from the face image to be identified.

S302: and generating a third random integer vector, and carrying out random transformation on the face features to be identified through the third random integer vector to obtain the face random vector to be identified.

And carrying out random transformation such as random scaling, symbol inversion and the like on the face features to be identified by the third random integer vector to obtain the face random vector to be identified, wherein the third random integer vector is composed of a plurality of groups of random integers. Performing a random transformation of the face features to be identified is advantageous for enhancing the security of the face information.

In some embodiments, the third random integer vector includes a fifth random integer sub-vector and a sixth random integer sub-vector equal to the number of face features to be identified, and a third normalization factor; the method comprises the steps of carrying out random transformation on the face features to be identified through a third random integer vector, and specifically comprising the following steps:

randomly transforming the face features to be identified through the fifth random integer sub-vector and the sixth random integer sub-vector;

and carrying out normalization processing on the face features to be identified after the random transformation by a third normalization factor to obtain a face random vector to be identified.

Firstly, determining the bit number of the face feature to be identified, generating two groups of random integers equal to the bit number to form a fifth random integer sub-vector and a sixth random integer sub-vector, carrying out random transformation on the face feature to be identified according to the fifth random integer sub-vector and the sixth random integer sub-vector, and setting a normalization factor; and then carrying out normalization processing on the face features to be identified after random transformation by using normalization factors so as to speed up the operation speed, thereby obtaining the random vectors of the faces to be identified. And the third random integer vector is encoded by a fifth random integer sub-vector, a sixth random integer sub-vector, and a third normalization factor.

S304: and encrypting the third random integer vector through a preset encryption algorithm to obtain third encryption information.

After random transformation is carried out on the face features to be identified, a third random integer vector which carries out random transformation is encrypted by using a preset encryption algorithm, and the face privacy can be protected in an indirect encryption mode, so that the server side can also carry out similarity calculation and face identification authentication under the condition that the face data cannot be decrypted, and the calculation efficiency and the security of the face identification authentication are improved.

S306: a face identification authentication request is initiated to a server, and a face random vector to be identified and third encryption information are sent to the server; the server terminal determines the similarity between the face feature to be recognized and the target face feature based on the face random vector to be recognized, the third encryption information, the target face random vector and the fourth encryption information which are stored in advance, and determines a face recognition authentication result based on the similarity.

The face recognition authentication request initiated by the client to the server side can comprise information such as user identity information initiating the request, an application platform requesting face recognition authentication service, and the type of service requested by the face recognition authentication, for example, the type of service requested by the face recognition authentication can be login account number, fund transaction, information authorization, and the like.

In some embodiments, before the face recognition authentication request is initiated to the server side, the method further includes:

acquiring target face characteristics;

In the registration stage of face recognition authentication, the client registers a plurality of groups of target faces with the server side for searching in the face recognition authentication process, and stores the target faces in a database of the server side. The target face is used for comparing with the input face to be recognized in the retrieval stage of face recognition authentication, and if the face to be recognized is successfully matched with any target face pre-stored in the server side, the face recognition authentication is passed; if the target face in the server cannot be successfully matched with the face to be recognized, the face recognition authentication is not passed. Since the target face features are processed by the client in steps similar to those of the processing of the face features to be recognized in S300-S304, they will not be described in detail herein.

acquiring a target face image;

Alternatively, a feature extraction network may be constructed based on a network structure such as ResNet, transformer to extract target face features from the target face image.

Specifically, the fourth random integer vector includes a seventh random integer sub-vector and an eighth random integer sub-vector that are equal to the number of feature bits of the target face, and a fourth normalization factor; the random transformation is carried out on the target face features through a fourth random integer vector, and the method specifically comprises the following steps:

randomly transforming the target face features through the seventh random integer sub-vector and the eighth random integer sub-vector;

and carrying out normalization processing on the target face characteristics after random transformation by a fourth normalization factor to obtain a target face random vector.

Specifically, encrypting the fourth random integer vector by an encryption algorithm to obtain fourth encrypted information, which specifically includes:

generating a fourth random sub-vector for each random integer in the fourth random integer vector;

encrypting the fourth random matrix by using an inverse matrix of the encryption matrix to obtain fourth encrypted data;

and signing the fourth encrypted data by using the private key to obtain fourth encrypted information.

Each random integer in the fourth random integer vector is randomly encoded, then the inverse matrix of the encryption matrix is used for encryption, and then the private key signature of the client is used for greatly enhancing the confidentiality of the fourth random integer vector, so that the difficulty of an attacker in restoring the target face features according to the fourth random integer vector is increased.

Correspondingly, encrypting the third random integer vector through an encryption algorithm to obtain third encrypted information, which specifically comprises the following steps:

Generating a third random sub-vector for each random integer in the third random integer vector;

encrypting the third random matrix by utilizing the encryption matrix to obtain third encrypted data;

and encrypting the third encrypted data by using the public key to obtain third encrypted information.

In the retrieval stage of face recognition authentication, a third random integer vector used for carrying out random transformation on the face features to be recognized is encrypted according to a step similar to a fourth random integer vector, wherein the retrieval stage directly uses an encryption matrix to encrypt the encoded third random integer vector, so that the addition of the encryption matrix does not influence the calculation of the similarity; and the client further encrypts with the public key during the retrieval phase.

The random face vector to be recognized and the random target face vector respectively contain the feature information of the face to be recognized and the feature information of the target face, and the third encryption information and the fourth encryption information respectively contain random integer vector information for executing random transformation on the feature of the face to be recognized and the feature of the target face.

Specifically, based on the random vector of the face to be identified, the third encryption information, the target random vector of the face and the fourth encryption information, the server side can determine the similarity of the random vector of the face to be identified and the target random vector of the face to be identified and the cosine similarity by calculating the inner product or the cosine similarity between the face features to be identified and the target face features.

In some embodiments, the server end determines the face recognition authentication result based on the similarity, and specifically includes:

S308: and obtaining a face recognition authentication result fed back by the server side.

In the registration stage of face recognition authentication, the client acquires the target face features, performs random transformation on the target face features, indirectly encrypts fourth encryption information for performing random transformation, and sends the acquired target face random vector and the fourth encryption information to the server for storage, so that the privacy security of the target face is ensured. In the retrieval stage of face recognition authentication, the client acquires the face features to be recognized, encrypts the face features by adopting the same method as the target face features to acquire random vectors of the face to be recognized and third encrypted information, and sends the random vectors and the third encrypted information to the server together with a face recognition authentication request, so that the server can perform similarity calculation under the condition that specific face information cannot be decrypted, and the server has high safety; and finally, receiving a face recognition authentication result returned by the server side.

The privacy protection method in face recognition authentication provided in the present specification will be further described in detail by specific embodiments, but the detailed description does not limit the present specification.

In this embodiment, the face feature to be recognized is set to x, and the target face feature is set to y, where x and y are feature vectors. The step of random face feature transformation will be described below by taking the face feature x to be identified as an example, and the target face feature y is only required to be referred to when being transformed randomly.

Specifically, the manner of randomly transforming the face feature x to be identified can be represented by the following formula:

wherein T is _x A first random integer vector for performing a random transformation on the face feature x to be identified; c _x And representing the random vector of the face to be recognized, which is obtained after the random transformation is carried out on the face feature x to be recognized.

First, the face feature x to be identified may be divided into K parts, as follows:

then respectively generating first random integer sub-vectorsA second random integer sub-vectorEach random integer sub-vector is composed of K random integers, the range of all random integers is set to be more than or equal to 0 and less than or equal to L-1, wherein L is a super-parameter, and the debugging is carried out manually.

Then, the random integer sub-vector is utilized to carry out random transformation on x to obtain an intermediate vector b _x ：

Wherein b _x The ith component of (a)Can be calculated using the following formula:

wherein,and->Respectively representing the corresponding ith random integer in the random integer sub-vector;Representing an ith component in the face feature x to be identified; l, M is radix Ginseng Rubra.

Finally, the random vector c of the face to be identified can be obtained by the following method _x Is marked as

W _x ＝||b _x ||

In addition, W can be represented by the following formula _x Quantized to an integer to obtain a first normalization factor w _x ：

From which a first random integer vector T can be obtained _x Which is composed of a first random integer sub-vectorSecond random integer sub-vector->And a first normalization factor w _x The composition is as follows:

similarly, a second random integer vector T for performing random transformation on the target face feature y can be obtained _y And a target face random variable c obtained after random transformation _y ：

If the similarity between the face feature x to be identified and the target face feature y needs to be determined, preferably, x can be calculated ^T y determines the inner product or cosine similarity between the two, based on the random transformation method described above, then (T can be used _x +T _y ,c _x ,c _y ) To calculate x ^T The specific formula of y is as follows:

as can be seen from the above, the server is connected to the server at a known (T _x +T _y ,c _x ,c _y ) Can be used for face feature similarity calculation without obtaining specific random integer vector T _x Or T _y Thus, when the random face to be recognized is directly transformed into the random vector c _x And a target face random vector c _y When the target face feature is sent to the server, in order to enable the server to be unable to restore the face feature x to be recognized and the target face feature y before random transformation while the similarity calculation can be performed, it is necessary to ensure that the server knows T _x +T _y But cannot obtain a specific T _x Or T _y Such that even an untrusted server may be used for face recognition authentication.

To further enhance the confidentiality of the face features, a first random integer variable T may be encrypted based on a function _x And a second random integer becomes T _y Encryption is performed, T which needs to be summed _x And T _y Respectively replaced by [1, T ] _x ]And [ T ] _y ,1]Then:

T _x +T _y ＝[1,T _x ] ^T [T _y ,1]

thus the server side calculates T _x +T _y It becomes more difficult to obtain a specific T _x Or T _y Is a value of (2).

Specifically, first, the client obtains the public key mpk and the private key issued by the trusted third partyA key msk and a reversible encryption matrix B. In the registration stage of face recognition authentication, the target face feature y passes through a second random integer vector T _y Transformed into a target face random vector c _y Thus for T _y Each random number in (a)Sampling a second random sub-vector r _i ∈Z ^q And (3) making:

thereby taking the second random integer vector T _y Encoded as a second random matrix

Next, a second random matrix is encrypted using the inverse of encryption matrix BAnd signs by using the private key msk to obtain second encryption information +.>The formula is as follows:

after the client initiates a registration request to the server, the target face random vector and the second encryption information are obtainedAnd sending the target face registration information to a server for storage, and finishing target face registration.

In the retrieval stage of face recognition authentication, the face feature x to be recognized passes through a first random integer vector T _x Is converted into a face random vector c to be identified _x Thus let:

thereby taking the first random integer vector T _x Encoded as a first random matrix

Next, the first random matrix is encrypted using encryption matrix BAnd encrypting with public key mpk to obtain first encryption information +.>The formula is as follows:

after the client initiates a face recognition authentication request to the server, the random vector of the face to be recognized and the first encryption information, namelyIs sent to the server side so that the server side firstly carries out the encryption according to the second encryption information>First encryption information- >Determining a first random integer vector T _x And a second random integer vector T _y Sum vector T of _x +T _y The following formula: />

Then according to T _x +T _y Human face random vector c to be identified _x Target face random vector c _y The similarity x between the face feature x to be identified and the target face feature y can be calculated ^T And y, the server further determines a face recognition authentication result according to the similarity calculation result.

In the above embodiment, the server may obtain the second encrypted data obtained by encrypting the second random matrix with the inverse matrix of the encryption matrix, that isHowever, in the case where B cannot be decrypted, the server cannot recover the second random integer vector T _y Even if the linear relationship between the second random matrices can be obtained, it is not significant, and thus the information of the target face feature y cannot be further obtained. And the first random matrix is multiplied by the encryption matrix and then encrypted by the public key to obtain first encryption information +.>Without decryption key, the server side cannot decrypt +.>More unable to obtain the first random integer vector T _x Therefore, the face feature x to be recognized cannot be restored.

In addition, a second random sub-vector r _i Is randomly generated, so that the security and the cracking difficulty of the second random matrix and the encryption matrix can be enhanced.

An embodiment of the present specification further provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a privacy protection method applied to server-side face recognition authentication as set forth in any one of the above.

One embodiment in the present specification also provides an electronic device, including

One or more processors; and

An embodiment in the present specification further provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a privacy protection method applied to face recognition authentication of a client as set forth in any one of the above.

One or more processors; and

Fig. 5 exemplarily shows a block diagram of an electronic device provided in an embodiment of the present disclosure, which shows a schematic structural diagram of a computer system 400 of a terminal device or a server suitable for implementing an embodiment of the present invention. The terminal device or server shown in fig. 5 is only an example, and should not impose any limitation on the functions and scope of use of the embodiments of the present invention.

In a typical configuration, computer 400 includes one or more processors (CPUs) 402, a memory 404, a network interface 406, an input interface 408, and an output interface 410.

Memory 404 may include non-volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, etc., such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.

In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flow chart. The above-described functions defined in the method of the present invention are performed when the computer program is executed by a Central Processing Unit (CPU) 402.

The computer readable medium shown in the present invention may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

The foregoing describes specific embodiments of the present disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous. It will also be noted that each block of the figures, and combinations of blocks in the figures, can be implemented by special purpose hardware-based systems which perform the specified functions or operations, or combinations of special purpose hardware and computer instructions.

It should be noted that the above-mentioned embodiments are merely examples of the present invention, and it is obvious that the present invention is not limited to the above-mentioned embodiments, and many similar variations are possible. All modifications attainable or obvious from the present disclosure set forth herein should be deemed to be within the scope of the present disclosure.

Claims

1. A privacy protection method in face recognition authentication is applied to a server side and comprises the following steps:

2. The method of claim 1, the first random integer vector comprising first and second random integer sub-vectors equal to the number of face features to be identified, and a first normalization factor; transforming the face features to be identified through the first random integer vector specifically comprises the following steps:

3. The method of claim 1, the second random integer vector comprising a third random integer sub-vector and a fourth random integer sub-vector equal to the number of bits of the target face feature, and a second normalization factor; transforming the target face feature through the second random integer vector specifically includes:

4. The method of claim 1, encrypting the second random integer vector by the encryption algorithm, comprising in particular:

5. The method of claim 4, encrypting the first random integer vector by the encryption algorithm, comprising:

6. The method according to claim 1, wherein determining the similarity between the face feature to be identified and the target face feature based on the face random vector to be identified, the first encryption information, the target face random vector, and the second encryption information specifically comprises:

7. The method of claim 1, determining a face recognition authentication result based on the similarity, specifically comprising:

8. A privacy protection method in face recognition authentication is applied to a client and comprises the following steps:

acquiring face characteristics to be identified;

9. The method of claim 8, wherein the obtaining the face feature to be identified specifically comprises:

acquiring a face image to be identified;

10. The method of claim 8, the third random integer vector comprising a fifth random integer sub-vector and a sixth random integer sub-vector equal to the number of face features to be identified, and a third normalization factor; the method for carrying out random transformation on the face features to be identified through the third random integer vector specifically comprises the following steps:

11. The method of claim 8, further comprising, prior to initiating a face recognition authentication request to the server side:

acquiring target face characteristics;

12. The method of claim 11, acquiring target face features, specifically comprising:

acquiring a target face image;

13. The method of claim 11, the fourth random integer vector comprising a seventh random integer sub-vector and an eighth random integer sub-vector equal to the number of bits of the target face feature, and a fourth normalization factor; the random transformation is carried out on the target face features through the fourth random integer vector, and the method specifically comprises the following steps:

14. The method of claim 11, wherein encrypting the fourth random integer vector by the encryption algorithm results in the fourth encrypted information, specifically comprising:

15. The method of claim 14, wherein the third encryption information is obtained by encrypting the third random integer vector by the encryption algorithm, and specifically includes

16. A face recognition authentication system, comprising: a client and a server;

the server side is configured to implement the privacy protection method in face recognition authentication according to any one of claims 1 to 7 in the face recognition authentication process;

the client is configured to implement the privacy protection method in face recognition authentication according to any one of claims 8 to 15 in the face recognition authentication process.

17. A computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the method of any of claims 1 to 7.

18. An electronic device comprising

One or more processors; and

a memory associated with the one or more processors for storing program instructions that, when read for execution by the one or more processors, perform the steps of the method of any of claims 1 to 7.

19. A computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the method of any of claims 8 to 15.

20. An electronic device comprising

One or more processors; and

a memory associated with the one or more processors for storing program instructions that, when read for execution by the one or more processors, perform the steps of the method of any of claims 8 to 15.