WO2023040202A1

WO2023040202A1 - Face recognition method and apparatus, electronic device, and storage medium

Info

Publication number: WO2023040202A1
Application number: PCT/CN2022/078372
Authority: WO
Inventors: 邵巾芮; 蔺颖
Original assignee: 上海商汤智能科技有限公司
Priority date: 2021-09-17
Filing date: 2022-02-28
Publication date: 2023-03-23
Also published as: CN113807253A

Abstract

A face recognition method and apparatus, an electronic device, and a storage medium. The method comprises: determining, by means of a terminal device, a face image to be recognized and a face recognition mode of the face image to be recognized; sending, to a server according to the face recognition mode, the face image to be recognized and/or a first face feature obtained by performing feature extraction on the face image to be recognized as face recognition information; and after performing face recognition according to the face recognition information, the server returning the obtained face recognition result to the terminal device. During a face recognition process, a user himself/herself can determine whether to upload a face image to a server; and in the case that the user does not allow the uploading of the face image, a feature extracted from the face image can be uploaded to the server for face recognition, so that an accurate face recognition process can also be achieved.

Description

Face recognition method and device, electronic device and storage medium

This application claims the priority of the Chinese patent application filed on September 17, 2021, with the application number 202111092935.3, and the title of the invention is "Face recognition method and device, electronic equipment and storage medium", the entire content of which is incorporated by reference in this application middle.

technical field

The present disclosure relates to the field of computer technology, and in particular to a face recognition method and device, electronic equipment, and a storage medium.

Background technique

At present, face recognition technology is widely used in various fields. Due to the limitation of the computing power of the terminal equipment, the existing face recognition technology directly collects the user's face information and uploads it to the server for face recognition without considering the user's wishes, which has the hidden danger of user privacy leakage.

Contents of the invention

The present disclosure proposes a face recognition method and device, electronic equipment, and a storage medium, aiming at performing the face recognition process under the premise of respecting user privacy choices and ensuring the accuracy of face recognition results.

According to a first aspect of the present disclosure, a face recognition method is provided, the method comprising:

Determine the face image to be recognized;

Determine the face recognition mode of the face image to be recognized;

Send face recognition information to the server according to the face recognition mode, the face recognition information includes a face image to be recognized, and/or a first face feature obtained by feature extraction of the face image to be recognized ;

and receiving a face recognition result returned by the server, where the face recognition result is obtained by performing face recognition on the face recognition information.

In a possible implementation manner, the determining the face recognition mode of the face image to be recognized includes:

Displaying a face recognition mode selection page, the selection page including a first recognition control and/or a second recognition control;

In response to the triggering of the first recognition control, determine the face recognition mode of the face image to be recognized as the first recognition mode, and the first recognition mode is used to send the face image to be recognized;

In response to the triggering of the second recognition control, it is determined that the face recognition mode of the face image to be recognized is the second recognition mode, and the second recognition mode is used to send a feature extraction function to the face image to be recognized. Get the first face features.

In a possible implementation manner, the sending face recognition information to the server according to the face recognition mode includes:

In response to the face recognition mode being the second recognition mode, inputting the face image to be recognized into a first feature extraction model obtained through pre-training, and determining that the output first face feature is face recognition information;

Send the face recognition information to the server.

In a possible implementation manner, the determination process of the face recognition result includes:

Input the face recognition information into the pre-trained second feature extraction model through the server, and output the second face feature;

According to the matching of the second face feature with the pre-stored user face feature set, the face recognition result is obtained, and the user face feature set includes at least one user face feature corresponding to the user face feature user information.

In a possible implementation, the method further includes:

In response to the face recognition result being a non-registered user, determine current user information;

Send the current user information to the server, so as to store the current user information and the corresponding second facial features in the user facial feature set.

According to a second aspect of the present disclosure, a face recognition method is provided, the method comprising:

receiving face recognition information sent by the terminal device, the face recognition information including the face image to be recognized, and/or the first face feature obtained by feature extraction of the face image to be recognized by the terminal device;

performing face recognition according to the face recognition information to obtain a face recognition result;

Sending the face recognition result to the terminal device.

In a possible implementation manner, performing face recognition according to the face recognition information to obtain a face recognition result includes:

performing feature extraction on the face recognition information to obtain a second face feature;

Performing face recognition according to the second face features to obtain a face recognition result.

In a possible implementation manner, the performing feature extraction on the face recognition information to obtain the second face features includes:

Inputting the first human face feature or the to-be-recognized human face image into the pre-trained second feature extraction model, and outputting the second human face feature.

In a possible implementation manner, performing face recognition according to the second face feature, and obtaining a face recognition result includes:

Determining a user face feature set, the user face feature set including at least one user face feature, and user information corresponding to the user face feature;

matching according to the second facial features and the user's facial features to obtain a matching result of the user's facial features;

In response to the matching result of the existence of the user's face feature being a successful match, the user information corresponding to the user's face feature is used as the face recognition result;

In response to the fact that the matching result of no facial features of the user is successful, it is determined that the facial recognition result is a non-registered user.

In a possible implementation, the method further includes:

In response to receiving the current user information returned by the terminal device when the face recognition result is a non-registered user, storing the current user information and the corresponding second face feature in the user face feature gather.

According to a third aspect of the present disclosure, a face recognition device is provided, the device comprising:

An image determination module, configured to determine a face image to be recognized;

A recognition mode determination module, configured to determine the face recognition mode of the face image to be recognized;

An information sending module, configured to send face recognition information to the server according to the face recognition mode, the face recognition information includes a face image to be recognized, and/or a feature extracted from the face image to be recognized first face feature;

The result receiving module is configured to receive the face recognition result returned by the server, the face recognition result is obtained according to face recognition on the face recognition information.

In a possible implementation manner, the identification mode determination module includes:

The selection page display submodule is used to display the face recognition mode selection page, and the selection page includes the first recognition control and/or the second recognition control;

The first mode determination submodule is configured to determine the face recognition mode of the face image to be recognized as the first recognition mode in response to the first recognition control being triggered, and the first recognition mode is used to send the Face image to be recognized;

The second mode determination submodule is used to determine the face recognition mode of the face image to be recognized as the second recognition mode in response to the triggering of the second recognition control, and the second recognition mode is used to send the Describe the first human face feature obtained by performing feature extraction on the face image to be recognized.

In a possible implementation manner, the information sending module includes:

The information determination sub-module is used to input the face image to be recognized into the first feature extraction model obtained by pre-training in response to the face recognition mode being the second recognition mode, and determine that the output first face feature is a face identifying information;

The information sending sub-module is configured to send the face recognition information to the server.

In a possible implementation manner, the device further includes:

A user information determination submodule, configured to determine current user information in response to the face recognition result being a non-registered user;

The user information sending sub-module is configured to send the current user information to the server, so as to store the current user information and the corresponding second facial features in the user facial feature set.

According to a fourth aspect of the present disclosure, a face recognition device is provided, the device comprising:

The information receiving module is used to receive the face recognition information sent by the terminal device, the face recognition information includes the face image to be recognized, and/or the first person obtained by feature extraction of the face image to be recognized by the terminal device facial features;

A face recognition module, configured to perform face recognition according to the face recognition information to obtain a face recognition result;

A result sending module, configured to send the face recognition result to the terminal device.

In a possible implementation, the face recognition module includes:

The feature extraction submodule is used to perform feature extraction on the face recognition information to obtain a second face feature;

The face recognition sub-module is configured to perform face recognition according to the second face feature to obtain a face recognition result.

In a possible implementation, the feature extraction submodule includes:

A model extraction unit, configured to input the first facial feature or the facial image to be recognized into a pre-trained second feature extraction model, and output the second facial feature.

In a possible implementation, the face recognition submodule includes:

A set determining unit, configured to determine a user face feature set, the user face feature set including at least one user face feature, and user information corresponding to the user face feature;

A feature matching unit, configured to match the second facial features with the user's facial features to obtain a matching result of the user's facial features;

The first result determination unit is configured to use the user information corresponding to the user's face feature as the face recognition result in response to the matching result of the user's face feature being successful;

The second result determination unit is configured to determine that the face recognition result is a non-registered user in response to the matching result of no user's face feature being a successful match.

In a possible implementation manner, the device further includes:

A feature storage module, configured to store the current user information and the corresponding second facial features in the The set of facial features of the user.

According to a fifth aspect of the present disclosure, there is provided an electronic device, including: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to call the instructions stored in the memory to execute the above method.

According to a sixth aspect of the present disclosure, there is provided a computer-readable storage medium, on which computer program instructions are stored, and the above-mentioned method is implemented when the computer program instructions are executed by a processor.

According to a seventh aspect of the present disclosure, there is provided a computer program product, including computer readable codes, or a non-volatile computer readable storage medium bearing computer readable codes, when the computer readable codes are stored in an electronic device When running in the processor of the electronic device, the processor in the electronic device is used to implement the above method.

In the embodiment of the present disclosure, the user can decide whether to upload the face image to the server during the face recognition process, and can upload the feature obtained by extracting the face image when the user does not allow uploading the face image. Face recognition on the server side can also achieve accurate face recognition process.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the accompanying drawings.

Description of drawings

The accompanying drawings here are incorporated into the description and constitute a part of the present description. These drawings show embodiments consistent with the present disclosure, and are used together with the description to explain the technical solution of the present disclosure.

FIG. 1 shows a schematic diagram of an application system of a face recognition method according to an embodiment of the present disclosure;

FIG. 2 shows a flow chart of a face recognition method on the terminal device side according to an embodiment of the present disclosure;

Fig. 3 shows a schematic diagram of a face recognition mode selection page according to an embodiment of the present disclosure;

Fig. 4 shows a schematic diagram of a process of extracting first facial features according to an embodiment of the present disclosure;

FIG. 5 shows a signaling flow chart of a face recognition method according to an embodiment of the present disclosure;

FIG. 6 shows a signaling flow chart of another face recognition method according to an embodiment of the present disclosure;

Fig. 7 shows a signaling flow chart of a process of updating a user's face feature set according to an embodiment of the present disclosure;

FIG. 8 shows a flow chart of a face recognition method on the server side according to an embodiment of the present disclosure;

FIG. 9 shows a schematic diagram of a process of extracting second facial features according to an embodiment of the present disclosure;

FIG. 10 shows a schematic diagram of a face recognition device on the terminal device side according to an embodiment of the present disclosure;

FIG. 11 shows a schematic diagram of a face recognition device on the server side according to an embodiment of the present disclosure;

Fig. 12 shows a schematic diagram of an electronic device according to an embodiment of the present disclosure;

Fig. 13 shows a schematic diagram of an electronic device according to an embodiment of the present disclosure.

Detailed ways

Various exemplary embodiments, features, and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numbers in the figures indicate functionally identical or similar elements. While various aspects of the embodiments are shown in drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as superior or better than other embodiments.

The term "and/or" in this article is just an association relationship describing associated objects, which means that there can be three relationships, for example, A and/or B can mean: A exists alone, A and B exist simultaneously, and there exists alone B these three situations. In addition, the term "at least one" herein means any one of a variety or any combination of at least two of the more, for example, including at least one of A, B, and C, which may mean including from A, Any one or more elements selected from the set formed by B and C.

In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific implementation manners. It will be understood by those skilled in the art that the present disclosure may be practiced without some of the specific details. In some instances, methods, means, components and circuits that are well known to those skilled in the art have not been described in detail so as to obscure the gist of the present disclosure.

Fig. 1 shows a schematic diagram of an application system of a face recognition method according to an embodiment of the present disclosure. As shown in FIG. 1 , the system applying the face recognition method of the embodiment of the present disclosure may include a terminal device 10 and a server 11 connected through a network. Among them, the terminal device 10 is used to determine the face image to be recognized and the face recognition mode, and directly send the face image to be recognized to the server 11 for face recognition according to the face recognition mode. Or extract the first facial feature from the image to be processed according to the face recognition mode, and send the extracted first facial feature to the server 11 for face recognition. The server 11 returns the face recognition result to the terminal device 10 after obtaining the face recognition result based on the received face image to be recognized or the first face feature. Further, in the embodiment of the present disclosure, the face image to be recognized and the first face feature may also be sent to the server 11 for face recognition.

In a possible implementation, the terminal device 10 may be a user equipment (User Equipment, UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (Personal Digital Assistant, PDA), a handheld device , a computing device, a vehicle-mounted device, a wearable device, etc., the subject of execution of the steps of the method may be executed by hardware, or executed by a processor running computer-executable codes. For example, the face recognition method of the embodiments of the present disclosure may be implemented by a processor calling computer-readable instructions stored in a memory. The server 11 can be a single server, or a server cluster composed of multiple servers.

The face recognition method of the embodiments of the present disclosure can be applied to any face recognition scene, such as an application scene in which a smartphone is unlocked through face recognition, or an application scene such as opening a car door by collecting face information through a camera and acquisition device built in a vehicle terminal.

Fig. 2 shows a flow chart of a face recognition method on a terminal device side according to an embodiment of the present disclosure. As shown in FIG. 2, in a possible implementation manner, the face recognition method on the terminal device side of the embodiment of the present disclosure includes the following steps:

Step S10, determining the face image to be recognized.

In a possible implementation manner, a face image to be recognized that requires face recognition is determined. Optionally, the face image to be recognized may be a face image directly collected by a mobile terminal implementing a face recognition method, or an image obtained after preprocessing the collected user's face image. Alternatively, the method of determining the face image to be recognized may also be to receive the face image sent by other electronic devices.

Optionally, when the face image is directly captured by the mobile terminal executing the face recognition method, the face capture page can be displayed on the display device first. When the user triggers the face acquisition control in the display device, the camera device is turned on, and the user's face image is collected as the face image to be recognized.

Step S20, determining the face recognition mode of the face image to be recognized.

In a possible implementation manner, a face recognition mode of the face image to be recognized is determined, and the face recognition mode is used to characterize a recognition manner of the face image to be recognized. Optionally, the face recognition mode may be determined according to attribute information of the face image to be recognized, user information of the user in the face image to be recognized, and the like. For example, while obtaining the face image to be recognized, determine the geographical location where the face image to be recognized is obtained, and when the legal regulations corresponding to the geographical location do not allow the upload of the face image to be recognized, determine that the face recognition mode is the first Two modes. Further, when the legal provisions corresponding to the geographic location allow uploading of the face image to be recognized, the face recognition mode is determined to be the first mode.

Further, in the embodiments of the present disclosure, the user can also select the face recognition mode by himself through human-computer interaction with the user. For example, a face recognition mode selection page may be displayed through a display device of the mobile terminal, and the selection page includes a first recognition control and a second recognition control. In response to the triggering of the first recognition control, it is determined that the face recognition mode of the face image to be recognized is the first recognition mode, and the first recognition mode is used to send the face image to be recognized. In response to the triggering of the second recognition control, it is determined that the face recognition mode of the face image to be recognized is the second recognition mode, and the second recognition mode is used to send the first face feature obtained by feature extraction from the face image to be recognized. That is to say, it is up to the user to choose whether to upload the face image to be recognized to the server for face recognition.

Optionally, if permitted by law, the embodiment of the present disclosure may also determine a face recognition mode that simultaneously sends the face image to be recognized and the first face feature as the third recognition mode.

Fig. 3 shows a schematic diagram of a face recognition mode selection page according to an embodiment of the present disclosure. As shown in FIG. 3 , the terminal device executing the face recognition method can display a face recognition mode selection page 30 through a display device, which includes a first recognition control 31 and a second recognition control 32 . When the user triggers the first recognition control 31 by clicking or other means, the terminal device determines that the user allows the upload of the face image to be recognized, that is, determines that the face recognition mode is the first recognition mode for sending the face image to be recognized. When the user triggers the second recognition control 32 by clicking or other means, the terminal device determines that the user is not allowed to upload the face image to be recognized, that is, determines that the face recognition mode is to send the face image to be recognized and extract the second face feature to obtain the first face feature. recognition pattern.

Further, the face recognition mode selection page 30 may only include one recognition control. When the user triggers the recognition control by clicking or other means, the terminal device determines that the user allows uploading the face image to be recognized and the first face feature, and directly The face recognition mode is determined as the third recognition mode in which the image of the face to be recognized and the first face feature are simultaneously sent.

In a possible implementation manner, the embodiment of the present disclosure may also determine the face recognition mode by combining the foregoing multiple manners. For example, while obtaining the face image to be recognized, determine the geographical location where the face image to be recognized is obtained, and when the legal regulations corresponding to the geographical location do not allow the upload of the face image to be recognized, determine that the face recognition mode is the first Two modes. When the legal provisions corresponding to the geographic location allow uploading of facial images to be recognized, the facial recognition mode is determined to be the first mode. Further, when the legal provisions corresponding to the geographic location require the consent of the user to upload the face image to be recognized, the face recognition mode selection page is displayed, and the user can select the required face recognition mode by himself.

Step S30, sending face recognition information to the server according to the face recognition mode.

In a possible implementation manner, after determining the face recognition mode, the terminal device may determine face recognition information according to the face recognition mode, and send the face recognition information to the server for face recognition. Wherein, the face recognition information may include a face image to be recognized, and/or a first face feature obtained by feature extraction from the face image to be recognized. For example, when the face recognition mode corresponding to the face image to be recognized is the first recognition mode, the face recognition information may be directly determined as the face image to be recognized. When the face recognition mode corresponding to the face image to be recognized is the second recognition mode, it may be determined that the face recognition information is the first face feature extracted from the face image to be recognized. Further, when the face recognition mode corresponding to the face image to be recognized is the third recognition mode, face recognition information including the face image to be recognized and the first face feature may also be determined.

That is to say, when the face recognition mode is the first recognition mode, the face image to be recognized may be sent to the server. When the face recognition mode is the second recognition mode, the first face feature is sent to the server. Or when the face recognition mode is the third recognition mode, send the face image to be recognized and the first face feature to the server. Wherein, the first face feature may be determined through a first feature extraction model obtained in advance through training in the terminal device. For example, in response to the face recognition mode being the second recognition mode, the face image to be recognized is input into the pre-trained first feature extraction model, the first face feature output is determined to be face recognition information, and the face recognition information is sent to the server. information.

Fig. 4 shows a schematic diagram of a process of extracting first facial features according to an embodiment of the present disclosure. As shown in FIG. 4 , when the face recognition mode corresponding to the face image to be recognized is the first recognition mode, the terminal device inputs the face image 40 to be recognized into the first feature extraction model 41 obtained by pre-training for preliminary feature extraction. , to output the first human face feature 42 representing the key point feature of the human face.

Step S40, receiving the face recognition result returned by the server.

In a possible implementation manner, after receiving the face recognition information, the server performs face recognition according to the face recognition information to obtain a face recognition result. Further, the obtained face recognition result is returned to the terminal device. Wherein, the process of face recognition by the server may be to input the face recognition information into the pre-trained second feature extraction model through the server, output the second face features, and The face feature set is matched to obtain a face recognition result. The user face feature set includes at least one user face feature and user information corresponding to the user face feature.

That is to say, the server pre-stores a user face feature set including at least one user face feature and user information corresponding to the user face feature. After receiving the face recognition information, the server can input the face recognition information into the pre-trained second feature extraction model, and output the second face features. Wherein, for face recognition information with different contents, the second feature extraction models for feature extraction are the same or different. Further, when the second feature extraction model is the same and the face recognition information is a face image to be recognized, the server can also perform the first feature extraction model obtained through pre-training before inputting the second feature extraction model. feature extraction. When the face recognition information is the first face feature, the server performs feature extraction on the first face feature again through the second feature extraction model to obtain a second face feature with higher accuracy.

In the embodiment of the present disclosure, when the face recognition information transmitted by the terminal device is the first face feature, the second face feature is extracted from the first face feature, and the second face feature with higher accuracy is obtained according to the second face feature extraction. Face recognition, improve the accuracy of face recognition results.

Optionally, after obtaining the second facial feature corresponding to the image to be recognized, calculate the similarity between the second facial feature and each user's facial feature in the user's facial feature set to determine the face recognition result. Wherein, when there is a user's face feature whose similarity is greater than a threshold, it is determined that the user's face feature matches the second face feature, and the user information corresponding to the user's face feature is returned as a face recognition result. When there is no facial feature of the user whose similarity is greater than the threshold, it is determined that no facial feature of the user matches the second facial feature, and the face recognition result is returned as a non-registered user.

Fig. 5 shows a signaling flowchart of a face recognition method according to an embodiment of the present disclosure. As shown in Figure 5, in a possible implementation, the face recognition method may include the following steps:

Step 50. Determine the face image to be recognized through the terminal device.

Step 51. Determine the face recognition mode of the face image to be recognized as the first recognition mode through the terminal device.

Step 52, directly send the face image to be recognized to the server through the terminal device.

Step 53, extracting high-precision second facial features according to the facial image to be recognized through the server.

Step 54: The server matches the second face feature with the pre-stored user face feature set to obtain a face recognition result.

Step 55: Return the face recognition result to the terminal device through the server.

The above face recognition method can directly upload images for feature extraction when permitted by law and users, respects user privacy and ensures the accuracy of recognition results.

Fig. 6 shows a signaling flowchart of another face recognition method according to an embodiment of the present disclosure. As shown in Figure 6, in another possible implementation, the face recognition method may also include the following steps:

Step 60. Determine the face image to be recognized through the terminal device.

Step 61. Determine, through the terminal device, that the face recognition mode of the face image to be recognized is the second recognition mode.

Step 62: Perform preliminary feature extraction on the face image to be recognized through the terminal device to obtain the first face feature.

Step 63: Send the first face feature to the server through the terminal device.

Step 64: Perform secondary feature extraction on the first facial features through the server to obtain high-precision second facial features.

Step 65: The server matches the second face feature with the pre-stored user face feature set to obtain a face recognition result.

Step 66: Return the face recognition result to the terminal device through the server.

The above-mentioned face recognition method can upload the first face feature obtained by image feature extraction for face recognition without the permission of the user, and ensure the accuracy of the recognition result by performing secondary feature extraction on the first face feature at the server end. degree of accuracy.

Further, when the face recognition result is a non-registered user, the terminal device may store the user's current user information and the corresponding second face feature to the server, so as to add the current user as a registered user. For example, the current user information is determined in response to the face recognition result being a non-registered user. The current user information is sent to the server, so that the current user information and the corresponding second facial features are stored in the user facial feature set. Optionally, the way to determine the current user information may be to display the user information collection page through the display device of the terminal device, and after the user enters the user information through the user information collection page, the user information is determined as the current user information. Further, the second face feature corresponding to the face image to be recognized is stored in the user face feature set as the user face feature corresponding to the current user information.

Fig. 7 shows a signaling flowchart of a process of updating a user's face feature set according to an embodiment of the present disclosure. As shown in FIG. 7, in a possible implementation, the face recognition method of the embodiment of the present disclosure may also include the following steps:

Step 70: Determine through the terminal device that the face recognition result is a non-registered user.

Step 71. If the face recognition result is a non-registered user, determine the current user information through the terminal device.

Step 72: Send the determined current user information to the server through the terminal device.

Step 73: Update the current user information and the second facial feature corresponding to the image to be recognized to the user facial feature set through the server.

In the face recognition process, the embodiment of the present disclosure can determine whether to upload the face image to the server according to legal regulations, or the user can decide whether to upload the face image to the server, so as to ensure compliance with the law and respect for user privacy. Moreover, when the user is not allowed to upload the face image, the features extracted from the face image can be uploaded to the server, and the server can perform face recognition after secondary feature extraction to accurately determine the face recognition result.

Fig. 8 shows a flow chart of a face recognition method on the server side according to an embodiment of the present disclosure. As shown in FIG. 8, in a possible implementation manner, the face recognition method on the server side of the embodiment of the present disclosure may include the following steps:

Step S50, receiving face recognition information sent by the terminal device.

In a possible implementation manner, the face recognition information includes a face image to be recognized, and/or a first face feature obtained by feature extraction of the face image to be recognized by the terminal device. Wherein, when the face recognition mode corresponding to the face image to be recognized is the first face recognition mode, the face recognition information includes the face image to be recognized. When the face recognition mode corresponding to the face image to be recognized is the second face recognition mode, the face recognition information includes the first face feature obtained by the preliminary feature extraction of the face image to be recognized.

Step S60, perform face recognition according to the face recognition information, and obtain a face recognition result.

In a possible implementation manner, due to the limitation of the computing power of the terminal device, the first face feature obtained by preliminary extraction of the face image to be recognized is a face feature with low precision. Therefore, when the face recognition information includes the face image to be recognized or the first face feature, it is necessary to perform feature extraction on the face recognition information before performing face recognition. For example, feature extraction is performed on the face recognition information to obtain the second face feature. Face recognition is performed according to the second face feature to obtain a face recognition result.

Optionally, the feature extraction process for face recognition information may be to input the first face feature or the face image to be recognized into a pre-trained second feature extraction model, and output the second face feature. That is to say, when the face recognition information is a face image to be recognized, the high-precision second face feature is directly extracted through the second feature extraction model of the server. When the face recognition information is the first face feature with low precision, the first face feature is extracted again through the second feature extraction model of the server to obtain the second face feature with high precision.

Further, after obtaining the second facial feature, the server determines the user facial feature set, which includes at least one user facial feature and user information corresponding to the user facial feature. Matching is performed based on the second facial features and the user's facial features to obtain a matching result of the user's facial features. In response to the fact that there is a matching result of the user's face feature, the matching result is successful, and the user information corresponding to the user's face feature is used as the face recognition result. In response to the fact that the matching result of no facial features of the user is successful, it is determined that the facial recognition result is a non-registered user. Optionally, the manner of matching the second facial feature with the user's facial feature may be to calculate the similarity between the second facial feature and each user's facial feature.

Fig. 9 shows a schematic diagram of a process of extracting second facial features according to an embodiment of the present disclosure. As shown in FIG. 9 , after receiving the face recognition information 90 , input the face recognition information 90 into the pre-trained second feature extraction model 91 , and output the high-precision second face feature 92 .

Step S40, sending the face recognition result to the terminal device.

In a possible implementation manner, the server returns the face recognition result to the terminal device after determining the face recognition result. Further, when the face recognition result is a non-registered user, the terminal device may also determine the current user information of the user, so as to update the user's face feature set. That is to say, in response to receiving the current user information returned by the terminal device when the face recognition result is a non-registered user, the current user information and the corresponding second face feature may be stored in the user face feature set.

It can be understood that the above-mentioned method embodiments mentioned in this disclosure can all be combined with each other to form a combined embodiment without violating the principle and logic. Due to space limitations, this disclosure will not repeat them. Those skilled in the art can understand that, in the above method in the specific implementation manner, the specific execution order of each step should be determined according to its function and possible internal logic.

In addition, the present disclosure also provides face recognition devices, electronic equipment, computer-readable storage media, and programs, all of which can be used to implement any face recognition method provided in the present disclosure. For the corresponding technical solutions and descriptions, refer to the method section Corresponding records are not repeated here.

Fig. 10 shows a schematic diagram of a face recognition device on the terminal device side according to an embodiment of the present disclosure. As shown in Fig. 10 , the device includes:

An image determination module 100, configured to determine a face image to be recognized;

A recognition mode determination module 101, configured to determine the face recognition mode of the face image to be recognized;

The information sending module 102 is configured to send face recognition information to the server according to the face recognition mode, the face recognition information includes a face image to be recognized, and/or the face image to be recognized is obtained by feature extraction The first face feature of

The result receiving module 103 is configured to receive the face recognition result returned by the server, and the face recognition result is obtained by performing face recognition on the face recognition information.

In a possible implementation manner, the information sending module includes:

In a possible implementation manner, the device further includes:

FIG. 11 shows a schematic diagram of a face recognition device on the server side according to an embodiment of the present disclosure. As shown in FIG. 11 , the device includes:

The information receiving module 110 is configured to receive the face recognition information sent by the terminal device, the face recognition information includes the face image to be recognized, and/or the first feature extracted by the terminal device from the face image to be recognized. facial features;

A face recognition module 111, configured to perform face recognition according to the face recognition information to obtain a face recognition result;

The result sending module 112 is configured to send the face recognition result to the terminal device.

In a possible implementation, the face recognition module includes:

In a possible implementation, the feature extraction submodule includes:

In a possible implementation, the face recognition submodule includes:

In a possible implementation manner, the device further includes:

In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the methods described in the method embodiments above, and its specific implementation can refer to the description of the method embodiments above. For brevity, here No longer.

Embodiments of the present disclosure also provide a computer-readable storage medium, on which computer program instructions are stored, and the above-mentioned method is implemented when the computer program instructions are executed by a processor. Computer readable storage media may be volatile or nonvolatile computer readable storage media.

An embodiment of the present disclosure also proposes an electronic device, including: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to invoke the instructions stored in the memory to execute the above method.

An embodiment of the present disclosure also provides a computer program product, including computer-readable codes, or a non-volatile computer-readable storage medium carrying computer-readable codes, when the computer-readable codes are stored in a processor of an electronic device When running in the electronic device, the processor in the electronic device executes the above method.

Electronic devices may be provided as terminals, servers, or other forms of devices.

Fig. 12 shows a schematic diagram of an electronic device 800 according to an embodiment of the present disclosure. For example, the electronic device 800 may be a terminal such as a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, or a personal digital assistant.

12, electronic device 800 may include one or more of the following components: processing component 802, memory 804, power supply component 806, multimedia component 808, audio component 810, input/output (I/O) interface 812, sensor component 814 , and the communication component 816.

The processing component 802 generally controls the overall operations of the electronic device 800, such as those associated with display, telephone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above method. Additionally, processing component 802 may include one or more modules that facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802 .

The memory 804 is configured to store various types of data to support operations at the electronic device 800 . Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phonebook data, messages, pictures, videos, and the like. The memory 804 can be implemented by any type of volatile or non-volatile storage device or their combination, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), Magnetic Memory, Flash Memory, Magnetic or Optical Disk.

The power supply component 806 provides power to various components of the electronic device 800 . Power components 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for electronic device 800 .

The multimedia component 808 includes a screen providing an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may not only sense a boundary of a touch or swipe action, but also detect duration and pressure associated with the touch or swipe action. In some embodiments, the multimedia component 808 includes a front camera and/or a rear camera. When the electronic device 800 is in an operation mode, such as a shooting mode or a video mode, the front camera and/or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have focal length and optical zoom capability.

The audio component 810 is configured to output and/or input audio signals. For example, the audio component 810 includes a microphone (MIC), which is configured to receive external audio signals when the electronic device 800 is in operation modes, such as call mode, recording mode and voice recognition mode. Received audio signals may be further stored in memory 804 or sent via communication component 816 . In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

The I/O interface 812 provides an interface between the processing component 802 and a peripheral interface module, which may be a keyboard, a click wheel, a button, and the like. These buttons may include, but are not limited to: a home button, volume buttons, start button, and lock button.

Sensor assembly 814 includes one or more sensors for providing status assessments of various aspects of electronic device 800 . For example, the sensor component 814 can detect the open/closed state of the electronic device 800, the relative positioning of components, such as the display and the keypad of the electronic device 800, the sensor component 814 can also detect the electronic device 800 or a Changes in position of components, presence or absence of user contact with electronic device 800 , electronic device 800 orientation or acceleration/deceleration and temperature changes in electronic device 800 . Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects in the absence of any physical contact. Sensor assembly 814 may also include an optical sensor, such as a complementary metal-oxide-semiconductor (CMOS) or charge-coupled device (CCD) image sensor, for use in imaging applications. In some embodiments, the sensor component 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.

The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as a wireless network (WiFi), a second generation mobile communication technology (2G) or a third generation mobile communication technology (3G), or a combination thereof. In an exemplary embodiment, the communication component 816 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (BT) technology and other technologies.

In an exemplary embodiment, electronic device 800 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable A programmable gate array (FPGA), controller, microcontroller, microprocessor or other electronic component implementation for performing the methods described above.

In an exemplary embodiment, there is also provided a non-volatile computer-readable storage medium, such as the memory 804 including computer program instructions, which can be executed by the processor 820 of the electronic device 800 to implement the above method.

FIG. 13 shows a schematic diagram of an electronic device 1900 according to an embodiment of the present disclosure. For example, electronic device 1900 may be provided as a server. Referring to FIG. 13 , electronic device 1900 includes processing component 1922 , which further includes one or more processors, and a memory resource represented by memory 1932 for storing instructions executable by processing component 1922 , such as application programs. The application programs stored in memory 1932 may include one or more modules each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

Electronic device 1900 may also include a power supply component 1926 configured to perform power management of electronic device 1900, a wired or wireless network interface 1950 configured to connect electronic device 1900 to a network, and an input-output (I/O) interface 1958 . The electronic device 1900 can operate based on the operating system stored in the memory 1932, such as the Microsoft server operating system (Windows Server ^TM ), the graphical user interface-based operating system (Mac OS X ^TM ) introduced by Apple Inc., and the multi-user and multi-process computer operating system (Unix ^™ ), a free and open source Unix-like operating system (Linux ^™ ), an open source Unix-like operating system (FreeBSD ^™ ), or the like.

In an exemplary embodiment, there is also provided a non-transitory computer-readable storage medium, such as the memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the electronic device 1900 to implement the above method.

The present disclosure can be a system, method and/or computer program product. A computer program product may include a computer readable storage medium having computer readable program instructions thereon for causing a processor to implement various aspects of the present disclosure.

A computer readable storage medium may be a tangible device that can retain and store instructions for use by an instruction execution device. A computer readable storage medium may be, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or flash memory), static random access memory (SRAM), compact disc read only memory (CD-ROM), digital versatile disc (DVD), memory stick, floppy disk, mechanically encoded device, such as a printer with instructions stored thereon A hole card or a raised structure in a groove, and any suitable combination of the above. As used herein, computer-readable storage media are not to be construed as transient signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., pulses of light through fiber optic cables), or transmitted electrical signals.

Computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a respective computing/processing device, or downloaded to an external computer or external storage device over a network, such as the Internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers. A network adapter card or a network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing/processing device .

Computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or Source or object code written in any combination, including object-oriented programming languages—such as Smalltalk, C++, etc., and conventional procedural programming languages—such as the “C” language or similar programming languages. Computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server implement. In cases involving a remote computer, the remote computer can be connected to the user computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (such as via the Internet using an Internet service provider). connect). In some embodiments, an electronic circuit, such as a programmable logic circuit, field programmable gate array (FPGA), or programmable logic array (PLA), can be customized by utilizing state information of computer-readable program instructions, which can Various aspects of the present disclosure are implemented by executing computer readable program instructions.

Aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It should be understood that each block of the flowcharts and/or block diagrams, and combinations of blocks in the flowcharts and/or block diagrams, can be implemented by computer-readable program instructions.

These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine such that when executed by the processor of the computer or other programmable data processing apparatus , producing an apparatus for realizing the functions/actions specified in one or more blocks in the flowchart and/or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause computers, programmable data processing devices and/or other devices to work in a specific way, so that the computer-readable medium storing instructions includes An article of manufacture comprising instructions for implementing various aspects of the functions/acts specified in one or more blocks in flowcharts and/or block diagrams.

It is also possible to load computer-readable program instructions into a computer, other programmable data processing device, or other equipment, so that a series of operational steps are performed on the computer, other programmable data processing device, or other equipment to produce a computer-implemented process , so that instructions executed on computers, other programmable data processing devices, or other devices implement the functions/actions specified in one or more blocks in the flowcharts and/or block diagrams.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, a portion of a program segment, or an instruction that includes one or more Executable instructions. In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved. It should also be noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented by a dedicated hardware-based system that performs the specified function or action , or may be implemented by a combination of dedicated hardware and computer instructions.

The computer program product can be specifically realized by means of hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK) etc. wait.

Having described various embodiments of the present disclosure above, the foregoing description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and alterations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principle of each embodiment, practical application or improvement of technology in the market, or to enable other ordinary skilled in the art to understand each embodiment disclosed herein.

Claims

A face recognition method applied to a terminal device, characterized in that the method includes:

Determine the face image to be recognized;

Determine the face recognition mode of the face image to be recognized;

Send face recognition information to the server according to the face recognition mode, the face recognition information includes a face image to be recognized, and/or a first face feature obtained by feature extraction of the face image to be recognized ;

and receiving a face recognition result returned by the server, where the face recognition result is obtained by performing face recognition on the face recognition information.
The method according to claim 1, wherein the determining the face recognition mode of the face image to be recognized comprises:

Displaying a face recognition mode selection page, the selection page including a first recognition control and/or a second recognition control;

In response to the triggering of the first recognition control, determine the face recognition mode of the face image to be recognized as the first recognition mode, and the first recognition mode is used to send the face image to be recognized;

In response to the triggering of the second recognition control, it is determined that the face recognition mode of the face image to be recognized is the second recognition mode, and the second recognition mode is used to send a feature extraction function to the face image to be recognized. The obtained first face features.
The method according to claim 1 or 2, wherein the sending face recognition information to the server according to the face recognition mode comprises:

In response to the face recognition mode being the second recognition mode, inputting the face image to be recognized into a first feature extraction model obtained through pre-training, and determining that the output first face feature is face recognition information;

Send the face recognition information to the server.
The method according to any one of claims 1-3, wherein the determination process of the face recognition result comprises:

Input the face recognition information into the pre-trained second feature extraction model through the server, and output the second face feature;

According to the matching of the second face feature with the pre-stored user face feature set, the face recognition result is obtained, and the user face feature set includes at least one user face feature corresponding to the user face feature user information.
According to the method described in claim 4, it is characterized in that, described method also comprises:

In response to the face recognition result being a non-registered user, determine current user information;

Send the current user information to the server, so as to store the current user information and the corresponding second facial features in the user facial feature set.
A face recognition method applied to a server, characterized in that the method comprises:

receiving face recognition information sent by the terminal device, the face recognition information including the face image to be recognized, and/or the first face feature obtained by feature extraction of the face image to be recognized by the terminal device;

performing face recognition according to the face recognition information to obtain a face recognition result;

Sending the face recognition result to the terminal device.
The method according to claim 6, wherein performing face recognition according to the face recognition information to obtain a face recognition result comprises:

performing feature extraction on the face recognition information to obtain a second face feature;

Performing face recognition according to the second face features to obtain a face recognition result.
method according to claim 7, is characterized in that, described described facial recognition information is carried out feature extraction, obtains the second facial features and comprises:

Inputting the first human face feature or the to-be-recognized human face image into the pre-trained second feature extraction model, and outputting the second human face feature.
The method according to claim 7 or 8, wherein performing face recognition according to the second face feature, and obtaining a face recognition result comprises:

Determining a user face feature set, the user face feature set including at least one user face feature, and user information corresponding to the user face feature;

matching according to the second facial features and the user's facial features to obtain a matching result of the user's facial features;

In response to the matching result of the existence of the user's face feature being a successful match, the user information corresponding to the user's face feature is used as the face recognition result;

In response to the fact that the matching result of no facial features of the user is successful, it is determined that the facial recognition result is a non-registered user.
The method according to claim 9, characterized in that the method further comprises:

In response to receiving the current user information returned by the terminal device when the face recognition result is a non-registered user, storing the current user information and the corresponding second face feature in the user face feature gather.
A face recognition device, characterized in that the device comprises:

An image determination module, configured to determine a face image to be recognized;

A recognition mode determination module, configured to determine the face recognition mode of the face image to be recognized;

An information sending module, configured to send face recognition information to the server according to the face recognition mode, the face recognition information includes a face image to be recognized, and/or a feature extracted from the face image to be recognized first face feature;

The result receiving module is configured to receive the face recognition result returned by the server, the face recognition result is obtained according to face recognition on the face recognition information.
A face recognition device, characterized in that the device comprises:

The information receiving module is used to receive the face recognition information sent by the terminal device, the face recognition information includes the face image to be recognized, and/or the first person obtained by feature extraction of the face image to be recognized by the terminal device facial features;

A face recognition module, configured to perform face recognition according to the face recognition information to obtain a face recognition result;

A result sending module, configured to send the face recognition result to the terminal device.
An electronic device, characterized in that it comprises:

processor;

memory for storing processor-executable instructions;

Wherein, the processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1-10.
A computer-readable storage medium, on which computer program instructions are stored, wherein, when the computer program instructions are executed by a processor, the method according to any one of claims 1 to 10 is implemented.
A computer program product, comprising computer-readable codes, or a non-volatile computer-readable storage medium bearing computer-readable codes, characterized in that, when the computer-readable codes are run in a processor of an electronic device , the processor in the electronic device executes the method for realizing any one of claims 1-10.