CN116071804A

CN116071804A - Face recognition method and device and electronic equipment

Info

Publication number: CN116071804A
Application number: CN202310101512.6A
Authority: CN
Inventors: 吴筝
Original assignee: Liulv Technology Co ltd
Current assignee: Liulv Technology Co ltd
Priority date: 2023-01-18
Filing date: 2023-01-18
Publication date: 2023-05-05

Abstract

The application provides a face recognition method, a face recognition device and electronic equipment, which can improve the one-time passing rate of face recognition. The method comprises the steps of obtaining a first face image and a target face image of a user; extracting an image of a five-sense organ region from the first face image and the target face image; taking the image with the similarity meeting the threshold requirement in the image of the five sense organs as a candidate image of the five sense organs; taking the first face image as a base map, and fusing the candidate facial region image with the base map in a corresponding region in the base map to generate a second face image; and identifying the user according to the second face image. In the embodiment of the application, when a single image is identified, low one-time passing rate caused by large difference between part of the five sense organs and the target image can be avoided, and user experience can be improved.

Description

Face recognition method and device and electronic equipment

Technical Field

The present invention relates to the field of image processing, and in particular, to a method and apparatus for face recognition, and an electronic device.

Background

With the development of computer and internet technologies, face recognition (face recognition) technology brings great convenience for user authentication.

In the related art, the face recognition method generally performs face detection and feature extraction on the whole face image, performs similarity measurement according to the extracted features, and performs face image verification according to a set threshold.

However, in the process of recognition, the user cannot perceive the shooting time, so that the shot facial image of the user may have expression changes such as eye closure, or light changes, facial angle changes and the like in the image, which may cause a large difference between a partial region in the image and the target image, thereby reducing the recognition rate, and even if the user adjusts the gesture for many times, the user cannot pass verification, thereby bringing bad experience to the user.

Therefore, how to improve the recognition once-through rate of the face recognition technology is a technical problem to be solved.

Disclosure of Invention

The application provides a face recognition method, a face recognition device and electronic equipment, which can improve the one-time passing rate of face recognition and improve user experience.

In a first aspect, a face recognition method is provided, the method including acquiring a first face image and a target face image of a user, the first face image including at least two face images of the user; extracting images of five sense organ areas from the first face image and the target face image, wherein the five sense organ areas comprise at least two areas of left eyebrow, right eyebrow, left eye, right eye, nose and mouth; based on the image characteristic information of the five-sense organ region, determining the similarity between the image of the five-sense organ region in the first face image and the image of the region corresponding to the target face image, and taking the image of which the similarity meets the threshold requirement in the image of the five-sense organ region as a candidate five-sense organ region image; taking the first face image as a base map, and fusing the candidate facial region image with the base map in a corresponding region in the base map to generate a second face image; and identifying the user according to the second face image.

Therefore, in the embodiment of the application, the five sense organs regions with high similarity with the target image in the face images can be identified after fusion processing, so that the problem that the identification rate is low due to the fact that the difference between part of regions in the images and the target image is large when the original face images are identified is avoided, the one-time passing rate of face identification can be improved, and the user experience is improved.

With reference to the first aspect, in certain implementation manners of the first aspect, the extracting an image of a five-sense organ region from the first face image and the target face image includes: constructing a five-sense organ detector by utilizing a self-adaptive enhancement algorithm, wherein the five-sense organ detector is a screening type cascade classifier; inputting the first face image and the target face image into the five sense organs detector, wherein the five sense organs detector sequentially detects different five sense organs areas from the face image; and extracting from the first face image and the target face image according to the position and the size of the five-sense organ region to obtain images of different five-sense organ regions.

Therefore, in the embodiment of the application, the self-adaptive enhancement algorithm is utilized to construct the facial feature detector, the facial feature region image can be accurately obtained, the facial feature region with high similarity with the target image in the plurality of face images is fused, the problem that the recognition rate is low due to the fact that the difference between part of the region in the image and the target image is large when the original face image is recognized is avoided, the once-through rate of face recognition can be improved, and the user experience is improved.

With reference to the first aspect, in certain implementation manners of the first aspect, the extracting an image of a five-sense organ region from the first face image and the target face image includes: acquiring face features, wherein the face features are the features of the first face image and the target face image; taking the face features as input of a convolutional neural network extraction model, taking a parameter set of five sense organs as weight of the convolutional neural network extraction model, and obtaining the five sense organs features detected by the convolutional neural network extraction model in the face features, wherein the parameter set of the five sense organs features comprises five sense organ feature parameters of different five sense organs, and the five sense organ feature parameters corresponding to the five sense organs are used when the features of the different five sense organs are detected; and extracting the image of the five sense organs according to the features of the five sense organs.

With reference to the first aspect, in certain implementation manners of the first aspect, the image feature information of the five-element region includes a five-element contour line in an image of the five-element region, where the determining, based on the image feature information of the five-element region, a similarity between the image of the five-element region in the first face image and the image of the region corresponding to the target face image includes: and determining the similarity of the image of the five-sense organ region in the first face image and the image of the region corresponding to the target face image based on the five-sense organ contour line in the image of the five-sense organ region, wherein the five-sense organ contour line is a closed two-dimensional curve surrounding the five-sense organ region.

With reference to the first aspect, in certain implementation manners of the first aspect, the determining, based on a facial contour line in the image of the facial region, a similarity between the image of the facial region in the first face image and the image of the region corresponding to the target face image includes: calculating the curvature K (t, sigma) of the outline of the five sense organs through the outline of the five sense organs,

wherein t is any parameter, x (t, sigma) and y (t, sigma) are respectively the results of convolution of the abscissa x (t) of each point on the five-sense organ contour line, the ordinate y (t) of each point on the five-sense organ contour line and the Gaussian function G (t, sigma), and x _t (t, sigma) and x _tt (t, sigma) is a first and second derivative of x (t, sigma) to t, y, respectively _t (t, sigma) and y _tt (t, sigma) is a first-order derivative function and a second-order derivative function of y (t, sigma) to t respectively, sigma is a standard deviation in a Gaussian function, v is a parameter, deltav is a step length, and x (v) and y (v) are functions of v and the transverse and longitudinal coordinates on the outline of the five sense organs respectively; determining an extreme point set c on the facial contour line in the first face image through curvature detection ₁ And an extreme point set c on the facial contour line in the target face image ₂ The method comprises the steps of carrying out a first treatment on the surface of the According to the extreme point set c ₁ And the extreme point set c ₂ The hausdorff distance between determines the similarity.

With reference to the first aspect, in certain implementation manners of the first aspect, the taking the first face image as a base map includes: and taking one first face image with the largest candidate facial region image as the base map.

In a second aspect, there is provided an apparatus for face recognition, the apparatus comprising: the image acquisition module is used for acquiring a first face image and a target face image of a user, wherein the first face image comprises at least two face images of the user; a processor, configured to extract an image of a five-sense organ region from the first face image and the target face image, where the five-sense organ region includes at least two regions of a left eyebrow, a right eyebrow, a left eye, a right eye, a nose, and a mouth; the processor is further configured to determine, based on image feature information of the five-element region, a similarity between an image of the five-element region in the first face image and an image of a region corresponding to the target face image, and use an image of the five-element region, in which the similarity meets a threshold requirement, as a candidate five-element region image; the processor is further configured to fuse the candidate facial region image with the base map in a corresponding region in the base map to generate a second facial image by using the first facial image as the base map; the processor is further configured to identify the user according to the second face image.

With reference to the second aspect, in some implementations of the second aspect, the extracting an image of a five-element region from the first face image and the target face image, the processor is specifically configured to: constructing a five-sense organ detector by utilizing a self-adaptive enhancement algorithm, wherein the five-sense organ detector is a screening type cascade classifier; inputting the first face image and the target face image into the five sense organs detector, wherein the five sense organs detector sequentially detects different five sense organs areas from the face image; and extracting from the first face image and the target face image according to the position and the size of the five-sense organ region to obtain images of different five-sense organ regions.

With reference to the second aspect, in some implementations of the second aspect, the extracting an image of a five-element region from the first face image and the target face image, the processor is specifically configured to: acquiring face features, wherein the face features are the features of the first face image and the target face image; taking the face features as input of a convolutional neural network extraction model, taking a parameter set of five sense organs as weight of the convolutional neural network extraction model, and obtaining the five sense organs features detected by the convolutional neural network extraction model in the face features, wherein the parameter set of the five sense organs features comprises five sense organ feature parameters of different five sense organs, and the five sense organ feature parameters corresponding to the five sense organs are used when the features of the different five sense organs are detected; and extracting the image of the five sense organs according to the features of the five sense organs.

With reference to the second aspect, in certain implementation manners of the second aspect, the image feature information of the five-element region includes a five-element contour line in an image of the five-element region, wherein the determining, based on the image feature information of the five-element region, a similarity between the image of the five-element region in the first face image and an image of a region corresponding to the target face image is specifically configured to: and determining the similarity of the image of the five-sense organ region in the first face image and the image of the region corresponding to the target face image based on the five-sense organ contour line in the image of the five-sense organ region, wherein the five-sense organ contour line is a closed two-dimensional curve surrounding the five-sense organ region.

With reference to the second aspect, in some implementations of the second aspect, the determining, based on a facial contour line in the image of the facial region, a similarity between the image of the facial region in the first face image and the image of the region corresponding to the target face image is specifically configured to: calculating the curvature K (t, sigma) of the outline of the five sense organs through the outline of the five sense organs,

/>

wherein t is any parameter, x (t, sigma) and y (t, sigma) are respectively the results of convolution of the abscissa x (t) of each point on the five-sense organ contour line, the ordinate y (t) of each point on the five-sense organ contour line and the Gaussian function G (t, sigma), and x _t (t, sigma) and x _tt (t, sigma) is a first and second derivative of x (t, sigma) to t, y, respectively _t (t, sigma) and y _tt (t, sigma) is a first-order derivative function and a second-order derivative function of y (t, sigma) to t, respectively, sigma is the standard deviation in the Gaussian function, v is a parameter, deltav is a step size, x (v)And y (v) is a function of v and the abscissa on the outline of the five sense organs, respectively; determining an extreme point set c on the facial contour line in the first face image through curvature detection ₁ And an extreme point set c on the facial contour line in the target face image ₂ The method comprises the steps of carrying out a first treatment on the surface of the According to the extreme point set c ₁ And the extreme point set c ₂ The hausdorff distance between determines the similarity.

With reference to the second aspect, in some implementations of the second aspect, the first face image is taken as a base map, and the processor is specifically configured to: and taking one first face image with the largest candidate facial region image as the base map.

In a third aspect, a computer readable medium is provided, on which a computer program is stored, characterized in that the program, when executed by a computer, performs the method of face recognition in any one of the possible implementations of the first aspect or the first aspect.

In a fourth aspect, a computer program product is provided which, when executed by a computer, implements the method of face recognition in the first aspect or any one of the possible implementations of the first aspect.

In a fifth aspect, there is provided an electronic device comprising means for implementing face recognition in the second aspect or any one of the possible implementations of the second aspect.

In a sixth aspect, there is provided an electronic device comprising: a storage device having a computer program stored thereon; processing means for executing said computer program in said storage means to carry out the steps of the first aspect or any of the possible implementations of the first aspect.

In a seventh aspect, a chip system is provided, where the chip system includes a processor and a data interface, where the processor reads instructions stored on a memory through the data interface to implement the method of the first aspect and any implementation manner thereof.

Drawings

Fig. 1 is a schematic flow chart of a face recognition method according to an embodiment of the present invention.

Fig. 2 is a flow chart of a method of extracting an image of a five sense organ region.

Fig. 3 is a schematic diagram of a process of extracting an image of a five-element region using an extraction model according to one embodiment of the present application.

Fig. 4 is a schematic diagram of a process of extracting an image of a five-element region using an extraction model according to another embodiment of the present application.

Fig. 5 is a schematic flow chart diagram of a method of determining similarity according to one embodiment of the present application.

Fig. 6 is a schematic flow chart of a method for generating a second face image based on convolutional neural network image fusion model fusion.

Fig. 7 is a schematic block diagram of a face recognition device according to one embodiment of the present application.

Fig. 8 is a schematic hardware structure of a face recognition device according to an embodiment of the present application.

Fig. 9 is a schematic block diagram of an electronic device in accordance with one embodiment of the present application.

Detailed Description

The technical solutions in the present application will be described below with reference to the accompanying drawings. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be noted that, for convenience of description, only the portions related to the present invention are shown in the drawings.

It should be understood that the various steps recited in the method embodiments of the present application may be performed in a different order and/or performed in parallel. Furthermore, method embodiments may include additional steps and/or omit performing the illustrated steps. The scope of the present application is not limited in this respect.

The term "including" and variations thereof as used herein are intended to be open-ended, i.e., including, but not limited to. The term "based on" is based at least in part on. The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments. Related definitions of other terms will be given in the description below.

It should be noted that the terms "first," "second," and the like herein are merely used for distinguishing between different devices, modules, or units and not for limiting the order or interdependence of the functions performed by such devices, modules, or units.

It should be noted that references to "one" or "a plurality" in this application are intended to be illustrative rather than limiting, and those of ordinary skill in the art will appreciate that "one or more" is intended to be interpreted as "one or more" unless the context clearly indicates otherwise.

The names of messages or information interacted between the various devices in the embodiments of the present application are for illustrative purposes only and are not intended to limit the scope of such messages or information.

It should also be understood that the various embodiments described in this specification may be implemented alone or in combination, and that the examples herein are not limited in this regard.

Face recognition (face recovery) technology can be used for user authentication in shopping, security inspection and other fields. Specifically, face recognition may be applied to perform unlocking, payment, and the like, and may also be applied to various aspects of entertainment games, and the like. The intelligent terminal equipment, such as mobile phones, tablet computers, televisions and the like, is mostly provided with cameras, and after images containing faces are collected by the cameras, the images can be used for face detection and recognition, so that other related applications can be further executed by using recognition results. However, in the process of shooting a face image of a user, the user cannot perceive shooting time, expression changes such as eye closure may occur during shooting, or conditions such as light changes, face angle changes and the like may occur in the image, so that a part of areas in the image are greatly different from a target image, the recognition rate is reduced, and even if the user adjusts the gesture for many times, the user cannot pass verification, thereby bringing bad experience to the user.

The application firstly provides a facial recognition method and device based on facial image fusion, which utilize the extracted facial image with higher fitting degree with a target image to carry out fusion processing with a facial image and carry out facial recognition based on the processed image, so that the one-time passing rate in the facial recognition can be improved. Further, on the basis of the identification result, the application provides a task execution method of the electronic device and the electronic device, and different operations such as verification, unlocking, payment and the like can be executed by utilizing the identification result of the face identification method. Embodiments of the present application are described in detail below with reference to the specific drawings.

Fig. 1 is a flow chart illustrating a face recognition method according to an exemplary embodiment of the disclosure, and for convenience of understanding, a method for face recognition based on facial image fusion will be briefly described with reference to fig. 1.

As shown in fig. 1, the method comprises the steps of:

100, acquiring a first face image and a target face image of a user, wherein the first face image comprises at least two face images of the user.

The first face image may be photographed by a terminal having a photographing function. The face image may be captured by one camera or by a plurality of cameras.

The target face image can be identity document information or image information stored in advance, and can be used for comparison and reference after image processing.

200, extracting images of five sense organ areas from the first face image and the target face image, wherein the five sense organ areas comprise at least two areas of left eyebrow, right eyebrow, left eye, right eye, nose and mouth.

In some embodiments, before the extracting the image of the five-element region from the first face image and the target face image, it may be further verified whether the users in the first face image and the target face image are the same person.

Through preliminary verification of the first face image, follow-up image processing process of non-target face users can be prevented, and efficiency in verification is improved.

In one possible implementation, the facial geometry of the facial image may be obtained based on an adaptive enhancement (adaptive enhancement) algorithm, facial key points are detected on the facial image, and the facial position and range are determined, where the facial region may include at least two regions of the left eyebrow, the right eyebrow, the left eye, the right eye, the nose, and the mouth.

Specifically, the five-sense organ detector is constructed by utilizing an adaptive enhancement method, the Cascade classifier is organized by utilizing a Cascade (Cascade) algorithm classifier as a screening type, and each node of the Cascade is a strong classifier obtained by adaptive enhancement training. A threshold b is set at each node of the cascade such that almost all corresponding facial samples can pass, while most non-corresponding facial samples cannot. The nodes are arranged from simple to complex, and the nodes positioned farther back are more complex, namely, the nodes contain more weak classifiers. This minimizes the amount of computation in rejecting images but areas, informing of high detection rate and low rejection rate of guaranteed classifiers. And inputting the face image into the facial feature detector, sequentially detecting the images by the facial feature detector, determining different facial feature areas, and extracting the facial image according to the position and the size of the facial feature areas to obtain images of at least two facial feature areas.

In a possible implementation manner, an active shape model (Active Shape Model ) method may also be used to locate the five-sense organ area of the face image, which is not limited in this application.

In one possible implementation manner, a feature point positioning algorithm of an active shape model is adopted to determine shape models of different five sense organs in a first face image and a target face image;

according to the shape model of different five sense organs, gabor wavelet transform, PCA (Principal Component Analysis ) and LDA (Linear DiscriminantAnalysis, linear discriminant analysis) are sequentially carried out on the different five sense organs to obtain the characteristic information of the images of the different five sense organs.

In practical application, when a feature point positioning algorithm based on an active shape model is adopted to determine the shape model of the five-sense organ image, initial positioning is performed in the image, and then the accurate position of each feature point is searched in the image and corrected according to the gray scale model of each feature point aiming at each feature point of the initial positioning. The determined shape model can better reflect the features of the five sense organs through multiple searches and corrections.

In some embodiments, the face keypoint identification model is obtained by training the following steps:

Acquiring a training sample set, wherein the training sample comprises a face image sample and a face key point information sample marked in advance for the face image sample;

for training samples in the training sample set, the following steps are performed:

inputting a face image sample in the training sample into a face feature recognition model; acquiring image features extracted by a feature extraction layer of a face feature recognition model as image feature samples, and forming a new training sample by utilizing the acquired image feature samples and the training sample; and using a machine learning method, taking a face image sample and an image feature sample which are included in a training sample in the new training sample as input, taking a face key point information sample corresponding to the input face image sample and the image feature sample as expected output, and training to obtain a face key point recognition model.

Alternatively, as an embodiment, an image of the five-element region may be extracted from the first face image and the target face image based on a convolutional neural network image extraction model.

The convolutional neural network extraction model comprises a set of five-sense organ extraction parameters, and the set of extraction parameters in the convolutional neural network extraction model used in the process of extracting images of different five-sense organ areas is different.

The convolutional neural network (convolutional neuron network, CNN) is a deep neural network with a convolutional structure. The convolutional neural network comprises a feature extractor consisting of a convolutional layer and a sub-sampling layer, which can be regarded as a filter. The convolution layer refers to a neuron layer in the convolution neural network, which performs convolution processing on an input signal. In the convolutional layer of the convolutional neural network, one neuron may be connected with only a part of adjacent layer neurons. A convolutional layer typically contains a number of feature planes, each of which may be composed of a number of neural elements arranged in a rectangular pattern. Neural elements of the same feature plane share weights, where the shared weights are convolution kernels. Sharing weights can be understood as the way image information is extracted is independent of location. The convolution kernel can be initialized in the form of a matrix with random size, and reasonable weight can be obtained through learning in the training process of the convolution neural network. In addition, the direct benefit of sharing weights is to reduce the connections between layers of the convolutional neural network, while reducing the risk of overfitting.

Hereinafter, as an example and not by way of limitation, the image extraction method of the five sense organs region according to the embodiment of the present application will be described in detail with reference to the specific example of fig. 2. The method as shown in fig. 2 may be performed by an apparatus for extracting an image of the five sense organs.

Accordingly, the device for extracting the facial image (hereinafter referred to as "extracting device") in the embodiment of the present application may be a device having a facial image extracting function, including but not limited to a monitoring device, a smart phone, a camera, a video camera, etc., and the embodiment of the present application is not limited thereto, as long as the extracting device can implement the method for extracting the facial image. Alternatively, the extraction device may also be a chip.

And 210, acquiring face features, wherein the face features are the features of the first face image and the target face image.

In some implementations, the first face image and the target face image may be input into a neural network, and feature extraction is performed, where the extracted features are called face features. The feature extraction network may be a general-purpose neural network for extracting features, or may be a redesigned neural network capable of extracting features.

One expression form of the face features is a matrix, and if the matrix corresponding to the face features is a one-dimensional matrix, the expression form of the face features is a vector, and the vector can be called a face feature vector. In one example, a face feature vector may be 256-dimensional, each of which is 32-bit floating point number (fp 32) data.

220, taking the face feature as input of a convolutional neural network extraction model, taking a parameter set of the five-element feature as weight of the convolutional neural network extraction model, and obtaining the five-element feature detected by the convolutional neural network extraction model in the face feature, wherein the parameter set of the five-element feature comprises five-element feature parameters of different five elements, and the five-element feature parameters of corresponding five elements are used when the features of different five elements are detected.

In some implementations, the facial feature parameters in the set of facial feature parameters may be feature parameters of facial feature image samples pre-configured according to facial feature types, where the types of facial feature image samples may be manually calibrated, for example, may be pre-defined by a professional according to previous experience. Alternatively, the type of the facial feature image sample may be predetermined by the extraction device or other apparatus according to a predefined rule, and the embodiment of the present application is not limited thereto.

Alternatively, the five sense organs may include the left eyebrow, right eyebrow, left eye, right eye, nose, mouth.

Specifically, in the embodiment of the present application, the input of the convolutional neural network (convolutional neural networks, CNN) image extraction model (hereinafter referred to as "extraction model") is a face feature extracted from each face image, and the output of the extraction model is a five-element feature of an image of a different five-element region in each face image. In other words, after the face image is input into the extraction model, the image of the five sense organs area in the face image is finally output through the processing of each layer in the extraction model.

Alternatively, when the type of the image is left eye, the first set of five sense organ feature parameters may be corresponding; when the type of the image block is right eye, the second set of five sense organ feature parameters may be corresponding, and so on. Accordingly, the first set of facial feature parameters, the second set of facial feature parameters, and the third set of facial feature parameters … may be facial feature parameters corresponding to different facial regions, respectively.

230, extracting the image of the five sense organs according to the features of the five sense organs.

In the embodiment of the application, after the output of the extraction model is obtained, the region corresponding to the facial image of the facial feature can be determined based on the output of the extraction model, and the image of the facial region is extracted from the facial image according to the region where the facial feature is located.

It should be understood that the selection of the facial feature in the region corresponding to the facial image may be the smallest image region containing the facial feature.

Therefore, in the embodiment of the application, different facial feature parameters are used for extracting different types of facial regions through the extraction model, so that the defect of unified processing of the whole facial image is avoided, corresponding extraction can be performed for different facial regions, and the processing efficiency and accuracy can be improved.

It should be understood that the extraction model in the embodiment of the present application may be a shallow CNN model, and the embodiment of the present application is not limited thereto.

It should be appreciated that in embodiments of the present application, the extraction model may be pre-trained. For example, the extraction model may be pre-trained by the extraction device. Alternatively, the extraction model may be obtained by the extraction device from another device, where the extraction model is trained in advance by another device, and the other device may be another extraction device or a device specifically used for training the extraction model, and the embodiment of the present application is not limited thereto. The feature parameters of the five sense organs corresponding to different five sense organs can be obtained by training the extraction model.

The specific process by which the extraction model is trained by the extraction device or by other devices, respectively, is described in detail below.

Case one: the extraction model is trained by the extraction device.

In this case, in an embodiment of the present application, before the extracting device extracts the image of the five-element region from the first face image and the target face image, the method further includes:

the extraction model is trained.

Specifically, the method for training the extraction model as shown in fig. 3 includes:

And 310, acquiring different facial image samples and facial image samples of a predefined type.

It should be appreciated that the predefined type of facial feature sample may be a facial feature image sample of which the type is pre-defined, e.g., the type of facial feature image sample may be manually calibrated, e.g., may be pre-defined by a professional based on previous experience. Alternatively, the type of the facial feature image sample may be predetermined by the extraction device or other apparatus according to a predefined rule, and the embodiment of the present application is not limited thereto.

Specifically, as shown in fig. 3, the extraction device first obtains different facial image samples and facial feature image samples of a predefined type. For example, the facial feature image sample predefined type includes a left eyebrow object, a right eyebrow object, a left eye object, a right eye object, a nose object, or a mouth object. The facial feature image samples can randomly collect 300-500 sample pictures in a sample database to form a training set, and the facial object characteristics (including left eyebrow object, right eyebrow object, left eye object, right eye object, nose object or mouth object) of all the sample pictures in the training set are manually marked.

320 training the extraction model using the different face image samples and the facial feature image samples of a predefined type, respectively.

The extraction device inputs the different facial image samples and the facial five-sense organ image samples of the predefined types into the extraction model for parameter training, and specifically, corresponding parameters in each layer in the extraction model can be trained. For example, the convolution kernel in the convolution layer and the weight parameter in the full-connection layer of the extraction model, and the offset values respectively corresponding to the convolution kernel and the weight parameter are not limited thereto. After the extraction model is trained, in practical application, the extraction device inputs the face image to be extracted into the extraction model, and the images of different five sense organs in the face image are finally extracted through real-time processing of the extraction model.

It should be understood that, after the extraction device trains the extraction model, when the face image is extracted later, the extraction model can be directly used to extract the image of the facial region, and the extraction model does not need to be trained. Alternatively, the extraction device may only need to train and refine the extraction model periodically, and the extraction model need not be trained each time an image is extracted, which is not limited in this application.

And a second case: the extraction model is trained by other means.

the extraction model is obtained.

Specifically, the extraction device first acquires the extraction model from the other device. In this case, the extraction model may be trained in advance by the other apparatus or the extraction model may be acquired by the other apparatus from another device, the extraction model being trained by the other device, and the embodiment of the present application is not limited thereto.

As shown in fig. 4, the extraction device first acquires an extraction model (the extraction model is a model that has been completed training) from the other device. In practical application, the extraction device inputs the face image to be extracted into the extraction model, and the image of different five sense organs in the face image is finally extracted through real-time processing of the extraction model.

300, determining the similarity of the image of the five-sense organ region in the first face image and the image of the region corresponding to the target face image based on the image characteristic information of the five-sense organ region, and taking the image of which the similarity meets the threshold requirement in the image of the five-sense organ region as a candidate five-sense organ region image.

Alternatively, the image feature information may be feature points or feature vectors extracted from the image, which is not limited in this application.

Alternatively, the similarity may be determined by at least one selected from the group consisting of Euclidean distance (Euclidean metric: the true distance between two points in an m-dimensional space, or the natural length of a vector), cosine similarity (cosine similarity: evaluating the similarity of two vectors by calculating their angle cosine value; plotting the vectors into a vector space, such as the most common two-dimensional space, according to coordinate values), relative entropy (Kullback-Leibler divergence, an asymmetry metric of the difference between two probability distributions).

It should be appreciated that the threshold requirement of the similarity may be arbitrarily selected, and preferably, one image with the highest similarity between the image of the facial region in the first face image and the image of the region corresponding to the target face image may be used as the candidate facial region image.

Optionally, the similarity between the image of the five-sense organ region in the first face image and the image of the region corresponding to the target face image is determined according to a cosine value between the image of the five-sense organ region and the feature vector of the image of the region corresponding to the target face image.

Specifically, cosine distances between feature vectors of different five sense organs in the first face image and feature vectors of different five sense organs in the target face image can be calculated according to a cosine similarity formula, so that the similarity of the different five sense organs in the first face image and the target face image is obtained.

The cosine similarity calculation formula is as follows:

wherein n represents feature vectors of five sense organs in n target face images; a is that _i An i-th item representing a feature vector of a five sense organs in the first face image; w (W) _i An i-th item representing a feature vector of a five sense organs in the target face image; a is that ^T Representing a transpose of facial feature vectors in the first face image; the A represents the five sense organ vector in the first face image2 norms of A, namely the square sum root number of each element in the five sense organs feature vector A in the first face image; the I and W I represent 2 norms of the five-sense organ feature vector W in the target face image, namely the square sum root number of each element in the five-sense organ feature vector W in the target face image; cos θ represents the cosine similarity value of the facial feature vector in the first face image and the facial feature vector in the target face image. The closer the cos θ is to 1, the higher the similarity between the facial feature vector in the first face image and the facial feature vector in the target face image is.

In some embodiments, the image feature information of the facial region includes a facial feature contour line within an image of the facial region, wherein the determining, based on the image feature information of the facial region, a similarity of the image of the facial region in the first face image to the image of the region corresponding to the target face image includes: and determining the similarity of the image of the five-sense organ region in the first face image and the image of the region corresponding to the target face image based on the five-sense organ contour line in the image of the five-sense organ region, wherein the five-sense organ contour line is a closed two-dimensional curve surrounding the five-sense organ region.

It should be understood that the facial contours in the images of different facial regions may be identified, and the similarity between the image of the facial region in the first face image and the image of the region corresponding to the target face image may be determined according to the hausdorff distance between the feature points on the facial contours.

Specifically, the outline of the five elements is a closed two-dimensional curve surrounding the region of the five elements.

Optionally, the image of the five-element region may be processed to obtain a five-element contour line on the five-element, where the five-element contour line is a boundary line of the five-element with obvious contrast difference and large gray gradient change in the image.

Specifically, the Hausdorff distance is a maximum-minimum distance defined on two point sets, given two finite point sets a= { n1, n2, …, a _p Sum b= { B1, B2, …, B _q Then the hausdorff distance between a, B is defined as:

H(A,B)＝max{h(A，B),h(B，A)}

where h (A, B) is the directed Haoskov distance from point set A to point set B, h (B, A) can be analogized.

Hereinafter, as an example and not by way of limitation, a method for determining similarity between an image of the facial region in the first face image and an image of a region corresponding to the target face image according to an embodiment of the present application will be described in detail with reference to a specific example of fig. 5.

FIG. 5 is a schematic flow chart diagram of a method of determining similarity, as shown in FIG. 5, according to one embodiment of the present application, where the method shown in FIG. 5 may be performed by a processor. The method comprises the following steps:

510, calculating the curvature of the outline of the five sense organs.

In one possible embodiment, the facial contours may be the contours of the lips, the contours of the eyes, or the boundary lines of the eyebrows, which is not limited in this application.

Specifically, the point taking is performed on the five-element contour line function r= [ x (t), y (t) ] where t is an arbitrary parameter, x (t) is the abscissa of each point on the five-element contour line, and y (t) is the ordinate of each point on the five-element contour line.

Convolving the abscissa x (t) and the ordinate y (t) of each point on the outline of the five sense organs with the Gaussian function G (t, sigma) to obtain r _σ = (x (t, σ), y (t, σ)), the curvature of the facial contours after noise removal is calculated:

wherein K (t, sigma) is a function taking the parameter t of the profile and the standard deviation sigma in the Gaussian function as independent variables, x _t (t, sigma) and x _tt (t, sigma) is a first and second derivative of x (t, sigma) to t, y, respectively _t (t, sigma) and y _tt (t, sigma) is a first-order derivative function and a second-order derivative function of y (t, sigma) to t respectively, sigma is a standard deviation in a Gaussian function, v is a parameter, deltav is a step length, x (v) and y (v) are functions of v and the horizontal and vertical coordinates on the facial contours respectively, and when the standard deviation sigma in the Gaussian function is reduced from small to large, the reserved details are correspondingly reduced gradually.

And 520, determining characteristic points on the outline of the five sense organs.

Searching the maximum point P of curvature on the outline of the five sense organs _max Its curvature value is recorded as K _max And let P _max =2; in the clockwise direction, P _max The next point is used as a starting point P, the curvature K is sequentially compared with the curvature K of the following point, if the curvature of the following point Pi is still smaller than the curvature K of the current point, iterative comparison is still carried out until the curvature of the following point is larger than the curvature of the current point P, and the current point is a local extreme point and is recorded as LP _i The curvature is denoted as K (LP) _i ). If the curvature K (LP) _i )<0, let LP _i -2; if K (LP) _i ) > 0, let LP _i ＝1；

When the local extreme point LP _i When = -2, let the subsequent point be the starting point P _i+1 The curvature is compared with the curvature of the next point one by one. The previous alternate comparison process is repeated until a subsequent point curvature to the current point P occurs _i+1 Curvature is small, then this current point P _i+1 Is a local extreme point, denoted as LP _i+1 The curvature is denoted as K (LP) _i+1 ). If the curvature K (LP) _i+1 )<0, let LP _i+1 =2; if the curvature is smaller than K (LP) _i+1 )<0, let LP _i+1 ＝-1；

If LP _i =2, and one of the following two conditions is satisfied, this point is regarded as a feature point:

1. this point K (LP _i+1 )>value (value is a threshold value), and the curvature of two adjacent points is smaller than 0;

2. this point K (LP _i+1 )>value, is bigger than 2 times of the minimum value of curvature of two adjacent points at the same time;

if LP _i+1 =2, and one of the following two conditions is satisfied, this point is regarded as a feature point:

1. this point K (LP _i+1 )<Value, while the curvature of both its adjacent points is greater than 0;

2. this point K (LP _i+1 )<Value, simultaneously less than 2 times the maximum value of curvature of its two adjacent points.

Specifically, the contour lines may be subjected to similarity matching according to the curvature of the feature points on the contour lines.

Respectively taking extreme point sets c on the facial contour lines in the first face image ₁ And an extreme point set c on the facial contour line in the target face image ₂ Point set c ₁ And c ₂ The Haoskov distance between:

530, determining the similarity according to the hausdorff distance.

Specifically, haoskov distance H (c ₁ ,c ₂ ) The smaller the facial contour line is, the higher the similarity between the facial contour line and the facial contour line in the target face image is.

Therefore, the geometric characteristic information of the outline of the five sense organs is fully utilized when the similarity of the five sense organs is judged, and the recognition efficiency is improved.

400, taking the first face image as a base map, and fusing the candidate facial region image with the base map in a corresponding region in the base map to generate a second face image.

In one possible embodiment, when selecting the base map, one of the first face images may be optionally selected as the base map. This embodiment is not limited thereto.

Alternatively, one first face image having the largest number of the candidate facial region images may be used as the base map.

In one possible implementation, when the candidate five-sense organ region image and the base map are fused, the candidate five-sense organ region image and the base map can be matched according to the same characteristic information, for example, when the open eye region image is fused to the base map with the closed eye region, moles or wrinkles of the region can be used as positioning, so that the common characteristic information is used as a reference to be fused, and the precision of generating the image is higher.

As an optional mode, the feature points of the five sense organs in the target face image and the first face image can be utilized to find out the corresponding feature point pairs in the images to be fused, and the matching fusion position of the two images is determined based on the feature point pairs. The fusion position can be an overlapping region of the candidate five-sense organ region image and the base map, and for two images without overlapping regions, the fusion position is determined by calculating relevant feature points. The feature points are points used to measure the effect of a particular marker in the image and may be implemented using a related digital image processing algorithm or neural network model.

In one possible implementation manner, when the candidate facial region images are fused at the facial region positions corresponding to the base map to generate the second facial image, a convolutional neural network image fusion model may be based.

Specifically, as shown in fig. 6, the method for generating the second face image based on the fusion of the convolutional neural network image fusion model includes:

a training dataset is acquired 610.

Specifically, the training data set may include facial image samples of different facial image samples and their corresponding facial image samples of different poses.

And 620, constructing a convolutional neural network image fusion model.

Specifically, the image features of the facial images with different faces and the facial images with different corresponding facial poses can be obtained based on the feature information extracted in the step 300; fusing (can directly connect in series) the extracted features, and inputting the fused features into a registration decoder network, so as to obtain registration parameters; transforming the region corresponding to the facial image on the facial image by using the obtained registration parameters; inputting the transformed facial images into an encoder network for encoding; inputting the coded transformed facial image and the coded target face image into a fusion layer for fusion; and inputting the obtained fused data into a reconstruction decoder network, thereby obtaining a final fused image.

In particular, the reconstruction decoder network may include a first convolution kernel, a second convolution kernel, a third convolution kernel, and a fourth convolution kernel; the first convolution kernel, the second convolution kernel, the third convolution kernel and the fourth convolution kernel are sequentially connected in series; the first convolution kernel has a size of 64 x 3; the size of the second convolution kernel is 64 x 32 x 3; large of third convolution kernel the small is 32 x 16 x 3; of a third convolution kernel size is 16 x 1 x 3; the parameters are defined as the number of input channels of the convolution kernel, the number of output channels, the length, and the width.

630, training the convolutional neural network image fusion model by adopting the training data set to obtain an image fusion model;

it should be understood that after the convolutional neural network image fusion model is trained, the fusion model can be directly used for fusing the facial image and the face image when the facial image and the face image are fused later, and the extraction model is not required to be trained. Alternatively, the fusion device only needs to train and perfect the fusion model periodically, and does not need to train and perfect the fusion model every time the image is fused, and the embodiment of the application is not limited thereto.

640, inputting the candidate five sense organs region image and the base map into the image fusion model to complete image fusion.

Specifically, for example, if an eye region image of the same user's open eyes is fused with a face image of the closed eyes, the face image of the user's open eyes can be obtained by using the image fusion model. Other facial region images of the face image can be adjusted through the image fusion model, and the embodiment of the application is not limited to this.

Therefore, the face recognition method and device can comprehensively generate the face image with high similarity to each five-sense organ area of the target face image aiming at the problem of low face recognition once-through rate caused by the possible situations of eye closure, eyeglass reflection, mouth opening, illumination and the like in the original face image, can improve the face recognition once-through rate, and improve user experience.

And 500, identifying the user according to the second face image.

In one possible implementation, the face recognition process is performed by acquiring feature information in the second face image and comparing the feature information with the target face image. The embodiments of the present application are not limited thereto.

Alternatively, the recognition of the user according to the second face image may be performed on another device, which is not limited in the embodiment of the present application.

Based on the technical scheme, at least the following technical effects can be realized:

the method and the device can acquire the face image of the user, select the candidate facial feature region image with the similarity exceeding the set value from the face image of the user, and obtain the face image of the user with high similarity to the face image of the target by fusing the candidate facial feature region image with the base map, so that the primary pass rate of the face recognition process can be improved, the recognition accuracy rate is lower due to the condition that the extracted face image has the expression change such as eye closing and the like, the primary pass rate of the recognition can be improved, and the user experience is improved.

The embodiment of the invention also provides a schematic block diagram of a face recognition device, as shown in fig. 7, the device 700 includes:

The image acquisition module 710 is configured to acquire a first face image of a user and a target face image, where the first face image includes at least two face images of the user.

Alternatively, the image acquisition module 710 may be any device that acquires images, such as a video camera, a camera, or the like.

A processor 720, configured to extract an image of a five-sense organ region from the first face image and the target face image, where the five-sense organ region includes at least two regions of a left eyebrow, a right eyebrow, a left eye, a right eye, a nose, and a mouth;

optionally, the processor 720 is further configured to determine, based on the image feature information of the five-element region, a similarity between an image of the five-element region in the first face image and an image of a region corresponding to the target face image, and use, as a candidate five-element region image, an image of the five-element region in which the similarity meets a threshold requirement;

optionally, the processor 720 is further configured to fuse the pictures, specifically: taking the first face image as a base map, and fusing the candidate facial region image with the base map in a corresponding region in the base map to generate a second face image;

optionally, the processor 720 is further configured to identify the user according to the second face image.

In a possible implementation manner, the processor 720 is specifically configured to extract an image of a five-element region from the first face image and the target face image based on an adaptive enhancement algorithm, and includes: constructing a five-sense organ detector by utilizing a self-adaptive enhancement algorithm, wherein the five-sense organ detector is a screening type cascade classifier; inputting the first face image and the target face image into the five sense organs detector, wherein the five sense organs detector sequentially detects different five sense organs areas from the face image; and extracting from the first face image and the target face image according to the position and the size of the five-sense organ region to obtain images of different five-sense organ regions.

Optionally, the extracting an image of a five-element region from the first face image and the target face image, the processor 720 is further configured to: acquiring face features, wherein the face features are the features of the first face image and the target face image; taking the face features as input of a convolutional neural network extraction model, taking a parameter set of five sense organs as weight of the convolutional neural network extraction model, and obtaining the five sense organs features detected by the convolutional neural network extraction model in the face features, wherein the parameter set of the five sense organs features comprises five sense organ feature parameters of different five sense organs, and the five sense organ feature parameters corresponding to the five sense organs are used when the features of the different five sense organs are detected; and extracting the image of the five sense organs according to the features of the five sense organs.

Optionally, the image feature information of the facial region includes a facial feature contour line in an image of the facial region, wherein the processor 720 is further configured to determine, based on the image feature information of the facial region, a similarity between the image of the facial region in the first face image and the image of the region corresponding to the target face image: and determining the similarity of the image of the five-sense organ region in the first face image and the image of the region corresponding to the target face image based on the five-sense organ contour line in the image of the five-sense organ region, wherein the five-sense organ contour line is a closed two-dimensional curve surrounding the five-sense organ region.

In a possible implementation manner, the determining the similarity between the image of the facial region in the first face image and the image of the corresponding region of the target face image based on the facial contour line in the image of the facial region is further specifically configured to: calculating the curvature K (t, sigma) of the outline of the five sense organs through the outline of the five sense organs,

/>

Optionally, the processor 720 may specifically be configured to: and taking one first face image with the largest candidate facial region image as a base map.

the face image of the user can be obtained, the candidate facial feature region image with high similarity with the target face image is selected from the face image, the face image of the user with high similarity with the target face image is obtained by fusing the candidate facial feature region image with the base map, the once-through rate of the face recognition process can be improved by using the processed face image of the user for face recognition, the recognition accuracy rate is low due to the condition that the extracted face image has expression changes such as eye closing and the like is prevented, the recognition accuracy rate can be improved, and the user experience is improved.

Fig. 8 is a schematic hardware structure of a face recognition device according to an embodiment of the present application. The face recognition device 800 shown in fig. 8 includes a memory 801, a processor 802, a communication interface 803, and a bus 804. Wherein the memory 801, the processor 802, and the communication interface 803 are communicatively connected to each other through a bus 804.

The memory 801 may be a Read Only Memory (ROM), a static storage device, a dynamic storage device, or a random access memory (random access memory, RAM). The memory 801 may store a program, and when the program stored in the memory 801 is executed by the processor 802, the processor 802 and the communication interface 803 are used to perform the respective steps of the face recognition apparatus of the embodiment of the present application.

The processor 802 may employ a general-purpose central processing unit (central processing unit, CPU), microprocessor, application specific integrated circuit (application specific integrated circuit, ASIC), graphics processor (graphics processing unit, GPU) or one or more integrated circuits for executing associated programs to perform functions required by the units in the image extraction apparatus of the embodiments of the present application or to perform the face recognition method of the embodiments of the present application.

The processor 802 may also be an integrated circuit chip with signal processing capabilities. In implementation, the steps of the face recognition method according to the embodiment of the present application may be completed by an integrated logic circuit of hardware in the processor 802 or an instruction in the form of software.

The processor 802 may also be a general purpose processor, a digital signal processor (digital signal processing, DSP), an ASIC, an off-the-shelf programmable gate array (field programmable gate array, FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components. The general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in connection with the embodiments of the present application may be embodied directly in hardware, in a decoded processor, or in a combination of hardware and software modules in a decoded processor. The software modules may be located in a random access memory, flash memory, read only memory, programmable read only memory, or electrically erasable programmable memory, registers, etc. as well known in the art. The storage medium is located in the memory 801, and the processor 802 reads information in the memory 801, and in combination with hardware thereof, performs functions required to be performed by units included in the face recognition device of the embodiment of the present application, or performs the face recognition method of the embodiment of the method of the present application.

Communication interface 803 enables communication between apparatus 800 and other devices or communication networks using a transceiver apparatus such as, but not limited to, a transceiver. For example, an image to be processed may be acquired through the communication interface 803.

Bus 804 may include a path for transferring information between various components of device 800 (e.g., memory 801, processor 802, communication interface 803).

It should be noted that although the apparatus 800 described above shows only a memory, a processor, a communication interface, in a particular implementation, those skilled in the art will appreciate that the apparatus 800 may also include other devices necessary to achieve proper operation. Also, as will be appreciated by those of skill in the art, the apparatus 800 may also include hardware devices that implement other additional functions, as desired. Furthermore, it will be appreciated by those skilled in the art that the apparatus 800 may also include only the devices necessary to implement the embodiments of the present application, and not necessarily all of the devices shown in FIG. 8.

As shown in fig. 9, the embodiment of the present application further provides an electronic device 900, where the electronic device may include the apparatus 700 of the present application, for example, the electronic device 900 is a smart door lock, a mobile phone, a computer, an access control system, or the like, where face recognition needs to be applied. The apparatus 700 includes software and hardware means for face recognition in the electronic device 900.

The terminal device in the embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. The electronic device shown in fig. 9 is only an example, and should not impose any limitation on the functions and scope of use of the embodiments of the present application.

It should be appreciated that the processor of the embodiments of the present application may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method embodiments may be implemented by integrated logic circuits of hardware in a processor or instructions in software form. The processor may be a general purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), an off-the-shelf programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components. The disclosed methods, steps, and logic blocks in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in connection with the embodiments of the present application may be embodied directly in hardware, in a decoded processor, or in a combination of hardware and software modules in a decoded processor. The software modules may be located in a random access memory, flash memory, read only memory, programmable read only memory, or electrically erasable programmable memory, registers, etc. as well known in the art. The storage medium is located in a memory, and the processor reads the information in the memory and, in combination with its hardware, performs the steps of the above method.

It will be appreciated that the face recognition device of the embodiments of the present application may also include memory, which may be volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory. The nonvolatile Memory may be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable EPROM (EEPROM), or a flash Memory. The volatile memory may be random access memory (Random Access Memory, RAM) which acts as an external cache. By way of example, and not limitation, many forms of RAM are available, such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (Double Data Rate SDRAM), enhanced SDRAM (ESDRAM), synchronous DRAM (SLDRAM), and Direct RAM (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to comprise, without being limited to, these and any other suitable types of memory.

Embodiments of the present application also provide a computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a portable electronic device, comprising a plurality of application programs, enable the portable electronic device to perform the methods of the embodiments shown in fig. 1-6.

It should be noted that the computer readable medium described in the present application may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present application, however, a computer-readable signal medium may include a data signal that propagates in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, fiber optic cables, RF (radio frequency), and the like, or any suitable combination of the foregoing.

In some implementations, the clients, servers may communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol ), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the internet (e.g., the internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed networks.

The computer readable medium may be contained in the electronic device; or may exist alone without being incorporated into the electronic device.

The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: acquiring at least two internet protocol addresses; sending a node evaluation request comprising the at least two internet protocol addresses to node evaluation equipment, wherein the node evaluation equipment selects an internet protocol address from the at least two internet protocol addresses and returns the internet protocol address; receiving an Internet protocol address returned by the node evaluation equipment; wherein the acquired internet protocol address indicates an edge node in the content distribution network.

Alternatively, the computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: receiving a node evaluation request comprising at least two internet protocol addresses; selecting an internet protocol address from the at least two internet protocol addresses; returning the selected internet protocol address; wherein the received internet protocol address indicates an edge node in the content distribution network.

Computer program code for carrying out operations of the present application may be written in one or more programming languages, including, but not limited to, an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The present embodiments also provide a computer program comprising instructions which, when executed by a computer, cause the computer to perform the method of the embodiments shown in fig. 1-6.

For example, embodiments of the present application include a computer program product comprising a computer program embodied on a non-transitory computer readable medium, the computer program comprising program code for performing the method shown in the flow chart.

The embodiment of the application also provides a chip, which comprises an input-output interface, at least one processor, at least one memory and a bus, wherein the at least one memory is used for storing instructions, and the at least one processor is used for calling the instructions in the at least one memory to execute the method of the embodiment shown in fig. 1-6. For example, the chip may be a field-programmable gate array (field-programmable gate array, FPGA), an application-specific integrated chip (application specific integrated circuit, ASIC), a system on chip (SoC), a central processing unit (central processor unit, CPU), a network processor (network processor, NP), a digital signal processing circuit (digital signal processor, DSP), a microcontroller (micro controller unit, MCU), a programmable controller (programmable logic device, PLD) or other integrated chip.

It should be understood that the processor may be implemented in hardware or in software, and when implemented in hardware, the processor may be a logic circuit, an integrated circuit, or the like; when implemented in software, the processor may be a general-purpose processor, implemented by reading software code stored in a memory, the memory may be integrated with the processor, for example, the memory may be integrated in the processor, and the memory may also be located outside the processor, and may exist separately.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, and are not repeated herein.

In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.

The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the present application may be embodied in essence or a part contributing to the prior art or a part of the technical solutions, or in the form of a software product, which is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

It should be appreciated that reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present application. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. It should be understood that, in various embodiments of the present application, the sequence numbers of the foregoing processes do not mean the order of execution, and the order of execution of the processes should be determined by the functions and internal logic thereof, and should not constitute any limitation on the implementation process of the embodiments of the present application.

As used in this specification, the terms "component," "module," "system," and the like are intended to refer to a computer-related entity, either hardware, firmware, a combination of hardware and software, or software in execution. For example, a component may be, but is not limited to being, a process running on a processor, an object, an executable, a thread of execution, a program, and/or a computer. By way of illustration, both an application running on a computing device and the computing device can be a component. One or more components may reside within a process and/or thread of execution and a component may be localized on one computer and/or distributed between 2 or more computers. Furthermore, these components can execute from various computer readable media having various data structures stored thereon.

It should also be understood that the first, second, third, fourth, and various numerical numbers referred to herein are merely descriptive convenience and are not intended to limit the scope of embodiments of the present application.

It should be understood that the term "and/or" is merely an association relationship describing the associated object, and means that three relationships may exist, for example, a and/or B may mean: a exists alone, A and B exist together, and B exists alone.

Those of ordinary skill in the art will appreciate that the various illustrative logical blocks (illustrative logical block) and steps (steps) described in connection with the embodiments disclosed herein can be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions (programs). When the computer program instructions (program) are loaded and executed on a computer, the processes or functions described in accordance with the embodiments of the present application are fully or partially produced. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another, for example, by wired (e.g., coaxial cable, optical fiber, digital Subscriber Line (DSL)), or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid State Disk (SSD)), etc.

The foregoing is merely specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily think about changes or substitutions within the technical scope of the present application, and the changes and substitutions are intended to be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

In summary, the foregoing description is only a preferred embodiment of the technical solution of the present application, and is not intended to limit the scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A face recognition method, comprising:

acquiring a first face image and a target face image of a user, wherein the first face image comprises at least two face images of the user;

extracting images of five sense organ areas from the first face image and the target face image, wherein the five sense organ areas comprise at least two areas of left eyebrow, right eyebrow, left eye, right eye, nose and mouth;

based on the image characteristic information of the facial region, determining the similarity between the image of the facial region in the first face image and the image of the region corresponding to the target face image;

Taking the image with the similarity meeting the threshold requirement in the image of the five sense organs as a candidate image of the five sense organs;

taking the first face image as a base map, and fusing the candidate facial region image with the base map in a corresponding region in the base map to generate a second face image;

and identifying the user according to the second face image.

2. The method of claim 1, wherein the extracting an image of a five-element region from the first face image and the target face image comprises:

constructing a five-sense organ detector by utilizing a self-adaptive enhancement algorithm, wherein the five-sense organ detector is a screening type cascade classifier;

inputting the first face image and the target face image into the five sense organs detector, wherein the five sense organs detector sequentially detects different five sense organs areas from the face image;

and extracting from the first face image and the target face image according to the position and the size of the five-sense organ region to obtain images of different five-sense organ regions.

3. The method of claim 1, wherein the extracting an image of a five-element region from the first face image and the target face image comprises:

Acquiring face features, wherein the face features are the features of the first face image and the target face image;

taking the face features as input of a convolutional neural network extraction model, taking a parameter set of five sense organs as weight of the convolutional neural network extraction model, and obtaining the five sense organs features detected by the convolutional neural network extraction model in the face features, wherein the parameter set of the five sense organs features comprises five sense organ feature parameters of different five sense organs, and the five sense organ feature parameters corresponding to the five sense organs are used when the features of the different five sense organs are detected;

and extracting the image of the five sense organs according to the features of the five sense organs.

4. A method according to any one of claims 1 to 3, wherein the image characteristic information of the five sense organ region includes a five sense organ contour line within an image of the five sense organ region;

the determining, based on the image feature information of the facial region, a similarity between the image of the facial region in the first face image and the image of the region corresponding to the target face image includes:

and determining the similarity of the image of the five-sense organ region in the first face image and the image of the region corresponding to the target face image based on the five-sense organ contour line in the image of the five-sense organ region, wherein the five-sense organ contour line is a closed two-dimensional curve surrounding the five-sense organ region.

5. The method of claim 4, wherein determining the similarity of the image of the facial region in the first face image to the image of the region corresponding to the target face image based on facial contours within the image of the facial region comprises:

calculating the curvature K (t, sigma) of the outline of the five sense organs through the outline of the five sense organs,

wherein t is any parameter, x (t, sigma) and y (t, sigma) are respectively the results of convolution of the abscissa x (t) of each point on the five-sense organ contour line, the ordinate y (t) of each point on the five-sense organ contour line and the Gaussian function G (t, sigma), and x _t (t, sigma) and x _tt (t, sigma) is a first and second derivative of x (t, sigma) to t, y, respectively _t (t, sigma) and y _tt (t, sigma) is y (t, sigma) to t, respectivelyA first-order derivative function and a second-order derivative function, wherein sigma is the standard deviation in a Gaussian function, v is a parameter, deltav is a step length, and x (v) and y (v) are functions of v and the abscissa and the ordinate on the outline of the five sense organs respectively;

determining an extreme point set c on the facial contour line in the first face image through curvature detection ₁ And an extreme point set c on the facial contour line in the target face image ₂ ；

According to the extreme point set c ₁ And the extreme point set c ₂ And determining the similarity between the image of the five sense organs region in the first face image and the image of the region corresponding to the target face image.

6. The method according to any one of claims 1 to 5, wherein said taking the first face image as a base map comprises:

and taking one first face image with the largest candidate facial region image as the base map.

7. A face recognition device, comprising:

the image acquisition module is used for acquiring a first face image and a target face image of a user, wherein the first face image comprises at least two face images of the user;

a processor, configured to extract an image of a five-sense organ region from the first face image and the target face image, where the five-sense organ region includes at least two regions of a left eyebrow, a right eyebrow, a left eye, a right eye, a nose, and a mouth;

the processor is further configured to determine, based on image feature information of the facial region, a similarity between an image of the facial region in the first face image and an image of a region corresponding to the target face image;

the processor is further configured to use, as a candidate five-element region image, an image in which the similarity satisfies a threshold requirement in the image of the five-element region;

the processor is further configured to fuse the candidate facial region image with the base map in a corresponding region in the base map to generate a second facial image by using the first facial image as the base map;

The processor is further configured to identify the user according to the second face image.

8. The apparatus of claim 7, wherein the extracting the image of the five-element region from the first face image and the target face image, the processor is specifically configured to:

9. The apparatus of claim 7, wherein the extracting the image of the five-element region from the first face image and the target face image, the processor is specifically configured to:

10. The apparatus according to any one of claims 7 to 9, wherein the image feature information of the facial region includes a facial feature contour within an image of the facial region, wherein the processor is specifically configured to determine, based on the image feature information of the facial region, a similarity of the image of the facial region in the first face image to the image of the target face image corresponding region:

11. The apparatus according to claim 10, wherein the processor is configured to determine a similarity between the image of the facial region in the first face image and the image of the region corresponding to the target face image based on a facial contour within the image of the facial region, and wherein the processor is configured to:

/>

Wherein t is any parameter, x (t, sigma) and y (t, sigma) are respectively the results of convolution of the abscissa x (t) of each point on the five-sense organ contour line, the ordinate y (t) of each point on the five-sense organ contour line and the Gaussian function G (t, sigma), and x _t (t, sigma) and x _tt (t, sigma) is a first and second derivative of x (t, sigma) to t, y, respectively _t (t, sigma) and y _tt (t, sigma) is a first-order derivative function and a second-order derivative function of y (t, sigma) to t respectively, sigma is a standard deviation in a Gaussian function, v is a parameter, deltav is a step length, and x (v) and y (v) are functions of v and the transverse and longitudinal coordinates on the outline of the five sense organs respectively;

According to the extreme point set c ₁ And the extreme point set c ₂ The hausdorff distance between determines the similarity.

12. The apparatus according to any one of claims 7 to 11, wherein the first face image is taken as a base map, and the processor is specifically configured to:

13. A computer readable medium having stored thereon a computer program which, when executed by a computer, performs a method of face recognition according to any one of claims 1-6.

14. An electronic device, comprising:

an apparatus for face recognition according to any one of claims 7 to 12.

15. A computer program product, characterized in that the computer program product comprises a computer program or instructions which, when run on a computer, cause the method according to any one of claims 1 to 6 to be performed.

16. A chip system comprising a processor and a data interface, the processor reading instructions stored on a memory via the data interface to perform the method of any one of claims 1 to 6.