CN114783031A

CN114783031A - Face recognition method, face recognition apparatus, face recognition system, computer device, and medium

Info

Publication number: CN114783031A
Application number: CN202210463737.1A
Authority: CN
Inventors: 胡伟阳; 祖春山
Original assignee: BOE Technology Group Co Ltd
Current assignee: BOE Technology Group Co Ltd
Priority date: 2022-04-29
Filing date: 2022-04-29
Publication date: 2022-07-22

Abstract

The invention discloses a face recognition method, a face recognition device, a face recognition system, computer equipment and a medium, wherein the face recognition method of one embodiment comprises the following steps: using a human face feature extraction model to extract human face features of the received image to be detected and outputting a first human face feature matrix; according to a second face feature matrix with a plurality of face features of a preset face feature library, carrying out face search with the first face feature matrix and outputting a face similarity matrix of a face to be detected and the plurality of face features; and acquiring the maximum similarity between the human face and the human face to be detected according to the human face similarity matrix, and outputting the identification information of the human face to be detected according to the human face feature library if the maximum similarity meets a preset identification threshold. The embodiment provided by the invention completes face search through one-step matrix operation to obtain the similarity between the face characteristics of the face to be detected and each face characteristic in the face characteristic library, can avoid long-time high-complexity cyclic nesting operation in the prior art, and effectively improves the operation speed.

Description

Face recognition method, face recognition apparatus, face recognition system, computer device, and medium

Technical Field

The present invention relates to the field of face recognition technologies, and in particular, to a face recognition method, a face recognition apparatus, a face recognition system, a computer device, and a storage medium.

Background

With the development of image processing and face recognition technology, face recognition is increasingly applied to daily life, however, the face recognition method in the prior art usually has the problems of long time consumption and poor real-time performance.

Disclosure of Invention

In order to solve at least one of the above problems, a first embodiment of the present invention provides a face recognition method, including:

using a human face feature extraction model of a data processor to extract human face features of the received image to be detected and outputting a first human face feature matrix of the human face to be detected;

according to a second face feature matrix with a plurality of face features of a preset face feature library, carrying out face search on the second face feature matrix and the first face feature matrix through a data processor and outputting a face similarity matrix of the face to be detected and the face features;

and acquiring the maximum similarity between the human face and the human face to be detected according to the human face similarity matrix, and outputting the identification information of the human face to be detected according to the human face feature library if the maximum similarity meets a preset identification threshold, wherein the identification information comprises human face information corresponding to the maximum similarity in the human face feature library.

For example, in a face recognition method provided in some embodiments of the present application, the performing, by using a face feature extraction model of a data processor, face feature extraction on a received image to be detected and outputting a first face feature matrix of the face to be detected further includes:

loading the image to be detected into a data processor;

detecting the image to be detected by using a human face detection model and outputting human face image data of the human face to be detected;

and extracting the human face features of the human face image data by using a human face feature extraction model and outputting the first human face feature matrix.

For example, in a face recognition method provided in some embodiments of the present application, after the detecting the image to be detected using the face detection model and outputting the face image data of the face to be detected, before the extracting the face features of the face image data using the face feature extraction model and outputting the first face feature matrix, the face recognition method further includes:

extracting key points of the face image data by using a face key point extraction model and outputting position coordinates of each key point;

according to the position coordinates of each key point of the face image data, carrying out affine transformation on the face image data by using a face alignment model so as to align the face image data;

and filtering the face image data by using a face quality model according to a preset face image quality threshold value.

For example, in a face recognition method provided by some embodiments of the present application, after the performing face feature extraction on the face image data by using a face feature extraction model and outputting the first face feature matrix, the face recognition method further includes:

and carrying out normalization processing on the first face characteristic matrix.

For example, in a face recognition method provided in some embodiments of the present application, before performing, by a data processor, a face search on a second face feature matrix having a plurality of face features according to a preset face feature library and the first face feature matrix, and outputting a face similarity matrix between the face to be detected and the plurality of face features, the face recognition method further includes:

transposing a second face feature matrix of the face feature library;

and loading the face feature library into the data processor.

For example, in a face recognition method provided in some embodiments of the present application, the face feature library includes a plurality of sub-face feature libraries, and the face recognition method includes:

selecting a sub-face feature library;

the transposing of the second face feature matrix of the face feature library further comprises: transposing a second face feature matrix of the selected sub-face feature library;

said loading said library of facial features into a data processor further comprises: loading the transposed sub-facial feature library into the data processor;

the step of performing face search on the second face feature matrix and the first face feature matrix through a data processor according to a second face feature matrix with a plurality of face features in a preset face feature library and outputting a face similarity matrix of the face to be detected and the plurality of face features further comprises: performing face search on a second face feature matrix of the transposed sub-face feature library and the first face feature matrix and outputting a sub-face similarity matrix of the face to be detected and the plurality of face features;

the obtaining the maximum similarity with the face to be detected according to the face similarity matrix, and if the maximum similarity meets a preset identification threshold, outputting the identification information of the face to be detected according to the face feature library further includes: obtaining the maximum similarity with the face to be detected according to the face similarity matrix,

if the maximum similarity meets the preset recognition threshold, outputting the recognition information of the face to be detected according to the sub-face feature library, and finishing face recognition,

and if the maximum similarity does not meet the preset recognition threshold, judging whether a non-transposed sub-face feature library exists in the plurality of sub-face feature libraries, if so, skipping to the selected sub-face feature library, otherwise, finishing the face recognition.

For example, in a face recognition method provided in some embodiments of the present application, before the transposing the second face feature matrix of the face feature library, the face recognition method further includes:

establishing the face feature library, including: and performing face detection and feature extraction according to the received plurality of face images to form a second face feature matrix with a plurality of face features and face information in one-to-one correspondence with each face feature of the second face feature matrix.

A second embodiment of the present invention provides a face recognition apparatus, which includes a central processing unit and a data processor, wherein the central processing unit is configured to:

transmitting the received image to be detected to the data processor, enabling the data processor to use a human face feature extraction model to extract human face features of the image to be detected and output a first human face feature matrix of the human face to be detected, and according to a second human face feature matrix with a plurality of human face features of a preset human face feature library and the first human face feature matrix, performing human face search and outputting a human face similarity matrix of the human face to be detected and the human face features;

For example, in some embodiments of the present application, the face recognition apparatus further includes an internal memory, and the central processor is further configured to:

loading the image to be detected stored in the internal memory into the data processor, enabling the data processor to detect the image to be detected by using a face detection model and output face image data of the face to be detected, performing face feature extraction on the face image data by using a face feature extraction model and outputting a first face feature matrix, and performing normalization processing on the first face feature matrix;

transposing a second face feature matrix of the face feature library stored in the internal memory, and loading the face feature library into the data processor, so that the data processor performs face search on the second face feature matrix of the face feature library and the first face feature matrix and outputs a face similarity matrix of the face to be detected and the plurality of face features;

For example, in some embodiments of the present application, the face recognition apparatus further includes a face key point extraction model, a face alignment model, and a face quality model, where the face key point extraction model, the face alignment model, and the face quality model are provided

The face key point extraction model is used for extracting key points of the face image data and outputting position coordinates of the key points;

the face alignment model is used for carrying out affine transformation on the face image data according to the position coordinates of each key point of the face image data so as to align the face image data;

the face quality model is used for filtering the face image data according to a preset face image quality threshold value.

A third embodiment of the present invention provides a face recognition system, including a plurality of face recognition devices as described in the second embodiment, wherein the face recognition devices are connected via a network, and a face feature library of each face recognition device forms a total face feature library of the face recognition system, and a central processing unit of each face recognition device is configured to:

loading the received image to be detected to the data processor, so that the data processor uses a human face feature extraction model to extract human face features of the image to be detected and outputs a first human face feature matrix of the human face to be detected;

selecting a face feature library;

transposing a second face feature matrix of the selected face feature library;

loading the transposed face feature library into the data processor;

performing face search on a second face feature matrix of the transposed face feature library and the first face feature matrix and outputting a face similarity matrix of the face to be detected and the plurality of face features;

acquiring the maximum similarity between the human face and the human face to be detected according to the human face similarity matrix,

if the maximum similarity meets the preset recognition threshold, outputting the recognition information of the face to be detected according to the face feature library, and finishing face recognition,

and if the maximum similarity does not meet the preset recognition threshold, judging whether a non-transposed face feature library exists in the total face feature library, if so, skipping to the selected face feature library, otherwise, finishing the face recognition.

A fourth embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, wherein the program, when executed by a processor, implements the face recognition method according to the first embodiment.

A fifth embodiment of the present invention provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the face recognition method according to the first embodiment.

The invention has the following beneficial effects:

the invention aims at the existing problems at present, establishes a face recognition method, a recognition device and a recognition system, the face feature of the face to be detected is matrixed, matrix operation is carried out according to the face feature matrix of the face to be detected and the face feature matrix of the face feature library so as to complete face search and output a face similarity matrix, the face information which has the maximum similarity in the face similarity matrix and meets the recognition threshold value is taken as recognition information and output, the face search is completed through one-step matrix operation so as to obtain the similarity calculation between the face feature of the face to be detected and each face feature in the face feature library, therefore, the method avoids long-time high-complexity loop nesting operation in the prior art, effectively improves the operation speed, can realize real-time efficient, rapid and accurate face recognition, overcomes the problems in the prior art, and has wide application prospect.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

FIG. 1 is a flow chart of a face recognition method according to an embodiment of the invention;

2a-2b show block diagrams of the structures of the face recognition apparatus according to an embodiment of the present invention;

FIG. 3 is a block diagram of a face recognition system according to an embodiment of the present invention;

FIG. 4 is a schematic diagram of a computer device according to another embodiment of the present invention;

fig. 5 shows a flow chart of a face recognition method according to another embodiment of the invention.

Detailed Description

In order to more clearly illustrate the invention, the invention is further described below with reference to preferred embodiments and the accompanying drawings. Similar components in the figures are denoted by the same reference numerals. It is to be understood by persons skilled in the art that the following detailed description is illustrative and not restrictive, and is not to be taken as limiting the scope of the invention.

At present, the development of the face recognition technology is rapidly advanced, and a face recognition method in the prior art utilizes the high reliability, the high calculation performance and the strong image processing capability of a chip and combines a face recognition algorithm to retrieve the face information of a video image in real time and extract a characteristic value, so that repeated face recognition is avoided. The face recognition method can realize real-time processing of the acquired image to a certain extent, however, it needs to be pointed out that the face recognition method realizes the cosine similarity calculation between the face to be recognized and the face picture by using the matrix function interface of the operation base of the chip, namely, the face recognition method realizes the matrix operation by the chip, but the cosine similarity of the face to be verified can only be realized by calling the operation interface of the chip once, and when the matrix operation between the face to be recognized and the face feature base with massive face features needs to be calculated, the face recognition method still faces the operation of circularly calling the chip by using the CPU, namely, the problems of high CPU occupancy rate and low speed also exist.

In view of the above situation, the inventors have made extensive research and experiments to propose a face recognition method, as shown in fig. 1, a face recognition method provided by an embodiment of the present invention includes:

using a human face feature extraction model to extract human face features of the received image to be detected and outputting a first human face feature matrix of the human face to be detected;

according to a second face feature matrix with a plurality of face features of a preset face feature library, carrying out face search with the first face feature matrix and outputting a face similarity matrix of the face to be detected and the face features;

The method comprises the steps of matrixing the face characteristics of a face to be detected, carrying out matrix operation according to the face characteristic matrix of the face to be detected and the face characteristic matrix of a face characteristic library to complete face search and output a face similarity matrix, and taking the face information which has the maximum similarity in the face similarity matrix and meets a recognition threshold value as recognition information and outputting the recognition information. In the embodiment, the face search is completed through one-step matrix operation to obtain the similarity calculation between the face features of the face to be detected and each face feature in the face feature library, so that the long-time high-complexity loop nesting operation in the prior art is avoided, the operation speed is effectively improved, the face recognition can be realized efficiently, quickly and accurately in real time, the problems in the prior art are solved, and the method has a wide application prospect.

In a specific example, as shown in fig. 2a, the structural block diagram of the face recognition apparatus includes a central processing unit and a data processor, where the central processing unit is a control core of the face recognition apparatus and is configured to control and coordinate each component of the face recognition apparatus according to preset operation steps; the data processor is a special arithmetic processor, such as a graphic processor GPU, a tensor processor TPU, a neural network processor NPU and the like, wherein the graphic processor GPU is provided with a plurality of computing units and an ultra-long pipeline and is good at the operation acceleration in the image processing field, the tensor processor TPU is provided with an application-specific integrated circuit which can realize higher processing speed and lower energy consumption, and the neural network processor NPU adopts a data-driven parallel computing architecture to accelerate the operation speed of a neural network and improve the operation efficiency.

As shown in fig. 5, a specific face recognition process is described as an example.

The method comprises the following steps that firstly, a central processing unit extracts face features of a received image to be detected through a face feature extraction model of a data processor and outputs a first face feature matrix of the face to be detected.

In this embodiment, the extracting the face features of the image to be detected by the face feature extraction model disposed in the data processor specifically includes:

firstly, a central processing unit loads the image to be detected into a data processor.

In this embodiment, the central processing unit receives an image to be detected, where the image to be detected may be an image input from the outside or an image collected by a connected image collecting device, and the central processing unit stores the image to be detected in the internal memory. In order to facilitate the data processor to read the image to be tested quickly, the central processing unit loads the image to be tested stored in the internal memory into the data processor.

Secondly, the data processor detects the image to be detected according to the face detection model and outputs the face image data of the face to be detected.

In this embodiment, the face detection model uses retinaFace with high face recognition accuracy, the hardware uses a development kit of a bit continental SE5 edge calculation box, and the face detection model is subjected to platform optimization for a Linux system. Specifically, systematic framework processing can be performed on the face detection model according to different operating systems to adapt to each system, and the face detection model is converted to migrate to the system platform, in this embodiment, the face detection model is converted into a format required by a Linux system by using sdk of a bitfield SE5 edge calculation box, so that hardware performance is exerted to the maximum extent, for example, INT8 quantization is performed on the model by using a corresponding tool of the system platform, so that on one hand, accuracy is controlled within a preset loss range to ensure detection accuracy, and on the other hand, the operating efficiency of the model is effectively improved. In this embodiment, a face detection model with platform optimization is used to detect an image to be detected, and if no face is detected, the image is exited without processing, and a next image to be detected is waited for; and if the face is recognized, outputting face image data, wherein the face image data is face image data of 112 × 112 pixels. It should be noted that, if there are multiple faces in the image to be measured, the face image data of each face is output, that is, the face image data of multiple 112 × 112 pixels are output.

It should be noted that, in the present application, the face detection model is not specifically limited, and the hardware and software system used are not specifically limited, and those skilled in the art should select a suitable face detection model, hardware and software system according to the actual application requirements to implement the face detection function as the design criterion, which is not described herein again.

In view of further improving the accuracy of face feature extraction, in an optional embodiment, the method further includes:

performing affine transformation on the face image data by using a face alignment model according to the position coordinates of each key point of the face image data so as to align the face image data;

In this embodiment, to further improve the accuracy of face feature extraction, a plurality of operations are used, wherein:

the first operation is to extract key points of the face, extract the key points by using a face key point extraction model aiming at face image data, and output position coordinates of five parts of a left eye, a right eye, a nose tip, a left mouth corner and a right mouth corner in the face image data so as to facilitate the application of face alignment in the subsequent steps. It should be noted that, in the present application, the number and the positions of the key points are not specifically limited, and those skilled in the art should select the key points with appropriate number and positions to extract according to the actual application requirements, which is not described herein again.

The second operation is to align the face, extract the position coordinates of the key parts output by the model according to the key points of the face, perform affine transformation on the face image, and align the face image according to the position coordinates of each part in the face, for example, aligning the face inclined at a certain angle in the face image data, so as to facilitate the use of the subsequent steps.

The third operation is to filter the face quality, judge the quality of the face image data by using a face quality model aiming at the aligned face image data and output floating point numbers between 0 and 1 to represent the image quality, wherein the floating point data specifically relates to information such as image definition, face angle and the like, and judge the output floating point data according to a preset face quality threshold value: when the output floating point data is larger than the human face quality threshold value, the human face image data is considered to be high-quality data, and human face recognition can be carried out to ensure the accuracy of human face recognition; when the output floating point data is smaller than the human face quality threshold value, the human face image data is considered to be low-quality data, and the low-quality data is discarded; namely, the face quality model is used for filtering the data of each face image so as to ensure the accuracy of face recognition.

The embodiment further screens the face image data through the operations of face feature point extraction, face alignment and face quality filtering, so that the quality of the face image data can be effectively improved, and the accuracy of face recognition is improved.

And finally, carrying out face feature extraction on the face image data through a face feature extraction model of the data processor and outputting a first face feature matrix of the face to be detected.

In this embodiment, according to the facial image data, especially the high-quality facial image data after the facial feature point extraction, the facial alignment and the facial quality filtering operations, the facial feature extraction model in the data processor is used to perform feature extraction, and the first facial feature matrix of the one-dimensional floating-point number vector of 1 × 512 is output and stored in the memory of the data processor.

Specifically, in this embodiment, first, a memory space is applied in the memory of the data processor: and according to the array size of the first face feature matrix, applying for a memory space in a memory of the data processor by using a corresponding function in the BMsdk and acquiring the address of the memory space. Second, the first face feature matrix is copied into the memory of the data processor using the s2d function in BMsdk and accessed via a pointer to the device memory address. In this embodiment, the first face feature matrix is loaded into the data processor, so that the data processor can quickly acquire the first face feature matrix from the memory, and the overall speed of face recognition can be increased.

Secondly, the central processing unit searches the human face through the data processor according to a second human face feature matrix with a plurality of human face features in a preset human face feature library and the first human face feature matrix and outputs a human face similarity matrix of the human face to be detected and the human face features;

in this embodiment, a first face feature matrix of a face to be detected is compared with a large number of face features in a preset face feature library, specifically, a face search is performed by comparing euclidean distances between the first face feature matrix and each face feature in the face feature library.

In an alternative embodiment, when the facial feature library is externally input to the central processor and stored in the internal memory, the facial feature library stored in the internal memory is loaded into the data processor for the convenience of reading by the data processor. Specifically, a data structure is constructed in a memory of a data processor, and then a matrix with N rows and 512 columns is constructed; then applying for a space with a corresponding size in a memory of the data processor, and acquiring a memory address; copying the human face feature library stored in the internal memory of the central processing unit into the memory space of the data processor, and accessing through the memory address of the data processor. Unlike loading the face to be detected, the face feature library is usually loaded only once, that is, the face feature library is copied into the memory of the data processor, so as to facilitate face search for many times.

In an optional embodiment, the face recognition method further includes: establishing the face feature library, including: and performing face detection and feature extraction according to the received plurality of face images to form a second face feature matrix with a plurality of face features and face information in one-to-one correspondence with each face feature of the second face feature matrix.

In this embodiment, in consideration of the establishment of a face feature library, the addition of new face images, and other application scenarios, face detection and feature extraction are performed on a large number of received face images in batch, for example, face detection is performed on the large number of received face images, each piece of face data in the face images is separated, then feature extraction is performed respectively by using a face feature extraction model, for example, a face feature matrix of each face is generated in a cyclic mode, each face feature matrix is a one-dimensional floating-point number vector of 1 × 512, and the generated second face feature matrix is a floating-point matrix of N × 512, where N is the number of face features in the face feature library. In order to identify each face feature, the face feature library further includes face information corresponding to each face feature of the second face feature matrix, for example, the face information includes a face name corresponding to each face feature and an identification ID representing each face.

Because the first face feature matrix of the face to be detected is a one-dimensional matrix of 1 × 512, and the second face feature matrix of the face feature library is a floating-point matrix of N × 512, the euclidean distances between the first face feature of the face to be detected and each face feature in the face feature library can be obtained after matrix operation.

Considering that each face feature is characterized by a one-dimensional floating-point matrix of 1 × 512, in order to further speed up the matrix operation, in an alternative embodiment, the central processing unit transposes the second face feature matrix of the face feature library before loading the face feature library stored in the central processing unit into the data processor.

In this embodiment, the second face feature matrix is transposed, that is, the second face feature matrix of N × 512 is converted into a matrix of 512 × N, so that in the matrix operation, the first face feature matrix 1 × 512 and the second face feature matrix 512 × N of the face to be detected are multiplied, that is, a 1 × N face similarity matrix is formed, thereby effectively improving the operation efficiency and increasing the operation speed.

And thirdly, a central processing unit acquires the maximum similarity with the face to be detected according to the face similarity matrix, and outputs the identification information of the face to be detected according to the face feature library if the maximum similarity meets a preset identification threshold.

In this embodiment, the central processing unit receives a face similarity matrix output by the data processor, and finds a face most similar to the face features of the face to be detected in the face feature library, that is, a number with the largest similarity value, by traversing the values of the similarities in the face similarity matrix; then, judging through a preset recognition threshold, if the maximum similarity is greater than or equal to the recognition threshold, determining that the person corresponding to the face feature with the maximum similarity is the person corresponding to the face to be detected, namely finishing face recognition, and outputting recognition information which is face information corresponding to the face feature with the maximum similarity; and if the maximum similarity is smaller than the recognition threshold, the face to be detected is not recognized.

In consideration of setting of the recognition threshold, in an alternative embodiment, after the first face feature matrix is obtained, the first face feature matrix is normalized.

In the embodiment, the normalization processing is performed on the first face feature matrix, that is, 512 floating-point numbers are normalized into 512 data between 0 and 1; meanwhile, each face feature in the face feature library is normalized, and the obtained similarity value is a numerical value between 0 and 1 in the budget of matrix multiplication. At this time, the recognition threshold is also a value between 0 and 1, for example, if the recognition threshold is set to 0.7, the similarity with the maximum similarity value in the similarity matrix is greater than or equal to 0.7, the face to be detected is considered to be recognized, otherwise, the face to be detected is considered to be not recognized.

Thus, the face recognition of the face to be detected is completed. As shown in fig. 5, in the present embodiment, a central processing unit receives a large number of face images input from outside, and a data processor generates a second face feature matrix; receiving an externally input image to be detected of the face to be detected by the central processing unit, and generating a first face characteristic matrix by the data processor; namely, two sets of data flows are completed through the central processing unit and the data processor. And then, a face similarity matrix comprising Euclidean distances between the first face feature matrix and each face feature in the face feature library can be quickly obtained through one-step matrix operation of the data processor, the central processing unit carries out data analysis on the face similarity matrix and judges whether the face to be detected is identified or not, and the identified identification information of the face to be detected is output, so that face identification is completed.

In this embodiment, the face detection step, the face feature extraction step and the face search step in the face recognition process are set in the data processor to be completed, on one hand, the operation of large data volume is rapidly realized by using the data processor, on the other hand, the problem of long operation time caused by the circular operation of the central processing unit in the prior art can be solved, the operating pressure of the central processing unit is effectively relieved, a large amount of resources of the central processing unit are avoided being occupied, and the occupancy rate of the central processing unit is reduced.

It should be noted that fig. 2b is a block diagram of a face recognition apparatus according to another embodiment, which includes a central processing unit 1, a central processing unit 2 and a data processor, wherein the central processing unit 1 is a control core of the face recognition apparatus, such as a local processor, and is used for controlling and coordinating various components of the face recognition apparatus according to preset operation steps; the central processing unit 2 is used for loading the face feature library into a data processor, such as a cloud processor, so that the occupancy rate of the central processing unit 1 is effectively reduced, the memory and the computing power are saved, and the recognition efficiency is further improved. The data processor is the same as the foregoing embodiments, and is not described herein again.

Considering that the face feature library includes a huge number of face features, in an optional embodiment, the face feature library includes a plurality of sub-face feature libraries, and the face recognition method includes:

selecting a sub-face feature library;

the transposing the second face feature matrix of the face feature library further comprises: transposing a second face feature matrix of the selected sub-face feature library;

the step of performing face search with the first face feature matrix according to a second face feature matrix with a plurality of face features in a preset face feature library and outputting a face similarity matrix of the face to be detected and the plurality of face features further comprises: performing face search on a second face feature matrix of the transposed sub-face feature library and the first face feature matrix and outputting a sub-face similarity matrix of the face to be detected and the plurality of face features;

the obtaining the maximum similarity with the face to be detected according to the face similarity matrix, and if the maximum similarity meets a preset identification threshold, outputting the identification information of the face to be detected according to the face feature library further includes: acquiring the maximum similarity with the face to be detected according to a sub-face similarity matrix, outputting the identification information of the face to be detected according to the sub-face feature library if the maximum similarity meets a preset identification threshold, finishing face identification, judging whether non-transposed sub-face feature libraries exist in the sub-face feature libraries or not if the maximum similarity does not meet the preset identification threshold, jumping to the selected sub-face feature library if the non-transposed sub-face feature libraries exist, and finishing face identification if the non-transposed sub-face feature libraries exist.

In this embodiment, the face feature library is partitioned into blocks, the blocks are loaded into the data processor, and face search is performed sequentially with the first face feature matrix of the face to be detected. Specifically, a second face feature matrix of each partitioned sub-face feature library is transposed, then the transposed face is loaded into a data processor, and is subjected to matrix operation with a first face feature matrix of a face to be detected to obtain a sub-face similarity matrix and output to a central processing unit, the central processing unit traverses similarity values in the sub-face similarity matrix to obtain a maximum similarity value, and judges whether the similarity value of a face feature in the sub-face feature library, which is most similar to the first face feature matrix, is an identification value by using a preset identification threshold, if the similarity value meets the identification threshold, the face to be detected is identified and identification information is output, and the face identification is finished; otherwise, another sub-face feature library is selected, face searching and maximum similarity judgment are carried out according to the steps until the face to be detected is identified, or face searching of all the sub-face feature libraries, namely the face feature library, is completed. In this embodiment, the face feature library is partitioned into blocks, so that face search of the face feature library with an ultra-large capacity can be satisfied, for example, a 1G memory can store 100 ten thousand pieces of face feature information, the 100 ten thousand pieces of face feature information are partitioned into 10 blocks, and face search is performed according to the blocks, so that the recognition speed of face recognition is further increased.

It should be noted that, the present application does not specifically limit the partitioning of the face feature library, and the partitioning may be performed by the central processing unit when receiving the transmitted face feature library, or may be performed by the central processing unit receiving a plurality of partitioned sub-face feature libraries.

In a specific embodiment, the above-mentioned face recognition method is used for searching people, when people are searched in a large area or city by outdoor monitoring, for example, the family members of people to be searched see that the people to be searched is at the exit of the railway station for the last time, and then the people are lost and not found and alarm, at this time, the periphery of the railway station needs to be searched. The method comprises the steps that mass data exist in a region around a railway station, the region is divided into a plurality of sub-regions according to the region, a face feature library of a face recognition device comprises a plurality of sub-face feature libraries corresponding to the sub-regions, a photo of a person to be searched is input into a central processing unit of the face recognition device, and the data processor is used for carrying out face recognition on the person to be searched according to the sub-face feature libraries to find the person to be searched. Specifically, the method comprises the following steps:

the first step, the central processing unit receives the input image to be detected of the person to be searched and transmits the image to the data processor.

In the second step, the data processor performs face detection and face feature recognition according to the image to be detected, and the method specifically comprises the following steps:

detecting the image to be detected according to a face detection model and outputting face image data of the face to be detected;

filtering the face image data by using a face quality model according to a preset face image quality threshold;

and extracting the face features of the face image data through a face feature extraction model, outputting a first face feature matrix of the face to be detected, outputting a first face feature matrix of a 1 x 512 one-dimensional floating-point number vector, and storing the first face feature matrix in a memory of a data processor.

And step three, respectively carrying out face search on the first face feature matrix and the sub-face feature library to identify the person to be searched.

The central processing unit selects a sub-face feature library from a plurality of sub-face feature libraries of the face feature library, transposes the sub-face feature library, and loads the transposed sub-face feature library into the data processor; the data processor carries out face search on a second face feature matrix of the transposed sub-face feature library and the first face feature matrix and outputs a sub-face similarity matrix of the face to be detected and the plurality of face features; the central processing unit obtains the maximum similarity with the face to be detected according to the sub-face similarity matrix: if the maximum similarity meets a preset recognition threshold, outputting the recognition information of the face to be detected according to the sub-face feature library, and finishing face recognition; and if the maximum similarity does not meet the preset recognition threshold, judging whether a non-transposed sub-face feature library exists in the plurality of sub-face feature libraries, if so, skipping to the selected sub-face feature library, otherwise, finishing the face recognition.

According to the method, the first face feature matrix and the plurality of sub-face feature libraries are used for face searching, so that people can be quickly found in a plurality of sub-areas around the railway station, the operation speed is effectively increased through one-step matrix operation of face searching, real-time, efficient, quick and accurate face recognition can be realized, the problems in the prior art are solved, and the method has practical application value and wide application prospect. Corresponding to the face recognition method provided in the foregoing embodiments, an embodiment of the present application further provides a face recognition apparatus using the face recognition method, and since the face recognition apparatus provided in the embodiment of the present application corresponds to the face recognition methods provided in the foregoing embodiments, the foregoing embodiment is also applicable to the face recognition apparatus provided in the present embodiment, and detailed description is not given in this embodiment.

As shown in fig. 2a, an embodiment of the present application further provides a face recognition apparatus applying the face recognition method, including a central processing unit and a data processor, where the central processing unit is configured to:

transmitting the received image to be detected to the data processor, enabling the data processor to use a human face feature extraction model to extract human face features of the image to be detected and output a first human face feature matrix of the human face to be detected, and according to a second human face feature matrix with a plurality of human face features of a preset human face feature library and the first human face feature matrix, performing human face search and outputting human face similarity matrixes of the human face to be detected and the human face features;

The face feature of the face to be detected is matrixed, the matrix operation is carried out according to the face feature matrix of the face to be detected and the face feature matrix of the face feature library to complete face search and output a face similarity matrix, the face information which has the maximum similarity in the face similarity matrix and meets the recognition threshold is taken as the recognition information and output, and the face search is completed through one-step matrix operation to obtain the similarity calculation between the face feature of the face to be detected and each face feature in the face feature library, so that the long-time high-complexity cyclic nesting operation in the prior art is avoided, the operation speed is effectively increased, real-time, efficient, rapid and accurate face recognition can be realized, the problems in the prior art are solved, and the method has a wide application prospect. For the specific implementation of this embodiment, reference is made to the foregoing embodiments, which are not described herein again.

In an optional embodiment, the face recognition apparatus further comprises an internal memory, and the central processor is further configured to:

loading the image to be detected stored in the internal memory into the data processor, enabling the data processor to detect the image to be detected by using a face detection model and output face image data of the face to be detected, using a face key point extraction model to extract key points of the face image data and output position coordinates of each key point, performing affine transformation on the face image data by using a face alignment model according to the position coordinates of each key point of the face image data to align the face image data, filtering the face image data by using a face quality model according to a preset face image quality threshold value, extracting face features of the face image data by using a face feature extraction model, outputting a first face feature matrix, and performing normalization processing on the first face feature matrix;

In the embodiment, an image to be detected stored in an internal memory is loaded into a data processor, face detection is carried out through a face detection model of the data processor, key point extraction is carried out through a face key point extraction model, face alignment is carried out through a face alignment model, high-quality face image data are output through filtering of a face image quality threshold value, finally, face feature extraction is carried out through a face feature extraction model to obtain a first face feature matrix of the face to be detected, and normalization processing is carried out on the first face feature matrix; the first face feature matrix after normalization processing and the second face feature matrix after conversion of the face feature library are subjected to face search, a face similarity matrix is obtained through one-step matrix operation, then the maximum similarity value is screened out, and the maximum similarity value is compared with the recognition threshold value to judge whether the face to be detected is recognized or not.

Based on the above face recognition device, as shown in fig. 3, an embodiment of the present application further provides a face recognition system, which includes a plurality of face recognition devices, each face recognition device is connected via a network, and the face feature library of each face recognition device forms a total face feature library of the face recognition system, wherein the central processing unit of each face recognition device is configured to:

selecting a face feature library;

transposing a second face feature matrix of the selected face feature library;

loading the transposed face feature library into the data processor;

and acquiring the maximum similarity with the face to be detected according to the face similarity matrix, outputting the identification information of the face to be detected according to the face feature library and finishing face identification if the maximum similarity meets a preset identification threshold, judging whether a non-transposed face feature library exists in the total face feature library if the maximum similarity does not meet the preset identification threshold, and jumping to the selected face feature library if the non-transposed face feature library exists, otherwise finishing face identification.

The face recognition system of this embodiment comprises a plurality of face recognition devices, for example, face recognition devices disposed in a campus or at different locations in a certain area, each face recognition device comprising a face feature library stored in its internal memory, the plurality of face recognition devices connected together forming a total face feature library, so as to obtain the maximum similarity with the face to be detected by sequentially using the plurality of face feature libraries and a first face feature matrix of the face to be detected, for example, using the face feature library of its own and then the face feature libraries of other face recognition devices, and using a recognition threshold to determine whether the face to be detected is recognized, so as to complete face search by one-step matrix operation to obtain the similarity between the face features of the face to be detected and each face feature in the face feature library, thereby avoiding long-time high-complexity cyclic nesting operation in the prior art, the operation speed is effectively improved, and the face recognition can be realized efficiently, quickly and accurately in real time.

Another embodiment of the present invention provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements: using a human face feature extraction model to extract human face features of the received image to be detected and outputting a first human face feature matrix of the human face to be detected; according to a second face feature matrix with a plurality of face features of a preset face feature library, carrying out face search with the first face feature matrix and outputting a face similarity matrix of the face to be detected and the face features; and acquiring the maximum similarity between the human face and the human face to be detected according to the human face similarity matrix, and outputting the identification information of the human face to be detected according to the human face feature library if the maximum similarity meets a preset identification threshold, wherein the identification information comprises human face information corresponding to the maximum similarity in the human face feature library.

In practice, the computer readable storage medium may take any combination of one or more computer readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present embodiment, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).

As shown in fig. 4, another embodiment of the present invention provides a schematic structural diagram of a computer device. The computer device 12 shown in FIG. 4 is only one example and should not bring any limitations to the functionality or scope of use of embodiments of the present invention.

As shown in FIG. 4, computer device 12 is in the form of a general purpose computing device. The components of computer device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that couples various system components including the system memory 28 and the processing unit 16.

Bus 18 represents one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures. By way of example, such architectures include, but are not limited to, Industry Standard Architecture (ISA) bus, micro-channel architecture (MAC) bus, enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.

Computer device 12 typically includes a variety of computer system readable media. Such media may be any available media that is accessible by computer device 12 and includes both volatile and nonvolatile media, removable and non-removable media.

The system memory 28 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM)30 and/or cache memory 32. The computer device 12 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 34 may be used to read from and write to non-removable, nonvolatile magnetic media (not shown in FIG. 4, and commonly referred to as a "hard drive"). Although not shown in FIG. 4, a magnetic disk drive for reading from and writing to a removable, nonvolatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, nonvolatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 by one or more data media interfaces. Memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the invention.

A program/utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which examples or some combination thereof may comprise an implementation of a network environment. Program modules 42 generally carry out the functions and/or methodologies of embodiments of the invention as described.

Computer device 12 may also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), with one or more devices that enable a user to interact with computer device 12, and/or with any devices (e.g., network card, modem, etc.) that enable computer device 12 to communicate with one or more other computing devices. Such communication may be through an input/output (I/O) interface 22. Also, computer device 12 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network such as the Internet) via network adapter 20. As shown in FIG. 4, the network adapter 20 communicates with the other modules of the computer device 12 via the bus 18. It should be appreciated that although not shown in FIG. 4, other hardware and/or software modules may be used in conjunction with computer device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, to name a few.

The processor unit 16 executes various functional applications and data processing, such as implementing a face recognition method provided by an embodiment of the present invention, by running a program stored in the system memory 28.

The invention aims at the existing problems to establish a face recognition method, a recognition device and a recognition system, the face feature of the face to be detected is matrixed, matrix operation is carried out according to the face feature matrix of the face to be detected and the face feature matrix of the face feature library so as to complete face search and output a face similarity matrix, the face information which has the maximum similarity in the face similarity matrix and meets the recognition threshold value is taken as recognition information and output, the face search is completed through one-step matrix operation so as to obtain the similarity calculation between the face feature of the face to be detected and each face feature in the face feature library, therefore, the method avoids long-time high-complexity loop nesting operation in the prior art, effectively improves the operation speed, can realize real-time efficient, rapid and accurate face recognition, overcomes the problems in the prior art, and has wide application prospect.

It should be understood that the above-mentioned embodiments of the present invention are only examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention, and it will be obvious to those skilled in the art that other variations or modifications may be made on the basis of the above description, and all embodiments may not be exhaustive, and all obvious variations or modifications may be included within the scope of the present invention.

Claims

1. A face recognition method, comprising:

2. The method of claim 1, wherein the performing face feature extraction on the received image to be tested by using a face feature extraction model of the data processor and outputting a first face feature matrix of the face to be tested further comprises:

loading the image to be detected into a data processor;

3. The method according to claim 2, wherein after the detecting the image to be detected using the face detection model and outputting the face image data of the face to be detected, before the performing face feature extraction on the face image data using the face feature extraction model and outputting the first face feature matrix, the method further comprises:

4. The method according to claim 2, wherein after the performing face feature extraction on the face image data using a face feature extraction model and outputting the first face feature matrix, the method further comprises:

and carrying out normalization processing on the first face feature matrix.

5. The face recognition method according to claim 2, wherein before the second face feature matrix having a plurality of face features according to the preset face feature library, performing face search on the second face feature matrix and the first face feature matrix through a data processor, and outputting a face similarity matrix between the face to be detected and the plurality of face features, the face recognition method further comprises:

transposing a second face feature matrix of the face feature library;

and loading the face feature library into the data processor.

6. The face recognition method according to claim 5, wherein the face feature library comprises a plurality of sub-face feature libraries, and the face recognition method comprises:

selecting a sub-face feature library;

said loading said facial feature library into a data processor further comprises: loading the transposed sub-facial feature library into the data processor;

7. The face recognition method of claim 5, wherein before the transposing the second face feature matrix of the face feature library, the face recognition method further comprises:

8. A face recognition apparatus comprising a central processor and a data processor, wherein the central processor is configured to:

9. The face recognition device of claim 8, further comprising an internal memory, the central processor further configured to:

loading the image to be detected stored in the internal memory into the data processor, so that the data processor detects the image to be detected by using a human face detection model and outputs human face image data of the human face to be detected, performs human face feature extraction on the human face image data by using a human face feature extraction model and outputs a first human face feature matrix, and performs normalization processing on the first human face feature matrix;

10. The face recognition apparatus according to claim 9, wherein the face recognition apparatus further comprises a face key point extraction model, a face alignment model and a face quality model, wherein

11. A face recognition system comprising a plurality of face recognition devices according to any one of claims 8-10, each face recognition device being connected via a network, the face feature libraries of each face recognition device forming an overall face feature library of the face recognition system, wherein the central processor of each face recognition device is configured to:

selecting a face feature library;

transposing a second face feature matrix of the selected face feature library;

loading the transposed face feature library into the data processor;

if the maximum similarity does not meet the preset recognition threshold, judging whether a non-transposed face feature library exists in the total face feature library, if so, jumping to the selected face feature library, otherwise, finishing face recognition.

12. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the face recognition method according to any one of claims 1 to 7.

13. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the face recognition method according to any one of claims 1-7 when executing the program.