[ summary of the invention ]
The invention mainly aims to provide a face recognition method of an embedded platform based on deep learning and embedded equipment.
The invention further aims to provide a face recognition system based on an embedded platform of deep learning and embedded equipment.
In order to achieve the above main object, the face recognition method of the embedded platform provided by the invention comprises the steps of acquiring a peripheral deviceThe collected face image information is preprocessed; constructing a feature fusion network, and establishing a face feature recognition model; inputting the preprocessed face image information A and the preprocessed face image information B into a feature fusion network, and extracting feature vectors of the face image information A and the face image information B through a trained face feature recognition model; obtaining a face feature vector F of face image information A1And face feature vector F of face image information B2(ii) a Calculating face feature vector F1And face feature vector F2And outputting the identification result according to the cosine distance.
The method comprises the following steps of inputting face image information into a face detection network for face detection, detecting the area where a face is located in the face image information and the coordinates of five key points including a left eye, a right eye, a nose, a left mouth corner and a right mouth corner on the face, and carrying out affine transformation according to the coordinates of the five key points to obtain a corrected face recognition cutting image, wherein the face detection network is an MTCNN network.
In a further aspect, the feature fusion network is a MobileNetV2 deep network.
A further scheme is that an MS-Celeb-1M data set is constructed as a face training data set, the face training data set is input into a softmax loss function for training, and a pre-training model is obtained through the ArcFace loss function for training; and training the pre-training model by the acquired Asian face data set through an ArcFace loss function, and further obtaining a face feature recognition model.
The method further comprises the steps of determining that the face image information A and the face image information B are the same person if the cosine distance is larger than a face similarity threshold, and determining that the face image information A and the face image information B are different persons if the cosine distance is smaller than the face similarity threshold.
Therefore, the method can input the shot face picture into the depth recognition network for recognition after the face picture is subjected to face detection and position adjustment through affine transformation, so that the whole recognition system is lighter and has no loss of precision, and the method has higher practicability.
In addition, because the public data set of the face recognition mainly comes from European and American countries, the invention adopts Asian face data to train in order to improve the precision of the model facing Asian faces.
Therefore, the face recognition method of the invention adopts a deep network suitable for embedded equipment, and improves the model precision by utilizing Asian face data.
In order to achieve the other object, the invention further provides a face recognition system of an embedded platform, which comprises a preprocessing module, a face recognition module and a face recognition module, wherein the face recognition module is used for acquiring face image information acquired by peripheral equipment and preprocessing the face image information; the model training module is used for constructing a feature fusion network and establishing a human face feature recognition model; a feature extraction module used for inputting the preprocessed face image information A and the face image information B into a feature fusion network and extracting feature vectors of the face image information A and the face image information B through a trained face feature recognition model, and the feature extraction module is also used for obtaining the face feature vector F of the face image information A1And face feature vector F of face image information B2(ii) a A recognition result output module for calculating the face feature vector F1And face feature vector F2And outputting the identification result according to the cosine distance.
The method comprises the following steps of inputting face image information into a face detection network for face detection, detecting the area where a face is located in the face image information and the coordinates of five key points including a left eye, a right eye, a nose, a left mouth corner and a right mouth corner on the face, and carrying out affine transformation according to the coordinates of the five key points to obtain a corrected face recognition cutting image, wherein the face detection network is an MTCNN network.
In a further aspect, the feature fusion network is a MobileNetV2 deep network.
A further scheme is that an MS-Celeb-1M data set is constructed as a face training data set, the face training data set is input into a softmax loss function for training, and a pre-training model is obtained through the ArcFace loss function for training; and training the pre-training model by the acquired Asian face data set through an ArcFace loss function, and further obtaining a face feature recognition model.
The method further comprises the steps of determining that the face image information A and the face image information B are the same person if the cosine distance is larger than a face similarity threshold, and determining that the face image information A and the face image information B are different persons if the cosine distance is smaller than the face similarity threshold.
Therefore, the face recognition system provided by the invention can realize face recognition under a deep neural network, is a high-precision face recognition system facing embedded equipment, and a shot face picture can be input into the deep recognition network for recognition after being subjected to face detection and position adjustment through affine transformation, so that the whole recognition system is lighter in weight, has no loss of precision and has higher practicability.
In addition, because the public data set of the face recognition mainly comes from European and American countries, the invention adopts Asian face data to train in order to improve the precision of the model facing Asian faces.
Therefore, the system of the invention adopts a deep network suitable for embedded equipment, and improves the model precision by utilizing Asian face data.
[ detailed description ] embodiments
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention.
The embodiment of a face recognition method of an embedded platform comprises the following steps:
referring to fig. 1, when detecting and recognizing the face of a user, in the face recognition method of this embodiment, first, step S1 is executed to acquire face image information acquired by a peripheral device, and pre-process the face image information, where in the embodiment of the present invention, the peripheral device may be a mobile phone, a camera, an access control system, or other devices with an image acquisition function.
Specifically, in the face recognition method of this embodiment, when face image information is preprocessed, the face image information is input into a face detection network for face detection, coordinates of five key points including a left eye, a right eye, a nose, a left mouth corner and a right mouth corner on a face and an area where the face is located in the face image information are detected, and affine transformation is performed according to the coordinates of the five key points, so that a corrected face recognition tangential graph can be obtained, wherein the face detection network is an MTCNN network.
Therefore, in the preprocessing part, in order to improve the speed of face detection, the MTCNN network with the cascade concept is used to detect the face image and obtain 5 contour points, namely, a left eye point, a right eye point, a nose tip, a left mouth corner and a right mouth corner. And adopting a fixed template for the contour points of the target face, solving an affine transformation equation of the image through the relation between the detection points and the target points, and then obtaining a corrected face recognition cutting image through affine transformation. The size of the face recognition cut is 112x 112.
Then, step S2 is executed to construct a feature fusion network and establish a face feature recognition model. Wherein, the feature fusion network is a MobileNet V2 deep network. In the face feature extraction part of the embodiment, MobileNetV2 is used as a backbone network, and the network adopts a depth separable convolution and an inventedresiduals module, so that the speed and the precision are well balanced. In addition, the global average pooling layer is adopted to replace the full-connection layer of the original MobileNet V2, so that the model parameter quantity can be reduced, and the calculation speed is improved.
When the face feature recognition model is established, firstly, an MS-Celeb-1M data set is established as a face training data set, the face training data set is input into a softmax loss function for training, and a pre-training model is obtained through the training of the ArcFace loss function; and training the pre-training model by the acquired Asian face data set through an ArcFace loss function, and further obtaining a face feature recognition model.
Specifically, the training method of the face feature recognition model of the embodiment is as follows: firstly, network training is performed on an MS-Celeb-1M data set, parameters of the network are initialized, and the network is not easy to overfit on a single data set. Then, fine tuning is carried out on the self-collected Asian face data set, so that the model learns the distribution on the data set, and the Asian face-oriented scene has higher precision. Specifically, the method can be divided into two stages, in the first stage, an MS-Celeb-1M data set is adopted, in order to enable the network to be easy to converge, a softmax loss function is used for training, and then an ArcFace loss function is used for training, so that the features extracted by the network can obtain smaller intra-class distance and larger inter-class distance. In the second stage, an Asian face data set acquired by self is adopted, the model obtained in the first stage is used as a pre-training model, and training is carried out through an ArcFace loss function, so that the distribution of the model is closer to the Asian face data.
Then, step S3 is executed, the preprocessed face image information a and face image information B are input into the feature fusion network, and feature vector extraction is performed on the face image information a and face image information B through the trained face feature recognition model.
Then, step S4 is executed to obtain the face feature vector F of the face image information a1And face feature vector F of face image information B2。
Therefore, in the face feature extraction part, the deep network designed and trained by the invention is used for extracting the feature vector from the face recognition tangent image, the length of the feature vector is 128, and the feature vector contains enough information for describing the face attribute.
Then, step S5 is executed to calculate a face feature vector F1And face feature vector F2And outputting the identification result according to the cosine distance.
Specifically, if the cosine distance is greater than the face similarity threshold, it is determined that the face image information a and the face image information B are the same person, and if the cosine distance is less than the face similarity threshold, it is determined that the face image information a and the face image information B are different persons.
It can be seen that, in the feature comparison part, the cosine distances of two feature vectors obtained by calculating the two face recognition tangent maps are calculated, if the distance is greater than the face similarity threshold (such as 0.6), the two face recognition tangent maps are judged to be the same person, and if the distance is less than the face similarity threshold, the two face recognition tangent maps are judged to be different persons.
Therefore, the invention can realize face recognition under a deep neural network, is a high-precision face recognition algorithm which can be oriented to embedded equipment, and can input a face picture obtained by shooting into the deep recognition network for recognition after the face picture is subjected to face detection and position adjustment through affine transformation, so that the whole recognition system is lighter in weight and has high practicability without losing precision.
In addition, because the public data set of the face recognition mainly comes from European and American countries, the invention adopts Asian face data to train in order to improve the precision of the model facing Asian faces.
Therefore, the face recognition method of the invention adopts a deep network suitable for embedded equipment, and improves the model precision by utilizing Asian face data.
The embodiment of the face recognition system of the embedded platform comprises the following steps:
as shown in fig. 2, fig. 2 is a schematic block diagram of an embodiment of a face recognition system of an embedded platform according to the present invention. The system comprises a preprocessing module 10, a model training module 20, a feature extraction module 30 and a recognition result output module 40. The face recognition system further comprises an upper computer and an embedded device end (such as a Rayleigh micro RK3399CPU), wherein the upper computer is used for transplanting a driving program and a preset face recognition program to the embedded device end, and the embedded device end is used for running the face recognition program and displaying a recognition result on a display. The face recognition program comprises the steps of obtaining face image information, preprocessing the face image information, establishing a face feature recognition model, extracting face feature vectors and outputting a face recognition result.
The preprocessing module 10 is configured to acquire face image information acquired by a peripheral device, and preprocess the face image information.
The model training module 20 is used for constructing a feature fusion network and establishing a face feature recognition model.
The feature extraction module 30 is configured to input the preprocessed face image information a and face image information B into a feature fusion network, and perform feature vector extraction on the face image information a and the face image information B through a trained face feature recognition model.
The feature extraction module 30 is further configured to obtain a face feature vector F of the face image information a1And face feature vector F of face image information B2。
The recognition result output module 40 is used for calculating the face feature vector F1And face feature vector F2And outputting the identification result according to the cosine distance.
Further, the preprocessing module 10 is configured to preprocess the face image information, and includes: inputting the face image information into a face detection network for face detection, detecting the area of the face in the face image information and the coordinates of five key points including a left eye, a right eye, a nose, a left mouth corner and a right mouth corner on the face, and performing affine transformation according to the coordinates of the five key points to obtain a corrected face identification cutting image, wherein the face detection network is an MTCNN network.
Further, the feature fusion network is a MobileNetV2 deep network.
Further, the model training module 20 is configured to establish a face feature recognition model, including: an MS-Celeb-1M data set is constructed as a face training data set, the face training data set is input into a softmax loss function for training, and a pre-training model is obtained through the training of the ArcFace loss function; and training the pre-training model by the acquired Asian face data set through an ArcFace loss function, and further obtaining a face feature recognition model.
Further, the recognition result output module 40 is configured to output a recognition result according to the cosine distance, and includes: and if the cosine distance is greater than the face similarity threshold, determining that the face image information A and the face image information B are the same person, and if the cosine distance is less than the face similarity threshold, determining that the face image information A and the face image information B are different persons.
Specifically, the accuracy of 3000+ positive face pair judgment and 3000+ negative face pair judgment constructed from an actual data set reaches 97.3%, and the actual measurement speed on an embedded equipment end Rui core micro RK3399CPU reaches about 50 ms.
Therefore, the face recognition system provided by the invention can realize face recognition under a deep neural network, is a high-precision face recognition system facing embedded equipment, and a shot face picture can be input into the deep recognition network for recognition after being subjected to face detection and position adjustment through affine transformation, so that the whole recognition system is lighter in weight, has no loss of precision and has higher practicability.
In addition, because the public data set of the face recognition mainly comes from European and American countries, the invention adopts Asian face data to train in order to improve the precision of the model facing Asian faces.
Therefore, the system of the invention adopts a deep network suitable for embedded equipment, and improves the model precision by utilizing Asian face data.
It should be noted that the above is only a preferred embodiment of the present invention, but the design concept of the present invention is not limited thereto, and any insubstantial modifications made by using the design concept also fall within the protection scope of the present invention.