CN108681704A - A kind of face identification system based on deep learning - Google Patents

Abstract

A face recognition system based on deep learning belongs to the technical field of deep learning and image processing. Including PC-side network training module, system-on-chip and peripheral peripheral modules: PC-side network training module includes data set processing, CNN training, initialization, gradient return, backpropagation and forward propagation units; first in the PC-side network module Construct the network structure, collect enough face training sets at the same time, and train a usable deep learning face recognition network; the system on chip includes image acquisition module, fifo control module, deep learning face recognition module, personnel information storage module and image display module. The recognition speed can reach 400FPS under the 50MHz clock; the recognition accuracy is as high as 99.25%, which is comparable to the human eye recognition accuracy; the training set covers various lighting conditions, and the recognition function can be completed under most lighting conditions, with good robustness .

Description

A face recognition system based on deep learning

technical field

The invention relates to a face recognition system based on deep learning, belonging to the technical field of digital image processing.

Background technique

Human faces are highly non-rigid, and there are a large number of details that reflect individual differences. Face recognition is the process of finding a matching face by comparing the face image detected from a static image or dynamic video with the face image in the database. It is usually used for identification and identification purposes. It belongs to the field of biometric identification.

There are many traditional face recognition methods, which can be roughly divided into two categories. One is based on local methods, such as using local descriptor Gabor, local binary patterns, etc. for recognition; the other is based on global methods, including Classic face recognition algorithms, such as subspace learning algorithms such as eigenface method and linear discriminant analysis method, and popular learning algorithms such as locality preserving projection algorithm.

At present, the research in the field of face recognition at home and abroad is mainly focused on principal component analysis, LBP algorithm, and deep learning algorithm. These three methods are better than other algorithms in terms of success rate and implementation difficulty of face recognition.

Deep learning has strong robustness in the field of face recognition, as well as a very high recognition rate, and the recognition effect is far superior to other algorithms. But at this stage, face recognition on the FPGA platform mainly uses principal component analysis and LBP algorithm.

Although these two algorithms are easier to implement on the FPGA platform compared with the deep learning algorithm, due to the limitations of the algorithm itself (poor robustness, low recognition rate, and increased time consumption for preprocessing), the speed of face recognition and In terms of accuracy, these two algorithms cannot meet the technical requirements of frame-by-frame real-time recognition and adaptation to various lighting conditions. The literature shows that most of the scientific research results of face recognition on the FPGA platform have a detection speed of less than 10FPS, and the fastest results at this stage can only reach 25FPS. Most of the deep learning algorithms are implemented by using CPU, GPU, and TPU, and there are relatively few researches on the FPGA platform. Using FPGA to design a deep learning network for face recognition is relatively blank, and there are also some scientific researchers. Some attempts have been made in this regard, but there has been no mature scientific research results.

The present invention implements a deep learning face recognition system on the FPGA platform. The system works at a clock frequency of 50MHz, and the recognition speed can reach 400FPS, far exceeding some existing achievements. The recognition accuracy is comparable to the recognition accuracy of human eyes, as high as 99.25%. The training set covers a variety of lighting conditions, and the recognition function can be completed under most lighting conditions, with good robustness.

Contents of the invention

The purpose of the present invention is to propose a face recognition system based on deep learning for the defects of poor real-time performance, low detection accuracy and poor robustness of the face recognition technology on the FPGA platform.

A face recognition system based on deep learning consists of a PC-side network training module, a system-on-chip and peripheral peripheral modules:

Among them, the PC-side network training module includes: dataset processing unit, CNN training unit, initialization unit, gradient return unit, backpropagation unit, and forward-propagation unit; according to actual needs, first build the network structure in the PC-side network module , and collect enough face training sets at the same time to train an available deep learning face recognition network, wherein; the activation function used by the face recognition network is a sigmoid function;

The system on chip includes image acquisition module, fifo control module, deep learning face recognition module, personnel information storage module and image display module;

Peripheral peripheral modules include camera sub-module, VGA digital-to-analog conversion module and local display;

Among them, the camera sub-module mainly includes multiple cameras and camera connection lines;

The connection relationship of each module in a face recognition system based on deep learning is as follows:

The multi-channel camera in the camera sub-module connects multiple data channels to the image acquisition module through the camera cable, the image acquisition module is connected to the fifo control module, the fifo control module is connected to the deep learning face recognition module, and is connected to the image display module at the same time , the deep learning face module is connected to the personnel information storage module, the personnel information storage module is integrated in the fifo control module, controlled by the fifo control module, and the result is output to the image display module; the image display module then transmits the digital signal to the VGA digital The analog conversion module is output to the local display after digital-to-analog conversion;

The functions of each module are as follows:

The multi-channel camera completes the image acquisition function and transmits it to the image acquisition module;

The function of the image acquisition module is to receive the input data of the video stream and transmit it to the fifo control module;

The fifo control module caches the incoming video signal to ensure the integrity of the output image data, and performs real-time downsampling on the incoming picture, downsampling to a pixel size suitable for network recognition and storing it in the fifo;

The function of the deep learning face recognition module is to realize the accelerated processing of the face recognition function. Its input is the down-sampled face image. After the module completes the face recognition, it outputs the face result;

The personnel information is stored in the personnel information storage module, which receives the result from the deep learning face recognition module, and outputs the specific information of the person to the image display module, which is integrated in the fifo control module; controlled by the fifo control module, output to the image display module;

The function of the image display module is to receive the cached video stream data, generate a standardized video stream timing and match the corresponding pixel data, and output it to the off-chip VGA digital-to-analog conversion module. In addition, this module also has the function of automatic data alignment, which can overcome the The display misalignment problem caused by the lag of the input video stream data after the local timing is generated;

The functions of each module of the peripheral peripheral module are as follows:

The function of the camera is to capture the external scene and generate a stable digital video stream;

The function of the VGA digital-to-analog conversion module is to receive the digital signal and line-field synchronization signal from the image display module, and perform digital-to-analog conversion on the digital signal and transmit it to the local display; the function of the local display is to collect images and display the recognition results in real time;

A working process of a face recognition system based on deep learning, including building a face recognition network, training a face recognition network, building an FPGA platform, and updating network parameters. The specific steps are as follows:

Step 1. Build a convolutional neural network for face recognition in the PC-side network training module;

Step 2, use the pre-collected face library to train the face recognition network to obtain the trained network parameters;

Step 3. Build a face recognition network based on the FPGA platform;

Step 4. Import the network parameters trained in step 2 into the face recognition network built in step 3.

Beneficial effect

A face recognition system based on deep learning, compared with the prior art, has the following beneficial effects:

1. In the prior art, there is no design that combines deep learning and face recognition and realizes it on the FPGA platform, but the present invention successfully realizes a face recognition network based on deep learning on the FPGA platform;

2. compared with existing face recognition technology, recognition speed is improved: the processing speed of existing relevant face recognition system is generally lower than 25FPS, the present invention is by adopting the CNN model in deep learning technology field to gather the image Training, greatly improving the image processing speed;

3. Compared with the existing face recognition, the recognition accuracy has been improved, which is specifically reflected in the following: the face recognition method based on the FPGA platform is mainly the principal component analysis method and the LBP algorithm, and the recognition accuracy can only reach 80%. %; and the deep learning algorithm recognition rate adopted by the present invention can reach more than 99%, which can be compared with the accuracy of human eye recognition.

Description of drawings

Fig. 1 is a structural block diagram of a face recognition system based on deep learning of the present invention;

FIG. 2 is a schematic diagram of a network structure in an embodiment of a face recognition system based on deep learning in the present invention.

Detailed ways

The present invention will be further illustrated and described in detail below in conjunction with the accompanying drawings and embodiments.

Example 1

This embodiment describes the structure and specific implementation of a face recognition system based on deep learning in the present invention.

As shown in Figure 1, it is a structural block diagram of a face recognition system based on deep learning. Based on this structural block diagram, this embodiment was produced.

This embodiment is realized by using the PGT180H device of Ziguang Tongchuang Company, and the size of the image processed by the whole system is 540X480 pixels.

First, in the PC-side network training module, a 6-layer CNN convolutional neural network was designed and trained with matlab. Here we used the CMU_PIE face database from Carnegie Mellon University and some face databases collected by ourselves. After testing on the test set, the 6-layer network can achieve a recognition rate of 99.25% for 20 people in the face database.

The 6-layer convolutional neural network consists of one input layer, two convolutional layers, two pooling layers and one fully connected layer. The input layer is an image with a size of 32x32 pixels. After 4 hidden layers, it is connected to the last fully connected layer and the result is output. The network uses the activation function as the sigmoid function;

Among them, the 4 hidden layers include the first convolutional layer, the first pooling layer, the second convolutional layer and the second pooling layer;

Build a 6-layer network structure on the FPGA platform, as shown in Figure 2, import and train the network structure parameters.

What embodiment system camera adopts is MT9V034 gray-scale image sensor, and this is a kind of global exposure CMOS sensor, and the speed is 60FPS when full pixel (752Hx480V) output, and SoC system on chip configures its output as 540x480 size when initializing, and then device Automatically collect and input continuous data (including field/line synchronization) to the image acquisition module of the system;

This example builds a 6-layer convolutional neural network on the FPGA platform. The network structure is the same as the network structure trained on the PC side. It consists of one layer of input layer, two layers of convolutional layer, two layers of pooling layer, and one layer of fully connected layer. The sigmoid activation function adopts 6 segmental fitting to implement. Under the clock frequency of 50MHz, the system completes face recognition within 2.4ms, and the number of people recognized is 20. After the algorithm processing is completed, the identification result is transmitted to the fifo control module, and the fifo control module controls the personnel information storage module to output personnel information;

The fifo control module transmits the real-time cached images and personnel information collected by the camera to the image display module, and the image display module generates corresponding line and field synchronization signals, which are output to the peripheral VGA digital-to-analog conversion module. The VGA digital-to-analog conversion module completes the conversion of digital signals to analog signals, and connects to the VGA display to complete the real-time acquisition and display of images and the display of recognition results.

The operation process of the present invention: the system is powered on, the bit file is loaded, and the configuration of the system is completed. The recognition target enters the image acquisition area, and the target recognition result is displayed in real time on the local monitor.

The system described in this example builds a 20-person face recognition network under the condition that the image acquisition speed is fast enough. With a clock frequency of 50MHz as the system clock, the face recognition can be completed within 2.4ms, and the recognition speed can reach up to 400FPS. Due to the limitation of the number of frames of the MT9V034 grayscale image sensor, in this example, real-time face recognition at 60FPS is realized without delay. Compared with the processing speed of the existing design (less than 25FPS), the real-time performance is improved. And the recognition accuracy rate is as high as 99.25%, which is comparable to the recognition accuracy rate of human eyes and far exceeds some current designs based on principal component analysis and LBP algorithm. This example is tested under different lighting environments, and the face recognition can be accurately completed, and the robustness to lighting is strong.

The above descriptions are preferred embodiments of the present invention, and the present invention should not be limited to the content disclosed in the embodiments and accompanying drawings. All equivalents or modifications accomplished without departing from the disclosed spirit of the present invention fall within the protection scope of the present invention.

Claims (1)

1. A face recognition system based on deep learning, characterized in that: comprising a PC end network training module, a system-on-chip and peripheral peripheral modules; Among them, the PC-side network training module includes: dataset processing unit, CNN training unit, initialization unit, gradient return unit, backpropagation unit, and forward-propagation unit; according to actual needs, first build the network structure in the PC-side network module , and collect enough face training sets at the same time to train an available deep learning face recognition network, wherein; the activation function used by the face recognition network is a sigmoid function; The system on chip includes image acquisition module, fifo control module, deep learning face recognition module, personnel information storage module and image display module; Peripheral peripheral modules include camera sub-module, VGA digital-to-analog conversion module and local display; Among them, the camera sub-module mainly includes multiple cameras and camera connection lines; The connection relationship of each module in a face recognition system based on deep learning is as follows: The multi-channel camera in the camera sub-module connects multiple data channels to the image acquisition module through the camera cable, the image acquisition module is connected to the fifo control module, the fifo control module is connected to the deep learning face recognition module, and is connected to the image display module at the same time , the deep learning face module is connected to the personnel information storage module, the personnel information storage module is integrated in the fifo control module, controlled by the fifo control module, and the result is output to the image display module; the image display module then transmits the digital signal to the VGA digital The analog conversion module is output to the local display after digital-to-analog conversion; The functions of each module are as follows: The multi-channel camera completes the image acquisition function and transmits it to the image acquisition module; The function of the image acquisition module is to receive the input data of the video stream and transmit it to the fifo control module; The fifo control module caches the incoming video signal to ensure the integrity of the output image data, and performs real-time downsampling on the incoming picture, downsampling to a pixel size suitable for network recognition and storing it in the fifo; The function of the deep learning face recognition module is to realize the accelerated processing of the face recognition function. Its input is the down-sampled face image. After the module completes the face recognition, it outputs the face result; The personnel information is stored in the personnel information storage module, which receives the result from the deep learning face recognition module, and outputs the specific information of the person to the image display module, which is integrated in the fifo control module; controlled by the fifo control module, output to the image display module; The function of the image display module is to receive the cached video stream data, generate a standardized video stream timing and match the corresponding pixel data, and output it to the off-chip VGA digital-to-analog conversion module. In addition, this module also has the function of automatic data alignment, which can overcome the The display misalignment problem caused by the lag of the input video stream data after the local timing is generated; The functions of each module of the peripheral peripheral module are as follows: The function of the camera is to capture the external scene and generate a stable digital video stream; The function of the VGA digital-to-analog conversion module is to receive the digital signal and line-field synchronization signal from the image display module, and perform digital-to-analog conversion on the digital signal and transmit it to the local display; the function of the local display is to collect images and display the recognition results in real time; A working process of a face recognition system based on deep learning, including four parts: building a face recognition network, training a face recognition network, building an FPGA platform, and updating network parameters. The specific steps are as follows: Step 1. Build a convolutional neural network for face recognition in the PC-side network training module; Step 2, use the pre-collected face library to train the face recognition network to obtain the trained network parameters; Step 3. Build a face recognition network based on the FPGA platform; Step 4. Import the network parameters trained in step 2 into the face recognition network built in step 3.