TWI731511B - Finger vein or palm vein identification processing and neural network training method - Google Patents

Abstract

This patent proposes a new training method called “Ripple-Freeze,” which distributes different gradient descent rates to different layers. It mitigates gradients in several layer changes under different epochs. The shallow layer passes a small oscillation to the deep layer and makes the parameters change more aggressively. Experimental results indicate that the proposed training method can effectively accelerate the training procedure, with acceleration rates reaching as much as 20%. The results indicate the feasibility of this training method and take a step forward for practical application of palm vein identification systems.

Description

Finger/palm vein recognition processing and its neural network training method

The present invention is related to finger/palm vein recognition processing. More specifically, it is related to deep neural network training applied to finger/palm vein recognition processing. The present invention names it as Ripple-Freeze Training (Ripple-Freeze Training, RFT).

With the advancement of technology, regardless of airport immigration clearance, bank ATM operation, company access control, and unlocking of various technological products, in order to ensure safety, the use of biometrics for identification has become the current mainstream. Among them, the use of palm veins for body discrimination is an important branch in the field of biometrics. This method uses a non-contact identification method, which is not only convenient and sanitary, but the biggest difference from other biological identification methods is that the vein is buried under the skin, which is more difficult to forge than other identification methods, and the physiological tissue of the vein changes little with the passage of time , The subject of identification is sufficiently stable.

Recognition of finger veins or palm veins with vein images involves the following steps: obtaining near-infrared images, extracting features, obtaining ROI images, and then performing feature classification to delineate the vein features, and use the geometry of the vein features The shape or topological structure is compared for one-to-many identification (Identification).

Feature classification is to send the ROI image data set into the neural network for training to perform prediction, statistics and accuracy evaluation. One of the strategies of Transfer Learning neural network training is to train a pretrained model again to apply it to a new task. Common transfer learning training strategies are decided to freeze at the beginning of training Calculate the weights of which convolutional layers. Once the decision is made, from the first epoch (period, one epoch=a forward pass and a reverse pass of all training samples.) to the end of the training, the entire training process will not be changed.

In a neural network, the bottom-level features are relatively low-level and general, the top-level features are relatively high-level and special, and the high-level and special features are closer to the fully connected layer. Therefore, it can be reasonably speculated that the neural network is mainly classified by high-level features. The high-level features are formed from the middle-level features, and the middle-level features are formed from the low-level features. If the traditional transfer learning training strategy is adopted, the low-level or middle-level features change too sharply, which will make the high-level features difficult to form.

The present invention proposes a palm vein recognition system combined with a deep learning neural network, using the innovative training method of the present invention "Ripple-Freeze Training method" (RFT), which is a volume of convolutional neural network The multi-layer training method in which the weights are gradually frozen in the order of the bottom layer, the middle layer, and the top layer. First of all, the weights of all convolutional layers will not be frozen. After training for several epochs, the lower convolutional layer will be frozen to a greater degree to train the middle convolutional layer; after training for a few more epochs, the middle convolutional layer will also be processed Freeze to train the top convolutional layer. The neural network is assigned different gradient descent speeds layer by layer to avoid excessive changes in the gradient of the layer (bottom layer, middle layer, and top layer) between different epochs (periods), and the parameters of the bottom layer are oscillated to the top layer to cause more changes. Violent shock. The experimental results show that the training method of the present invention can effectively accelerate the training of the neural network, the acceleration rate can reach 20%, and the Top-1 accuracy (classification accuracy) can reach 99%.

The present invention is a neural network training method for finger/palm vein recognition and processing. The method trains a convolutional neural network. The convolutional neural network is used for the data of finger/palm vein infrared images. The material set is feature-classified; the convolutional neural network includes a plurality of convolutional layers, and the convolutional layers are arranged in a sequence of bottom, middle, and top layers; the training method includes:

Step 1: All the weights of the convolutional layer are trained for a predetermined number of epochs (epochs) to obtain the initial weights;

Step 2: Perform weight freeze on the bottom layer;

Step 3: The middle layer performs training for a predetermined number of epochs, and the training ignores the variables of the bottom layer due to the result of step 2.

Step 4: The middle layer performs weight freeze;

Step 5: The top layer is trained for a predetermined number of epochs, and because of the result of step 4, the training of step 5 ignores the variables of the middle layer, and classifies the high-level features of the finger/palm vein infrared image.

The finger/palm vein recognition processing of the present invention includes:

STP-1, to obtain near-infrared image of finger/palm vein (NIR image);

STP-2, the NIR image performs image pre-processing (Image Pre-processing);

STP-3, which performs Feature Extraction on the pre-processed image;

STP-4, cut the image after feature extraction to define the palm vein recognition area (Region-of-Interest, ROI);

STP-5, the palm vein recognition area performs data augmentation processing (Date Augmentation);

STP-6, which performs data pre-processing on the data obtained after data enhancement processing, and generates a training dataset, a validation dataset, and a test Data set (Testing dataset);

STP-7, input all the above data sets (dataset) into the convolutional neural network for training; training methods include:

STP-7.1: Arrange the bottom, middle, and top layers of the convolutional layers of the convolutional neural network in a sequence;

STP-7.2, all the weights of the convolutional layer are trained for a predetermined number of epochs to obtain the initial weights;

STP-7.3, the bottom layer performs weight freezing;

STP-7.4, the middle layer is trained for a predetermined number of epochs, and due to the result of STP-7.3, the training ignores the variables of the bottom layer;

STP-7.5, the middle layer performs weight freeze;

In STP-7.6, the top layer is trained for a predetermined number of epochs, and due to the result of STP-7.5, the training ignores the variables of the middle layer, and classifies the high-level features of the finger/palm vein infrared image.

STP-8, performs one-to-many identification comparison of classified high-level features.

Figure 1 is a schematic diagram of the Casia multi-spectral imaging device used in the present invention.

Figure 2 shows the number of images obtained at each stage of the palm vein recognition system using Casia in the present invention.

Figure 3 shows the accuracy learning curve on ResNet-50 using the 100-person data set of the present invention.

Figure 4 shows the loss learning curve on ResNet-50 using the 100-person data set according to the present invention.

Figure 5 is the learning rate decay curve on ResNet-50 using the 100-person data set according to the present invention.

Figure 6 is the accuracy learning curve of Retrain plus RFT of the present invention.

Figure 7 is the loss learning curve of Retrain plus RFT of the present invention.

Figure 8 shows the ROC curve of Retrain plus RFT of the present invention.

Figure 9 is the learning rate decay curve of each convolutional layer after Retrain plus RFT of the present invention.

Figure 10 is the learning rate decay curve of each batch normalization layer after Retrain plus RFT of the present invention.

The new training method proposed by the present invention is applied to palm vein or finger vein recognition processing. The finger/palm vein recognition process can adopt the known standard process of vein image recognition process, including:

Perform Image Pre-processing on the obtained infrared vein image (NIR image). The pre-processing is mainly image binarization to obtain a binary image. Binary image ) Perform Edge Detection and Isolation/Enhancement respectively.

Perform Feature Extraction on the pre-processed image. Feature extraction methods include but are not limited to repeated line tracking, maximum curvature, wide line detector and Gabor filter (Gabor filter).

The feature extraction image is processed by image cutting to define the palm vein recognition area (Region-of-Interest, ROI), through the palm center tracking step (Palm center tracking), finger valley tracking step (Finger valley tracking), and repeated sampling steps (Repeated sampling) Obtain ROI images.

Perform Data Augmentation on the ROI image, and use the data to augment The strong (Data Augmentation) method performs various transformations on the ROI image, such as image blur (Gaussian Blur), image sharpening (Sharpen), affine transform (Affine transform), image shaking (Shake), adding Gaussian noise By adding Gaussian Noise (Gaussian Noise) and adding Coarse Dropout, these data are processed to increase the data volume and diversity of the training data set, which can prevent the neural network from overfitting during the training phase.

The data obtained after data enhancement processing is normalized (Data Normalization), standardization (Standardization), and labeling (Data Pre-Processing); it can help optimize the algorithm in the neural network in high-level processing. The rate of descent in the dimensional feature space; this section will generate training datasets, validation datasets, and testing datasets.

The processed data set and labels are input into the neural network for training, performance evaluation, and prediction classification.

The present invention uses two deep convolutional neural network architectures ResNet-50 (Residual Network, residual network) and DenseNet (Dense Convolutional Network, dense convolutional neural network) for training as a verification case.

The present invention proposes a new training method. In the training process, first, the weights of all convolutional layers will not be frozen. After training for several epochs, compare the freezing of the bottom convolutional layer to train the middle convolutional layer. After training for a few more epochs, the middle convolutional layer is also frozen to train the top convolutional layer. The basic concept is to train the middle network from the previous network, then train the latter network from the middle network, and finally train the training method of the classifier.

The difference from the two training strategies of Transfer Learning is that the training strategy proposed by the present invention gradually freezes the weights during the training process. Generally speaking, you can It is observed that in the neural network, the bottom-level features are relatively low-level and general, the top-level features are relatively high-level and special, and the high-level features are closer to the fully connected layer. Therefore, it can be reasonably speculated that the neural network is mainly classified by high-level features. The high-level features are formed from the middle-level features, and the middle-level features are formed from the low-level features. Therefore, it can be inferred that if the low- and middle-level features change too sharply, the high-level features will not be easily formed; on the contrary, if the low- and middle-level features are given a certain time after the training is completed, The freezing allows the training of neural networks to focus on shaping high-level features, and it is speculated that it should speed up the training of neural networks.

In practice, there are many ways to freeze the weights of a convolutional layer. The present invention freezes the weights by changing the learning rate.

&

在訓練開始初期，特別是衰減率(decay rate)很小時，v _t 與s _t 特別偏向0，所以需要做個偏差修正。η表示為學習率，W _t 表示為調整前權重係數，W _t+1表示為調整後權重係數，ε表示為平滑值，

與

表示為t時間內的阻力係數。 In each backpropagation, after calculating the gradient, it will first multiply the learning rate, and then update the weight. The original weight update formula is as shown in formula (1.1), v _t is the first moment estimation of the gradient at time t, which represents the mean, and s _t is the second moment estimation of the gradient at time t, which represents the variance.

&

In the early beginning of the training, especially the rate of decay (decay rate) is very small, v _t and s _t 0 special favor, so we need to be offset correction. η is the learning rate, W _t is the weight coefficient before adjustment, W _{t +1} is the weight coefficient after adjustment, and ε is the smooth value.

versus

Expressed as the drag coefficient in t time.

&

在訓練開始初期，特別是衰減率(decay rate)很小時，v _t 與s _t 特別偏向0，所以需要做個偏差修正。η表示為學習率，W _t 表示為調整前權重係數，W _t+1表示為調整後權重係數，ε表示為平滑值。 The training method proposed by the present invention will additionally multiply the learning rate η by a parameter α _l , as shown in formula (1.2), and define the global learning rate η originally belonging to the global variable as an independent layer of each convolutional layer Learning rate (layer-wise learning rate) η _l . The neural network reuses formula (1.3) to update the weight. α _l is a random parameter, the global learning rate η originally belonging to the global variable is defined as the independent layer-wise learning rate η _{l of} each convolutional layer, and s _t is the time t The second moment estimation of the gradient represents the variance,

&

In the early beginning of the training, especially the rate of decay (decay rate) is very small, v _t and s _t 0 special favor, so we need to be offset correction. η is the learning rate, W _t is the weight coefficient before adjustment, W _{t +1} is the weight coefficient after adjustment, and ε is the smooth value.

η _l =η×α _l (1.2)

Regardless of equation (1.1) or equation (1.3), the learning rate attenuation method remains the same. If the observation loss (loss) does not continue to decrease during the training process, the learning rate attenuation is performed once, as in equation (1.4). The subscript T represents an epoch in which loss is not continuously reduced, and the next epoch will reduce the learning rate. In practice, the initial global learning rate is set to 10 ^-3 and the minimum global learning rate is 10 ^-5 . The subscript T represents an epoch in which loss is not continuously reduced, and the next epoch will reduce the learning rate. In practice, the initial global learning rate is set to 10 ^-3 and the minimum global learning rate is 10 ^-5 .

The training method proposed by the present invention adjusts the learning rate in the optimization algorithm, and each neural network must use the optimization algorithm to update the weights, so this training method is not limited to a specific neural network architecture. From the experimental results, the learning rate of different convolutional layers It is the vertical axis, and the epoch is the horizontal axis, which depicts the changes in the learning rate during the entire training process. It looks like a ripple. This training method is named Ripple-Freeze Training (Ripple-Freeze Training) in the present invention. ,RFT).

The far-infrared palm image of the present invention is taken from the Multi-Sprctral Palmprint Image Database provided by The Institute of Automation (Chinese Academy of Sciences, CASIA). This data set contains 100 subjects with 6 wavelengths of light, including 460nm, 630nm, 700nm, 850nm, 940nm and white light. Each subject has 6 grayscale images in each band, and the imaging equipment is designed by CASIA, as shown in Figure 1. In the present invention, 6 gray-scale images in the 850nm band are classified as training dataset by each subject with images numbered 1 to 4, and image number 5 is the Validation dataset, number 6 The image is a testing dataset. The number of images output at each stage is shown in Figure 2. The data results after each image pre-processing are shown in Table 1 below.

Table 1 Dataset

Use CASIA 850nm 100-person database and ResNet-50 architecture without using Ripple-Freeze Training. The training results are shown in Table 2. The accuracy learning curve of the training process is shown in Figure 3, the loss learning curve is shown in Figure 4, and the learning rate decay curve is shown in Figure 5.

Table 2 Training results of 100 CASIA people on ResNet-50

The comparison between the experimental results and the state of the art is shown in Table 3. Compared with other palm vein recognition systems, this system has an accuracy rate of 99%. In [48], LBP and LDP are used to encode the vein image, and then the Hamming distance is calculated with the data set image. Compared with this system, the accuracy rate of this system is increased by 1.8%, Precision by 0.49%, and Recall by 1.8%. In the literature [49], Matched Filter is used to extract vein features, and then the image is XOR matching. Compared with this system, the accuracy rate of this system is increased by 0.2%, Precision is slightly decreased by 0.83%, and Recall is increased by 0.2%.

Reference [48]: L. Mirmohamadsadeghi and A. Drygajlo, "Palm vein recognition with local binary patterns and local derivative patterns," in Proc. 2011 International Joint Conference on Biometrics (IJCB), Washington, DC, 2011, pp. 1- 6.

Literature [49]: Y.-B. Zhang, Q. Li, J. You, and P. Bhattacharya, “Palm vein extraction and matching for personal authentication,” Advances in Visual Information Systems, vol. 4781, 2007, pp. 154-164.

Table 3. Comparison table of ResNet-50 experimental results and state of the art

Under the Retrain strategy, consider whether Ripple-Freeze Training is used or not for training. The learning rate of the convolutional layer and the batch normalization layer is set, and the pooling layer has no learning rate can be set. The setting method is shown in Table 4. The comparison of the accuracy of the experimental results is shown in Table 5, and the comparison of the training time is shown in Table 6. From Table 5 and Table 6, it can be observed that in the 60-person application scenario, the accuracy of using Ripple-Freeze Training is increased by 0.03%, and the training time is shortened from 4 hours and 17 minutes to 3 hours and 42 minutes, and the acceleration rate is about 13.48%. The calculation method of acceleration rate is as formula (1.5) . t represents the training time without Ripple-Freeze Training, t _RF represents the training time with Ripple-Freeze Training.

In the application scenario of 80 people, the accuracy rate of using Ripple-Freeze Training is reduced by 1.25%, and the training time is shortened from 5 hours and 10 minutes to 4 hours and 25 minutes, and the acceleration rate is about 11.85%. In the application scenario of 100 people, this is the maximum number of categories that the experiment can classify. The accuracy rate of using Ripple-Freeze Training has not been reduced at all. The training time has been shortened from 11 hours and 5 minutes to 8 hours and 48 minutes, and the acceleration rate is about 20.61%.

From Table 5 and Table 6, it can be observed that in the application scenario of 60 people, the accuracy of using Ripple-Freeze Training has increased by 0.03%, and the training time has been shortened from 4 hours and 17 minutes to 3 hours and 42 minutes. rate) is about 13.48%. The calculation method of acceleration rate is as formula (1.6) . t represents the training time without Ripple-Freeze Training, t _RF represents the training time with Ripple-Freeze Training.

In the application scenario of 80 people, the accuracy rate of using Ripple-Freeze Training is reduced by 1.25%, and the training time is shortened from 5 hours and 10 minutes to 4 hours and 25 minutes, and the acceleration rate is about 11.85%. In the application scenario of 100 people, this is the maximum number of categories that the experiment can classify. The accuracy rate of using Ripple-Freeze Training has not been reduced at all. The training time has been shortened from 11 hours and 5 minutes to 8 hours and 48 minutes, and the acceleration rate is about 20.61%.

Table 4 RFT learning rate setting on ResNet-50

Table 5 Accuracy comparison of Retrain plus RFT

Table 6 Comparison of training time between Retrain and RFT

Finally, the experimental results after Ripple-Freeze Training are added. The accuracy learning curve is shown in Figure 6, the loss learning curve is shown in Figure 7, and the learning attenuation curve of the convolutional layer is shown in Figure 9. The learning of the batch normalization layer The attenuation curve is shown in Figure 10, and the ROC curve is shown in Figure 8.

Claims (10)

A neural network training method for finger/palm vein recognition. The neural network training method trains a convolutional neural network to classify features of a data set of infrared images of finger/palm veins; the convolutional neural network It includes a plurality of convolutional layers, and the convolutional layers are arranged in a sequence of the bottom, middle, and top layers; the neural network training method includes: Step 1, the weights of all the convolutional layers are performed for a predetermined number of epochs (epoch) Training to obtain initial weights; step two, the bottom layer performs weight freezing; step three, the middle layer performs training for a predetermined number of epochs, and the training ignores the variables of the bottom layer due to the result of step two; step four, The middle layer is weight frozen; step 5, the top layer is trained for a predetermined number of epochs, and because of the result of step 4, the training of step 5 ignores the variables of the middle layer, based on the data of the infrared image of the finger/palm vein The set performs a high-level feature classification; among them, the weight freezing of step 2 and step 4 is performed by changing the learning rate. The neural network training method for finger/palm vein recognition as described in claim 1, wherein the weight freezing of step 2 and step 4 is performed by changing the learning rate, which is performed at each In a backpropagation, after adjusting the weight coefficient ( W _t ) to calculate the gradient, it will first multiply it by a learning rate, and then update the weight to obtain an adjusted weight coefficient ( W _{t +1} ), where the learning rate is changed ( The method of learning rate is that the bottom, middle, and top layers of the convolutional layers have different learning rates. The neural network training method for finger/palm vein recognition as described in claim 2, wherein the method of changing the learning rate is to enable the global learning of the bottom, middle, and top layers of the convolutional layers. The global learning rate η is multiplied by a parameter α _l , which is defined as the independent layer-wise learning rate η _{l of} each convolutional layer. The neural network training method for finger/palm vein recognition described in claim 3, wherein the weight freezing method weight freezing is performed by changing the learning rate, adjusting the weight coefficient ( W _t ) before And the adjusted weight coefficient ( W _{t +1} ) by using formula (1.3) to update the weight;

among them,

Is the first moment estimation of the gradient at time t after the deviation correction, which represents the mean,

Is the second moment estimation of the gradient at time t after the deviation is corrected, which represents the variance; ε is the smooth value; where in formula (1.3), the attenuation of these learning rates remains the same . The neural network training method for finger/palm vein recognition described in claim 4, in which, if the observation loss (loss) does not continue to decrease during the training process, the next epoch (epoch) is Decreased learning rate

Among them, the subscript T represents the number of epochs in which the loss (loss) has not continued to decrease. A method for identifying and processing finger/palm veins, including: step 1, obtaining a near-infrared image (NIR image) of the finger/palm vein; step 2, performing image pre-processing (Image Pre-processing) on the NIR image; Step 3: Perform Feature Extraction on the pre-processed image; Step 4: Cut the image after feature extraction to define the palm vein recognition region (Region-of-Interest, ROI); Step 5, the Palm vein identification area for data augmentation processing (Date Augmentation); step 6, the data obtained after data augmentation processing for data pre-processing, and generate training data set (Training dataset), validation data set (Validation dataset) and test data Set (Testing dataset); step 7, input all the above-mentioned data sets (dataset) into the convolutional neural network for a training; the training includes: step 7.1, the plurality of convolutional layers of the convolutional neural network are performed in a sequence order Arrangement of the bottom layer, middle layer, and top layer; step 7.2, all the weights of the convolutional layer are trained for a predetermined number of epochs to obtain the initial weight; step 7.3, the bottom layer is weight frozen; step 7.4, the middle layer is performed for a predetermined period Epoch training, and because of the result of step 7.3, the training ignores the variables of the bottom layer; step 7.5, the middle layer performs weight freezing; step 7.6, the top layer performs training for a predetermined number of epochs, and because of step 7.5 As a result, the training ignores the middle-level variables and classifies the high-level features of the finger/palm vein infrared image; step 8, performs one-to-many identification comparison on the classified high-level features; Among them, the weight freezing of step 7.3 and step 7.5 is performed by changing the learning rate. The finger/palm vein recognition processing method described in claim 6, wherein the weight freezing of step 7.3 and step 7.5 is performed by changing the learning rate, which is performed by each backpropagation, After adjusting the weight coefficient ( W _t ) to calculate the gradient, it will first multiply by a learning rate, and then update the weight to obtain an adjusted weight coefficient ( W _{t +1} ). The method of changing the learning rate (learning rate) is The bottom, middle, and top layers of these convolutional layers have different learning rates. For example, the finger/palm vein recognition processing method of claim 7, wherein the method of changing the learning rate is to make the global learning rate of the bottom, middle, and top layers of the convolutional layers (global learning rate) η is multiplied by a parameter α _l and defined as the independent layer-wise learning rate η _{l of} each convolutional layer. The finger/palm vein identification processing method described in claim 8, wherein the weight freezing method is performed by changing the learning rate, and the weight coefficient before adjustment ( W _t ) and the weight coefficient after adjustment ( W _{t +1} ) update the weight by using formula (1.3);

among them,

Is the first moment estimation of the gradient at time t after the deviation correction, which represents the mean,

Is the second moment estimation of the gradient at time t after the deviation is corrected, which represents the variance; ε is the smooth value; where in formula (1.3), the attenuation of these learning rates remains the same . The finger/palm vein recognition processing method described in claim 9, wherein, if the observation loss (loss) does not continue to decrease during the training process, the learning rate of the next epoch is reduced by formula (1.4)

Among them, the subscript T represents the number of epochs in which the loss (loss) has not continued to decrease.

Images