CN115966029A

CN115966029A - Offline signature authentication method and system based on attention mechanism

Info

Publication number: CN115966029A
Application number: CN202310221689.XA
Authority: CN
Inventors: 廖万里; 陈灵; 林智聪; 陈焯辉
Original assignee: Zhuhai Kingsware Information Technology Co Ltd
Current assignee: Zhuhai Kingsware Information Technology Co Ltd
Priority date: 2023-03-09
Filing date: 2023-03-09
Publication date: 2023-04-14
Anticipated expiration: 2043-03-09
Also published as: CN115966029B

Abstract

The invention discloses an offline signature authentication method and system based on an attention mechanism, wherein the method comprises the following steps: (1) Acquiring a Chinese handwritten signature data set, and dividing the data set into a training set, a verification set and a test set; (2) Constructing an off-line signature authentication network, training the network by using a training set, and judging the fitting condition of the network by using a verification set; (3) And acquiring the signature to be authenticated, finding a real signature in the database, and putting the signature to be authenticated and the real signature into a network for prediction. The invention applies the training mode of the sample pair to the problem of off-line signature authentication, can accurately and efficiently realize the off-line signature authentication task, can reduce the information lost by the convolution layer in the characteristic extraction process as little as possible, strengthens the characteristic extraction effect of the encoder, improves the accuracy of the off-line signature authentication task, and provides guarantee for the safety of industries such as finance, banking, legal business and the like.

Description

Offline signature authentication method and system based on attention mechanism

Technical Field

The invention relates to the field of computer vision and pattern recognition, in particular to an offline signature authentication method and system based on an attention mechanism.

Background

Verifying the authenticity of signatures is a significant challenge as there are a large number of important financial, business and forensic documents signed daily throughout the world today. This presents a serious violation risk to financial, banking, and governmental agencies as benefit-driven events that result in the falsely signed signature occur at times. Meanwhile, as more and more commercial documents are provided, the speed and accuracy of manual identification are difficult to guarantee. Therefore, it is becoming more and more important to develop an automatic, accurate, and efficient signature verification technique.

Although the past decades have witnessed developments and advances in signature verification, signature verification presents the following challenges in the chinese world:

first, the lack of a proper and public chinese handwritten signature dataset hinders the research and application of chinese offline signature verification.

Second, chinese strokes are more complex than other languages, and have higher requirements for feature extraction on the network.

Third, the signature style of most people is relatively random, which makes the same person's signature style significantly different on different occasions, and some intentionally forged signatures closely resemble real signatures.

For the above problems, the conventional offline signature authentication method achieves a good effect at present, but ignores the loss of the convolutional layer to the feature information, which may prevent the network from learning effective track features and cause the performance of the network to be reduced.

Disclosure of Invention

The invention aims to overcome the defects of the prior art and provide an offline signature authentication method based on an attention mechanism.

Another object of the present invention is to provide an offline signature authentication system based on attention mechanism.

The purpose of the invention is realized by the following technical scheme:

an offline signature authentication method based on an attention mechanism comprises the following steps:

(1) Acquiring a Chinese handwritten signature data set, and dividing the data set into a training set, a verification set and a test set;

(2) Constructing an off-line signature authentication network, training the network by using a training set, and judging the fitting condition of the network by using a verification set;

the step (2) comprises the following steps:

(2.1) building an encoder consisting of five convolution layers and four pooling layers;

(2.2) building a decoder consisting of three full-connection layers and three two-dimensional attention machine modules;

(2.3) connecting the first three convolution layers of the encoder and the three full-connection layers of the decoder by using a two-dimensional attention mechanism module, thereby completing the construction of the network;

(2.4) loading a network, and setting network parameters for training;

(2.5) extracting a batch of sample pairs from the training set each time, putting the sample pairs into a network for training, putting the output of a network encoder and a label into a loss function for calculating loss, and updating network parameters through back propagation;

(2.6) putting the verification set of each batch into the network when the last batch of the training set is extracted;

(2.7) repeating the steps (2.5) and (2.6) until the last batch in the training set is trained by the network

Sub, wherein>

Is the number of training sessions of the network;

(3) And acquiring the signature to be authenticated, finding a real signature in the database, and putting the signature to be authenticated and the real signature into a network for prediction.

The step (1) comprises the following steps:

(1.1) carrying out gray processing on the Chinese handwritten signature image;

(1.2) carrying out black-white overturning treatment on the grayed Chinese handwritten signature image;

(1.3) using random elastic deformation for forged signatures;

(1.4) constructing a positive and negative sample pair on the processed image;

and (1.5) splitting all samples into a training set, a verification set and a test set according to a preset proportion.

The step (1.4) comprises:

(1.4.1) for the real signature of the same signer, combining each two different real signature images as a positive sample pair, and assigning a label as 1;

(1.4.2) for the forged signature of the same signer, combining each two different real signature images and forged signature images as a negative sample pair, and assigning a value of a label to be 0;

(1.4.3) randomly selecting the same number of samples as the number of pairs of positive samples in the negative sample pair.

The step (1.4.1) is specifically as follows:

the formula for calculating the number of positive samples is as follows:

；

wherein

For a number of positive sample pairs>

The number of signers;

the step (1.4.2) is specifically as follows:

the formula for calculating the number of negative examples is as follows:

；

wherein

Is a negative sample pair number, is selected>

The number of signers.

The step (2.5) is specifically as follows:

the network uses a binary cross entropy function as a loss function, and the formula is as follows:

；

wherein

Means the sample size of the data set, based on the comparison of the sample size and the sample size>

And &>

Respectively, the true class and the predicted class of the sample pair.

The step (3) comprises the following steps:

(3.1) carrying out gray processing on the signature image to be authenticated;

(3.2) carrying out black and white overturning treatment on the grayed signature image to be authenticated;

(3.3) finding a real signature left by the signer before in the database;

(3.4) forming a sample pair by the signature to be authenticated and the real signature, and putting the sample pair into the network

And (6) line prediction.

The step (3.4) is specifically as follows:

if the output value of the network prediction is more than or equal to 0.5, the signature to be authenticated is a real signature; otherwise, the signature to be authenticated is a fake signature.

The other purpose of the invention is realized by the following technical scheme:

an offline signature authentication system based on an attention mechanism comprises a preprocessing module, a training module and a prediction module; wherein:

the system comprises a preprocessing module, a test module and a data processing module, wherein the preprocessing module is used for acquiring a Chinese handwritten signature data set and dividing the data set into a training set, a verification set and a test set;

the training module is used for building an off-line signature authentication network, training the network by using a training set and judging the fitting condition of the network by using a verification set;

and the prediction module is used for acquiring the signature to be authenticated, finding a real signature in the database, and putting the signature to be authenticated and the real signature into a network for prediction.

A server comprising a processor and a memory, the memory having stored therein at least one program which is loaded and executed by the processor to implement the method of attention-based offline signature authentication.

A computer readable storage medium having stored therein at least one program, which is loaded and executed by a processor to implement the method for attention-based offline signature authentication.

Compared with the prior art, the invention has the following advantages and beneficial effects:

1. the invention applies the training mode of the sample pair to the off-line signature authentication task, uses the signature to be authenticated and the real signature as the sample pair, and puts the sample pair into the network for training, thereby predicting whether the signature to be authenticated is the personal signature.

2. The invention provides an attention mechanism-based offline signature authentication network (att-OfSVNet), which fuses the features extracted by an encoder and a decoder by using a two-dimensional attention mechanism, reduces the information lost by a convolutional layer in the feature extraction process as little as possible, strengthens the effect of encoder feature extraction and improves the accuracy of an offline signature authentication task.

3. The invention provides a set of off-line signature authentication process and sets up an off-line signature authentication system, thereby providing guarantee for business safety of industries such as finance, banking and legal business and the like.

Drawings

FIG. 1 is an example of a Chinese handwritten signature image;

FIG. 2 is an example of random elastic deformation;

FIG. 3 is a network architecture diagram of att-OfSVNet in accordance with the present invention;

FIG. 4 is a two-dimensional attention model module layout.

Detailed Description

The present invention will be described in further detail with reference to examples and drawings, but the embodiments of the present invention are not limited thereto.

(1) A Chinese handwritten signature data set is obtained, and the data set is divided into a training set, a verification set and a test set.

In the embodiment, an OpenCV technology is adopted to process the image, an offline signature authentication network is built by using a PyTorch deep learning framework, and a programming experiment is performed by combining a Python language.

And carrying out gray processing on the Chinese handwritten signature image.

In the embodiment, the OpenCV technology is used for converting the Chinese handwritten signature image from the RGB image into the gray image, and the formula is as follows:

；/>

wherein, gray is the pixel value of the Chinese handwritten signature image after graying, B is the pixel value of a blue channel in the RGB image, G is the pixel value of a green channel in the RGB image, and R is the pixel value of a red channel in the RGB image. The results of graying are shown in fig. 1.

And performing black-white inversion processing on the gray Chinese handwritten signature image.

In this example, the grayed pixel value of the Chinese handwritten signature image is subtracted from 255, and the formula is as follows:

；

wherein, inverse is the pixel value of the Chinese handwritten signature image after black and white inversion. Therefore, the black and white inversion of the Chinese handwritten signature image is realized.

Random elastic deformation is used for forged signatures.

Because the diversity of the forged signature is not enough, the forged signature is subjected to random elastic deformation, so that the diversity of the forged signature is increased, and the performance of the network is improved. This example uses a smaller size of 3 for each signer's forged signature image to perform random elastic deformation, as shown in fig. 2. Elastic deformation is often used for image scenes of non-fixed nature, and is simply a scene that may receive small external factors and the image part area is simply changed, for example, in outdoor flower recognition and the like. The use of elastic deformation is not common in the field of signature authentication, but the small-range elastic deformation adopted by the invention is a scene which is very consistent with off-line signature authentication, because the signatures at different moments of a signature are not identical, and a small part of the signatures may be different, but the fact that the signatures are real signatures is not influenced. The invention adopts elastic deformation to play an important role in changing the situation.

And constructing positive and negative sample pairs for the processed image.

The signature image data used in this example is a CNSig chinese handwritten signature dataset collected by itself, which includes 836 signers. For a true signature, there are 10 true signature images per signer. For a fake signature, there are 10 authentic signature images per signer. The CNSig data set includes 8360 real signature images and 8360 fake signature images in total. For the true signature of the same signer, each two different true signature images are combined as a positive sample pair, the label is assigned 1, and there are 836 × 45=37620 pairs. For the forged signature of the same signer, combining every two different real signature images and forged signature images as a negative sample pair, and assigning the label as 0. Then, in order to avoid the fact that the network learns wrong a priori information, the sample quantities of the positive sample pairs and the negative sample pairs are balanced, 45 pairs of samples with the same quantity as the positive sample pairs are randomly selected from the 10 × 10=100 pairs of negative samples, and 836 × 45=37620 pairs are used as the negative sample pairs.

The training set, validation set and test set were split for all sample pairs according to the ratio of 8.

In this example, for all constructed pairs of samples, a training set, a validation set and a test set are split according to the proportion of 8.

(2) The invention builds an off-line signature authentication network called att-OfSVNet.

The structure of att-OfSVNet is shown in FIG. 3. att-OfSVNet is a new network defined by the invention, takes a true signature image after black and white inversion and a signature image to be authenticated after black and white inversion as input, and comprises two encoders, two decoders and six two-dimensional attention mechanisms. Firstly, a real signature image and a signature image to be authenticated are respectively input into two encoders with shared weights, and two characteristic graphs are obtained. The encoding process is shown in the following formula:

；

；

wherein the Encoder is an Encoder.

Is a pixel value of a real signature image after a black-and-white inversion>

Is the calculation result of the first convolution layer in the encoder when the true signature image after black and white inversion is inputted. />

The feature map of the true signature image after black and white inversion is the calculation result of the last maximum pooling layer in the encoder. />

For a pixel value of a signature image to be authenticated after black and white inversion, based on the comparison result>

When the signature image to be authenticated after black and white inversion is input, the calculation result of the first convolution layer in the encoder is obtained. />

The characteristic diagram of the signature image to be authenticated after black and white inversion is a calculation result of the last maximum pooling layer in the encoder.

Then, the two feature maps are respectively input into two decoders with shared weights, so as to obtain two vectors with the length of 32, and the decoding process is shown as the following formula:

；

；

wherein the Decoder is a Decoder.

The length of the vector is 32, which is output by a decoder after the characteristic diagram of the true signature image after black and white inversion is input into the decoder. />

After the feature map of the signature image to be authenticated after black and white inversion is input into a decoder, the length of the vector output by the decoder is 32.

And finally, splicing the two vectors with the length of 32, taking a full-connection layer with the number of hidden neurons being 1 as an output layer, sending the spliced vectors into the output layer, and obtaining the probability that the signature to be authenticated is a real signature and a fake signature through a Sigmoid activation function, wherein the formula is as follows:

；

；

；

where Concat is the stitching operation of two vectors,

spliced length 64 vectors for two length 32 vectors, based on length x>

Is the weight of the output layer>

For the bias of the output layer>

Is the input of the Sigmoid function.

An encoder is built which consists of five convolutional layers and four pooling layers.

The att-OfSVNet encoder is composed of five convolutional layers and four pooling layers and can play a role in extracting features. The first convolutional layer receives an input size of 128 x 256, black and white inverted signature image, and has 32 convolutional kernels, thereby extending the feature map from a single channel to 32 channels. The second pooling layer takes as input the output of the first convolutional layer, uses a receptive field of size 2 × 2, and has a step size of 2 pixels, and crops the feature map. One convolutional layer and one max pooling layer are used together as a module, which is repeated three times until the third max pooling layer. The number of convolution kernels of the first three convolution layers is respectively 32, 64 and 128, and the convolution kernels are gradually increased. The output of the third pooling layer is used as input to the fourth convolutional layer and the fifth convolutional layer, respectively, and the fourth convolutional layer has 64 convolutional kernels, which reduces the number of channels to 64. While the fifth convolutional layer has 128 convolutional kernels, boosting the number of channels to match the input of the third convolutional layer. The result of merging the fourth convolutional layer with the fifth convolutional layer in the channel dimension, i.e., the number of channels becomes 192. And finally, passing through a maximum pooling layer with the receptive field size of 2 multiplied by 2 and the step length of 2 pixels, wherein the output of the fourth maximum pooling layer is the output of the encoder. All convolutional layers are followed by a BN layer and a ReLU activation function, and the convolutional kernel size in all convolutional layers is 3 × 3, step size is 1 pixel.

The calculation formula of the BN layer is as follows:

；

wherein the content of the first and second substances,

is the output of BN layer>

Is the input of BN layer, is>

Is->

Is based on the mean value of>

Is->

Standard deviation of (4), based on the measured value>

Is residual error, is based on>

、/>

Are parameters.

The calculation formula of the ReLU activation function is as follows:

；

wherein

Is the input of the ReLU function.

And constructing a decoder consisting of three full connection layers and three two-dimensional attention mechanism modules.

The att-OfSVNet decoder is composed of three fully connected layers and three two-dimensional attention modules, and aims to perform dimension reduction on the feature map. The decoder firstly inputs the output of the encoder into a full-connection layer with the neuron number of 128 of a hidden layer, and then the decoder and the convolution result of the third layer in the encoder are put into a two-dimensional attention module for fusion. And inputting the fused result into a full-link layer with 64 hidden layer neurons, and then putting the fused result and the result after the second layer convolution into a two-dimensional attention module for fusion. And finally, inputting the fused result into a full-link layer with the neuron number of the hidden layer being 32, and then putting the fused result and the result after the convolution of the third layer into a two-dimensional attention module for fusion, wherein the fused result is the output of the decoder.

And connecting the first three convolution layers of the encoder and the three full-connection layers of the decoder by using a two-dimensional attention mechanism module, thereby completing the construction of the network.

The structure of the two-dimensional attention mechanism module is shown in fig. 4. The input of the two-dimensional attention mechanism module is a three-dimensional tensor and a one-dimensional vector, the three-dimensional tensor is firstly input into a convolution layer with the convolution kernel size of 3 multiplied by 3 and the filling size of 1, so that the number of channels is halved, and then the height dimension and the width dimension of the image are combined to be reduced into a two-dimensional tensor. Meanwhile, the one-dimensional vector is input into a full connection layer, so that the length is halved, and then the broadcast operation is carried out, so that the one-dimensional vector can be added with the two-dimensional tensor element by element. And sequentially inputting the added tensor into a tanh activation function, a full connection layer and a Softmax activation function to obtain a result of matrix multiplication of the three-dimensional tensor after conversion. And finally, outputting a vector with the same shape as the input one-dimensional vector, thereby completing the two-dimensional attention mechanism.

The two-dimensional attention mechanism fuses the encoder and decoder using the formula given below:

；

；

；

wherein

For merging results of convolutional layers and fully-linked layers>

For the attention matrix, is selected>

For the output of the two-dimensional attention mechanism, flatten flattens the height and width dimensions of the image in the three-dimensional tensor, conv is convolution operation, and/or the length of the image is greater than the maximum value>

For a three-dimensional tensor input, is asserted>

For a one-dimensional tensor input>

Is the first->

Weight of a fully connected layer, < > >>

Is the first->

Biasing of the full connection layers.

the tan h activation function is calculated as follows:

；

wherein

Is the input to the tanh function.

The calculation formula of the Softmax activation function is as follows:

；

wherein

Is input->

In a first or second section>

Number of components->

Is the total number of categories.

The invention realizes that the information of the convolutional layer in the extraction process is lost as little as possible and the effect of extracting the characteristics by the encoder is strengthened through correspondingly fusing the output of the convolutional layer of the first three layers in the encoder and the output of the three full-connection layers in the decoder through the two-dimensional attention mechanism module, thereby improving the performance of the network and more accurately authenticating the off-line handwritten signature.

And loading the network and setting the parameters of the training network.

The example sets up 100 periods of network training, 32 samples per batch. The learning rate is updated using Adam optimizer and cosine annealing decay method, as follows:

；

wherein, the first and the second end of the pipe are connected with each other,

for learning rate, is selected>

For the total number of training sessions, is selected>

To stop the updated learning rate.

The initial learning rate is set to 0.01, the learning rate at which updating is stopped is set to 0.1, and the weight attenuation coefficient is set to 1e-6.

And extracting a batch of sample pairs from the training set each time, putting the sample pairs into att-OfSVNet for training, putting the output of the att-OfSVNet encoder and the label into a loss function for calculating loss, and updating network parameters through back propagation.

The present example uses a binary cross entropy function as the loss function, with the formula:

；

wherein

And &>

Respectively, the true class and the predicted class of the sample pair.

And when the last batch of the training set is extracted, putting the verification set of each batch into att-OfSVNet for verification, thereby judging the fitting condition of att-OfSVNet.

If the loss value of the training set passing through the network is basically equal to the loss value of the verification set passing through the network, the network performance is better; if the loss value of the training set passing through the network is far smaller than that of the verification set passing through the network, the overfitting degree of the network is serious.

Repeating the steps (2.5) and (2.6) until the last batch in the training set is trained by the network

Therein of

Is the number of training sessions of the network.

And carrying out gray processing on the signature image to be authenticated.

In the embodiment, the OpenCV technology is used for converting the to-be-authenticated signature image from the RGB image into the grayscale image, and the formula is as follows:

；

wherein, gray is the pixel value of the Chinese handwritten signature image after graying, B is the pixel value of a blue channel in the RGB image, G is the pixel value of a green channel in the RGB image, and R is the pixel value of a red channel in the RGB image.

And carrying out black-white overturning treatment on the grayed signature image to be authenticated.

In this example, the grayed pixel value of the signature image to be authenticated is subtracted from 255, and the formula is as follows:

；

wherein, inverse is the pixel value of the Chinese handwritten signature image after black and white inversion. Therefore, black and white inversion of the signature image to be authenticated is realized.

A true signature left by the signer before is found in the database.

And randomly selecting a real signature left by the signer before in the database as a reference object of the signature to be authenticated.

And forming a sample pair by the signature to be authenticated and the real signature, and putting the sample pair into the network for prediction.

And constructing a sample pair by using the signature to be authenticated and the real signature as the input of att-OfSVNet. If the output value of the network prediction is more than or equal to 0.5, the signature to be authenticated is a real signature; otherwise, the signature to be authenticated is a fake signature.

the training module is used for constructing an off-line signature authentication network, training the network by using a training set and judging the fitting condition of the network by using a verification set;

The above embodiments are preferred embodiments of the present invention, but the present invention is not limited to the above embodiments, and any other changes, modifications, substitutions, combinations, and simplifications which do not depart from the spirit and principle of the present invention should be construed as equivalents thereof, and all such changes, modifications, substitutions, combinations, and simplifications are intended to be included in the scope of the present invention.

Claims

1. An offline signature authentication method based on an attention mechanism is characterized by comprising the following steps:

the step (2) comprises the following steps:

(2.4) loading a network, and setting network parameters for training;

Wherein

Is the number of training sessions of the network;

2. The attention mechanism-based offline signature authentication method of claim 1, wherein the step (1) comprises:

(1.1) carrying out gray processing on the handwritten Chinese signature image;

(1.3) using random elastic deformation for forged signatures;

(1.4) constructing a positive and negative sample pair on the processed image;

3. The attention mechanism-based offline signature authentication method of claim 2, wherein said step (1.4) comprises:

(1.4.3) randomly selecting the same number of samples as the number of pairs of positive samples in the negative sample pairs.

4. The attention mechanism-based offline signature authentication method according to claim 3, wherein the step (1.4.1) is specifically as follows:

the formula for calculating the number of positive samples is as follows:

；

wherein

Is a positive sample pair number, is asserted>

The number of signers;

the step (1.4.2) is specifically as follows:

the negative examples are calculated as follows:

；

wherein

For a negative sample pair number>

The number of signers.

5. The attention-based offline signature authentication method of claim 1, wherein the step (2.5) is specifically:

；

wherein

And &>

Respectively, the true class and the predicted class of the sample pair.

6. The attention mechanism-based offline signature authentication method of claim 1, wherein the step (3) comprises:

(3.1) carrying out gray processing on the signature image to be authenticated;

(3.3) finding a real signature left by the signer before in the database;

And (6) line prediction.

7. The attention-based offline signature authentication method of claim 6, wherein the step (3.4) is specifically:

8. An offline signature authentication system based on an attention mechanism is characterized by comprising a preprocessing module, a training module and a prediction module; wherein:

9. A server, characterized by: the server comprises a processor and a memory, wherein at least one program is stored in the memory, and the program is loaded by the processor and executed to realize the offline signature authentication method based on the attention mechanism according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: the storage medium has at least one program stored therein, which is loaded and executed by a processor to implement the method for attention-based offline signature authentication according to any one of claims 1-7.