CN111275070A - Signature verification method and device based on local feature matching - Google Patents
Signature verification method and device based on local feature matching Download PDFInfo
- Publication number
- CN111275070A CN111275070A CN201911387311.7A CN201911387311A CN111275070A CN 111275070 A CN111275070 A CN 111275070A CN 201911387311 A CN201911387311 A CN 201911387311A CN 111275070 A CN111275070 A CN 111275070A
- Authority
- CN
- China
- Prior art keywords
- signature
- feature
- verified
- template
- local feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012795 verification Methods 0.000 title claims abstract description 132
- 238000000034 method Methods 0.000 title claims abstract description 59
- 238000000605 extraction Methods 0.000 claims abstract description 100
- 238000012549 training Methods 0.000 claims description 41
- 238000010586 diagram Methods 0.000 description 19
- 235000019580 granularity Nutrition 0.000 description 13
- 239000013598 vector Substances 0.000 description 9
- 238000012545 processing Methods 0.000 description 8
- 238000013527 convolutional neural network Methods 0.000 description 5
- 238000011176 pooling Methods 0.000 description 4
- 238000007781 pre-processing Methods 0.000 description 3
- 238000012935 Averaging Methods 0.000 description 2
- 238000006073 displacement reaction Methods 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Theoretical Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Artificial Intelligence (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Collating Specific Patterns (AREA)
Abstract
The invention relates to a signature verification method based on local feature matching, which comprises the steps of inputting a signature picture to be verified into a feature extraction network, and performing feature extraction on the signature picture to be verified to obtain a signature feature picture to be verified; inputting N template signature pictures of the user into a feature extraction network one by one, and outputting N template signature feature pictures after feature extraction; n is a natural number and is more than or equal to 1; extracting m partial feature graphs of the signature to be verified with the same size from the feature graph of the signature to be verified, and respectively extracting m partial feature graphs of the template signature with the same size from each feature graph of the template signature; respectively performing feature matching on the local feature graph of the signature to be verified and the N template signature local feature graphs to obtain N × m local feature matching graphs; and inputting the N x m local feature matching graphs into a classification network, and obtaining the probability that the signature to be verified is a real signature through classification verification. The invention realizes the sharing of all users of one verification model and has high accuracy of verification results.
Description
Technical Field
The invention relates to a signature verification method and device based on local feature matching, and belongs to the field of signature verification.
Background
The signature verification problem belongs to one of classical pattern matching problems and aims to judge whether a current signature belongs to a specific signer or not. Because signatures of different people have different characteristics and the signatures of the same person have high common similarity, the signatures can be used as personal identity information in many scenes. Signature verification methods also have wide application, such as signature verification of credit cards and contracts, note authentication in judicial programs, and the like.
According to different extraction modes of signature features, signature verification schemes can be divided into 2 types: an online signature verification scheme and an offline signature verification scheme. The online signature verification scheme needs a signature image and process information of signing and writing by a signer, and further extracts sequence characteristics such as stroke order, signature strength change and the like from the signature image and the process information. An offline signature verification scheme then only requires a signature image. Because the online signature needs to record the process of the signature in detail, which cannot be realized in many application scenarios, the offline signature verification scheme is still the currently widely adopted solution, and the invention also belongs to the offline signature verification scheme.
The traditional off-line signature verification method adopts a Scale-invariant feature transform (Scale-invariant feature transform, SIFT) algorithm for extracting local features of an image, a maximum extreme Stable region (MSER) algorithm for performing speckle detection on a binary signature image based on the thought of watershed and the like to extract signature feature points, and then judges whether the signature is true or false through feature point matching, so that the signature has high recognition rate. With the development and wide application of the convolutional neural network, the method of extracting the features of the signature picture by using the deep convolutional network and then training a classification recognizer for each user by using an SVM algorithm at present exceeds the identification accuracy of the traditional SIFT (scale invariant feature transform) and other methods. The method separates feature extraction from classification verification, but the method needs to train a classification recognizer for each user, which causes a large problem, and when the number of users increases, an SVM recognizer needs to be trained for each user. For example, in the application scenario of a bank, there are hundreds of millions of customers, and training hundreds of millions of SVM recognizers for these customers is obviously not feasible.
In the existing off-line verification method, the following method is also adopted: and representing the signature picture to be verified and the template signature picture of the user by using a feature vector with uniform length, and then calculating the similarity between the feature vectors, wherein the high similarity indicates that the signature is the user signature, and the low similarity indicates that the signature is a false signature. However, the signature picture to be verified is usually shifted or scaled from the template signature picture, and cannot be aligned precisely, which may cause the accuracy of the existing method to decrease.
Disclosure of Invention
In order to solve the technical problems, the invention provides a signature verification method based on local feature matching, an end-to-end verification model is formed by a feature extraction network, the local feature matching and a classification network, all users share the model, the model is not influenced by the number of system users, a large amount of user data can be supported, the verification result is high in accuracy, and the signature verification method is simple, effective and strong in practicability.
The first technical scheme of the invention is as follows:
a signature verification method based on local feature matching comprises the following steps: inputting the picture of the signature to be verified into a feature extraction network, and performing feature extraction on the picture of the signature to be verified to obtain a signature feature map to be verified; inputting template signature pictures of a user corresponding to N signatures to be verified into a feature extraction network one by one, and outputting N template signature feature pictures after feature extraction; n is a natural number and is more than or equal to 1; extracting m signature local feature graphs to be verified with the same size from the signature feature graph to be verified, and respectively extracting m template signature local feature graphs with the same size from each template signature feature graph; respectively performing feature matching on the local feature graph of the signature to be verified and the N template signature local feature graphs to obtain N × m local feature matching graphs; and inputting the N x m local feature matching graphs into a classification network, and obtaining the probability that the signature to be verified is a real signature through classification verification.
Preferably, the extracting step of the local feature map specifically comprises: setting a first sliding window with a configurable window size, wherein the first sliding window carries out local feature extraction on the signature feature graph to be verified according to a preset sliding step length and a sliding rule to obtain m signature local feature graphs to be verified, and m is determined by the sliding step length and the sliding rule; setting a second sliding window with the size capable of being configured with the window, and performing local feature extraction on each template signature feature graph by the second sliding window according to the sliding step length and the sliding rule to obtain N x m template signature local feature graphs; when the characteristics are matched, carrying out local characteristic pattern matching on m local characteristic patterns of the signature to be verified and m local characteristic patterns corresponding to any user template signature picture to obtain m local characteristic matching pictures of the signature to be verified and the user template signature picture; and by analogy, when the corresponding local feature graphs of the N user template signature pictures are matched with the local feature graph of the signature to be verified completely, obtaining N m local feature matching graphs.
Preferably, the size of the first sliding window is larger than that of the second sliding window, and when the local feature extraction is performed on the signature feature map to be verified through the first sliding window, the periphery of the signature feature map to be verified is filled to obtain m signature local feature maps to be verified.
Preferably, the signature picture to be verified is input into the feature extraction network, the output of the middle two convolution layers of the feature extraction network is extracted, and a coarse-grained signature feature picture to be verified and a fine-grained signature feature picture to be verified are obtained; inputting N template signature pictures of the user into a feature extraction network one by one, and extracting the output of two convolution layers in the middle of the feature extraction network to obtain a coarse-grained template signature feature picture and a fine-grained template signature feature picture corresponding to each template signature picture; respectively performing local feature extraction and local feature matching on the coarse-grained signature feature map and the template signature feature map to be verified and the fine-grained signature feature map and the template signature feature map to obtain N x m coarse-grained local feature matching maps and N x m fine-grained local feature matching maps; and inputting all coarse-grained local feature matching graphs and fine-grained local feature matching graphs into the classification network, and obtaining the probability that the signature to be verified is a real signature through classification verification.
Preferably, the verification model for executing the signature verification method is trained, the training samples are a plurality of signature pictures to be tested and a plurality of template signature pictures corresponding to each signature picture to be tested, a label is set on each signature picture to be tested, the authenticity of the signature to be tested is identified by the label, and the training step is as follows: training a first step, taking a signature picture to be tested and a corresponding template signature picture as input, then training a verification model by using the signature verification method and the like, taking another signature picture to be tested and the corresponding template signature picture as input, training the verification model, finishing training all the signature pictures to be tested and the corresponding template signature pictures, and judging high-imitation forged signatures by using the verification model; and training II, taking a signature to be detected and all or part of the template signature pictures which are not corresponding to the other signatures as input, then executing the signature verification method, repeating the execution in the same way until all the signature pictures to be detected are trained, and judging the random forged signature by the verification model.
The invention also provides signature verification equipment based on local feature matching.
The second technical scheme of the invention is as follows:
a signature verification device based on local feature matching comprises a processor and a memory, wherein the memory is stored with instructions, the processor executes the instructions to obtain a signature verification model, and the signature verification model executes the following steps: inputting the picture of the signature to be verified into a feature extraction network, and performing feature extraction on the picture of the signature to be verified to obtain a signature feature map to be verified; inputting template signature pictures of a user corresponding to N signatures to be verified into a feature extraction network one by one, and outputting N template signature feature pictures after feature extraction; n is a natural number and is more than or equal to 1; extracting m signature local feature graphs to be verified in the same size from the signature feature graph to be verified, and respectively extracting m template signature local feature graphs in the same size from each template signature feature graph; respectively performing feature matching on the local feature graph of the signature to be verified and the N template signature local feature graphs to obtain N × m local feature matching graphs; and inputting the N x m local feature matching graphs into a classification network, and obtaining the probability that the signature to be verified is a real signature through classification verification.
Preferably, the extracting step of the local feature map specifically comprises: setting a first sliding window with a configurable window size, wherein the first sliding window carries out local feature extraction on the signature feature graph to be verified according to a preset sliding step length and a sliding rule to obtain m signature local feature graphs to be verified, and m is determined by the sliding step length and the sliding rule; setting a second sliding window with the size capable of being configured with the window, and performing local feature extraction on each template signature feature graph by the second sliding window according to the sliding step length and the sliding rule to obtain N x m template signature local feature graphs; when the characteristics are matched, carrying out local characteristic pattern matching on m local characteristic patterns of the signature to be verified and m local characteristic patterns corresponding to any user template signature picture to obtain m local characteristic matching pictures of the signature to be verified and the user template signature picture; and by analogy, when the corresponding local feature graphs of the N user template signature pictures are matched with the local feature graph of the signature to be verified completely, obtaining N m local feature matching graphs.
Preferably, the size of the first sliding window is larger than that of the second sliding window, and when the local feature extraction is performed on the signature feature map to be verified through the first sliding window, the periphery of the signature feature map to be verified is filled to obtain m signature local feature maps to be verified.
Preferably, the signature picture to be verified is input into the feature extraction network, the output of the middle two convolution layers of the feature extraction network is extracted, and a coarse-grained signature feature picture to be verified and a fine-grained signature feature picture to be verified are obtained; inputting N template signature pictures of the user into a feature extraction network one by one, and extracting the output of two convolution layers in the middle of the feature extraction network to obtain a coarse-grained template signature feature picture and a fine-grained template signature feature picture corresponding to each template signature picture; respectively performing local feature extraction and local feature matching on the coarse-grained signature feature map and the template signature feature map to be verified and the fine-grained signature feature map and the template signature feature map to obtain N x m coarse-grained local feature matching maps and N x m fine-grained local feature matching maps; and inputting all coarse-grained local feature matching graphs and fine-grained local feature matching graphs into the classification network, and obtaining the probability that the signature to be verified is a real signature through classification verification.
Preferably, the training step of the signature verification model is as follows: the training samples are a plurality of signature pictures to be tested and a plurality of template signature pictures corresponding to the signature pictures to be tested, marks are arranged on the signature pictures to be tested, and the authenticity of the signatures to be tested is marked through the marks; training a first step of inputting a signature picture to be tested and a corresponding template signature picture, inputting the signature verification model for verification, and repeating the steps, inputting another signature picture to be tested and a corresponding template signature picture, training the verification model, finishing training all the signature pictures to be tested and the corresponding template signature pictures, wherein the verification model can judge high-imitation forged signatures; and training II, inputting a signature to be detected and all or part of the non-corresponding template signature pictures as input, inputting the signature verification model for verification, repeating the steps in the same way until all the signature pictures to be detected are trained, and judging the random forged signature by the verification model.
The invention has the following beneficial effects:
1. the signature verification method and the signature verification equipment based on local feature matching can support large-scale users only by one verification model, have good expandability, effectively reduce the cost of off-line signature verification, and can better adapt to the signature displacement problem and signature pictures to be verified of various sizes by adopting local feature extraction and local feature matching.
2. According to the signature verification method and device based on local feature matching, the signature feature maps are respectively extracted through the two sliding windows with the configurable window sizes, so that the verification model can adapt to the input of pictures with different scales, and the authenticity of the signature pictures with different signature lengths can be accurately verified.
3. According to the signature verification method and device based on local feature matching, the feature pictures of two different convolution layers are extracted for verification, and the accuracy of signature verification is greatly improved.
4. The signature verification method and device based on local feature matching have the advantages that the verification model is convenient to train, random counterfeit signatures and high-imitation counterfeit signatures can be verified, and the capability of the verification model for identifying real signatures is improved.
Drawings
FIG. 1 is a block diagram of a signature verification process of the present invention;
FIG. 2 is a flow chart of signature verification of the present invention;
FIG. 3 is a schematic diagram of local feature extraction and matching according to the present invention.
Detailed Description
The invention is described in detail below with reference to the figures and the specific embodiments.
Example one
Referring to fig. 1 and 2, a signature verification method based on local feature matching includes the following steps: inputting the picture to be verified and signed into a feature extraction network (such as a VGG network, a ResNet network and the like), and performing feature extraction on the picture to be verified and signed to obtain a feature picture of the signature to be verified; inputting template signature pictures of users corresponding to N signatures to be verified into a feature extraction network one by one, and outputting N template signature feature pictures after feature extraction, wherein the template signature pictures of the users corresponding to the N signatures to be verified are different signatures of the same person, for example, the N template signature pictures are multiple signature pictures of the principal of Wangming, and the more the template signature pictures are, the more accurate the verification result is; n is a natural number and is more than or equal to 1; referring to fig. 3, a plurality of partial feature maps of the signature to be verified with the same size are extracted from the feature map of the signature to be verified, that is, the size of the partial feature maps of the signature to be verified is the same, and a plurality of partial feature maps of the template signature with the same size are respectively extracted from each of the feature maps of the template signature; respectively carrying out local feature matching on the local feature graph of the signature to be verified and the N template signature local feature graphs to obtain N local feature matching graphs; and inputting the N local feature matching graphs into a classification network to obtain the probability that the signature to be verified is a real signature.
In this embodiment, before extracting features from a signature picture (including a signature picture to be verified and a template signature picture), some preprocessing may optionally be performed on the signature picture. For example, pre-background pre-processing including binarization, de-noising and de-blurring is optionally performed on the signature picture, and pre-processing including unifying to the same picture height or unifying to a fixed size is optionally performed on the picture size. For example, the uniform fixed size may be as follows: firstly, setting a unified fixed size, secondly, determining the length scaling ratio and the width scaling ratio of the picture, and selecting a small scaling ratio as the scaling ratio of the whole picture; and after zooming, if the dimension of one edge is smaller than a set value, directly filling the picture with a 0 pixel value, and finally obtaining the preset picture dimension. During model training, the pictures of the training or testing must be unified to a fixed size.
The feature extraction network can adopt a traditional method, such as SIFT algorithm, and can also adopt a deep learning network method, such as convolutional neural network. In the present embodiment, a convolutional neural network is used as an extractor of the signature feature map. The convolutional neural network backbone can be a common feature extraction network structure such as a residual error network (Resnet) and VGG. And taking the characteristic diagram or a plurality of intermediate layer characteristic diagrams output by one intermediate layer of the characteristic extraction network as the characteristic representation of the signature. The middle layer is a hierarchy in the feature extraction network except for the first and last layers. The input of the feature extraction network can be a preprocessed signature picture to obtain a corresponding feature map.
Referring to fig. 3, the step of extracting the local feature map specifically includes: setting a first sliding window k with configurable window size1The first sliding window k1Performing local feature extraction on the signature feature map to be verified according to a preset sliding step length and a sliding rule to obtain m (in fig. 3, m is H 'W') signature local feature maps to be verified, wherein m is determined by the sliding step length and the sliding rule; setting a second sliding window k with configurable window size2And the second sliding window performs local feature extraction on each template signature feature map (a total of N template signature feature maps, which are extracted one by one) according to the sliding step length and the sliding rule to obtain N × m template signature local feature maps, wherein in fig. 3 (× N) the N template signature feature maps are shown, and the template signature feature maps are extracted one by one. The sliding rule may be set according to the characteristics of the signature picture, and may be sliding extraction line by line and then sliding extraction line by line, or sliding extraction at alternate lines, or sliding extraction without endpoint, which is not limited herein.
In the local feature matching step in this embodiment, a plurality of local feature maps with fixed sizes are extracted from each signature feature map (including the signature feature map to be verified and the template signature feature map), and feature matching is performed on the local feature maps of the signature to be verified and the N user template signatures respectively. And the local feature matching process comprises the steps of inputting a plurality of local feature graphs of the signature picture to be verified each time and a plurality of local feature graphs corresponding to one user template signature picture, and outputting the local feature graphs as a local feature matching graph. Repeating the same local feature matching operation for N times (using a plurality of local feature maps corresponding to one user template signature image every time), so as to respectively obtain a signature feature map to be verified and a local feature matching map of the N template signature feature maps.
A specific example is given below to illustrate the implementation process of local feature matching:
firstly, local feature extraction: signature characteristic diagram of user template obtained by the characteristic extraction network (the resolution of the signature characteristic diagram of the template is assumed to beWidth w, height h, denoted w x h), a certain number (assumed to be m) of local feature maps with resolution are extracted using a configurable fixed-size second sliding window, where the sliding rule is set to extract row by row, and m is determined by w, h and the sliding step size. Optionally, a template signature local feature map is generated by extracting a feature matrix corresponding to the position of the sliding center point in the second sliding window each time. And (4) repeatedly executing N times of local feature extraction when N template signature feature graphs exist. And extracting m local feature maps with the size of m by using a first sliding window with fixed size for a feature matrix of the picture to be verified, which is obtained by the feature extraction network. Preferably, the size of the first sliding window is generally set to be larger than or equal to the size of the second sliding window. If the size of the first sliding window>And filling the periphery of the signature characteristic diagram to be verified by the size of the second sliding window so as to ensure that m local characteristic diagrams are obtained. Optionally, the local feature map corresponding to the position of the sliding center point in the first sliding window is extracted each time. And extracting the local features of the user template signature feature graph and the signature feature graph to be verified in no sequence. In the local feature matching operation, the sizes of the first sliding window and the second sliding window can be selected according to experience, and it is assumed that the sizes of the sliding windows of the signature feature map to be verified and the signature feature map of the user template are respectively k1And k2Then usually k1≥k2. If the value is a fixed non-configurable value, the scale of the model for inputting the signature picture must be fixed, but the signature length is very different for signatures with different characters, therefore, the embodiment provides a technical scheme for realizing the configurable size of the sliding window, the realization model can input pictures with different scales,
specifically, the method comprises the following steps: a) obtaining the length and width of the corresponding feature map according to the input picture scale; b) setting the scale of the local feature map; c) taking the scale of the feature map obtained in the step a) as the input of the convolutional layer, taking the scale of the local feature map set in the step b) as the output of the convolutional layer, and calculating the size of the sliding window according to the convolutional relation between the input and the output. When the input picture is the signature characteristic picture to be verified, countingCalculating a first sliding window k1When the input picture is the signature characteristic diagram of the user template, a second sliding window k is calculated2The size of (2). For example, the following steps are carried out: a) assuming an input picture scale to obtain the length and width H, W of the feature map; b) setting the number m of the local feature maps as H 'W', wherein fixed values are set; c) the sliding window size k ═ k needs to be obtained1，kw](wherein, k ishIs the length of the sliding window, kwWidth of the sliding window) satisfies the condition that the characteristic diagram with the width of W is pressed according to kwThe sliding window is transversely slid by the step length of 1, and W local features can be obtained; on a characteristic diagram with height H, according to k1The sliding window is longitudinally slid by the step length of 1 to obtain H local features, so that the size calculation formula of the sliding window is as follows
kW＝W1W′+1
k1＝H1H′+1
Typically, H ═ W, and H ═ W', so k is1＝kw＝k。
Secondly, local feature normalization: considering that the feature matrices of different signature pictures may have a larger difference in feature values, optionally, normalization processing is performed on each local feature picture before performing local feature matching.
Finally, local feature matching: and matching the local feature maps (m) of the normalized signature picture to be verified and the local feature maps (m) of any user template signature picture to obtain local feature matching maps (m in total) of the signature picture to be verified and the user template signature picture. Optionally, a local feature matching graph is generated, which may be obtained by convolving the local feature graphs corresponding to the m user template signatures with the local feature graph of the signature to be verified. And after all matching is finished, obtaining N x m local feature matching graphs. The matching local feature map may also be generated by other methods, for example, calculating a mean square error between the local feature map of the signature picture to be verified and the local feature map of the signature picture of the user template, and the like.
The input of the classification network is a local feature matching graph of the to-be-verified signature picture and all template signature pictures, and the output of the classification network is the authenticity probability of the to-be-verified signature picture (which can be a true/false binary result, can also be a probability that a real number between 0 and 1 represents that the to-be-verified signature picture is a true signature, and other various representation methods). The present embodiment will describe the processing of the classification verification by using the probability that a real number between 0 and 1 is output to represent that the picture to be tested is a real signature. The classification network may employ a 2-classifier.
In the classification verification process, the embodiment adopts a classification network to process the local feature matching image of each group of signatures to be verified and each template signature, and outputs the probability of matching the signature image to be verified and the corresponding template signature image; and then, processing the probability of matching the obtained signature to be verified and the N template signatures to obtain the probability of representing the authenticity of the signature picture to be verified. Other methods may be used for the classification verification process.
The authentication process of the classified network is illustrated first: the classification network is obtained by a downsampling operation and a full connection layer:
firstly, using down-sampling to process N x m local feature matching graphs to obtain N x m local feature matching vectors. The down-sampling can use methods such as maximum pooling (Max pooling), Average pooling (Average pooling), depth direction (Depthwise) convolution and common convolution; secondly, processing N x m matched local feature vectors by using a full-connection layer, outputting N real numbers, and representing the authenticity probability between the signature picture to be verified and each template signature picture; and finally, fusing by adopting methods such as voting or averaging and the like to obtain the probability that the signature to be verified is the real signature.
Compared with the existing machine learning method, the signature verification method based on local feature matching provided by the invention has the remarkable advantages that: 1. only one model is needed (the number of the models is irrelevant to the number of users needing to be identified), the training number of the models is reduced, the needed storage space and the management cost are very low, large-scale users can be supported, and the expandability is very good; 2. the local special graph is extracted for matching, so that the problem of signature displacement can be better adapted, the training is convenient, the accuracy rate is high, the practicability is high, and the cost of off-line signature verification is effectively reduced; 3. compared with the existing method, the method can further improve the accuracy of off-line signature verification.
Example two
On the basis of the first embodiment, the embodiment provides that two feature maps with different granularities are adopted for verification, and the verification accuracy is further improved.
Inputting the signature picture to be verified into a feature extraction network, extracting the output of two convolution layers in the middle of the feature extraction network, and obtaining a coarse-grained signature feature picture to be verified and a fine-grained signature feature picture to be verified; the features under different scales are extracted through a convolutional neural network, the larger the scale is, the finer the granularity of the features is, the smaller the scale is, and the coarser the granularity of the features is. For example, in the residual network Resnet, the characteristic diagram output by the third convolutional layer is thinner than the characteristic diagram output by the fourth convolutional layer. By utilizing the multi-size features, more accurate feature information can be extracted, and signature features can be better expressed. For example, feature maps of 2 middle layers (outputs of the third convolutional layer and the fourth convolutional layer) of the feature extraction network are optionally used as feature expressions of the signature. But may be other combinations of features of the convolutional layer output.
And the feature extraction network inputs one signature picture to be verified each time, extracts the output of the middle two convolution layers after operation to obtain a coarse-grained feature map and a fine-grained feature map of the signature picture, wherein the number of convolution layers is relatively small, the output is the fine-grained feature map, and the number of convolution layers is relatively large, so that the coarse-grained feature map is output. And respectively processing the signature picture to be verified and the N template signature pictures of the known user by using a feature extraction network to obtain a coarse-grained feature map and a fine-grained feature map corresponding to the signature picture to be verified. And inputting the N template signature pictures of the user into a feature extraction network one by one, and extracting the output of two convolution layers in the middle of the feature extraction network to obtain a coarse-grained feature map and a fine-grained feature map corresponding to each template signature picture.
When the local features are matched, extracting a plurality of local feature graphs with fixed sizes from the output feature graphs with coarse granularity and fine granularity, and respectively performing feature matching on the local feature graphs of the signature to be verified and the N user template signatures. The local feature matching process comprises the steps of processing a coarse-grained (or fine-grained) feature graph of a signature picture to be verified each time and a coarse-grained (or fine-grained) feature graph of a user template signature, and outputting the feature graph as a local feature matching graph (measurement similarity) between the feature graph of the signature picture and the feature graph of the user template signature. The local feature matching operations on the coarse-grained and fine-grained feature maps are the same, and the local feature matching operations do not distinguish coarse-grained and fine-grained local feature matching.
Specifically, when a sliding window is used for local feature extraction, local feature extraction is respectively carried out on feature maps with fine granularity and coarse granularity, namely, m pieces of signature local feature maps to be verified with fine granularity are correspondingly extracted from the feature map with fine granularity to be verified, and m pieces of signature local feature maps to be verified with coarse granularity are correspondingly extracted from the feature map with coarse granularity to be verified; and executing the same extraction operation on the coarse-grained characteristic diagram and the fine-grained characteristic diagram of each template signature, wherein m coarse-grained template signature local characteristic diagrams are extracted from the coarse-grained characteristic diagram of each template signature, and m fine-grained template signature local characteristic diagrams are extracted from the fine-grained characteristic diagram of each template signature. Since the spatial scale of the fine-grained feature map is large and the spatial scale of the coarse-grained feature map is small, empirically, k for a sliding window on the fine-grained feature map is generally set1、k2K greater than sliding window used on coarse-grained feature maps1、k2。
Optionally, before performing local feature matching, the fine-grained and coarse-grained local feature maps of the signature to be verified and the fine-grained and coarse-grained local feature maps of the template signature are normalized.
Finally, local feature matching: and matching the coarse-grained local feature maps (m) of the normalized signature picture to be verified and the coarse-grained local feature maps (m) corresponding to any user template signature picture to obtain coarse-grained local feature matching maps (m in total) of the signature picture to be verified and the user template signature picture, and similarly processing the fine-grained local feature maps to obtain the coarse-grained local feature matching maps (m in total) of the signature picture to be verified and the user template signature picture. And after all matching is finished, obtaining N x m coarse-granularity local feature matching graphs and N x m fine-granularity local feature matching graphs.
When verification is carried out through a classification network, 2N x m local feature matching graphs are processed by using downsampling to obtain 2N x m local feature matching vectors; then cascading the coarse-grained local feature matching vectors and the corresponding fine-grained local feature matching vectors together (concat operation) to obtain corresponding local feature matching vectors (N × m in total); secondly, processing N x m matched local feature vectors by using a full-connection layer, outputting N real numbers, and representing the authenticity probability between the signature picture to be verified and each template signature picture; and finally, fusing by adopting methods such as voting or averaging and the like to obtain the probability that the signature to be verified is the real signature.
In the second embodiment, two feature maps with different granularities are extracted, and on the basis, local feature extraction and local feature matching are respectively performed, and finally, classification verification is performed, so that the verification accuracy can be further improved.
EXAMPLE III
In training the signature verification models of the first and second embodiments, a hybrid template training method is adopted to enable the signature verification model to judge random forgery and high-imitation forgery, wherein the random forgery uses the signature of another person to impersonate a specific signer (for example, impersonate "liquad" with the signature of "zhang san" and the high-imitation forgery uses the handwriting of the specific signer to impersonate (for example, impersonate "liquad" with the signature of high imitation to impersonate the real "liquad").
The training samples are a plurality of signature pictures to be tested and a plurality of template signature pictures corresponding to the signature pictures to be tested, marks are arranged on the signature pictures to be tested, and the authenticity of the signatures to be tested is marked through the marks; training a first step of inputting a signature picture to be tested and a corresponding template signature picture, inputting the signature verification model for verification, and repeating the steps, wherein the other signature picture to be tested and the corresponding template signature picture are used as input, the verification model is trained, all the signature pictures to be tested and the corresponding template signature pictures are trained, and the verification model can judge high-imitation forged signatures; and training II, inputting a signature to be detected and all or part of the non-corresponding template signature pictures as input, inputting the signature verification model for verification, repeating the steps by analogy, and judging the random forged signature by the verification model until all the signature pictures to be detected are trained.
Example four
Referring to fig. 1 and fig. 2, a signature verification apparatus based on local feature matching includes a processor and a memory, where the memory stores instructions, and the processor executes the instructions to obtain a signature verification model, where the signature verification model performs the following steps: inputting the picture of the signature to be verified into a feature extraction network, and performing feature extraction on the picture of the signature to be verified to obtain a signature feature map to be verified; inputting the template signature pictures of the user corresponding to the N signatures to be verified into a feature extraction network one by one, and outputting N template signature feature pictures after feature extraction; n is a natural number and is more than or equal to 1; extracting m partial feature graphs of the signature to be verified with the same size from the feature graph of the signature to be verified, and respectively extracting m partial feature graphs of the template signature with the same size from each feature graph of the template signature; respectively performing feature matching on the local feature graph of the signature to be verified and the N template signature local feature graphs to obtain N × m local feature matching graphs; and inputting the N x m local feature matching graphs into a classification network, and obtaining the probability that the signature to be verified is a real signature through classification verification.
The extraction steps of the local feature map are as follows: setting a first sliding window with a configurable window size, and performing local feature extraction on the signature feature graph to be verified by the first sliding window according to a preset sliding step length and a sliding rule to obtain m signature local feature graphs to be verified, wherein m is determined by the sliding step length and the sliding rule; setting a second sliding window with the size capable of being configured with the window, and performing local feature extraction on each template signature feature graph by the second sliding window according to the sliding step length and the sliding rule to obtain N × m template signature local feature graphs; when the characteristics are matched, performing local characteristic pattern matching on m local characteristic patterns of the signature to be verified and m local characteristic patterns corresponding to any user template signature picture to obtain m local characteristic matching pictures of the signature to be verified and the user template signature picture; and by analogy, when the corresponding local feature graphs of the N user template signature pictures are matched with the local feature graph of the signature to be verified, obtaining N m local feature matching graphs.
Preferably, the size of the first sliding window is larger than that of the second sliding window, and when the local feature extraction is performed on the signature feature map to be verified through the first sliding window, the periphery of the signature feature map to be verified is filled to obtain m signature local feature maps to be verified.
In order to further improve the verification accuracy, two feature maps with different granularities can be adopted for verification, which is specifically as follows: inputting the signature picture to be verified into a feature extraction network, extracting the output of the middle two convolution layers of the feature extraction network, and obtaining a coarse-grained signature feature picture to be verified and a fine-grained signature feature picture to be verified; inputting N template signature pictures of the user into a feature extraction network one by one, extracting the output of two convolution layers in the middle of the feature extraction network, and obtaining a coarse-grained template signature feature picture and a fine-grained template signature feature picture corresponding to each template signature picture; respectively performing local feature extraction and local feature matching on the coarse-grained signature feature map and the template signature feature map to be verified and the fine-grained signature feature map and the template signature feature map to obtain N x m coarse-grained local feature matching maps and N x m fine-grained local feature matching maps; and inputting all coarse-grained local feature matching graphs and fine-grained local feature matching graphs into the classification network, and obtaining the probability that the signature to be verified is a real signature through classification verification.
The training steps of the signature verification model are as follows: the training samples are a plurality of signature pictures to be tested and a plurality of template signature pictures corresponding to the signature pictures to be tested, marks are arranged on the signature pictures to be tested, and the authenticity of the signatures to be tested is marked through the marks; training a first step of inputting a signature picture to be tested and a corresponding template signature picture, inputting the signature verification model for verification, and repeating the steps, inputting another signature picture to be tested and a corresponding template signature picture, training the verification model, finishing training all signature pictures to be tested and the corresponding template signature pictures, and judging high-imitation forged signatures by the verification model; and training II, inputting a signature to be detected and all or part of the non-corresponding template signature pictures, inputting the signature verification model for verification, repeating the steps by analogy, and judging the random forged signature by the verification model until all the signature pictures to be detected are trained.
The fourth embodiment of the present invention refers to the related description of the first to third embodiments of the method.
The above description is only an embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. A signature verification method based on local feature matching is characterized by comprising the following steps:
inputting the picture of the signature to be verified into a feature extraction network, and performing feature extraction on the picture of the signature to be verified to obtain a signature feature map to be verified; inputting template signature pictures of a user corresponding to N signatures to be verified into a feature extraction network one by one, and outputting N template signature feature pictures after feature extraction; n is a natural number and is more than or equal to 1;
extracting m partial feature graphs of the signature to be verified with the same size from the feature graph of the signature to be verified, and respectively extracting m partial feature graphs of the template signature with the same size from each feature graph of the template signature;
respectively performing feature matching on the local feature graph of the signature to be verified and the N template signature local feature graphs to obtain N × m local feature matching graphs;
and inputting the N x m local feature matching graphs into a classification network, and obtaining the probability that the signature to be verified is a real signature through classification verification.
2. The signature verification method based on local feature matching according to claim 1, wherein the extracting step of the local feature map specifically comprises:
setting a first sliding window with a configurable window size, wherein the first sliding window carries out local feature extraction on the signature feature graph to be verified according to a preset sliding step length and a sliding rule to obtain m signature local feature graphs to be verified, and m is determined by the sliding step length and the sliding rule;
setting a second sliding window with the size capable of being configured with the window, and performing local feature extraction on each template signature feature graph by the second sliding window according to the sliding step length and the sliding rule to obtain N x m template signature local feature graphs;
when the characteristics are matched, performing local characteristic pattern matching on m local characteristic patterns of the signature to be verified and m local characteristic patterns corresponding to any user template signature picture to obtain m local characteristic matching pictures of the signature to be verified and the user template signature picture; and by analogy, when the corresponding local feature graphs of the N user template signature pictures are matched with the local feature graph of the signature to be verified, obtaining N m local feature matching graphs.
3. The signature verification method based on local feature matching according to claim 1, wherein: and the size of the first sliding window is larger than that of the second sliding window, and when local feature extraction is carried out on the signature feature graph to be verified through the first sliding window, the periphery of the signature feature graph to be verified is filled to obtain m signature local feature graphs to be verified.
4. The signature verification method based on local feature matching according to claim 1, wherein: inputting the signature picture to be verified into a feature extraction network, extracting the output of two convolution layers in the middle of the feature extraction network, and obtaining a coarse-grained signature feature picture to be verified and a fine-grained signature feature picture to be verified; inputting N template signature pictures of the user into a feature extraction network one by one, extracting the output of two convolution layers in the middle of the feature extraction network, and obtaining a coarse-grained template signature feature picture and a fine-grained template signature feature picture corresponding to each template signature picture; respectively performing local feature extraction and local feature matching on the coarse-grained signature feature map and the template signature feature map to be verified and the fine-grained signature feature map and the template signature feature map to obtain N × m coarse-grained local feature matching maps and N × m fine-grained local feature matching maps; and inputting all coarse-grained local feature matching graphs and fine-grained local feature matching graphs into the classification network, and obtaining the probability that the signature to be verified is a real signature through classification verification.
5. The signature verification method based on local feature matching according to claim 1, wherein: training a verification model for executing the signature verification method according to any one of claims 1 to 4, wherein the training samples are a plurality of signature pictures to be tested and a plurality of template signature pictures corresponding to the signature pictures to be tested, a label is set on each signature picture to be tested, and the authenticity of the signature to be tested is identified by the label, and the training step comprises:
training a first step, taking a signature picture to be tested and a corresponding template signature picture as input, then executing the signature verification method of any one of claims 1 to 4, and so on, taking another signature picture to be tested and a corresponding template signature picture as input, training a verification model, finishing training all the signature pictures to be tested and the corresponding template signature pictures, and judging high-imitation forged signatures by the verification model;
training II, taking a signature to be detected and other non-corresponding whole or partial template signature pictures as input, then executing the signature verification method of any one of claims 1 to 4, repeating the steps in the same way until all the signature pictures to be detected are trained, and judging the random forged signature by the verification model.
6. A signature verification device based on local feature matching is characterized by comprising a processor and a memory, wherein the memory is stored with instructions, the processor executes the instructions to obtain a signature verification model, and the signature verification model executes the following steps: inputting the picture of the signature to be verified into a feature extraction network, and performing feature extraction on the picture of the signature to be verified to obtain a signature feature map to be verified; inputting template signature pictures of a user corresponding to N signatures to be verified into a feature extraction network one by one, and outputting N template signature feature pictures after feature extraction; n is a natural number and is more than or equal to 1;
extracting m partial feature graphs of the signature to be verified with the same size from the feature graph of the signature to be verified, and respectively extracting m partial feature graphs of the template signature with the same size from each feature graph of the template signature;
respectively performing feature matching on the local feature graph of the signature to be verified and the N template signature local feature graphs to obtain N × m local feature matching graphs;
and inputting the N x m local feature matching graphs into a classification network, and obtaining the probability that the signature to be verified is a real signature through classification verification.
7. The signature verification device based on local feature matching as claimed in claim 6, wherein the step of extracting the local feature map specifically comprises:
setting a first sliding window with a configurable window size, wherein the first sliding window carries out local feature extraction on the signature feature graph to be verified according to a preset sliding step length and a sliding rule to obtain m signature local feature graphs to be verified, and m is determined by the sliding step length and the sliding rule;
setting a second sliding window with the size capable of being configured with the window, and performing local feature extraction on each template signature feature graph by the second sliding window according to the sliding step length and the sliding rule to obtain N x m template signature local feature graphs;
when the characteristics are matched, performing local characteristic pattern matching on m local characteristic patterns of the signature to be verified and m local characteristic patterns corresponding to any user template signature picture to obtain m local characteristic matching pictures of the signature to be verified and the user template signature picture; and by analogy, when the corresponding local feature graphs of the N user template signature pictures are matched with the local feature graph of the signature to be verified, obtaining N m local feature matching graphs.
8. The signature verification device based on local feature matching as claimed in claim 6, wherein the size of the first sliding window is larger than that of the second sliding window, and when the local feature extraction is performed on the signature feature map to be verified through the first sliding window, the surroundings of the signature feature map to be verified are filled to obtain m signature local feature maps to be verified.
9. The signature verification device based on local feature matching as claimed in claim 6, wherein the signature picture to be verified is input into a feature extraction network, and the output of two convolution layers in the middle of the feature extraction network is extracted to obtain a coarse-grained signature feature picture to be verified and a fine-grained signature feature picture to be verified; inputting N template signature pictures of the user into a feature extraction network one by one, extracting the output of two convolution layers in the middle of the feature extraction network, and obtaining a coarse-grained template signature feature picture and a fine-grained template signature feature picture corresponding to each template signature picture; respectively performing local feature extraction and local feature matching on the coarse-grained signature feature map and the template signature feature map to be verified and the fine-grained signature feature map and the template signature feature map to obtain N × m coarse-grained local feature matching maps and N × m fine-grained local feature matching maps; and inputting all coarse-grained local feature matching graphs and fine-grained local feature matching graphs into the classification network, and obtaining the probability that the signature to be verified is a real signature through classification verification.
10. The signature verification device based on local feature matching as claimed in claim 6, wherein the training step of the signature verification model is: the training samples are a plurality of signature pictures to be tested and a plurality of template signature pictures corresponding to the signature pictures to be tested, marks are arranged on the signature pictures to be tested, and the authenticity of the signatures to be tested is marked through the marks;
training a first step of inputting a signature picture to be tested and a corresponding template signature picture, inputting the signature verification model for verification, and repeating the steps, wherein the other signature picture to be tested and the corresponding template signature picture are used as input, the verification model is trained, all the signature pictures to be tested and the corresponding template signature pictures are trained, and the verification model can judge high-imitation forged signatures;
and training II, inputting a signature to be detected and all or part of the non-corresponding template signature pictures as input, inputting the signature verification model for verification, repeating the steps in the same way until all the signature pictures to be detected are trained, and judging the random forged signature by the verification model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911387311.7A CN111275070B (en) | 2019-12-26 | 2019-12-26 | Signature verification method and device based on local feature matching |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911387311.7A CN111275070B (en) | 2019-12-26 | 2019-12-26 | Signature verification method and device based on local feature matching |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111275070A true CN111275070A (en) | 2020-06-12 |
| CN111275070B CN111275070B (en) | 2023-11-14 |
Family
ID=71000178
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911387311.7A Active CN111275070B (en) | 2019-12-26 | 2019-12-26 | Signature verification method and device based on local feature matching |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111275070B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112001298A (en) * | 2020-08-20 | 2020-11-27 | 佳都新太科技股份有限公司 | Pedestrian detection method, device, electronic equipment and storage medium |
| CN112784829A (en) * | 2021-01-21 | 2021-05-11 | 北京百度网讯科技有限公司 | Bill information extraction method and device, electronic equipment and storage medium |
| CN113408492A (en) * | 2021-07-23 | 2021-09-17 | 四川大学 | Pedestrian re-identification method based on global-local feature dynamic alignment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7542593B1 (en) * | 2008-04-30 | 2009-06-02 | International Business Machines Corporaiton | Offline signature verification using high pressure regions |
| CN106778586A (en) * | 2016-12-08 | 2017-05-31 | 武汉理工大学 | Offline handwriting signature verification method and system |
| CN108416295A (en) * | 2018-03-08 | 2018-08-17 | 天津师范大学 | A kind of recognition methods again of the pedestrian based on locally embedding depth characteristic |
-
2019
- 2019-12-26 CN CN201911387311.7A patent/CN111275070B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7542593B1 (en) * | 2008-04-30 | 2009-06-02 | International Business Machines Corporaiton | Offline signature verification using high pressure regions |
| CN106778586A (en) * | 2016-12-08 | 2017-05-31 | 武汉理工大学 | Offline handwriting signature verification method and system |
| CN108416295A (en) * | 2018-03-08 | 2018-08-17 | 天津师范大学 | A kind of recognition methods again of the pedestrian based on locally embedding depth characteristic |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112001298A (en) * | 2020-08-20 | 2020-11-27 | 佳都新太科技股份有限公司 | Pedestrian detection method, device, electronic equipment and storage medium |
| CN112784829A (en) * | 2021-01-21 | 2021-05-11 | 北京百度网讯科技有限公司 | Bill information extraction method and device, electronic equipment and storage medium |
| CN112784829B (en) * | 2021-01-21 | 2024-05-21 | 北京百度网讯科技有限公司 | Bill information extraction method and device, electronic equipment and storage medium |
| CN113408492A (en) * | 2021-07-23 | 2021-09-17 | 四川大学 | Pedestrian re-identification method based on global-local feature dynamic alignment |
| CN113408492B (en) * | 2021-07-23 | 2022-06-14 | 四川大学 | Pedestrian re-identification method based on global-local feature dynamic alignment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111275070B (en) | 2023-11-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Wu et al. | Busternet: Detecting copy-move image forgery with source/target localization | |
| Walia et al. | Fusion of handcrafted and deep features for forgery detection in digital images | |
| CN108345827B (en) | Method, system and neural network for identifying document direction | |
| CN111275070B (en) | Signature verification method and device based on local feature matching | |
| CN108154133B (en) | Face portrait-photo recognition method based on asymmetric joint learning | |
| CN111709313B (en) | Pedestrian re-identification method based on local and channel combination characteristics | |
| Yousry et al. | Currency Recognition System for Blind people using ORB Algorithm. | |
| Ramu et al. | Image forgery detection for high resolution images using SIFT and RANSAC algorithm | |
| Pham et al. | Banknote recognition based on optimization of discriminative regions by genetic algorithm with one-dimensional visible-light line sensor | |
| Sharma et al. | Deep convolutional neural network with ResNet-50 learning algorithm for copy-move forgery detection | |
| Sabeena et al. | Convolutional block attention based network for copy-move image forgery detection | |
| Ubul et al. | Off-line Uyghur signature recognition based on modified grid information features | |
| CN112070116B (en) | Automatic artistic drawing classification system and method based on support vector machine | |
| Singh et al. | Performance analysis of ELA-CNN model for image forgery detection | |
| Khuspe et al. | Robust image forgery localization and recognition in copy-move using bag of features and SVM | |
| Pushpalatha et al. | Offline signature verification based on contourlet transform and textural features using HMM | |
| CN113505716B (en) | Training method of vein recognition model, and recognition method and device of vein image | |
| Hamadene et al. | Off-line handwritten signature verification using contourlet transform and co-occurrence matrix | |
| Pawar | A survey on signature verification approaches | |
| Rajesh et al. | ICA and neural networks for Kannada signature identification | |
| CN112801950A (en) | Image adaptation quality evaluation method based on geometric distortion measurement | |
| Xie et al. | An optimal orientation certainty level approach for fingerprint quality estimation | |
| Desai et al. | Signature Verification and Forgery Recognition System Using KNN, Backpropagation and CNN | |
| Singh et al. | Computer vision based currency classification system | |
| Faheem et al. | DL-CMFD: Deep Learning-Based Copy-Move Forgery Detection Using Parallel Feature-Extractor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |