CN111275070B - Signature verification method and device based on local feature matching - Google Patents
Signature verification method and device based on local feature matching Download PDFInfo
- Publication number
- CN111275070B CN111275070B CN201911387311.7A CN201911387311A CN111275070B CN 111275070 B CN111275070 B CN 111275070B CN 201911387311 A CN201911387311 A CN 201911387311A CN 111275070 B CN111275070 B CN 111275070B
- Authority
- CN
- China
- Prior art keywords
- signature
- verified
- feature
- local feature
- template
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012795 verification Methods 0.000 title claims abstract description 128
- 238000000034 method Methods 0.000 title claims abstract description 56
- 238000000605 extraction Methods 0.000 claims abstract description 88
- KLPWJLBORRMFGK-UHFFFAOYSA-N Molindone Chemical compound O=C1C=2C(CC)=C(C)NC=2CCC1CN1CCOCC1 KLPWJLBORRMFGK-UHFFFAOYSA-N 0.000 claims abstract description 9
- 229940028394 moban Drugs 0.000 claims abstract description 9
- 238000012549 training Methods 0.000 claims description 40
- 238000010586 diagram Methods 0.000 claims description 22
- 235000019580 granularity Nutrition 0.000 description 34
- 239000013598 vector Substances 0.000 description 9
- 238000013527 convolutional neural network Methods 0.000 description 5
- 238000011176 pooling Methods 0.000 description 4
- 238000012545 processing Methods 0.000 description 3
- 238000006073 displacement reaction Methods 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Theoretical Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Artificial Intelligence (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Collating Specific Patterns (AREA)
Abstract
The invention relates to a signature verification method based on local feature matching, which comprises the steps of inputting a signature picture to be verified into a feature extraction network, and extracting features of the signature picture to be verified to obtain a signature feature picture to be verified; inputting the N template signature pictures of the user into a feature extraction network one by one, and outputting N template signature feature pictures after feature extraction; n is a natural number and is more than or equal to 1; extracting m to-be-verified signature local feature images with the same size from the to-be-verified signature feature images, and respectively extracting m template signature local feature images with the same size from each template signature feature image; respectively carrying out feature matching on the signature local feature map to be verified and the N Zhang Moban signature local feature map to obtain N x m local feature matching maps; inputting the N.m local feature matching graphs into a classification network, and obtaining the probability that the signature to be verified is a true signature through classification verification. The invention realizes sharing of all users of one verification model and has high verification result accuracy.
Description
Technical Field
The invention relates to a signature verification method and device based on local feature matching, and belongs to the field of signature verification.
Background
The signature verification problem belongs to one of classical pattern matching problems, and aims to judge whether a current signature belongs to a specific signer, a basic strategy is to extract characteristic information of the signature, compare the characteristic information with characteristic information of the specific signer in a template library, and judge that the current signature belongs to the signer when the two characteristics are similar. Because signatures of different persons have different characteristics, and signatures of the same person have high general similarity, in many scenes, we can use the signatures as personal identity information. Signature verification methods are also widely used, such as signature verification of credit cards and contracts, note identification in judicial procedures, and the like.
Signature verification schemes can be classified into 2 classes according to different extraction modes of signature features: an online signature verification scheme and an offline signature verification scheme. The online signature verification scheme requires signing images and process information of signing notes by a signer, and further extracts characteristics such as stroke order, signature force change and the like. The offline signature verification scheme then only needs to sign the image. Since online signature requires a process of recording the signature in detail, which cannot be realized in many application scenarios, the offline signature verification scheme is still a currently commonly adopted solution, and the present invention also belongs to an offline signature verification scheme.
The traditional offline signature verification method adopts a Scale-invariant feature transform (Scale-invariant feature transform, SIFT, extracting local features of an image, the features keep invariance to rotation, scale scaling and brightness change) algorithm, a maximum extremum stable region (Maximally Stable Extrernal Regions, MSER, and spot detection of a binarized signature picture based on the thought of watershed) algorithm and the like to extract signature feature points, and then judges the authenticity of the signature through feature point matching, so that the method has higher recognition rate. With development and wide application of convolutional neural networks, the method of extracting features of signature pictures by using a deep convolutional network and training a classification identifier for each user by using an SVM algorithm at present has exceeded the identification accuracy of the traditional SIFT-based method. Such methods separate feature extraction from classification verification, but they require training a classification identifier for each user, which can lead to a significant problem, as the number of users increases, requiring training an SVM identifier for each user. For example, in a banking application scenario, there are hundreds of millions of their customers for whom training hundreds of millions of SVM identifiers is clearly not feasible.
In the existing offline verification method, the following method is also adopted: and representing the signature picture to be verified and the template signature picture of the user by using feature vectors with uniform length, and then calculating the similarity between the feature vectors, wherein a high similarity indicates that the signature is a user signature, and a low similarity indicates that the signature is a false signature. However, the signature picture to be verified is usually displaced or zoomed with the template signature picture, and cannot be aligned accurately, so that the accuracy of the existing method is reduced.
Disclosure of Invention
In order to solve the technical problems, the invention provides a signature verification method based on local feature matching, which forms an end-to-end verification model through a feature extraction network, local feature matching and a classification network, and all users share the model without being influenced by the number of users of the system, so that a large amount of user data can be supported, and the verification result has high accuracy, simplicity, effectiveness and strong practicability.
The first technical scheme of the invention is as follows:
a signature verification method based on local feature matching comprises the following steps: inputting the picture of the signature to be verified into a feature extraction network, and extracting the features of the picture of the signature to be verified to obtain a feature diagram of the signature to be verified; inputting template signature pictures of users corresponding to N signatures to be verified into a feature extraction network one by one, and outputting N template signature feature pictures after feature extraction; n is a natural number and is more than or equal to 1; extracting m to-be-verified signature local feature images with the same size from the to-be-verified signature feature images, and respectively extracting m template signature local feature images with the same size from each template signature feature image; respectively carrying out feature matching on the signature local feature map to be verified and the N Zhang Moban signature local feature map to obtain N x m local feature matching maps; inputting the N.m local feature matching graphs into a classification network, and obtaining the probability that the signature to be verified is a true signature through classification verification.
More preferably, the step of extracting the local feature map specifically includes: setting a first sliding window with a configurable window size, and carrying out local feature extraction on the signature feature map to be verified according to a preset sliding step length and a sliding rule by the first sliding window to obtain m signature local feature maps to be verified, wherein m is determined by the sliding step length and the sliding rule; setting a second sliding window with a configurable window size, and carrying out local feature extraction on each template signature feature map by the second sliding window according to the sliding step length and the sliding rule to obtain N x m template signature local feature maps; when the feature matching is carried out, carrying out the feature matching on m to-be-verified signature local feature graphs and m local feature graphs corresponding to any one user template signature picture to obtain m to-be-verified signature pictures and m local feature matching graphs of the user template signature picture; and similarly, when the corresponding local feature images of the N user template signature images are matched with the to-be-verified signature local feature images, N x m local feature matching images are obtained.
More preferably, the size of the first sliding window is larger than the size of the second sliding window, and when the local feature extraction is performed on the signature feature map to be verified through the first sliding window, the periphery of the signature feature map to be verified is filled so as to obtain m signature local feature maps to be verified.
More preferably, inputting the signature picture to be verified into a feature extraction network, and extracting the output of two middle convolution layers of the feature extraction network to obtain a coarse-granularity signature feature picture to be verified and a fine-granularity signature feature picture to be verified; inputting the N template signature pictures of the user into a feature extraction network one by one, extracting the output of two middle convolution layers of the feature extraction network, and obtaining a coarse-granularity template signature feature picture and a fine-granularity template signature feature picture corresponding to each template signature picture; respectively executing local feature extraction and local feature matching on the signature feature map to be verified with coarse granularity and the template signature feature map to be verified with fine granularity to obtain N.m coarse granularity local feature matching maps and N.m fine granularity local feature matching maps; and inputting all the coarse-granularity local feature matching graphs and the fine-granularity local feature matching graphs into the classification network, and obtaining the probability that the signature to be verified is a true signature through classification verification.
More preferably, a verification model for executing a signature verification method is trained, a training sample is a plurality of signature pictures to be tested and a plurality of template signature pictures corresponding to the signature pictures to be tested, labels are set on the signature pictures to be tested, and the authenticity of the signature to be tested is identified through the labels, and the training steps are as follows: training one, taking a to-be-tested signature picture and a corresponding template signature picture as inputs, then taking the other to-be-tested signature picture and the corresponding template signature picture as inputs by the signature verification method and the like, training a verification model, and after all to-be-tested signature pictures and the corresponding template signature pictures are trained, judging a high-imitation counterfeit signature by the verification model; training two, taking all or part of template signature pictures which are not corresponding to the signature to be detected and other signatures to be detected as input, then executing the signature verification method, and repeating the method until all the signature pictures to be tested are trained, wherein the verification model can judge random fake signatures.
The invention also provides signature verification equipment based on local feature matching.
The technical scheme II of the invention is as follows:
the signature verification device based on local feature matching comprises a processor and a memory, wherein the memory is stored with instructions, the processor executes the instructions to obtain a signature verification model, and the signature verification model performs the following steps: inputting the picture of the signature to be verified into a feature extraction network, and extracting the features of the picture of the signature to be verified to obtain a feature diagram of the signature to be verified; inputting template signature pictures of users corresponding to N signatures to be verified into a feature extraction network one by one, and outputting N template signature feature pictures after feature extraction; n is a natural number and is more than or equal to 1; extracting m to-be-verified signature local feature images with the same size from the to-be-verified signature feature images, and respectively extracting m template signature local feature images with the same size from each template signature feature image; respectively carrying out feature matching on the signature local feature map to be verified and the N Zhang Moban signature local feature map to obtain N x m local feature matching maps; inputting the N.m local feature matching graphs into a classification network, and obtaining the probability that the signature to be verified is a true signature through classification verification.
More preferably, the step of extracting the local feature map specifically includes: setting a first sliding window with a configurable window size, and carrying out local feature extraction on the signature feature map to be verified according to a preset sliding step length and a sliding rule by the first sliding window to obtain m signature local feature maps to be verified, wherein m is determined by the sliding step length and the sliding rule; setting a second sliding window with a configurable window size, and carrying out local feature extraction on each template signature feature map by the second sliding window according to the sliding step length and the sliding rule to obtain N x m template signature local feature maps; when the feature matching is carried out, carrying out the feature matching on m to-be-verified signature local feature graphs and m local feature graphs corresponding to any one user template signature picture to obtain m to-be-verified signature pictures and m local feature matching graphs of the user template signature picture; and similarly, when the corresponding local feature images of the N user template signature images are matched with the to-be-verified signature local feature images, N x m local feature matching images are obtained.
More preferably, the size of the first sliding window is larger than the size of the second sliding window, and when the local feature extraction is performed on the signature feature map to be verified through the first sliding window, the periphery of the signature feature map to be verified is filled so as to obtain m signature local feature maps to be verified.
More preferably, inputting the signature picture to be verified into a feature extraction network, and extracting the output of two middle convolution layers of the feature extraction network to obtain a coarse-granularity signature feature picture to be verified and a fine-granularity signature feature picture to be verified; inputting the N template signature pictures of the user into a feature extraction network one by one, extracting the output of two middle convolution layers of the feature extraction network, and obtaining a coarse-granularity template signature feature picture and a fine-granularity template signature feature picture corresponding to each template signature picture; respectively executing local feature extraction and local feature matching on the signature feature map to be verified with coarse granularity and the template signature feature map to be verified with fine granularity to obtain N.m coarse granularity local feature matching maps and N.m fine granularity local feature matching maps; and inputting all the coarse-granularity local feature matching graphs and the fine-granularity local feature matching graphs into the classification network, and obtaining the probability that the signature to be verified is a true signature through classification verification.
More preferably, the training steps of the signature verification model are as follows: the training sample is a plurality of signature pictures to be tested and a plurality of template signature pictures corresponding to the signature pictures to be tested, labels are arranged on the signature pictures to be tested, and the authenticity of the signature to be tested is identified through the labels; training a first, taking a to-be-tested signature picture and a corresponding template signature picture as inputs, inputting the signature verification model for verification, and so on, taking another to-be-tested signature picture and a corresponding template signature picture as inputs, training a verification model, wherein all to-be-tested signature pictures and corresponding template signature pictures are trained, and the verification model can judge high-imitation counterfeit signatures; training II, taking all or part of template signature pictures which are not corresponding to the signature to be detected and other signatures to be detected as input, inputting the template signature pictures into the signature verification model for verification, and repeating the steps until all the signature pictures to be tested are trained, wherein the verification model can judge random fake signatures.
The invention has the following beneficial effects:
1. the signature verification method and the signature verification device based on the local feature matching can support large-scale users only by one verification model, are good in expandability, effectively reduce the cost of offline signature verification, and can better adapt to signature displacement problems and signature pictures to be verified in various sizes by adopting the local feature extraction and the local feature matching.
2. According to the signature verification method and device based on local feature matching, signature feature graphs are respectively extracted through the sliding windows with the size of the two configurable windows, so that a verification model can adapt to the input of pictures with different scales, and the authenticity of signature pictures with different signature lengths can be accurately verified.
3. According to the signature verification method and device based on local feature matching, the feature pictures of two different convolution layers are extracted for verification, so that the accuracy of signature verification is greatly improved.
4. According to the signature verification method and device based on local feature matching, the verification model is convenient to train, random fake signatures and high-imitation fake signatures can be verified, and the capability of the verification model for identifying real signatures is improved.
Drawings
FIG. 1 is a block diagram of a signature verification process of the present invention;
FIG. 2 is a flow chart of signature verification of the present invention;
fig. 3 is a schematic diagram of local feature extraction and matching according to the present invention.
Detailed Description
The invention will now be described in detail with reference to the drawings and to specific embodiments.
Example 1
Referring to fig. 1 and 2, a signature verification method based on local feature matching includes the following steps: inputting the picture of the signature to be verified into a feature extraction network (such as VGG network, resNet network and the like), and extracting the features of the picture of the signature to be verified to obtain a signature feature diagram to be verified; inputting template signature pictures of users corresponding to N signatures to be verified into a feature extraction network one by one, and outputting N template signature feature graphs after feature extraction, wherein the template signature pictures of the users corresponding to the N signatures to be verified are different signatures of the same person, for example, the N template signature pictures are multiple signature pictures of the king, the more the template signature pictures are, the more accurate the verification result is; n is a natural number and is more than or equal to 1; referring to fig. 3, extracting a plurality of signature local feature images with the same size from the signature feature image to be verified, that is, extracting a plurality of template signature local feature images with the same size from each template signature feature image, wherein the sizes of the local feature images of the signature to be verified are the same; respectively carrying out local feature matching on the signature local feature map to be verified and the N Zhang Moban signature local feature map to obtain N local feature matching maps; and inputting the N partial feature matching graphs into a classification network to obtain the probability that the signature to be verified is a true signature.
In this embodiment, before extracting features from the signature pictures (including the signature picture to be verified and the template signature picture), some preprocessing is optionally performed on the signature pictures. For example, the signature pictures may optionally be pre-processed in the foreground, including binarization, denoising, and deblurring, and the picture size may optionally be pre-processed, including unifying to the same picture height, or unifying to a fixed size. For example, the uniform fixed size may be in the following manner: firstly, setting a unified fixed size, secondly, determining the length scaling ratio and the width scaling ratio of the picture, and selecting the small scaling ratio as the scaling ratio of the whole picture; after scaling, if the dimension of one side is smaller than the set value, the picture is directly filled with 0 pixel value, and finally the preset picture dimension is obtained. During model training, the training or test pictures must be unified to a fixed size.
The feature extraction network may employ a conventional method, such as SIFT algorithm, or may employ a deep learning network method, such as convolutional neural network. In the present embodiment, a convolutional neural network is used as an extractor of the signature feature map. The convolutional neural network backbone can be a common feature extraction network structure such as a residual network (Resnet) and a VGG. And taking the characteristic diagram output by one middle layer of the characteristic extraction network or a plurality of middle layer characteristic diagrams as characteristic representation of the signature. The middle tier is a hierarchy in the feature extraction network other than the first and last tiers. The input of the feature extraction network can be a preprocessed signature picture, so as to obtain a corresponding feature map.
Referring to fig. 3, the local feature map extracting step specifically includes: setting a first sliding window k with a configurable window size 1 The first sliding window k 1 Extracting local features of the signature feature map to be verified according to a preset sliding step length and a preset sliding rule to obtain m (m=h 'W' in fig. 3) signature local feature maps to be verified, wherein m is determined by the sliding step length and the sliding rule; setting a second sliding window k with a configurable window size 2 And the second sliding window extracts local features of the template signature feature graphs (totally N template signature feature graphs are extracted one by one) according to the sliding step length and the sliding rule to obtain N.m template signature local feature graphs, wherein in fig. 3 (x N) the N template signature feature graphs are extracted one by one. The sliding rule can be set according to the characteristics of the signature picture, can be that the signature picture is extracted by sliding line by line and then by column, can also be extracted by sliding line by line and by column,endpoint slip extraction may also be removed, not limited herein.
In the step of matching local features in this embodiment, a plurality of local feature maps with fixed sizes are extracted from each signature feature map (including signature feature maps to be verified and template signature feature maps), and feature matching is performed on the signature to be verified and the local feature maps of the N user template signatures respectively. And the local feature matching process is used for outputting a local feature matching graph by a plurality of local feature graphs of the signature picture to be verified and a plurality of local feature graphs corresponding to the user template signature picture which are input each time. The same local feature matching operation is repeated for N times (a plurality of local feature graphs corresponding to one user template signature picture are used at a time), so that the local feature matching graphs of the signature feature graph to be verified and the N Zhang Moban signature feature graph can be obtained respectively.
A specific example is given below to illustrate the implementation of local feature matching:
first, local feature extraction: for a user template signature feature map obtained through a feature extraction network (assuming that the resolution of the template signature feature map is w, and the height h is denoted as w×h), a certain number (assumed to be m) of local feature maps with resolution are extracted by using a second sliding window with a configurable fixed size, wherein the sliding rule is set to be extracted row by row, and m is determined by w, h and a sliding step length. Optionally, a template signature local feature map is generated each time a feature matrix corresponding to the sliding center point position in the second sliding window is extracted. There are N template signature feature maps, and N times of local feature extraction are repeatedly performed. And extracting m partial feature graphs with the same size from the feature matrix of the picture to be verified, which is acquired through the feature extraction network, by using a first sliding window with the fixed size. Preferably, the size of the first sliding window is generally set to be equal to or larger than the size of the second sliding window. If the size of the first sliding window>And the second sliding window is required to be filled around the signature feature map to be verified so as to ensure that m local feature maps are obtained. Optionally, a local feature map corresponding to the sliding center point position in the first sliding window is extracted each time. And the local feature extraction operation of the user template signature feature map and the signature feature map to be verified is not sequenced. At the position of In the local feature matching operation, the sizes of the first sliding window and the second sliding window can be selected according to experience, and the sliding window sizes of the signature feature map to be verified and the signature feature map of the user template are assumed to be k respectively 1 And k 2 Then generally k 1 ≥k 2 . If the value is fixed and non-configurable, the scale of the signature picture input by the model is fixed, but the signature length is very different for signatures of different characters, so the embodiment provides a technical scheme for realizing the configurable size of the sliding window, realizes that the model can input pictures of different scales,
specifically: a) Obtaining the length and width of the corresponding feature map according to the input picture scale; b) Setting the scale of the local feature map; c) Taking the scale of the feature map obtained in the step a) as the input of the convolution layer, taking the scale of the local feature map set in the step b) as the output of the convolution layer, and calculating the size of the sliding window according to the convolution relation between the input and the output. When the input picture is the signature feature diagram to be verified, a first sliding window k is calculated 1 When the input picture is the signature feature picture of the user template, a second sliding window k is calculated 2 Is of a size of (a) and (b). Illustrating: a) Assuming the dimension of an input picture, obtaining the length and width H and W of a feature map; b) Setting the number m=H 'W' of the local feature map, wherein H 'and W' are set to fixed values; c) The sliding window size k= [ k ] needs to be obtained h ，k w ](wherein k h For the length of the sliding window, k w Width of sliding window) is satisfied on a characteristic map of width W, by k w The sliding window transversely slides with the step length of 1, so that W' local features can be obtained; on a feature map with height H, the height is expressed as k h The sliding window longitudinally slides with the step length of 1 to obtain H' local features, so that the sliding window size calculation formula is as follows
k w ＝W-W′+1
k h ＝H-H′+1
Typically, h=w, and H '=w', so k h ＝k w ＝k。
Secondly, local feature normalization: considering that the feature matrixes of different signature pictures may have larger differences of feature values, optionally, normalization processing is performed on each local feature graph before the local feature matching is performed subsequently.
Finally, local feature matching: and carrying out local feature map matching on the local feature map (m) of the normalized signature picture to be verified and the local feature map (m) of any user template signature picture to obtain local feature matching maps (m in total) of the signature picture to be verified and the user template signature picture. Alternatively, a local feature matching diagram is generated, which can be obtained by convolving the local feature diagrams corresponding to the m user template signatures with the local feature diagrams of the signature to be verified. And after all matching is completed, obtaining N.m local feature matching graphs. The matching local feature map may also be generated by other methods, for example, calculating the mean square error of the local feature map of the signature picture to be verified and the local feature map of the user template signature picture, etc.
The input of the classification network is a local feature matching graph of the to-be-verified signature picture and all template signature pictures respectively, and the output of the classification network is the authenticity probability (the probability can be a true/false binary result or a probability that a real number between 0 and 1 indicates that the to-be-verified signature picture is a true signature, and other various expression methods) of the to-be-verified signature picture. The present embodiment will explain the process of classification verification with the probability that one real number between 0 and 1 is output to represent the picture to be tested as a true signature. The classification network may employ a 2-classifier.
In the classifying verification process, the embodiment adopts a classifying network to process each group of signatures to be verified and the local feature matching graph of each template signature, and outputs the matching probability of the signature pictures to be verified and the corresponding template signature pictures; and then, the probability that the obtained signature to be verified is matched with the N Zhang Moban signature is processed, and the probability for representing the authenticity of the picture of the signature to be verified is obtained. Other methods may also be employed for the classification verification process.
The authentication process of the classification network is first illustrated: the classification network is obtained by a downsampling operation and a fully connected layer:
Firstly, using downsampling to process N.m local feature matching graphs to obtain N.m local feature matching vectors. The downsampling may use methods such as maximum pooling (Max pooling), average pooling (Average pooling), depth direction (Depthwise) convolution, and general convolution; secondly, using a full connection layer to process N.m matched local feature vectors, outputting the N real numbers to represent the authenticity probability between the signature picture to be verified and each template signature picture; finally, the fusion can be carried out by adopting methods such as voting or average value taking and the like, so that the probability that the signature to be verified is the true signature is obtained finally.
Compared with the existing machine learning method, the signature verification method based on local feature matching has the remarkable advantages that: 1. only one model is needed (the number of the models is irrelevant to the number of users to be identified), the training number of the models is reduced, the needed storage space and management cost are low, large-scale users can be supported, and the expandability is very good; 2. the local special graphs are extracted for matching, so that the problem of signature displacement can be better adapted, the training is convenient, the accuracy is high, the practicability is high, and the cost of offline signature verification is effectively reduced; 3. compared with the prior art, the method can further improve the accuracy of offline signature verification.
Example two
Based on the first embodiment, the embodiment provides that two feature maps with different granularities are adopted for verification, so that verification accuracy is further improved.
Inputting the signature picture to be verified into a feature extraction network, extracting the output of two middle convolution layers of the feature extraction network, and obtaining a coarse-granularity signature feature picture to be verified and a fine-granularity signature feature picture to be verified; features under different scales are extracted through a convolutional neural network, the larger the scale is, the finer the granularity of the features is, the smaller the scale is, and the coarser the granularity of the features is. For example, in the residual network Resnet, the characteristic diagram output by the third convolution layer is finer than the characteristic diagram output by the fourth convolution layer. By utilizing the multi-size features, more accurate feature information can be extracted, and signature features can be better expressed. For example, a feature map of 2 intermediate layers (outputs of the third convolution layer and the fourth convolution layer) of the feature extraction network is optionally used as the signature feature expression. But may be a combination of features of the output of other convolutional layers.
And the feature extraction network is used for inputting a signature picture to be verified each time, extracting the output of the middle two convolution layers after operation to obtain a coarse-granularity feature map and a fine-granularity feature map of the signature picture, outputting the fine-granularity feature map with relatively small convolution layers, outputting the coarse-granularity feature map with relatively large convolution layers. And processing the signature picture to be verified and N template signature pictures of the known user by using a feature extraction network respectively to obtain a coarse-granularity feature map and a fine-granularity feature map corresponding to the signature picture to be verified. And inputting the N template signature pictures of the user into a feature extraction network one by one, and extracting the output of two middle convolution layers of the feature extraction network to obtain a coarse-granularity feature picture and a fine-granularity feature picture corresponding to each template signature picture.
And extracting a plurality of local feature graphs with fixed sizes from the output coarse-granularity and fine-granularity feature graphs when the local features are matched, and respectively performing feature matching on the signature to be verified and the local feature graphs of the N user template signatures. The local feature matching process is to output a local feature matching graph (measure similarity) between the feature graph of the picture with the signature and the feature graph of the user template signature, wherein the feature graph of the rough granularity (or the fine granularity) of the picture to be verified and the feature graph of the rough granularity (or the fine granularity) of the user template signature are obtained each time. The operation of local feature matching on coarse-granularity and fine-granularity feature maps is the same, and the local feature matching operation does not distinguish between coarse-granularity and fine-granularity local feature matching.
Specifically, when the sliding window is used for carrying out local feature extraction, local feature extraction is respectively carried out on the feature images with fine granularity and coarse granularity, namely m pieces of signature local feature images to be verified with fine granularity are correspondingly extracted on the feature images with fine granularity to be verified, and m pieces of signature local feature images to be verified with coarse granularity are correspondingly extracted on the feature images with coarse granularity to be verified; the same extraction operation is carried out on the coarse-granularity characteristic diagram and the fine-granularity characteristic diagram of each template signature, so that the coarse-granularity characteristic diagram of each template signature is correspondingly extracted m template signature local feature maps with coarse granularity, wherein the fine granularity feature map of the template signature corresponds to m template signature local feature maps with fine granularity. Since the spatial scale of the fine-grained feature map is relatively large and the spatial scale of the coarse-grained feature map is relatively small, empirically, k for a sliding window on the fine-grained feature map is typically set 1 、k 2 Greater than k for sliding window on coarse-grained feature map 1 、k 2 。
Optionally, before local feature matching is performed, normalizing the to-be-verified signature local feature graphs with fine granularity and coarse granularity and the template signature local feature graphs with fine granularity and coarse granularity.
Finally, local feature matching: and carrying out local feature map matching on the coarse-granularity local feature map (m) of the normalized signature picture to be verified and the coarse-granularity local feature map (m) corresponding to any user template signature picture to obtain coarse-granularity local feature matching maps (m total) of the signature picture to be verified and the user template signature picture, and similarly, carrying out the same processing on the fine-granularity local feature map to obtain coarse-granularity local feature matching maps (m total) of the signature picture to be verified and the user template signature picture. And after all matching is finished, obtaining N.m coarse-granularity local feature matching graphs and N.m fine-granularity local feature matching graphs.
When verification is carried out through a classification network, 2 N.m local feature matching graphs are processed through downsampling, and 2 N.m local feature matching vectors are obtained; cascading the coarse-granularity local feature matching vector and the corresponding fine-granularity local feature matching vector (concat operation) together to obtain corresponding local feature matching vectors (n×m total); secondly, using a full connection layer to process N.m matched local feature vectors, outputting the N real numbers to represent the authenticity probability between the signature picture to be verified and each template signature picture; finally, the fusion can be carried out by adopting methods such as voting or average value taking and the like, so that the probability that the signature to be verified is the true signature is obtained finally.
In the second embodiment, two feature graphs with different granularities are extracted, local feature extraction and local feature matching are respectively performed on the basis, and finally classification verification is performed, so that the verification accuracy can be further improved.
Example III
In training the signature verification models of the first and second embodiments, the hybrid template training is used to enable the signature verification model to determine random forgery, which is the use of the signature of another person to impersonate a particular signer (e.g., impersonate "litz" with the signature of "Zhang Sano") and highly imitative forgery, which is the impersonation by impersonating the handwriting of a particular signer (e.g., impersonate "Lifour" with highly imitative "Lifour").
The training sample is a plurality of signature pictures to be tested and a plurality of template signature pictures corresponding to the signature pictures to be tested, labels are arranged on the signature pictures to be tested, and the authenticity of the signature to be tested is identified through the labels; training a first, taking a to-be-tested signature picture and a corresponding template signature picture as inputs, inputting the signature verification model for verification, and so on, taking another to-be-tested signature picture and a corresponding template signature picture as inputs, training a verification model, wherein all to-be-tested signature pictures and corresponding template signature pictures are trained, and the verification model can judge high-imitation counterfeit signatures; training II, taking all or part of template signature pictures which are not corresponding to the signature to be detected and other signatures to be detected as input, inputting the template signature pictures into the signature verification model for verification, and repeating the steps until all the signature pictures to be tested are trained, wherein the verification model can judge random fake signatures.
Example IV
Referring to fig. 1 and 2, a signature verification device based on local feature matching includes a processor and a memory, where the memory stores instructions, and the processor executes the instructions to obtain a signature verification model, where the signature verification model performs the following steps: inputting the picture of the signature to be verified into a feature extraction network, and extracting the features of the picture of the signature to be verified to obtain a feature diagram of the signature to be verified; inputting template signature pictures of users corresponding to N signatures to be verified into a feature extraction network one by one, and outputting N template signature feature pictures after feature extraction; n is a natural number and is more than or equal to 1; extracting m to-be-verified signature local feature images with the same size from the to-be-verified signature feature images, and respectively extracting m template signature local feature images with the same size from each template signature feature image; respectively carrying out feature matching on the signature local feature map to be verified and the N Zhang Moban signature local feature map to obtain N x m local feature matching maps; inputting the N.m local feature matching graphs into a classification network, and obtaining the probability that the signature to be verified is a true signature through classification verification.
The local feature map extracting step specifically comprises the following steps: setting a first sliding window with a configurable window size, and carrying out local feature extraction on the signature feature map to be verified according to a preset sliding step length and a sliding rule by the first sliding window to obtain m signature local feature maps to be verified, wherein m is determined by the sliding step length and the sliding rule; setting a second sliding window with a configurable window size, and carrying out local feature extraction on each template signature feature map by the second sliding window according to the sliding step length and the sliding rule to obtain N x m template signature local feature maps; when the feature matching is carried out, carrying out the feature matching on m to-be-verified signature local feature graphs and m local feature graphs corresponding to any one user template signature picture to obtain m to-be-verified signature pictures and m local feature matching graphs of the user template signature picture; and similarly, when the corresponding local feature images of the N user template signature images are matched with the to-be-verified signature local feature images, N x m local feature matching images are obtained.
Preferably, the size of the first sliding window is larger than the size of the second sliding window, and when the local feature extraction is performed on the signature feature map to be verified through the first sliding window, the periphery of the signature feature map to be verified is filled so as to obtain m signature local feature maps to be verified.
In order to further improve the verification accuracy, two feature graphs with different granularities can be adopted for verification, and the feature graphs are specifically as follows: inputting the signature picture to be verified into a feature extraction network, extracting the output of two middle convolution layers of the feature extraction network, and obtaining a coarse-granularity signature feature picture to be verified and a fine-granularity signature feature picture to be verified; inputting the N template signature pictures of the user into a feature extraction network one by one, extracting the output of two middle convolution layers of the feature extraction network, and obtaining a coarse-granularity template signature feature picture and a fine-granularity template signature feature picture corresponding to each template signature picture; respectively executing local feature extraction and local feature matching on the signature feature map to be verified with coarse granularity and the template signature feature map to be verified with fine granularity to obtain N.m coarse granularity local feature matching maps and N.m fine granularity local feature matching maps; and inputting all the coarse-granularity local feature matching graphs and the fine-granularity local feature matching graphs into the classification network, and obtaining the probability that the signature to be verified is a true signature through classification verification.
The training steps of the signature verification model are as follows: the training sample is a plurality of signature pictures to be tested and a plurality of template signature pictures corresponding to the signature pictures to be tested, labels are arranged on the signature pictures to be tested, and the authenticity of the signature to be tested is identified through the labels; training a first, taking a to-be-tested signature picture and a corresponding template signature picture as inputs, inputting the signature verification model for verification, and so on, taking another to-be-tested signature picture and a corresponding template signature picture as inputs, training a verification model, wherein all to-be-tested signature pictures and corresponding template signature pictures are trained, and the verification model can judge high-imitation counterfeit signatures; training II, taking all or part of template signature pictures which are not corresponding to the signature to be detected and other signatures to be detected as input, inputting the template signature pictures into the signature verification model for verification, and repeating the steps until all the signature pictures to be tested are trained, wherein the verification model can judge random fake signatures.
For specific steps in this embodiment, see the relevant descriptions of embodiments one through three.
The foregoing description is only illustrative of the present invention and is not intended to limit the scope of the invention, and all equivalent structures or equivalent processes or direct or indirect application in other related technical fields are included in the scope of the present invention.
Claims (8)
1. The signature verification method based on local feature matching is characterized by comprising the following steps:
inputting the picture of the signature to be verified into a feature extraction network, and extracting the features of the picture of the signature to be verified to obtain a feature diagram of the signature to be verified; inputting template signature pictures of users corresponding to N signatures to be verified into a feature extraction network one by one, and outputting N template signature feature pictures after feature extraction; n is a natural number and is more than or equal to 1;
extracting m to-be-verified signature local feature images with the same size from the to-be-verified signature feature images, and respectively extracting m template signature local feature images with the same size from each template signature feature image;
respectively carrying out feature matching on the signature local feature map to be verified and the N Zhang Moban signature local feature map to obtain N x m local feature matching maps;
Inputting the N x m partial feature matching graphs into a classification network, and obtaining the probability that the signature to be verified is a true signature through classification verification;
the method is characterized in that the local feature map extracting step specifically comprises the following steps:
setting a first sliding window with a configurable window size, and carrying out local feature extraction on the signature feature map to be verified according to a preset sliding step length and a sliding rule by the first sliding window to obtain m signature local feature maps to be verified, wherein m is determined by the sliding step length and the sliding rule;
setting a second sliding window with a configurable window size, and carrying out local feature extraction on each template signature feature map by the second sliding window according to the sliding step length and the sliding rule to obtain N x m template signature local feature maps;
when the feature matching is carried out, carrying out the feature matching on m to-be-verified signature local feature graphs and m local feature graphs corresponding to any one user template signature picture to obtain m to-be-verified signature pictures and m local feature matching graphs of the user template signature picture; and similarly, when the corresponding local feature images of the N user template signature images are matched with the to-be-verified signature local feature images, N x m local feature matching images are obtained.
2. The signature verification method based on local feature matching as claimed in claim 1, wherein: and when the size of the first sliding window is larger than that of the second sliding window, and the signature feature diagram to be verified is subjected to local feature extraction through the first sliding window, filling the periphery of the signature feature diagram to be verified so as to obtain m signature local feature diagrams to be verified.
3. The signature verification method based on local feature matching as claimed in claim 1, wherein: inputting the signature picture to be verified into a feature extraction network, extracting the output of two middle convolution layers of the feature extraction network, and obtaining a coarse-granularity signature feature picture to be verified and a fine-granularity signature feature picture to be verified; inputting the N template signature pictures of the user into a feature extraction network one by one, extracting the output of two middle convolution layers of the feature extraction network, and obtaining a coarse-granularity template signature feature picture and a fine-granularity template signature feature picture corresponding to each template signature picture; respectively executing local feature extraction and local feature matching on the signature feature map to be verified with coarse granularity and the template signature feature map to be verified with fine granularity to obtain N.m coarse granularity local feature matching maps and N.m fine granularity local feature matching maps; and inputting all the coarse-granularity local feature matching graphs and the fine-granularity local feature matching graphs into the classification network, and obtaining the probability that the signature to be verified is a true signature through classification verification.
4. The signature verification method based on local feature matching as claimed in claim 1, wherein: training a verification model for executing the signature verification method according to any one of claims 1 to 3, wherein the training sample is a plurality of signature pictures to be tested and a plurality of template signature pictures corresponding to the signature pictures to be tested, marks are arranged on the signature pictures to be tested, and the authenticity of the signature to be tested is identified through the marks, and the training steps are as follows:
training one, taking a to-be-tested signature picture and a corresponding template signature picture as inputs, then executing the signature verification method according to any one of claims 1 to 3, and the like, taking another to-be-tested signature picture and a corresponding template signature picture as inputs, training a verification model, wherein all to-be-tested signature pictures and corresponding template signature pictures are trained, and the verification model can judge highly imitated counterfeit signatures;
training two, taking all or part of template signature pictures which are not corresponding to a signature to be detected and other signatures to be detected as input, then executing the signature verification method according to any one of claims 1 to 3, and repeating the steps until all the signature pictures to be tested are trained, wherein the verification model can judge random fake signatures.
5. A signature verification device based on local feature matching, comprising a processor and a memory, wherein the memory has instructions stored thereon, the processor executes the instructions to obtain a signature verification model, and the signature verification model performs the steps of: inputting the picture of the signature to be verified into a feature extraction network, and extracting the features of the picture of the signature to be verified to obtain a feature diagram of the signature to be verified; inputting template signature pictures of users corresponding to N signatures to be verified into a feature extraction network one by one, and outputting N template signature feature pictures after feature extraction; n is a natural number and is more than or equal to 1;
extracting m to-be-verified signature local feature images with the same size from the to-be-verified signature feature images, and respectively extracting m template signature local feature images with the same size from each template signature feature image;
respectively carrying out feature matching on the signature local feature map to be verified and the N Zhang Moban signature local feature map to obtain N x m local feature matching maps;
inputting the N x m partial feature matching graphs into a classification network, and obtaining the probability that the signature to be verified is a true signature through classification verification;
the local feature map extracting step specifically comprises the following steps:
Setting a first sliding window with a configurable window size, and carrying out local feature extraction on the signature feature map to be verified according to a preset sliding step length and a sliding rule by the first sliding window to obtain m signature local feature maps to be verified, wherein m is determined by the sliding step length and the sliding rule;
setting a second sliding window with a configurable window size, and carrying out local feature extraction on each template signature feature map by the second sliding window according to the sliding step length and the sliding rule to obtain N x m template signature local feature maps;
when the feature matching is carried out, carrying out the feature matching on m to-be-verified signature local feature graphs and m local feature graphs corresponding to any one user template signature picture to obtain m to-be-verified signature pictures and m local feature matching graphs of the user template signature picture; and similarly, when the corresponding local feature images of the N user template signature images are matched with the to-be-verified signature local feature images, N x m local feature matching images are obtained.
6. The signature verification device based on local feature matching according to claim 5, wherein the size of the first sliding window is larger than the size of the second sliding window, and when the local feature extraction is performed on the signature feature map to be verified through the first sliding window, the periphery of the signature feature map to be verified is filled so as to obtain m signature local feature maps to be verified.
7. The signature verification device based on local feature matching according to claim 5, wherein the signature picture to be verified is input into a feature extraction network, and the output of two middle convolution layers of the feature extraction network is extracted to obtain a coarse-granularity signature feature map to be verified and a fine-granularity signature feature map to be verified; inputting the N template signature pictures of the user into a feature extraction network one by one, extracting the output of two middle convolution layers of the feature extraction network, and obtaining a coarse-granularity template signature feature picture and a fine-granularity template signature feature picture corresponding to each template signature picture; respectively executing local feature extraction and local feature matching on the signature feature map to be verified with coarse granularity and the template signature feature map to be verified with fine granularity to obtain N.m coarse granularity local feature matching maps and N.m fine granularity local feature matching maps; and inputting all the coarse-granularity local feature matching graphs and the fine-granularity local feature matching graphs into the classification network, and obtaining the probability that the signature to be verified is a true signature through classification verification.
8. The signature verification device based on local feature matching as recited in claim 5, wherein the training step of the signature verification model is: the training sample is a plurality of signature pictures to be tested and a plurality of template signature pictures corresponding to the signature pictures to be tested, labels are arranged on the signature pictures to be tested, and the authenticity of the signature to be tested is identified through the labels;
Training a first, taking a to-be-tested signature picture and a corresponding template signature picture as inputs, inputting the signature verification model for verification, and so on, taking another to-be-tested signature picture and a corresponding template signature picture as inputs, training a verification model, wherein all to-be-tested signature pictures and corresponding template signature pictures are trained, and the verification model can judge high-imitation counterfeit signatures;
training II, taking all or part of template signature pictures which are not corresponding to the signature to be detected and other signatures to be detected as input, inputting the template signature pictures into the signature verification model for verification, and repeating the steps until all the signature pictures to be tested are trained, wherein the verification model can judge random fake signatures.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911387311.7A CN111275070B (en) | 2019-12-26 | 2019-12-26 | Signature verification method and device based on local feature matching |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911387311.7A CN111275070B (en) | 2019-12-26 | 2019-12-26 | Signature verification method and device based on local feature matching |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111275070A CN111275070A (en) | 2020-06-12 |
| CN111275070B true CN111275070B (en) | 2023-11-14 |
Family
ID=71000178
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911387311.7A Active CN111275070B (en) | 2019-12-26 | 2019-12-26 | Signature verification method and device based on local feature matching |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111275070B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112001298B (en) * | 2020-08-20 | 2021-09-21 | 佳都科技集团股份有限公司 | Pedestrian detection method, device, electronic equipment and storage medium |
| CN112784829B (en) * | 2021-01-21 | 2024-05-21 | 北京百度网讯科技有限公司 | Bill information extraction method and device, electronic equipment and storage medium |
| CN113408492B (en) * | 2021-07-23 | 2022-06-14 | 四川大学 | Pedestrian re-identification method based on global-local feature dynamic alignment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7542593B1 (en) * | 2008-04-30 | 2009-06-02 | International Business Machines Corporaiton | Offline signature verification using high pressure regions |
| CN106778586A (en) * | 2016-12-08 | 2017-05-31 | 武汉理工大学 | Offline handwriting signature verification method and system |
| CN108416295A (en) * | 2018-03-08 | 2018-08-17 | 天津师范大学 | A kind of recognition methods again of the pedestrian based on locally embedding depth characteristic |
-
2019
- 2019-12-26 CN CN201911387311.7A patent/CN111275070B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7542593B1 (en) * | 2008-04-30 | 2009-06-02 | International Business Machines Corporaiton | Offline signature verification using high pressure regions |
| CN106778586A (en) * | 2016-12-08 | 2017-05-31 | 武汉理工大学 | Offline handwriting signature verification method and system |
| CN108416295A (en) * | 2018-03-08 | 2018-08-17 | 天津师范大学 | A kind of recognition methods again of the pedestrian based on locally embedding depth characteristic |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111275070A (en) | 2020-06-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Wu et al. | Busternet: Detecting copy-move image forgery with source/target localization | |
| CN110348319B (en) | Face anti-counterfeiting method based on face depth information and edge image fusion | |
| Walia et al. | Fusion of handcrafted and deep features for forgery detection in digital images | |
| CN111275070B (en) | Signature verification method and device based on local feature matching | |
| CN108197644A (en) | A kind of image-recognizing method and device | |
| CN103456003A (en) | Device and method for tracking object by using characteristic point descriptor, device and method for removing erroneous characteristic | |
| Yousry et al. | Currency Recognition System for Blind people using ORB Algorithm. | |
| Jaberi et al. | Improving the detection and localization of duplicated regions in copy-move image forgery | |
| Kruthi et al. | Offline signature verification using support vector machine | |
| Mshir et al. | Signature recognition using machine learning | |
| CN112597798A (en) | Method for identifying authenticity of commodity by using neural network | |
| Sharma et al. | Deep convolutional neural network with ResNet-50 learning algorithm for copy-move forgery detection | |
| Wyzykowski et al. | Multiresolution synthetic fingerprint generation | |
| Isaac et al. | A key point based copy-move forgery detection using HOG features | |
| Sabeena et al. | Convolutional block attention based network for copy-move image forgery detection | |
| Sedik et al. | AI-enabled digital forgery analysis and crucial interactions monitoring in smart communities | |
| Khuspe et al. | Robust image forgery localization and recognition in copy-move using bag of features and SVM | |
| CN113505716B (en) | Training method of vein recognition model, and recognition method and device of vein image | |
| Gowri et al. | Real Time Signature Forgery Detection Using Machine Learning | |
| Sabeena et al. | Digital Image Forgery Detection Using Local Binary Pattern (LBP) and Harlick Transform with classification | |
| Abraham | Digital image forgery detection approaches: A review and analysis | |
| Alqudah et al. | Efficient k-class approach for face recognition | |
| Lee et al. | Automatic writer verification algorithm for Chinese characters using semi-global features and adaptive classifier | |
| CN111931757A (en) | Finger vein quick sorting method and device based on MDLBP block histogram and PCA dimension reduction | |
| Xie et al. | An optimal orientation certainty level approach for fingerprint quality estimation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |