CN110399815B - VGG 16-based CNN-SVM handwritten signature recognition method - Google Patents
VGG 16-based CNN-SVM handwritten signature recognition method Download PDFInfo
- Publication number
- CN110399815B CN110399815B CN201910629487.2A CN201910629487A CN110399815B CN 110399815 B CN110399815 B CN 110399815B CN 201910629487 A CN201910629487 A CN 201910629487A CN 110399815 B CN110399815 B CN 110399815B
- Authority
- CN
- China
- Prior art keywords
- image
- len
- handwritten signature
- imagenet
- defining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 55
- 239000011159 matrix material Substances 0.000 claims abstract description 29
- 238000012549 training Methods 0.000 claims abstract description 28
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 23
- 230000009467 reduction Effects 0.000 claims abstract description 14
- 238000010606 normalization Methods 0.000 claims abstract description 10
- 238000007781 pre-processing Methods 0.000 claims abstract description 7
- 238000003062 neural network model Methods 0.000 claims abstract description 5
- 238000002372 labelling Methods 0.000 claims abstract description 4
- 239000013598 vector Substances 0.000 claims description 11
- 238000004422 calculation algorithm Methods 0.000 claims description 10
- 238000000605 extraction Methods 0.000 claims description 8
- 238000012545 processing Methods 0.000 claims description 6
- HCUOEKSZWPGJIM-YBRHCDHNSA-N (e,2e)-2-hydroxyimino-6-methoxy-4-methyl-5-nitrohex-3-enamide Chemical compound COCC([N+]([O-])=O)\C(C)=C\C(=N/O)\C(N)=O HCUOEKSZWPGJIM-YBRHCDHNSA-N 0.000 claims description 3
- 230000000694 effects Effects 0.000 abstract description 5
- 238000012706 support-vector machine Methods 0.000 description 12
- 238000012847 principal component analysis method Methods 0.000 description 10
- 230000004913 activation Effects 0.000 description 3
- 238000011176 pooling Methods 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000013526 transfer learning Methods 0.000 description 2
- OYPRJOBELJOOCE-UHFFFAOYSA-N Calcium Chemical compound [Ca] OYPRJOBELJOOCE-UHFFFAOYSA-N 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013145 classification model Methods 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000004883 computer application Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000003708 edge detection Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000004377 microelectronic Methods 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 210000002569 neuron Anatomy 0.000 description 1
- 238000000513 principal component analysis Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
- 230000003595 spectral effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
- G06F18/2135—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on approximation criteria, e.g. principal component analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/32—Digital ink
- G06V30/333—Preprocessing; Feature extraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/32—Digital ink
- G06V30/36—Matching; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The application discloses a VGG 16-based CNN-SVM handwritten signature recognition method, which comprises the following steps: labeling the handwritten signature image dataset; step two: preprocessing a data set sequentially through image graying, binarization and size normalization; step three: training a neural network model VGG16 by using a public data set ImageNet of a Kagle company to obtain a weight set; step four: migrating the weight set to CNN and training to obtain an initial feature matrix; step five: and (3) inputting the initial feature matrix into the SVM for training after PCA dimension reduction to obtain a handwritten signature image recognition result. The application improves CNN-SVM based on VGG16, effectively improves the handwriting signature recognition effect, and increases the use value of drawing signatures.
Description
Technical Field
The application belongs to the technical field of text image recognition, and particularly relates to a CNN-SVM handwriting signature recognition method based on VGG 16.
Background
In the handwriting character recognition work, the handwriting fonts of different writers have certain difference, aiming at the problem, the existing paper mainly starts from single handwriting Chinese character recognition, the deep learning model-based method achieves the effect of approaching to or even exceeding the human eye recognition performance, the offline handwriting signature recognition is less in research, and meanwhile, the handwriting signature has the characteristics of continuous writing, simple writing and the like, so that the recognition work is more difficult.
Feng Moli, zhu Quanyin et al have studied on the basis of: wanli Feng.research of theme statement extraction for chinese literature based on lexical chain. International Journal of Multimedia and Ubiquitous Engineering, vol.11, no.6 (2016), pp.379-388; wanli Feng, ying Li, shangbing Gao, yunyangYan, jianxun xue.a novel flame edge detection algorithm via a novel active contour model international Journal of Hybrid Information Technology, vol.9, no.9 (2016), pp.275-282; liu Jinling, feng Moli pattern matching method based on attribute dependency [ J ]. Microelectronics and computer, 2011,28 (12): 167-170; liu Jinling, feng Moli, zhang Yagong. Initializing text clusters for cluster class centers and reconstructed scale functions [ J ]. Computer application research 2011,28 (11): 4115-4117; liu Jinling, feng Moli, zhang Yagong computer engineering and applications based on rescaled text clustering method for chinese text [ J ], 2012,48 (21): 146-150; zhu Quanyin, pan Lu, liu Wenru, etc. the Web technology news classification extraction algorithm [ J ]. Huaiyin academy of engineering, 2015,24 (5): 18-24; li Xiang and Zhu Quanyin collaborative filtering recommendation [ J ] computer science and exploration, 2014,8 (6): 751-759; quanyin Zhu, sunqun Cao.A. Novel class identifier-independent Feature Selection Algorithm for Imbalanced data 2009, p:77-82; quanyin Zhu, yonyang Yan, jin Ding, jin Qian. The Case Study for Price Extracting of Mobile Phone Sell Online.2011, p:282-285; quanyin Zhu, suqun Cao, pei Zhou, yunyang Yan, hong Zhou. Integrated Price Forecast based on Dichotomy Backfilling and Disturbance Factor Algorithm. International Review on Computers and Software,2011, vol.6 (6): 1089-1093; zhu Quanyin, feng Moli et al, discloses and grants related patents: feng Moli, shao Heshuai, zhuang Jun an intelligent refrigerated truck status monitoring wireless network terminal device is CN203616634U [ P ].2014; zhu Quanyin, hu Rongjing, he Suqun, zhou Pei, etc. A commodity price prediction method based on linear interpolation and adaptive sliding window; zhu Quanyin, cao Suqun, yan Yunyang, hu Rongjing, etc., a commodity price prediction method based on binary data patching and disturbing factors; li Xiang, zhu Quanyin, hu Ronglin, zhou Hong. An intelligent recommendation method for cold chain logistics loading based on spectral clustering. Chinese patent publication No. CN105654267A,2016.06.08.
Principal component analysis method (PCA):
PCA (Principal ComponentAnalysis) is a widely used data dimension reduction algorithm. The main idea of PCA is to map m-dimensional features onto k-dimensions, which are completely new orthogonal features, also called principal components, and are k-dimensional features reconstructed on the basis of the original m-dimensional features. And calculating a covariance matrix of the sample data, solving eigenvalues and orthogonal unit eigenvectors of the covariance matrix, sequencing the eigenvalues from large to small, and selecting the largest k eigenvalues. And then respectively taking k eigenvectors corresponding to the eigenvectors as row vectors to form an eigenvector matrix J1, and finally obtaining a reduced-dimension data set.
Convolutional Neural Network (CNN):
convolutional neural networks are one method of image feature extraction that has been popular in recent years, and the main parameters of the convolutional neural networks are the size M, the number N and the interval stride of the convolutional kernels, where the stride is usually 1, and features are extracted by using the convolutional kernels, and these initialized convolutional kernels are updated again and again in iteration in the process of back propagation, and are infinitely approximated to true solutions. The size and the number of the convolution kernels are adjusted according to actual conditions so as to obtain better effects.
Support Vector Machine (SVM):
SVM (Support Vector Machines) is a small sample learning method based on a statistical learning theory proposed by Vapnik et al, has stronger generalization capability according to a structural risk minimization principle, and can ensure that a global optimal solution is found. The application adopts SVM to construct classifier, and training to obtain handwriting signature recognition model.
Deng Yue, ding Xiaoyong, gao Chenlan, etc.. Handwriting signature recognition method based on PCA method, chinese patent publication No.: CN105893952a,2016.08.24, performing image space dimension reduction according to the PCA method, and extracting a feature value and a feature vector, but the dimension reduction method also reduces the accuracy; huang Jinjie, hejie, rush Jiang Quandeng. Handwriting signature recognition system based on neural network chinese patent publication No.: CN109766825a,2019.05.17, performing multiple times of iterative training with or without supervision to save the model by using the built convolutional neural network algorithm model, and then identifying, but this method only uses a simple convolutional neural algorithm model, the complexity of the model is not enough, and it is difficult to accurately identify the complex signature; wang Min, liu Pengfei, yin Na, etc. a method and apparatus for automatically recognizing signature seal or handwritten signature: CN108921126a,2018.11.30 uses a feature matrix to determine whether a signature in a feature image or a handwritten signature is included in a target image, but this method simply determines whether a handwritten signature of a feature image is included through the feature matrix, and the model complexity of the method is not sufficient.
The traditional handwriting recognition is to firstly preprocess the acquired handwriting signature image, then extract the characteristics of Chinese characters, then classify the signatures, and finally realize the recognition of the handwriting signatures. The handwriting signature recognition difficulty is high, and the difficulty in Chinese character recognition is brought to the fact that the handwriting signature recognition is mainly due to the fact that the structure of the signature is complex, the variety of fonts is large, the quantity is large, the material quality of writing paper is different in colors of pens used for writing, the force is different in control during writing Chinese characters, and even the angle, pixels and light are different during shooting images.
Disclosure of Invention
The application aims to: aiming at the problems, the application provides a CNN-SVM handwritten signature recognition method based on VGG16, through the migration learning of VGG16 weight, the error existing in the feature vector extraction when the data volume is less is reduced, and the classification accuracy and speed are effectively improved.
The technical scheme is as follows: the application provides a VGG 16-based CNN-SVM handwritten signature recognition method, which comprises the following steps:
step 1: defining an original handwritten signature Image data set as Image0, and obtaining the handwritten signature Image data set Image1 through labeling processing, wherein the method specifically comprises the following steps of:
step 1.1: defining an original handwritten signature Image dataset Image0, image 0= { C 1 ,C 2 ,…,C m }, wherein C m For the mth group of handwritten signature Image datasets in Image0, the global variable mE [1,100]Define id and num as C respectively m Number of images and number of images satisfying C m ＝{id,num}；
Step 1.2: defining a loop variable i to traverse Image0, wherein i is an initial value of 1, i epsilon [1, len (Image 0) ], and len (Image 0) is the number of handwritten signature images in Image 0;
step 1.3: if i is less than or equal to len (Image 0), executing step 1.4, otherwise executing step 1.6;
step 1.4: definition of id1, label i 、C i Respectively a sequence number, a label and a single handwritten signature set, for C i Adding a corresponding label i ；
Step 1.5: let i=i+1, execute step 1.3;
step 1.6: obtaining a handwriting signature Image data set Image1, wherein Image 1= { id1 and label id1, C id1 }。
Step 2: preprocessing a handwritten signature Image data set Image1 sequentially through an Image graying method, a binarization method and a size normalization method to obtain a handwritten signature Image data set Image2, wherein the specific method comprises the following steps of:
step 2.1: defining a loop variable i0 to traverse Image1, wherein the initial value of i0 is 1, i0 epsilon [1, len (Image 1) ], and len (Image 1) is the number of handwritten signature images in Image1;
step 2.2: if i0 is less than or equal to len (Image 1), executing step 2.3, otherwise executing step 2.5;
step 2.3: defining an image pixel set image = { a 1 ,a 2 ,…,a m And }, wherein a m = { R, G, B }, graying is performed using a weighted average method, and the specific formula is:
Ga＝0.299*R+0.578*G+0.114*B，
wherein Ga is a pixel value subjected to graying treatment;
step 2.4: let i0=i0+1, execute step 2.2;
step 2.5: obtain a grayscaled pixel set g_image= { Ga 1 ,Ga 2 ,…,Ga m }；
Step 2.6: defining a loop variable i1 to traverse g_image, wherein i1 is initialized to be 1, i1 epsilon [1, len (g_image) ] and len (g_image) is the length of g_image;
step 2.7: if i1 is equal to or less than len (g_image), executing step 2.8, otherwise executing step 2.14;
step 2.8: defining a pixel value constant x, x being assigned 125, and Ga i1 Comparing;
step 2.9: if Ga i1 And (3) not less than x, executing the step 2.10, otherwise executing the step 2.11;
step 2.10: let Ga i1 =0, step 2.12 is performed;
step 2.11: let Ga i1 =255, step 2.12 is performed;
step 2.12: binarizing the g_image by an Otsu algorithm (OTSU), setting a threshold value to 125, and setting a pixel value to 0 if the threshold value is larger than the threshold value; if the pixel value is smaller than the threshold value, setting the pixel value to 255;
step 2.13: let i1=i1+1, execute step 2.7;
step 2.14: obtain a binarized image set t_image= { Ta 1 ,Ta 2 ,…,Ta m }, wherein Ta m ＝{0,255}；
Step 2.15: defining a loop variable i2 to traverse t_image, wherein i2 is initialized to be 1, i2 epsilon [1, len (t_image) ], and len (t_image) is the length of t_image;
step 2.16: if i2 is not greater than len (t_image), executing step 2.17, otherwise executing step 2.19;
step 2.17: setting the length and width of a single image subjected to graying and binarization to be 28 x 28 through size normalization processing;
step 2.18: let i2=i2+1, execute step 2.16;
step 2.19: obtaining a preprocessed handwritten signature Image dataset Image2, wherein Image 2= { nor 1 ,nor 2 ,…,nor m }, where nor m Is a single image after graying, binarization and size normalization.
Step 3: training a neural network model VGG16 by using a public data set ImageNet of Kagle corporation, and training to obtain a weight set G, wherein the specific method comprises the following steps:
step 3.1: input data set ImageNet, imageNet = { IN 0 ,IN 1 ,…,IN D }, where IN D Is a single sheetGrouping image datasets of the same label;
step 3.2: defining a loop variable i3 to traverse the ImageNet, wherein the initial value of i3 is 1, i3 epsilon [1, len (ImageNet) ], and len (ImageNet) is the number of images in the ImageNet;
step 3.3: if i3 is less than or equal to len (ImageNet), executing step 3.4, otherwise executing step 3.6;
step 3.4: will IN i3 Inputting ImgeNet for training, wherein IN i3 An i3 rd image in the ImageNet;
step 3.5: let i3=i3+1, execute step 3.3;
step 3.6: obtaining a trained model M0, and obtaining a weight set G= { G 1 ,g 2 ,…,g N }, wherein g N Is the nth weight in G.
Step 4: migrating the weight set G to a CNN model, and inputting the data set Image2 to the trained CNN model to obtain an initial feature matrix J, wherein the specific method comprises the following steps of:
step 4.1: migrating the weights in the set G to a CNN model, and training to obtain a model M;
step 4.2: defining a loop variable i4 to traverse Image2, wherein the initial value of i4 is 1, i4 epsilon [1, len (Image 2) ], and len (Image 2) is the number of handwritten signature images in Image 2;
step 4.3: if i4 is less than or equal to len (Image 2), executing step 4.4, otherwise executing step 4.6;
step 4.4: will nor i4 Performing feature extraction on an input model to obtain an initial feature vector v, wherein the nor i4 An i4 th Image in Image 2;
step 4.5: let i4=i4+1, execute step 4.3;
step 4.6: obtaining an initial feature matrix J= [ v ] 1 ,v 2 ,…,v m ]And a corresponding set of eigenvalues f= { a 1 ,A 2, …,A m }, where v m Is the m-th eigenvector in the matrix J, A m V is m Corresponding initial characteristic values.
Step 5: the method comprises the steps of performing dimension reduction on a matrix J by using a PCA dimension reduction method to obtain a matrix J1, and training a feature matrix J1 by using an SVM to obtain a handwritten signature image recognition result, wherein the specific method comprises the following steps:
step 5.1: sorting the eigenvalues in the set F from large to small to obtain a set f1, f1= { λ 1 ,λ 2 ,...,λ m And }, where lambda m Is the m-th characteristic value in F1, meets lambda 1 ≥λ 2 ≥λ m ；
Step 5.2: selecting the feature vector (k) corresponding to the first k feature values (k<len (F1)), and performing PCA dimension reduction to obtain a feature matrix J1= [ u ] 1 ,u 2 ,...,u k ]Wherein u is k Representing the feature vector corresponding to the kth feature value in F1;
step 5.3: f1 is input into an SVM to obtain a recognition result R, R= { plabel, rlabel }, wherein the plabel, rlabel respectively represent a prediction category label and an actual category label.
The VGG 16-based CNN-SVM handwritten signature recognition method has important effect and significance on actual drawing signature recognition. Aiming at the handwriting recognition problem, researchers adopt methods such as preprocessing, feature extraction, feature dimension reduction, classifier design and the like to improve recognition effects. According to the application, the weight of VGG16 pre-training is migrated to the convolutional neural network, feature dimension reduction is performed by utilizing PCA, and an accurate and efficient recognition scheme is provided for a related system by combining the convolutional neural network and a support vector machine classifier algorithm.
Compared with the existing recognition method, the recognition method of the handwritten signature based on the CNN-SVM has certain advancement, but the training data set of the method is smaller, the degree of distinguishing the extracted features is low when the method is used independently, and the recognition requirement of the handwritten signature in practical application can not be met.
The application adopts the technical scheme and has the following beneficial effects:
the method is based on the existing gesture label image dataset, and utilizes the transfer learning of the convolutional neural network MobileNet and XGBoost to effectively classify the multi-label gesture images, and is specifically described as follows: according to the application, the characteristics are extracted by utilizing the MobileNet convolutional neural network architecture and the weights through transfer learning, and the algorithm is based on the existing weight files, so that the time for reconstructing the network architecture can be saved while the characteristics are accurately extracted; the XGBoost is used as a classification model, the extracted features are input into the XGBoost model, the XGBoost is utilized to automatically use the multithread of the CPU to perform parallel calculation and introduce regularization items, so that higher classification accuracy is realized, the calculation complexity of the model is reduced, the working time of practitioners is shortened, and the running efficiency of related products is improved.
Drawings
FIG. 1 is a general flow chart of the present application;
FIG. 2 is a flow chart of handwritten signature dataset production in an embodiment;
FIG. 3 is a flow chart of preprocessing a handwritten signature dataset in an embodiment;
FIG. 4 is a flow chart of VGG16 model training in an embodiment;
FIG. 5 is a flow chart of VGG16-CNN model training in an embodiment;
FIG. 6 is a flow chart of feature dimension reduction and classification in an embodiment.
Detailed Description
The present application is further illustrated below in conjunction with specific embodiments, it being understood that these embodiments are meant to be illustrative of the application and not limiting the scope of the application, and that modifications of the application, which are equivalent to those skilled in the art to which the application pertains, fall within the scope of the application defined in the appended claims after reading the application.
As shown in fig. 1 to 6, the CNN-SVM handwritten signature recognition method based on VGG16 according to the application comprises the following steps:
step 1: defining an original handwritten signature Image data set as Image0, and obtaining the handwritten signature Image data set Image1 through labeling processing, wherein the method specifically comprises the following steps of:
step 1.1: defining an original handwritten signature Image dataset Image0, image 0= { C 1 ,C 2 ,…,C m }, wherein C m For the mth group of handwritten signature Image datasets in Image0, the global variable mE [1,100]Define id and num as C respectively m Number of images and number of images satisfying C m ＝{id,num}；
Step 1.2: defining a loop variable i to traverse Image0, wherein i is an initial value of 1, i epsilon [1, len (Image 0) ], and len (Image 0) is the number of handwritten signature images in Image 0;
step 1.3: if i is less than or equal to len (Image 0), executing step 1.4, otherwise executing step 1.6;
step 1.4: definition of id1, label i 、C i Respectively a sequence number, a label and a single handwritten signature set, for C i Adding a corresponding label i ；
Step 1.5: let i=i+1, execute step 1.3;
step 1.6: obtaining a handwriting signature Image data set Image1, wherein Image 1= { id1 and label id1, C id1 }。
Step 2: preprocessing a handwritten signature Image data set Image1 sequentially through an Image graying method, a binarization method and a size normalization method to obtain a handwritten signature Image data set Image2, wherein the specific method comprises the following steps of:
step 2.1: defining a loop variable i0 to traverse Image1, wherein the initial value of i0 is 1, i0 epsilon [1, len (Image 1) ], and len (Image 1) is the number of handwritten signature images in Image1;
step 2.2: if i0 is less than or equal to len (Image 1), executing step 2.3, otherwise executing step 2.5;
step 2.3: defining an image pixel set image = { a 1 ,a 2 ,…,a m And }, wherein a m = { R, G, B }, graying is performed using a weighted average method, and the specific formula is:
Ga＝0.299*R+0.578*G+0.114*B，
wherein Ga is a pixel value subjected to graying treatment;
step 2.4: let i0=i0+1, execute step 2.2;
step 2.5: obtain a grayscaled pixel set g_image= { Ga 1 ,Ga 2 ,…,Ga m }；
Step 2.6: defining a loop variable i1 to traverse g_image, wherein i1 is initialized to be 1, i1 epsilon [1, len (g_image) ] and len (g_image) is the length of g_image;
step 2.7: if i1 is equal to or less than len (g_image), executing step 2.8, otherwise executing step 2.14;
step 2.8: defining a pixel value constant x, x being assigned 125, and Ga i1 Comparing;
step 2.9: if Ga i1 And (3) not less than x, executing the step 2.10, otherwise executing the step 2.11;
step 2.10: let Ga i1 =0, step 2.12 is performed;
step 2.11: let Ga i1 =255, step 2.12 is performed;
step 2.12: binarizing the g_image by an Otsu algorithm (OTSU), setting a threshold value to 125, and setting a pixel value to 0 if the threshold value is larger than the threshold value; if the pixel value is smaller than the threshold value, setting the pixel value to 255;
step 2.13: let i1=i1+1, execute step 2.7;
step 2.14: obtain a binarized image set t_image= { Ta 1 ,Ta 2 ,…,Ta m }, wherein Ta m ＝{0,255}；
Step 2.15: defining a loop variable i2 to traverse t_image, wherein i2 is initialized to be 1, i2 epsilon [1, len (t_image) ], and len (t_image) is the length of t_image;
step 2.16: if i2 is not greater than len (t_image), executing step 2.17, otherwise executing step 2.19;
step 2.17: setting the length and width of a single image subjected to graying and binarization to be 28 x 28 through size normalization processing;
step 2.18: let i2=i2+1, execute step 2.16;
step 2.19: obtaining a preprocessed handwritten signature Image dataset Image2, wherein Image 2= { nor 1 ,nor 2 ,…,nor m }, where nor m Is a single image after graying, binarization and size normalization.
Step 3: training a neural network model VGG16 by using a public data set ImageNet of Kagle corporation, and training to obtain a weight set G, wherein the specific method comprises the following steps:
step 3.1: input data set ImageNet, imageNet = { IN 0 ,IN 1 ,…,IN D }, where IN D Is of a single groupA labeled image dataset;
step 3.2: defining a loop variable i3 to traverse the ImageNet, wherein the initial value of i3 is 1, i3 epsilon [1, len (ImageNet) ], and len (ImageNet) is the number of images in the ImageNet;
step 3.3: if i3 is less than or equal to len (ImageNet), executing step 3.4, otherwise executing step 3.6;
step 3.4: will IN i3 Inputting ImgeNet for training, wherein IN i3 An i3 rd image in the ImageNet;
step 3.5: let i3=i3+1, execute step 3.3;
step 3.6: obtaining a trained model M0, and obtaining a weight set G= { G 1 ,g 2 ,…,g N }, wherein g N Is the nth weight in G.
Step 4: migrating the weight set G to a CNN model, and inputting the data set Image2 to the trained CNN model to obtain an initial feature matrix J, wherein the specific method comprises the following steps of:
step 4.1: migrating the weights in the set G to a CNN model, and training to obtain a model M;
step 4.2: defining a loop variable i4 to traverse Image2, wherein the initial value of i4 is 1, i4 epsilon [1, len (Image 2) ], and len (Image 2) is the number of handwritten signature images in Image 2;
step 4.3: if i4 is less than or equal to len (Image 2), executing step 4.4, otherwise executing step 4.6;
step 4.4: will nor i4 Performing feature extraction on an input model to obtain an initial feature vector v, wherein the nor i4 An i4 th Image in Image 2;
step 4.5: let i4=i4+1, execute step 4.3;
step 4.6: obtaining an initial feature matrix J= [ v ] 1 ,v 2 ,…,v m ]And a corresponding set of eigenvalues f= { a 1 ,A 2, …,A m }, where v m Is the m-th eigenvector in the matrix J, A m V is m Corresponding initial characteristic values;
specific parameters using convolutional neural networks are:
the first layer is a convolution layer, the number of filter kernels is 64, the size of the filter kernels is 3×3, the padding is the same, and the activation layer is a relu layer;
the second layer is a pooling layer, and the parameters of the core are 2 multiplied by 2;
the third layer is a convolution layer, the number of filter kernels is 128, the size of the filter kernels is 3×3, the padding is the same, and the activation layer is a relu layer;
the fourth layer is a pooling layer, and the parameters of the core are 2 multiplied by 2;
the fifth layer is a convolution layer, the number of filter kernels is 256, the size of the filter kernels is 3 multiplied by 3, the padding is the same, and the activation layer is a relu layer;
the sixth layer is a pooling layer, the parameters of the core are 2 multiplied by 2;
the seventh layer is a full-connection layer, and the number of neurons of the full-connection layer is 1024.
Step 5: the method comprises the steps of performing dimension reduction on a matrix J by using a PCA dimension reduction method to obtain a matrix J1, and training a feature matrix J1 by using an SVM to obtain a handwritten signature image recognition result, wherein the specific method comprises the following steps:
step 5.1: sorting the eigenvalues in the set F from large to small to obtain a set f1, f1= { λ 1 ,λ 2 ,...,λ m And }, where lambda m Is the m-th characteristic value in F1, meets lambda 1 ≥λ 2 ≥λ m ；
Step 5.2: selecting the feature vector (k) corresponding to the first k feature values (k<len (F1)), and performing PCA dimension reduction to obtain a feature matrix J1= [ u ] 1 ,u 2 ,...,u k ]Wherein u is k Representing the feature vector corresponding to the kth feature value in F1;
step 5.3: f1 is input into an SVM to obtain a recognition result R, R= { plabel, rlabel }, wherein the plabel, rlabel respectively represent a prediction category label and an actual category label.
The method comprises the steps of processing a data set containing 1000 handwritten signature images, migrating weights obtained by VGG16 pre-training to a CNN model, training the CNN model to obtain an initial feature matrix, performing PCA dimension reduction on the feature matrix, and combining an SVM classifier to realize handwritten signature recognition.
Table 1 variable description table
The application can be combined with a computer system to automatically complete the handwriting signature recognition in the field of character recognition.
The application creatively provides a method for combining weights, a CNN model and a support vector machine of VGG16 pre-training.
The application provides a VGG 16-based CNN-SVM handwriting signature recognition method which can be used for recognizing drawing signatures and other handwriting font application scenes.
The foregoing is merely exemplary of the present application and is not intended to limit the present application. All equivalents and alternatives falling within the spirit of the application are intended to be included within the scope of the application. What is not elaborated on the application belongs to the prior art which is known to the person skilled in the art.
Claims (3)
1. A VGG 16-based CNN-SVM handwritten signature recognition method is characterized by comprising the following steps:
(1) Defining an original handwritten signature Image data set as Image0, and obtaining a handwritten signature Image data set Image1 through labeling;
(2) Preprocessing a handwritten signature Image dataset Image1 sequentially through an Image graying method, a binarization method and a size normalization method to obtain a handwritten signature Image dataset Image2, wherein the preprocessing comprises the following steps:
(2.1) defining a loop variable i0 to traverse Image1, wherein the initial value of i0 is 1, i0 epsilon [1, len (Image 1) ], and len (Image 1) is the number of handwritten signature images in Image1;
(2.2) if i0 is not greater than len (Image 1), executing step (2.3), otherwise executing step (2.5);
(2.3) defining an image pixel set image = { a 1 ,a 2 ,…,a m And }, wherein a m = { R, G, B }, graying is performed using a weighted average method, and the specific formula is:
Ga＝0.299*R+0.578*G+0.114*B，
wherein Ga is a pixel value subjected to graying treatment;
(2.4) let i0=i0+1, and execute step (2.2);
(2.5) obtaining a grayed-out pixel set g_image= { Ga 1 ,Ga 2 ,…,Ga m }；
(2.6) defining a loop variable i1 to traverse the g_image, i1 being initially 1, i1 e [1, len (g_image) ], len (g_image) being the length of the g_image;
(2.7) if i1 is not greater than len (g_image), executing step (2.8), otherwise executing step (2.14);
(2.8) defining a pixel value constant x, x being assigned 125, and Ga i1 Comparing;
(2.9) if Ga i1 And (3) not less than x, executing the step (2.10), otherwise executing the step (2.11);
(2.10) Ga i1 =0, step (2.12) is performed;
(2.11) Ga i1 =255, step (2.12) is performed;
(2.12) binarizing the g_image by using an Otsu algorithm, setting a threshold value to 125, and setting a pixel value to 0 if the threshold value is larger than the threshold value; if the pixel value is smaller than the threshold value, setting the pixel value to 255;
(2.13) let i1=i1+1, execute step (2.7);
(2.14) obtaining a binarized image set t_image= { Ta 1 ,Ta 2 ,…,Ta m }, wherein Ta m ＝{0,255}；
(2.15) defining a loop variable i2 to traverse t_image, i2 having an initial value of 1, i2 e [1, len (t_image) ], len (t_image) being the length of t_image;
(2.16) if i2 is not greater than len (t_image), executing step (2.17), otherwise executing step (2.19);
(2.17) setting the length and width of the single image subjected to gray scale and binarization to 28 x 28 through size normalization processing;
(2.18) let i2=i2+1, execute step (2.16);
(2.19) obtaining a preprocessed handwritten signature Image dataset Image2, image 2= { nor 1 ,nor 2 ,…,nor m }, where nor m The single image is subjected to graying, binarization and size normalization;
(3) Training a neural network model VGG16 by adopting a data set ImageNet, and training to obtain a weight set G;
(4) Migrating the weight set G to a CNN model, and inputting the data set Image2 to the trained CNN model to obtain an initial feature matrix J, wherein the method comprises the following specific steps of:
(4.1) migrating the weights in the set G to a CNN model, and training to obtain a model M;
(4.2) defining a loop variable i4 to traverse Image2, wherein the initial value of i4 is 1, i4 epsilon [1, len (Image 2) ], and len (Image 2) is the number of handwritten signature images in Image 2;
(4.3) if i4 is not greater than len (Image 2), executing step (4.4), otherwise executing step (4.6);
(4.4) the nor i4 Performing feature extraction on an input model to obtain an initial feature vector v, wherein the nor i4 An i4 th Image in Image 2;
(4.5) let i4=i4+1, execute step (4.3);
(4.6) obtaining an initial feature matrix J= [ v ] 1 ,v 2 ,…,v m ]And a corresponding set of eigenvalues f= { a 1 ,A 2, …,A m }, where v m Is the m-th eigenvector in the matrix J, A m V is m Corresponding initial characteristic values;
(5) The method for reducing the dimension of the matrix J by using the PCA dimension reduction method to obtain a matrix J1, and training the matrix J1 by using the SVM to obtain a handwritten signature image recognition result R comprises the following steps:
(5.1) sorting the eigenvalues in the eigenvalue set F corresponding to the matrix J from large to small to obtain a set F1, F1 = { lambda% 1 ,λ 2 ,...,λ m And }, where lambda m Is the m-th characteristic value in F1, meets lambda 1 ≥λ 2 ≥λ m ；
(5.2) selecting the feature vector (k) corresponding to the first k feature values of F1 other than 0<len (F1)), and performing PCA dimension reduction to obtain a feature matrix J1= [ u ] 1 ,u 2 ,...,u k ]Wherein u is k Representing the feature vector corresponding to the kth feature value in F1;
and (5.3) inputting F1 into the SVM to obtain a recognition result R, R= { plabel and rlabel }, wherein the plabel and the rlabel respectively represent a prediction type label and an actual type label.
2. The CNN-SVM handwritten signature recognition method based on VGG16 according to claim 1, wherein the specific steps of obtaining the handwritten signature Image dataset Image1 in step (1) are as follows:
(1.1) defining an original handwritten signature Image dataset Image0, image 0= { C 1 ,C 2 ,…,C m }, wherein C m For the mth group of handwritten signature Image datasets in Image0, the global variable mE [1,100]Define id and num as C respectively m Number of images and number of images satisfying C m ＝{id,num}；
(1.2) defining a loop variable i to traverse Image0, wherein i is 1, i epsilon [1, len (Image 0) ], and len (Image 0) is the number of handwritten signature images in Image 0;
(1.3) if i is not greater than len (Image 0), executing step (1.4), otherwise executing step (1.6);
(1.4) definition of id1, label i 、C i Respectively a sequence number, a label and a single handwritten signature set, for C i Adding a corresponding label i ；
(1.5) let i=i+1, performing step (1.3);
(1.6) obtaining a handwritten signature Image dataset Image1, image 1= { id1, label id1, C id1 }。
3. The method for identifying the handwritten signature of the CNN-SVM based on VGG16 according to claim 1, wherein the specific steps of training the neural network model VGG16 by adopting a data set ImageNet in the step (3) and obtaining a weight set G by training are as follows:
(3.1) input data set ImageNet, imageNet = { IN 0 ,IN 1 ,…,IN D }, where IN D Image dataset for a single group of identical labels;
(3.2) defining a loop variable i3 to traverse ImageNet, wherein i3 has an initial value of 1, i3 e [1, len (ImageNet) ], and len (ImageNet) is the number of images in ImageNet;
(3.3) if i3 is not greater than len (ImageNet), executing step (3.4), otherwise executing step (3.6);
(3.4) the IN i3 Inputting ImgeNet for training, wherein IN i3 An i3 rd image in the ImageNet;
(3.5) let i3=i3+1, execute step (3.3);
(3.6) obtaining a trained model M0, and obtaining a weight set G= { G 1 ,g 2 ,…,g N }, wherein g N Is the nth weight in G.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910629487.2A CN110399815B (en) | 2019-07-12 | 2019-07-12 | VGG 16-based CNN-SVM handwritten signature recognition method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910629487.2A CN110399815B (en) | 2019-07-12 | 2019-07-12 | VGG 16-based CNN-SVM handwritten signature recognition method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110399815A CN110399815A (en) | 2019-11-01 |
| CN110399815B true CN110399815B (en) | 2023-09-26 |
Family
ID=68325412
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910629487.2A Active CN110399815B (en) | 2019-07-12 | 2019-07-12 | VGG 16-based CNN-SVM handwritten signature recognition method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110399815B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11600504B2 (en) * | 2020-06-29 | 2023-03-07 | Taiwan Semiconductor Manufacturing Company, Ltd. | Detecting damaged semiconductor wafers utilizing a semiconductor wafer sorter tool of an automated materials handling system |
| CN113158887B (en) * | 2021-04-19 | 2024-08-16 | 福建捷宇电脑科技有限公司 | Electronic signature authentication method and equipment for improving electronic signature recognition accuracy |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102254196B (en) * | 2011-06-22 | 2014-05-14 | 江苏奥博洋信息技术有限公司 | Method for identifying handwritten Chinese character by virtue of computer |
| CN107909095A (en) * | 2017-11-07 | 2018-04-13 | 江苏大学 | A kind of image-recognizing method based on deep learning |
| US10713563B2 (en) * | 2017-11-27 | 2020-07-14 | Technische Universiteit Eindhoven | Object recognition using a convolutional neural network trained by principal component analysis and repeated spectral clustering |
| CN108921029A (en) * | 2018-06-04 | 2018-11-30 | 浙江大学 | A kind of SAR automatic target recognition method merging residual error convolutional neural networks and PCA dimensionality reduction |
-
2019
- 2019-07-12 CN CN201910629487.2A patent/CN110399815B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN110399815A (en) | 2019-11-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Han et al. | Autoencoder inspired unsupervised feature selection | |
| Oquab et al. | Weakly supervised object recognition with convolutional neural networks | |
| Prince et al. | Probabilistic models for inference about identity | |
| CN112528928B (en) | Commodity identification method based on self-attention depth network | |
| Zhao et al. | Locality preserving hashing | |
| CN110390275B (en) | Gesture classification method based on transfer learning | |
| CN103258210B (en) | A kind of high-definition image classification method based on dictionary learning | |
| Zweig et al. | Hierarchical regularization cascade for joint learning | |
| Escalera et al. | Boosted Landmarks of Contextual Descriptors and Forest-ECOC: A novel framework to detect and classify objects in cluttered scenes | |
| Jiang et al. | DCT–CNN-based classification method for the Gongbi and Xieyi techniques of Chinese ink-wash paintings | |
| Islam et al. | A CNN based approach for garments texture design classification | |
| Shi et al. | Scene text recognition using structure-guided character detection and linguistic knowledge | |
| Rahman et al. | Two decades of bengali handwritten digit recognition: A survey | |
| Yang et al. | An ensemble classification algorithm for convolutional neural network based on AdaBoost | |
| CN110399815B (en) | VGG 16-based CNN-SVM handwritten signature recognition method | |
| Sampath et al. | Fuzzy-based multi-kernel spherical support vector machine for effective handwritten character recognition | |
| Ouyed et al. | Feature weighting for multinomial kernel logistic regression and application to action recognition | |
| Liu et al. | Evolutionary compact embedding for large-scale image classification | |
| Valem et al. | Unsupervised similarity learning through Cartesian product of ranking references | |
| Chen et al. | Face age classification based on a deep hybrid model | |
| Yang et al. | Multi-level contour combination features for shape recognition | |
| Bennour et al. | A deep learning framework for historical manuscripts writer identification using data-driven features | |
| CN108052867B (en) | Single-sample face recognition method based on bag-of-words model | |
| Raikar et al. | Efficiency comparison of supervised and unsupervised classifier on content based classification using shape, color, texture | |
| Sun et al. | Multiple-kernel, multiple-instance similarity features for efficient visual object detection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20231213 Address after: Room 1, No. 12 Tongyuan Road, Huai'an Economic and Technological Development Zone, Huai'an City, Jiangsu Province, 223200 Patentee after: Huai'an road data Co.,Ltd. Address before: 223005 A12-2, high tech Industrial Park, three East seven street, Hongze District, Huaian, Jiangsu (Hongze technology transfer center Hongze sub center) Patentee before: HUAIYIN INSTITUTE OF TECHNOLOGY |
|
| TR01 | Transfer of patent right |