CN111368632A - Signature identification method and device - Google Patents
Signature identification method and device Download PDFInfo
- Publication number
- CN111368632A CN111368632A CN201911380443.7A CN201911380443A CN111368632A CN 111368632 A CN111368632 A CN 111368632A CN 201911380443 A CN201911380443 A CN 201911380443A CN 111368632 A CN111368632 A CN 111368632A
- Authority
- CN
- China
- Prior art keywords
- character
- signature
- surname
- image
- signature image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 56
- 238000007781 pre-processing Methods 0.000 claims abstract description 12
- 230000000877 morphologic effect Effects 0.000 claims description 21
- 230000011218 segmentation Effects 0.000 claims description 19
- 241000949477 Toona ciliata Species 0.000 claims description 5
- 238000012545 processing Methods 0.000 claims description 5
- 238000011176 pooling Methods 0.000 claims description 4
- 230000015654 memory Effects 0.000 description 14
- 238000004422 calculation algorithm Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 7
- 238000004590 computer program Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000005291 magnetic effect Effects 0.000 description 4
- 238000001514 detection method Methods 0.000 description 3
- 101100440696 Caenorhabditis elegans cor-1 gene Proteins 0.000 description 2
- 150000001875 compounds Chemical class 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000001174 ascending effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
- 230000003936 working memory Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/30—Writer recognition; Reading and verifying signatures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/14—Image acquisition
- G06V30/148—Segmentation of character regions
- G06V30/153—Segmentation of character regions using recognition of characters or words
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
Abstract
The application aims to provide a signature identification method and equipment, and the signature identification method and equipment are used for identifying a signature by acquiring a signature image during signature; preprocessing the signature image in sequence and positioning and dividing single characters based on SSD to obtain single character images corresponding to the single characters in the signature image, wherein the single characters comprise surname single characters and name single characters; the method comprises the steps of identifying single surname characters according to a surname identification model based on a handwritten Chinese character identification network to obtain a classification result of the single surname characters, and identifying the single surname characters according to a first name identification model based on the handwritten Chinese character identification network to obtain a classification result of the single surname characters; the signature recognition result of the signature image is obtained based on the classification result of the surname single character and the classification result of the first name single character, the recognition of the single character in the signature image by fusing a surname recognition model and a first name recognition model is realized, the signature recognition of the signature image is completed cooperatively, and the accuracy of the signature recognition is improved.
Description
Technical Field
The present application relates to the field of computers, and in particular, to a signature identification method and apparatus.
Background
With the continuous development of electronic technology, the traditional handwritten signature cannot meet the signature requirements of people in daily life. In order to save office cost and improve signature efficiency, electronic dynamic signatures are gradually replacing traditional handwritten signatures and have the same legal effectiveness as traditional handwritten signatures. In order to identify an electronic signature to obtain a signature result, in the prior art, a method of first segmenting and then identifying or a method of overall identifying is generally used, where the first segmenting and then identifying refers to segmenting a single character image by using a conventional algorithm (such as connected domain threshold segmentation) or based on a deep learning algorithm (such as a multi-target detection algorithm), and then identifying the single character image, and the overall identifying refers to inputting a signature image into a Network based on a Recurrent Neural Network (RNN) or a variant thereof and outputting character categories included in the entire signature image by using a joint dominant Temporal Classification (CTC) loss function.
When the signature is divided into single characters and then recognized, if a traditional algorithm is adopted, due to the character spacing and the character inclination, the divided single characters are often interfered by other characters; in addition, the performance of the segmentation algorithm itself affects the accuracy of single-character recognition, and thus the accuracy of signature recognition. The problem of the method of outputting all the categories of single character strings contained in a signature image at one time by adopting a CTC loss function in a network based on RNN or a variant thereof is that the RNN network is based on the premise that the displayed semantic relation exists in the character strings, but in the character strings of the signature type, the characters are not in explicit semantic relation depending on the front and back. Therefore, how to efficiently identify the dynamic signature with Chinese characters becomes a problem that needs to be solved in the current industry.
Disclosure of Invention
An object of the present application is to provide a signature identification method and apparatus, so as to improve the accuracy of identifying a signature.
According to an aspect of the present application, there is provided a signature recognition method, wherein the method includes:
acquiring a signature image during signature;
preprocessing the signature image in sequence and positioning and dividing single characters based on a single-point multi-box detector SSD to obtain single character images corresponding to the single characters in the signature image, wherein the single characters comprise single surname characters and single name characters;
the method comprises the steps of recognizing single surname characters according to a surname recognition model based on a handwritten Chinese character recognition network to obtain a classification result of the single surname characters, and recognizing the single surname characters according to a first name recognition model based on the handwritten Chinese character recognition network to obtain a classification result of the single surname characters;
and obtaining a signature identification result of the signature image based on the classification result of the surname single character and the classification result of the first name single character.
Further, in the signature identification method, the preprocessing the signature image in sequence and the single-character positioning and segmentation based on the single-point multi-box detector SSD to obtain the single-character image corresponding to each single character in the signature image includes:
horizontally and vertically projecting the signature image to obtain the position information of a signature area in the signature image;
according to the position information of the signature region, cutting out a region of preset pixels around the position edge of the signature region from the signature image and carrying out binarization processing to obtain a cut signature image;
performing morphological operation of a preset area on the cut signature image to obtain a morphologically operated signature image;
zooming the signature image after the morphological operation, and placing the zoomed signature image in a preset area image with RGB three-channel pixel values all being preset pixel values to obtain a preprocessed signature image;
and carrying out single-character positioning segmentation on the preprocessed signature image based on the SSD to obtain a single-character image corresponding to each single character in the signature image.
Further, in the signature identification method, the scaling the signature image after the morphological operation and placing the scaled signature image in a preset region image in which RGB three-channel pixel values are all preset pixel values to obtain a preprocessed signature image includes:
if the length and width of the signature image after the morphological operation are w and h, respectively, the size of the preset area image is 400 × 80,
when in use
Scaling the morphological operated signature image from w × h to
On the scaled image, the length is 400 and the width is filled up and down, respectively
The RGB three-channel pixel values are pixels with preset pixel values;
when in use
Scaling the morphological operated signature image from w × h to
On the scaled image, the left and right fill lengths are respectively
The width is 80 and the RGB three-channel pixel values are all pixels with preset pixel values.
Further, in the signature identification method, the performing single-character positioning segmentation on the preprocessed signature image based on the SSD to obtain a single-character image corresponding to each single character in the signature image includes:
performing SSD-based single-character positioning segmentation on the preprocessed signature image to obtain a coordinate position of each candidate positioning frame in the signature image and a confidence degree of a single character contained in the candidate positioning frame, wherein the coordinate position of the candidate positioning frame is used for indicating the position of the candidate positioning frame in the preprocessed signature image;
determining the candidate positioning frame corresponding to the confidence coefficient greater than the preset confidence coefficient threshold value as a single character positioning frame;
and correspondingly cutting the preprocessed signature image according to the coordinate position of each single-character positioning frame to obtain a single-character image corresponding to each single character in the signature image.
Further, in the signature recognition method, the handwritten Chinese character recognition network comprises 19 layers, each of which comprises 10 convolution-batch normalization-linear unit layers with parameter correction, 5 pooling layers, 1 multi-scale hole convolution layer, 2 full-link layers and 1 regression model softmax layer;
the classification category number in the softmax layer in the surname recognition model based on the handwritten Chinese character recognition network is a first preset number, the classification category number in the softmax layer in the surname recognition model based on the handwritten Chinese character recognition network is a second preset number, and the first preset number is smaller than the second preset number.
Further, in the signature recognition method, the recognizing the surname single character according to a surname recognition model based on a handwritten chinese character recognition network to obtain a classification result of the surname single character includes:
identifying the surname single character according to a surname identification model based on a handwritten Chinese character identification network to obtain a plurality of surname classifications of the surname single character and a classification probability corresponding to each surname classification;
taking out a third preset number of surname categories with the highest probability from the surname categories;
and obtaining the classification result of the surname single characters based on the surname classes with the maximum probability and the corresponding classification probabilities thereof.
Further, in the signature recognition method, the recognizing the single name character according to a name recognition model based on a handwritten chinese character recognition network to obtain a classification result of the single name character includes:
identifying the single name character according to a name identification model based on a handwritten Chinese character identification network to obtain a plurality of name classifications of the single name character and a classification probability corresponding to each name classification;
taking out a third preset number of name categories with the highest probability from the plurality of name categories;
and obtaining the classification result of the single name character based on the third preset number of name categories with the maximum probability and the corresponding classification probability.
According to another aspect of the present application, there is also provided a computer readable medium having stored thereon computer readable instructions, which, when executed by a processor, cause the processor to implement the signature recognition method as described above.
According to another aspect of the present application, there is also provided a signature recognition apparatus, characterized in that the apparatus includes:
one or more processors;
a computer-readable medium for storing one or more computer-readable instructions,
when executed by the one or more processors, cause the one or more processors to implement the signature recognition method described above.
Compared with the prior art, the method and the device have the advantages that the signature image during signature is obtained firstly; then preprocessing the signature image in sequence and positioning and dividing Single characters based on a Single-point multi-box Detector (SSD) to obtain Single character images corresponding to the Single characters in the signature image, wherein the Single characters comprise Single characters of surname and Single characters of name; then, the surname single characters are identified according to a surname identification model based on a handwritten Chinese character identification network to obtain a classification result of the surname single characters, and the surname single characters are identified according to a surname identification model based on the handwritten Chinese character identification network to obtain a classification result of the surname single characters; and finally, obtaining a signature recognition result of the signature image based on the classification result of the surname single character and the classification result of the first name single character, realizing recognition of the single character in the signature image based on a surname recognition model of a handwritten Chinese character recognition network and a first name recognition model of the handwritten Chinese character recognition network, fusing the surname recognition model and the first name recognition model, and finishing signature recognition of the signature image in a cooperation manner, thereby improving the accuracy of signature recognition.
Drawings
Other features, objects and advantages of the present application will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings in which:
FIG. 1 illustrates a flow diagram of a signature recognition method in accordance with an aspect of the subject application;
FIG. 2 illustrates a schematic diagram of an original signature image in a signature recognition method according to an aspect of the subject application;
FIG. 3 is a diagram illustrating a partitioned single-character containing word single-character location box based on SSD after preprocessing a signature image in a signature recognition method according to an aspect of the present application;
FIG. 4 is a schematic diagram illustrating a network architecture of a handwritten Chinese character recognition network in a signature recognition method according to an aspect of the present application;
the same or similar reference numbers in the drawings identify the same or similar elements.
Detailed Description
The present application is described in further detail below with reference to the attached figures.
In a typical configuration of the present application, the terminal, the device serving the network, and the trusted party each include one or more processors (e.g., Central Processing Units (CPUs)), input/output interfaces, network interfaces, and memory.
The Memory may include volatile Memory in a computer readable medium, Random Access Memory (RAM), and/or nonvolatile Memory such as Read Only Memory (ROM) or flash Memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, Phase-Change RAM (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), flash Memory or other Memory technology, Compact Disc Read-Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, magnetic cassette tape, magnetic tape storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include non-transitory computer readable media (transmyedia), such as modulated data signals and carrier waves.
As shown in fig. 1, an aspect of the present application provides a schematic flowchart of a signature identification method, which is applied to a device for identifying each hanzi in a signature image containing the hanzi, where the device may be a mobile terminal with a touch screen function, and the mobile terminal includes a smart phone, an IPad computer, a personal computer, a mobile electronic touch device, and the like. The signature identification method comprises a step S11, a step S12, a step S13 and a step S14, wherein the method specifically comprises the following steps:
step S11, acquiring a signature image during signature; here, the signature image includes content of a signature performed by a user on a terminal device, for example, characters including a signature, such as kanji.
Aiming at the situations of insufficient generalization capability and missing detection and multiple detection of the traditional algorithm and the deep learning algorithm, step S12, sequentially preprocessing the signature image and positioning and dividing single characters based on a single-point multi-box detector SSD to obtain single character images corresponding to the single characters in the signature image, wherein the single characters comprise surname single characters and name single characters;
step S13, recognizing the surname single character according to a surname recognition model based on a handwritten Chinese character recognition network to obtain a classification result of the surname single character, and recognizing the surname single character according to a surname recognition model based on the handwritten Chinese character recognition network to obtain a classification result of the surname single character;
and step S14, obtaining a signature identification result of the signature image based on the classification result of the single surname character and the classification result of the single first name character.
Through the steps from S11 to S14, the recognition of the single character in the signature image by the surname recognition model based on the handwritten Chinese character recognition network and the name recognition model based on the handwritten Chinese character recognition network is realized, the two recognition models are fused, the signature recognition of the signature image is completed in a cooperation manner, and therefore the accuracy of the signature recognition is improved.
The specific steps of the signature recognition method of the present application are described in detail below from step S11 to step S14, respectively.
In step S11, a signature image I in the actual signature scene is acquired, and the length and width of the signature image I are W and H, respectively.
In the step S12, in the step S12, the signature image is sequentially preprocessed and single-character positioning and segmentation is performed based on a single-point multi-box detector SSD, so as to obtain a single-character image corresponding to each single character in the signature image, where the single character includes a single name character and a single name character, and the method specifically includes:
horizontally and vertically projecting the signature image to obtain the position information of a signature area in the signature image;
according to the position information of the signature region, cutting out a region of preset pixels around the position edge of the signature region from the signature image and carrying out binarization processing to obtain a cut signature image; here, the predetermined pixels include, but are not limited to, any number of pixels, and in a preferred embodiment of the present application, the predetermined pixels are preferably 5 pixels.
And performing morphological operation on the cut signature image to obtain a morphologically operated signature image, wherein the preset region includes but is not limited to a region with any range size and shape, and in a preferred embodiment of the present application, the preset region is preferably a 5 × 5 rectangular region.
Zooming the signature image after the morphological operation, and placing the zoomed signature image in a preset area image with RGB three-channel pixel values all being preset pixel values to obtain a preprocessed signature image;
and carrying out single-character positioning segmentation on the preprocessed signature image based on the SSD to obtain a single-character image corresponding to each single character in the signature image.
For example, in step S12, the signature image I is projected horizontally and vertically to locate the position of the signature region in the signature image I, where the position information of the signature region is [ x [ ]min，xmax，ymin，ymax]Wherein x isminIs the smallest abscissa value, x, of the abscissas of the signature regionmaxIs the maximum abscissa value, y, of the abscissas of the signature regionminIs the smallest abscissa value, y, in the ordinate of the signature regionmaxThe maximum ordinate value in the abscissa of the signature region.
Then, according to the position information of the signature area: [ x ] ofmin，xmax，ymin，ymax]Cutting out the signature area and presetting pixels around the edge of the signature area on a signature image ICarrying out binarization processing on the cut regions to obtain a cut signature image I corresponding to the signature image Icrop. In a preferred embodiment of the present application, the preset pixel is preferably 5 pixels, and then the clipped signature image I is obtainedcropExpressed as the following equation:
Icrop＝Ibinary[max(0，xmin-5)：min(xmax+5，W)，max(0，ymin-5)：min(ymax+ 5，H)]
then, in a preferred embodiment of the present application, the clipped signature image I is processedcropPerforming morphological operation on a rectangular area 5 × 5 to obtain a morphologically operated signature image I corresponding to the signature image IerodeWherein the morphologically manipulated signature image IerodeExpressed as the following equation:
Ierode＝f(Icrop，K(5，5))
where f (·) represents a morphological operation, and K (5, 5) represents a rectangular region with a kernel of 5 × 5 for the morphological operation.
Then, the signature image I after the morphological operation is carried outerodeZooming, and placing in a preset region image with RGB three-channel pixel values all being preset pixel values to obtain a preprocessed signature image Ipad。
Finally, the preprocessed signature image I is processedpadAnd performing single-character positioning segmentation based on the SSD to obtain a single-character image corresponding to each single character in the signature image, so as to sequentially and respectively perform preprocessing and SSD-based single-character positioning segmentation on the signature image I, thereby obtaining the single-character image corresponding to each single character in the signature image I.
In the step S12, when the signature image after the morphological operation is scaled and placed in a preset region image in which RGB three-channel pixel values are all preset pixel values to obtain a preprocessed signature image, the method specifically includes the following steps:
if the length and the width of the signature image after the morphological operation are dividedW and h, respectively, the size of the preset area image is 400 × 80, when
Scaling the morphological operated signature image from w × h to
On the scaled image, the length is 400 and the width is filled up and down, respectively
The RGB three-channel pixel values are pixels with preset pixel values; when in use
Scaling the morphological operated signature image from w × h to
On the scaled image, the left and right fill lengths are respectively
The width is 80 and the RGB three-channel pixel values are all pixels with preset pixel values.
Here, the preset pixel value includes, but is not limited to, any value of pixel value, and in a preferred embodiment of the present application, the preset pixel value is preferably 185, that is, the RGB three-channel pixel values of the filled pixel are all 185.
In the step S12, when the single-character positioning and segmentation based on the SSD is performed on the preprocessed signature image to obtain a single-character image corresponding to each single character in the signature image, the method specifically includes the following steps:
performing SSD-based single-character positioning segmentation on the preprocessed signature image to obtain a coordinate position of each candidate positioning frame in the signature image and a confidence degree of a single character contained in the candidate positioning frame, wherein the coordinate position of the candidate positioning frame is used for indicating the position of the candidate positioning frame in the preprocessed signature image;
determining the candidate positioning frame corresponding to the confidence coefficient greater than the preset confidence coefficient threshold value as a single character positioning frame; here, the preset confidence threshold includes, but is not limited to, any probability value greater than zero and less than 1, and in a preferred embodiment of the present application, the preset confidence threshold is preferably 0.2;
and correspondingly cutting the preprocessed signature image according to the coordinate position of each single-character positioning frame to obtain a single-character image corresponding to each single character in the signature image.
In the embodiment, the preprocessed signature image IpadInputting the preprocessed signature image I into an SSD networkpadPerforming SSD-based single-character positioning division, and outputting the coordinate position det of each candidate positioning frame after passing through the SSD networkcorAnd the confidence degree det of the single character (such as the character) contained in each candidate positioning frameconfWherein, the coordinate position det of each candidate positioning framecorExpressed as the following equation:
detcor＝[detcor_1，detcor_2，....]
confidence det of each candidate location boxconfExpressed as the following formula
detconf＝[detconf_1，detconf_2，....]
Wherein detcor_i＝[xi_min，xi_max，yi_min，yi_max]Signature image I representing ith candidate positioning frame after preprocessingpadPosition in the image, detconf_tRepresenting the probability of a single character contained in the ith candidate location box, the confidence det of said candidate location boxconf_iThe probability ranges are: det is not less than 0conf_i≤1。
In the preferred embodiment of the present application, the confidence degree det of each candidate location box is determinedconfThe coordinate position of the corresponding candidate location frame is determined to contain a single character (ratio) when the coordinate position is larger than 0.2Such as a word) and based on the smallest abscissa x in the coordinate position of each candidate position boxi_minAscending sorting is carried out to obtain the coordinate position det 'of each candidate positioning frame after sorting'corExpressed as the following equation:
det′cor＝[det′cor_1，det′cor_2，....](4)
respectively locating the positions of the single character locating boxes in the preprocessed signature image IpadThe corresponding image is cut out to obtain a divided single character image, the image cut out from each single character positioning frame corresponds to the single character image, as shown in fig. 2, fig. 2 is an original input signature image I, and each rectangular frame in fig. 3 is a single character positioning frame which is divided based on SSD to perform single character positioning and contains characters after preprocessing. Since the person name signature includes not only the last name but also the first name, the single character in the single character image obtained by the single character division in the step S12 includes not only the last name single character but also the first name single character.
After the single character image including the single character is divided in the signature image I in the step S12, before the single character image is input into the surname recognition model and the first name recognition model in the step S13, each single character image needs to be scaled to a preset pixel range, and the single character image of three channels RGB is converted into a single channel gray image to obtain the single character image for the input model to perform the single character recognition.
In the step S13, the Handwritten Chinese Character Recognition (HCCR) network includes 19 layers, each including 10 convolution-batch normalization-parameter-corrected linear unit layers, 5 pooling layers, 1 multi-scale hole convolution layer, 2 full-link layers, and 1 regression model softmax layer. The classification category number in the softmax layer in the surname recognition model based on the handwritten Chinese character recognition network is a first preset number, the classification category number in the softmax layer in the surname recognition model based on the handwritten Chinese character recognition network is a second preset number, and the first preset number is smaller than the second preset number. Here, the first preset number and the second preset number each include, but are not limited to, any number, and in a preferred embodiment of the present application, the first preset number may be preferably 430, that is, the number of classification categories in the softmax layer in the surname recognition model based on the handwritten chinese character recognition network is preferably 430; the second preset number is 7076, that is, the number of classification categories in the softmax layer in the name recognition model based on the handwritten Chinese character recognition network is 7076.
In the handwritten Chinese character recognition network shown in FIG. 4, 4 modules are designed, namely, a module Block A to a module Block D, wherein the module A is composed of a convolution (connected) -Batch Normalization (Batch Normalization) -parameter-corrected Linear Unit (Parametric corrected Linear Unit), and is followed by a pooling layer with a margin of 1 pixel and a convolution kernel size of 3 × and a step size of 2, the difference between the module A, the module B and the module C is that the number of convolution-Batch Normalization-parameter-corrected Linear units is 1, 2 and 3 in sequence, the convolution operations in the module A and the module C are all performed by using zero padding with a margin of 1 pixel, the convolution kernel size is 3 × and the step size is 1, the convolution module D is a multi-hole-scale convolution module, and is composed of multi-hole-scale convolution layer and multi-scale convolution layer feature map with a kernel of 1 pixel, the convolution kernel size is 3547, the convolution module D is a multi-hole-scale convolution module, and is composed of multi-hole-scale convolution layer and multi-scale convolution layer, and the feature map of the multi-scale convolution layer is composed of a multi-hole-scale convolution layer and multi-scale convolution layer, wherein the multi-hole-scale convolution kernel becomes a convolution kernel, the characteristic map is composed of a single-scale convolution kernel, the following convolution kernel, the convolution kernel becomes a convolution kernel, the calculation is performed by adding operation of a zero padding kernel, the number of a convolution kernel, the number of a single-scale convolution kernel, the convolution kernel, wherein the convolution kernel, the number of the module A, the convolution kernel, the zero padding kernel, the convolution kernel is respectively, the convolution kernel, the operation of the convolution kernel is respectively, the operation of the zero padding kernel is respectively, the operation of the zero padding of 1 pixel, the zero.
There are 430 categories for the counted surnames and 7076 categories for the first name in the signature identification process. That is, the category of non-surname can not appear in surname, and the accuracy of the signature is influenced by the accuracy of each single character of the signature; the number of classification categories of surnames in the signature is much smaller than that of the first names, and experiments show that the number of classification categories of surnames is: classification accuracy of 430 classes vs. number of classification classes of names: the accuracy rate of 7076 is about 5%. Therefore, a surname recognition model based on the HCCR19 network and a first name recognition model based on the HCCR19 network are trained. The surname recognition model and the first name recognition model are different in that in the process of classifying the number of categories based on the softmax function in the HCCR19 network, the surname recognition model based on the HCCR19 network and the first name recognition model based on the HCCR19 network are respectively classified into 430 categories and 7076 categories in sequence.
In step S13, the recognizing the surname single character according to a surname recognition model based on a handwritten chinese character recognition network to obtain a classification result of the surname single character specifically includes:
identifying the surname single character according to a surname identification model based on a handwritten Chinese character identification network to obtain a plurality of surname classifications of the surname single character and a classification probability corresponding to each surname classification;
taking out a third preset number of surname categories with the highest probability from the surname categories;
and obtaining the classification result of the surname single characters based on the surname classes with the maximum probability and the corresponding classification probabilities thereof.
In step S13, recognizing the single name character according to a name recognition model based on a handwritten chinese character recognition network to obtain a classification result of the single name character, which specifically includes:
identifying the single name character according to a name identification model based on a handwritten Chinese character identification network to obtain a plurality of name classifications of the single name character and a classification probability corresponding to each name classification;
taking out a third preset number of name categories with the highest probability from the plurality of name categories;
and obtaining the classification result of the single name character based on the third preset number of name categories with the maximum probability and the corresponding classification probability.
For example, a box det 'is located for the one-character'corEach of the one-character go-to boxes: det'cor＝[det′cor_1，det′cor_2，....]If the user signs according to the sequence from left to right, sorting the single character positioning boxes according to the ascending sequence from small to large of the minimum horizontal coordinate in each single character positioning box, and identifying the first single character positioning box (the surname is a single surname) or the first single character positioning boxes (the surname is a compound surname) by adopting a surname identification model based on an HCCR19 network to obtain a plurality of surname classifications of the surname single characters; and the single-character positioning boxes except the first or a plurality of single-character positioning boxes at the top in each single-character positioning box are subjected to name recognition by adopting a name recognition model based on HCCR19 network so as to obtain a plurality of name classifications of the name single characters. Certainly, if the user signs the signatures in the sequence from right to left, the single character positioning frames are sorted in descending order from large to small according to the maximum abscissa in each single character positioning frame, and the first single character positioning frame (the first surname is a single surname) or the first single character positioning frames (the last surname is a compound surname) are subjected to surname recognition by adopting a surname recognition model based on the HCCR19 network, so as to obtain a plurality of surname classifications of surname single characters; and divide each single character by the first most preceding in the boxOne-character positioning boxes except for one or more one-character positioning boxes are used for recognizing the first name by adopting a first name recognition model based on the HCCR19 network so as to obtain a plurality of first name classifications of the first name single character, and single-character recognition of the last name and the first name is realized.
The third predetermined number in this application includes, but is not limited to, any number, and in a preferred embodiment of this application, the third predetermined number is preferably 5. In the preferred embodiment, for a plurality of surname classifications of single surname characters and a plurality of first name classifications of single first name characters, respectively extracting 5 classes with the maximum classification probability in the plurality of classifications of each single character and the classification probability corresponding to the 5 classes as the classification result of the single character; the classification of each single character is represented as:
the classification probability corresponding to each classification of each single character is expressed as:
wherein staticvIndicates the classification of the single character in the v-th single character alignment box, p _ staticvThe classification probability of the classification of the single character in the v-th single-character orientation box is shown. Of course, the single character may be a single character of a surname or a single character of a first name.
In step S14, the classification result of the single surname character and the classification result of the single first name character are fused to obtain the signature recognition result of the signature image, so that the task of signature image recognition is completed in a manner of a surname and first name fusion model.
In the signature identification method provided by the embodiment of the application, signature identification of surnames and first names is fused, the preprocessed signature image is input into the SSD-based network to be subjected to single character positioning and segmentation, the coordinate position of a single character positioning frame is obtained, and the single character image is segmented according to the coordinate position, so that the SSD-based single character positioning and segmentation method has better robustness and higher segmentation accuracy. Meanwhile, on the basis of analyzing influence factors influencing the signature identification accuracy, the divided single-character images are sequenced according to the positions of the single-character positioning frames, the single characters in the signature images are identified by a surname identification model based on the HCCR19 network and a name identification model based on the HCCR19 network, the two identification models are fused, the signature identification of the signature images is completed in a cooperation mode, and therefore the signature identification accuracy is improved.
According to another aspect of the present application, there is also provided a computer readable medium having stored thereon computer readable instructions, which, when executed by a processor, cause the processor to implement the signature recognition method as described above.
According to another aspect of the present application, there is also provided a signature recognition apparatus, characterized by comprising:
one or more processors;
a computer-readable medium for storing one or more computer-readable instructions,
when executed by the one or more processors, cause the one or more processors to implement the signature recognition method described above.
Here, the details of each embodiment of the signature recognition device may specifically refer to the corresponding parts of the above-mentioned embodiments of the signature recognition method, and are not described herein again.
In summary, the present application first obtains a signature image during signature; then preprocessing the signature image in sequence and positioning and dividing single characters based on a single-point multi-box detector SSD to obtain single character images corresponding to the single characters in the signature image, wherein the single characters comprise single surname characters and single name characters; then, the surname single characters are identified according to a surname identification model based on a handwritten Chinese character identification network to obtain a classification result of the surname single characters, and the surname single characters are identified according to a surname identification model based on the handwritten Chinese character identification network to obtain a classification result of the surname single characters; and finally, obtaining a signature recognition result of the signature image based on the classification result of the surname single character and the classification result of the first name single character, realizing recognition of the single character in the signature image based on a surname recognition model of a handwritten Chinese character recognition network and a first name recognition model of the handwritten Chinese character recognition network, fusing the surname recognition model and the first name recognition model, and finishing signature recognition of the signature image in a cooperation manner, thereby improving the accuracy of signature recognition.
It should be noted that the present application may be implemented in software and/or a combination of software and hardware, for example, implemented using Application Specific Integrated Circuits (ASICs), general purpose computers or any other similar hardware devices. In one embodiment, the software programs of the present application may be executed by a processor to implement the steps or functions described above. Likewise, the software programs (including associated data structures) of the present application may be stored in a computer readable recording medium, such as RAM memory, magnetic or optical drive or diskette and the like. Additionally, some of the steps or functions of the present application may be implemented in hardware, for example, as circuitry that cooperates with the processor to perform various steps or functions.
In addition, some of the present application may be implemented as a computer program product, such as computer program instructions, which when executed by a computer, may invoke or provide methods and/or techniques in accordance with the present application through the operation of the computer. Program instructions which invoke the methods of the present application may be stored on a fixed or removable recording medium and/or transmitted via a data stream on a broadcast or other signal-bearing medium and/or stored within a working memory of a computer device operating in accordance with the program instructions. An embodiment according to the present application comprises an apparatus comprising a memory for storing computer program instructions and a processor for executing the program instructions, wherein the computer program instructions, when executed by the processor, trigger the apparatus to perform a method and/or a solution according to at least two embodiments of the present application.
It will be evident to those skilled in the art that the present application is not limited to the details of the foregoing illustrative embodiments, and that the present application may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the application being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned. Furthermore, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. At least two units or means recited in the apparatus claims may also be implemented by one unit or means in software or hardware. The terms first, second, etc. are used to denote names, but not any particular order.
Claims (9)
1. A signature recognition method, wherein the method comprises:
acquiring a signature image during signature;
preprocessing the signature image in sequence and positioning and dividing single characters based on a single-point multi-box detector SSD to obtain single character images corresponding to the single characters in the signature image, wherein the single characters comprise single surname characters and single name characters;
the method comprises the steps of recognizing single surname characters according to a surname recognition model based on a handwritten Chinese character recognition network to obtain a classification result of the single surname characters, and recognizing the single surname characters according to a first name recognition model based on the handwritten Chinese character recognition network to obtain a classification result of the single surname characters;
and obtaining a signature identification result of the signature image based on the classification result of the surname single character and the classification result of the first name single character.
2. The method of claim 1, wherein the sequentially preprocessing the signature image and the single-character positioning segmentation based on the single-point multi-box detector SSD to obtain a single-character image corresponding to each single character in the signature image, wherein the single character comprises a single surname character and a single first name character, comprises:
horizontally and vertically projecting the signature image to obtain the position information of a signature area in the signature image;
according to the position information of the signature region, cutting out a region of preset pixels around the position edge of the signature region from the signature image and carrying out binarization processing to obtain a cut signature image;
performing morphological operation of a preset area on the cut signature image to obtain a morphologically operated signature image;
zooming the signature image after the morphological operation, and placing the zoomed signature image in a preset area image with RGB three-channel pixel values all being preset pixel values to obtain a preprocessed signature image;
and carrying out single-character positioning segmentation on the preprocessed signature image based on the SSD to obtain a single-character image corresponding to each single character in the signature image.
3. The method according to claim 2, wherein the scaling the morphologically operated signature image and placing the scaled morphologically operated signature image in a preset region image with preset pixel values in RGB three channels to obtain a preprocessed signature image comprises:
if the length and width of the signature image after the morphological operation are w and h, respectively, the size of the preset area image is 400 × 80,
when in use
Scaling the morphological operated signature image from w × h to
After zoomingOn the image, the length is 400 and the width is 400 for the upward and downward filling respectively
The RGB three-channel pixel values are pixels with preset pixel values;
when in use
Scaling the morphological operated signature image from w × h to
On the scaled image, the left and right fill lengths are respectively
The width is 80 and the RGB three-channel pixel values are all pixels with preset pixel values.
4. The method of claim 2, wherein the performing single-character location segmentation based on the SSD on the preprocessed signature image to obtain a single-character image corresponding to each single character in the signature image comprises:
performing SSD-based single-character positioning segmentation on the preprocessed signature image to obtain a coordinate position of each candidate positioning frame in the signature image and a confidence degree of a single character contained in the candidate positioning frame, wherein the coordinate position of the candidate positioning frame is used for indicating the position of the candidate positioning frame in the preprocessed signature image;
determining the candidate positioning frame corresponding to the confidence coefficient greater than the preset confidence coefficient threshold value as a single character positioning frame;
and correspondingly cutting the preprocessed signature image according to the coordinate position of each single-character positioning frame to obtain a single-character image corresponding to each single character in the signature image.
5. The method of claim 1, wherein the handwritten chinese character recognition network comprises 19 layers, each comprising 10 convolutional-batch normalization-parametrically-modified linear unit layers, 5 pooling layers, 1 multi-scale hole convolutional layer, 2 fully-connected layers, and 1 regression model softmax layer;
the classification category number in the softmax layer in the surname recognition model based on the handwritten Chinese character recognition network is a first preset number, the classification category number in the softmax layer in the surname recognition model based on the handwritten Chinese character recognition network is a second preset number, and the first preset number is smaller than the second preset number.
6. The method of claim 1, wherein the recognizing the single surname character according to a surname recognition model based on a handwritten Chinese character recognition network to obtain a classification result of the single surname character comprises:
identifying the surname single character according to a surname identification model based on a handwritten Chinese character identification network to obtain a plurality of surname classifications of the surname single character and a classification probability corresponding to each surname classification;
taking out a third preset number of surname categories with the highest probability from the surname categories;
and obtaining the classification result of the surname single characters based on the surname classes with the maximum probability and the corresponding classification probabilities thereof.
7. The method of claim 1, wherein the recognizing the name single characters according to a name recognition model based on a handwritten Chinese character recognition network to obtain a classification result of the name single characters comprises:
identifying the single name character according to a name identification model based on a handwritten Chinese character identification network to obtain a plurality of name classifications of the single name character and a classification probability corresponding to each name classification;
taking out a third preset number of name categories with the highest probability from the plurality of name categories;
and obtaining the classification result of the single name character based on the third preset number of name categories with the maximum probability and the corresponding classification probability.
8. A computer readable medium having computer readable instructions stored thereon, which, when executed by a processor, cause the processor to implement the method of any one of claims 1 to 7.
9. A signature recognition device, characterized in that the device comprises:
one or more processors;
a computer-readable medium for storing one or more computer-readable instructions,
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911380443.7A CN111368632A (en) | 2019-12-27 | 2019-12-27 | Signature identification method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911380443.7A CN111368632A (en) | 2019-12-27 | 2019-12-27 | Signature identification method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111368632A true CN111368632A (en) | 2020-07-03 |
Family
ID=71212272
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911380443.7A Pending CN111368632A (en) | 2019-12-27 | 2019-12-27 | Signature identification method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111368632A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111753809A (en) * | 2020-07-10 | 2020-10-09 | 上海眼控科技股份有限公司 | Method and equipment for correcting handwritten signature |
| CN112580108A (en) * | 2020-12-10 | 2021-03-30 | 深圳证券信息有限公司 | Signature and seal integrity verification method and computer equipment |
| CN112651323A (en) * | 2020-12-22 | 2021-04-13 | 山东山大鸥玛软件股份有限公司 | Chinese handwriting recognition method and system based on text line detection |
| TWI777188B (en) * | 2020-07-07 | 2022-09-11 | 新光人壽保險股份有限公司 | Contract signature authentication method and device |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH06176206A (en) * | 1992-12-09 | 1994-06-24 | Casio Comput Co Ltd | Character recognizing device |
| CN107133616A (en) * | 2017-04-02 | 2017-09-05 | 南京汇川图像视觉技术有限公司 | A kind of non-division character locating and recognition methods based on deep learning |
| CN108229463A (en) * | 2018-02-07 | 2018-06-29 | 众安信息技术服务有限公司 | Character recognition method based on image |
| CN108898137A (en) * | 2018-05-25 | 2018-11-27 | 黄凯 | A kind of natural image character identifying method and system based on deep neural network |
| CN109102037A (en) * | 2018-06-04 | 2018-12-28 | 平安科技（深圳）有限公司 | Chinese model training, Chinese image-recognizing method, device, equipment and medium |
| CN109344883A (en) * | 2018-09-13 | 2019-02-15 | 西京学院 | Fruit tree diseases and pests recognition methods under a kind of complex background based on empty convolution |
| CN109740605A (en) * | 2018-12-07 | 2019-05-10 | 天津大学 | A kind of handwritten Chinese text recognition method based on CNN |
-
2019
- 2019-12-27 CN CN201911380443.7A patent/CN111368632A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH06176206A (en) * | 1992-12-09 | 1994-06-24 | Casio Comput Co Ltd | Character recognizing device |
| CN107133616A (en) * | 2017-04-02 | 2017-09-05 | 南京汇川图像视觉技术有限公司 | A kind of non-division character locating and recognition methods based on deep learning |
| CN108229463A (en) * | 2018-02-07 | 2018-06-29 | 众安信息技术服务有限公司 | Character recognition method based on image |
| CN108898137A (en) * | 2018-05-25 | 2018-11-27 | 黄凯 | A kind of natural image character identifying method and system based on deep neural network |
| CN109102037A (en) * | 2018-06-04 | 2018-12-28 | 平安科技（深圳）有限公司 | Chinese model training, Chinese image-recognizing method, device, equipment and medium |
| CN109344883A (en) * | 2018-09-13 | 2019-02-15 | 西京学院 | Fruit tree diseases and pests recognition methods under a kind of complex background based on empty convolution |
| CN109740605A (en) * | 2018-12-07 | 2019-05-10 | 天津大学 | A kind of handwritten Chinese text recognition method based on CNN |
Non-Patent Citations (3)
| Title |
|---|
| ZHUOYAO ZHONG ET AL.: "High Performance Offline Handwritten Chinese Character Recognition Using GoogLeNet and Directional Feature Maps", pages 1 - 5 * |
| 丁蒙 等: "卷积神经网络在手写字符识别中的应用研究", pages 40 - 42 * |
| 孙 俊 等: "空洞卷积结合全局池化的卷积神经网络识别作物幼苗与杂草", vol. 34, no. 11, pages 160 - 161 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TWI777188B (en) * | 2020-07-07 | 2022-09-11 | 新光人壽保險股份有限公司 | Contract signature authentication method and device |
| CN111753809A (en) * | 2020-07-10 | 2020-10-09 | 上海眼控科技股份有限公司 | Method and equipment for correcting handwritten signature |
| CN112580108A (en) * | 2020-12-10 | 2021-03-30 | 深圳证券信息有限公司 | Signature and seal integrity verification method and computer equipment |
| CN112580108B (en) * | 2020-12-10 | 2024-04-19 | 深圳证券信息有限公司 | Signature and seal integrity verification method and computer equipment |
| CN112651323A (en) * | 2020-12-22 | 2021-04-13 | 山东山大鸥玛软件股份有限公司 | Chinese handwriting recognition method and system based on text line detection |
| CN112651323B (en) * | 2020-12-22 | 2022-12-13 | 山东山大鸥玛软件股份有限公司 | Chinese handwriting recognition method and system based on text line detection |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109902622B (en) | Character detection and identification method for boarding check information verification | |
| CN109492643B (en) | Certificate identification method and device based on OCR, computer equipment and storage medium | |
| US7236632B2 (en) | Automated techniques for comparing contents of images | |
| WO2019174130A1 (en) | Bill recognition method, server, and computer readable storage medium | |
| CN111368632A (en) | Signature identification method and device | |
| WO2017202232A1 (en) | Business card content identification method, electronic device and storage medium | |
| CN109241861B (en) | Mathematical formula identification method, device, equipment and storage medium | |
| US20120224765A1 (en) | Text region detection system and method | |
| CN108154132A (en) | A kind of identity card text extraction method, system and equipment and storage medium | |
| CN111914698A (en) | Method and system for segmenting human body in image, electronic device and storage medium | |
| WO2021047484A1 (en) | Text recognition method and terminal device | |
| CN111552837A (en) | Animal video tag automatic generation method based on deep learning, terminal and medium | |
| CN110443184A (en) | ID card information extracting method, device and computer storage medium | |
| CN110796145B (en) | Multi-certificate segmentation association method and related equipment based on intelligent decision | |
| CN111415373A (en) | Target tracking and segmenting method, system and medium based on twin convolutional network | |
| CN111523537A (en) | Character recognition method, storage medium and system | |
| CN114581928A (en) | Form identification method and system | |
| CN108090728B (en) | Express information input method and system based on intelligent terminal | |
| CN112396060A (en) | Identity card identification method based on identity card segmentation model and related equipment thereof | |
| CN115953744A (en) | Vehicle identification tracking method based on deep learning | |
| CN113762027B (en) | Abnormal behavior identification method, device, equipment and storage medium | |
| US20220398399A1 (en) | Optical character recognition systems and methods for personal data extraction | |
| CN113780116A (en) | Invoice classification method and device, computer equipment and storage medium | |
| CN115424293A (en) | Living body detection method, and training method and device of living body detection model | |
| CN111213157A (en) | Express information input method and system based on intelligent terminal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |