WO2021164481A1 - Neural network model-based automatic handwritten signature verification method and device - Google Patents

Abstract

Disclosed in the present application are a neural network model-based automatic handwritten signature verification method and device. The method comprises: receiving a Chinese signature image input by a user, and preprocessing the Chinese signature image according to an image preprocessing model; placing in a convolutional neural network model to generate a first feature vector sequence; then inputting into a bidirectional recurrent neural network model to generate a second feature vector sequence; combining the first feature vector sequence and the second feature vector sequence in a preset combination mode to generate a third feature vector sequence; and placing in a recurrent neural network model to for classification and recognition to generate a name, and placing in a name verification model to verify the name. On the basis of the OCR technology, the present application solves the problem in the prior art that a handwritten signature of a user cannot be verified in a Chinese handwritten signature, and improves the automatic verification efficiency of Chinese handwritten signatures.

Description

Method and device for automatic verification of handwritten signature based on neural network model

This application claims the priority of the Chinese patent application filed with the Chinese Patent Office on February 18, 2020, the application number is 202010099130.0, and the invention title is "Method and Device for Automatic Verification of Handwritten Signature Based on Neural Network Model", all of which The content is incorporated in this application by reference.

Technical field

This application relates to the field of artificial intelligence technology, and in particular to a method and device for automatic verification of handwritten signatures based on a neural network model.

Background technique

At present, there are many technical methods for implementing electronic signatures, among which the pattern recognition of handwritten signatures belongs to the identification mark based on biometric statistics in electronic signatures. In the prior art, the signature scheme of the electronic signature link mainly includes two methods: check signature and handwritten signature on the tablet. The inventor found that checking the signature cannot verify whether the signature is the user's own signature; the current technology for writing tablet signatures only allows the user to sign by hand, but the content of the user's handwritten signature is not verified, and there is no verification method in the existing business scenario. The scenario of verifying whether the signature text in the user's handwritten signature is the name of the person.

Summary of the invention

The embodiments of the present application provide a method and device for automatic verification of handwritten signatures based on a neural network model, aiming to solve the problem of the inability to verify user handwritten signatures in Chinese handwritten signatures in the prior art.

In the first aspect, an embodiment of the present application provides a method for automatic verification of a handwritten signature based on a neural network model, which includes:

Receiving the Chinese signature image input by the user, and preprocessing the Chinese signature image according to the image preprocessing model to obtain the preprocessed Chinese signature image;

Placing the pre-processed Chinese signature image in a pre-trained convolutional neural network model to generate a first feature vector sequence;

Input the first feature vector sequence into a pre-trained bidirectional cyclic neural network model to generate a second feature vector sequence;

Splicing the first feature vector sequence and the second feature vector sequence according to a preset splicing manner to generate a third feature vector sequence;

Classifying and recognizing the third feature vector sequence according to the pre-trained recurrent neural network model, so as to recognize the name in the Chinese signature image;

The name is verified according to a preset name verification model to obtain a verification result of whether the Chinese signature image passes.

In the second aspect, an embodiment of the present application provides an automatic verification device of a handwritten signature based on a neural network model, which includes:

An image preprocessing unit for receiving a Chinese signature image input by a user, and preprocessing the Chinese signature image according to an image preprocessing model to obtain a preprocessed Chinese signature image;

A first feature vector sequence generating unit, configured to place the pre-processed Chinese signature image in a pre-trained convolutional neural network model to generate a first feature vector sequence;

The second feature vector sequence generating unit is configured to input the feature vector sequence into a pre-trained bidirectional cyclic neural network model and output it to generate a second feature vector sequence;

A third feature vector sequence generating unit, configured to combine the first feature vector sequence and the second feature vector sequence with the feature vectors at the corresponding positions according to a preset splicing manner to generate a third feature vector sequence;

The classification and recognition unit is configured to classify and recognize the third feature vector sequence according to the pre-trained recurrent neural network model, so as to recognize the name in the Chinese signature image;

The verification unit is configured to verify the name according to a preset name verification model to obtain a verification result of whether the Chinese signature image passes.

In a third aspect, an embodiment of the present application further provides a computer device, including a memory, a processor, and a computer program stored on the memory and running on the processor, wherein the processor executes the Perform the following steps in the computer program:

Receiving the Chinese signature image input by the user, and preprocessing the Chinese signature image according to the image preprocessing model to obtain the preprocessed Chinese signature image;

Placing the pre-processed Chinese signature image in a pre-trained convolutional neural network model to generate a first feature vector sequence;

Input the first feature vector sequence into a pre-trained bidirectional cyclic neural network model to generate a second feature vector sequence;

Splicing the first feature vector sequence and the second feature vector sequence according to a preset splicing manner to generate a third feature vector sequence;

Classifying and recognizing the third feature vector sequence according to the pre-trained recurrent neural network model, so as to recognize the name in the Chinese signature image;

The name is verified according to a preset name verification model to obtain a verification result of whether the Chinese signature image passes.

In a fourth aspect, the embodiments of the present application also provide a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor executes the following steps :

Receiving the Chinese signature image input by the user, and preprocessing the Chinese signature image according to the image preprocessing model to obtain the preprocessed Chinese signature image;

Placing the pre-processed Chinese signature image in a pre-trained convolutional neural network model to generate a first feature vector sequence;

Input the first feature vector sequence into a pre-trained bidirectional cyclic neural network model to generate a second feature vector sequence;

Splicing the first feature vector sequence and the second feature vector sequence according to a preset splicing manner to generate a third feature vector sequence;

Classifying and recognizing the third feature vector sequence according to the pre-trained recurrent neural network model, so as to recognize the name in the Chinese signature image;

The name is verified according to a preset name verification model to obtain a verification result of whether the Chinese signature image passes.

The embodiment of the application realizes end-to-end unconstrained Chinese character recognition under the premise of avoiding the segmentation of Chinese characters, solves the problem that the user's handwritten signature cannot be verified in the Chinese handwritten signature in the prior art, and improves The efficiency of automatic verification of Chinese handwritten signatures.

Description of the drawings

In order to explain the technical solutions of the embodiments of the present application more clearly, the following will briefly introduce the drawings used in the description of the embodiments. Obviously, the drawings in the following description are some embodiments of the present application. Ordinary technicians can obtain other drawings based on these drawings without creative work.

FIG. 1 is a schematic flowchart of a method for automatically verifying a handwritten signature based on a neural network model provided by an embodiment of the application;

2 is a schematic diagram of an application scenario of a method for automatic verification of a handwritten signature based on a neural network model provided by an embodiment of the application;

FIG. 3 is a schematic diagram of a sub-flow of a method for automatically verifying a handwritten signature based on a neural network model provided by an embodiment of the application;

4 is a schematic diagram of another sub-flow of the method for automatic verification of handwritten signatures based on a neural network model provided by an embodiment of the application;

FIG. 5 is a schematic diagram of another sub-flow of the method for automatic verification of handwritten signatures based on a neural network model provided by an embodiment of the application;

6 is a schematic diagram of another sub-flow of the method for automatic verification of handwritten signatures based on neural network models provided by an embodiment of the application;

FIG. 7 is a schematic block diagram of a device for automatic verification of handwritten signatures based on a neural network model provided by an embodiment of the application;

FIG. 8 is a schematic block diagram of subunits of a device for automatic verification of handwritten signatures based on a neural network model provided by an embodiment of the application;

9 is a schematic block diagram of another sub-unit of the device for automatic verification of handwritten signatures based on neural network model provided by an embodiment of the application;

10 is a schematic block diagram of another sub-unit of the device for automatic verification of handwritten signatures based on neural network model provided by an embodiment of the application;

11 is a schematic block diagram of another sub-unit of the apparatus for automatic verification of handwritten signatures based on neural network model provided by an embodiment of the application;

FIG. 12 is a schematic block diagram of a computer device provided by an embodiment of the application.

Detailed ways

The technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of them. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of this application.

Please refer to Figures 1 to 2. Figure 1 is a schematic flowchart of a method for automatic verification of a handwritten signature based on a neural network model provided by an embodiment of the application; Figure 2 is a handwritten signature based on a neural network model provided by an embodiment of the application Schematic diagram of the application scenario of the automatic verification method. The method for automatic verification of handwritten signatures based on the neural network model is applied to the management server 10, and multiple user terminals 20 and external service terminals 30 establish network connections with the management server 10 to perform automatic verification of Chinese handwritten signatures. Among them, the management server 10 is used to recognize the Chinese handwritten signature of the user terminal 20 and verify whether the user's signature is his own signature. This method is executed by the application software installed in the management server 10. It is converted into a picture and transmitted to the terminal device of the management server 10. The method for automatic verification of handwritten signatures based on the neural network model of this application is based on the neural network, which can realize the automatic recognition of signatures and then automatically verify the signature. This application can be applied to scenarios that require automatic verification of signature files, such as applications In the existing online loan process, the link of manually reviewing the contract signature can be omitted and can effectively prevent the signatory's handwritten signature from being inconsistent with the actual situation. While improving the user experience, this application can realize fully automatic credit business contract signing without manual review, thereby improving the accuracy of review and processing efficiency.

As shown in Fig. 1, the method includes steps S110 to S160.

S110: Receive a Chinese signature image input by a user, and preprocess the Chinese signature image according to an image preprocessing model to obtain a preprocessed Chinese signature image.

The Chinese signature image input by the user is received, and the Chinese signature image is preprocessed according to the image preprocessing model to obtain the preprocessed Chinese signature image. Because different users have different writing habits, even the same user, in different writing environments, the handwritten text written will be different, and different writing devices will also have an impact on the recognition of handwritten text. The image preprocessing model is used to adjust each character to a uniform proportion of height and width, while simulating the trajectory of the characters in the Chinese signature image to make it easier to recognize later, so the image preprocessing model performs the process on the Chinese signature image. Pretreatment is an essential step.

In an embodiment, the image preprocessing model includes nonlinear regularization rules and character segment interpolation processing rules. As shown in FIG. 3, step S110 includes substeps S111 and S112.

S111. Perform regularization processing on the Chinese signature image according to the non-linear regularization rule to enlarge or reduce the Chinese signature image.

Perform regularization processing on the Chinese signature image according to the non-linear regularization rule to enlarge or reduce the Chinese signature image. The non-linear regularization rule changes the size of the Chinese signature image while keeping the overall shape of the Chinese signature image unchanged, so that the characters in the Chinese signature image can be maintained to a greater extent. , The distortion is small. The conversion formula is:

Among them, W represents the original width of the character, H represents the original height of the character, W'represents the normalized width of the character, H'represents the normalized height of the character, and m represents The conversion ratio of the width of the character, n represents the conversion ratio of the height of the character.

Assuming that the character point coordinates in the Chinese signature image are (x, y), the linear regularization calculation formula corresponding to the point coordinates is:

Among them, (x, y) represents the original point coordinates of the character, (x′, y′) represents the original point coordinates of the character after regularization, m represents the conversion ratio of the width of the character, and n represents Is the conversion ratio of the height of the character.

S112: Perform interpolation processing on the normalized Chinese signature image according to the segment interpolation processing rule of the character to generate the preprocessed Chinese signature image.

The normalized Chinese signature image is subjected to interpolation processing according to the segmented interpolation processing rule of the character to generate the preprocessed Chinese signature image. Due to the user’s writing habits or the interference of the external environment, the written characters lack strokes or jitter, which makes the distance between the characters in the Chinese signature image inconsistent, which makes the handwritten text in the Chinese signature image more difficult in the subsequent process. Recognition, the segment interpolation processing rule of the character can make the handwritten text of the Chinese signature image easier to be recognized after the Chinese signature image is subjected to the interpolation processing.

The segment interpolation processing rule of the characters performs segment interpolation processing on the characters in the normalized Chinese signature image according to a specific function. The piecewise linear interpolation constructor is used to simulate the trajectory of characters, thereby improving the recognition rate of handwritten characters.

Suppose there is an interval [a, b], and there are points x ₀ , x ₁ , x ₂ ,...x _n on the interval, and the size is a=x ₀ ＜x ₁ ＜x ₂ ＜…＜x _n =b,f( x) is a function defined in the interval, and its corresponding function value is y ₀ , y ₁ , y ₂ , ... y _n , if the function φ(x) satisfies the following conditions: a. In the interval [a, b], φ (x) The function is a continuous function; b. In each subinterval [x _i , x _i+1 ] (i=0, 1, 2..., n-1), φ(x) is a polynomial of degree k; Then the φ(x) function is a piecewise k-th order interpolation polynomial of f(x) on the interval [a, b]. Among them, when k=1, it is piecewise linear interpolation, when k=2, it is piecewise parabolic interpolation.

Piecewise linear interpolation is also called piecewise first-order interpolation. In each subinterval [x _i , x _i+1 ] (i=0, 1, 2..., n-1), φ(x) is a first-order interpolation polynomial. The formula is:

Among them, x, x _i , x _i+1 are points on the interval [a, b], y, y _i , y _i+1 are functions f(x) corresponding to points x, x _i , x _i+1 The function value of φ(x) is a piecewise linear interpolation polynomial.

S120. Place the pre-processed Chinese signature image in a pre-trained convolutional neural network model to generate a first feature vector sequence.

The pre-processed Chinese signature image is placed in a pre-trained convolutional neural network model to generate a first feature vector sequence. Specifically, after the Chinese signature image is preprocessed, a pre-trained convolutional neural network model is used to perform feature extraction on it to generate a feature map of the preprocessed Chinese signature image, and the feature The feature vector corresponding to the image is classified according to the sequence of feature extraction of the pre-processed Chinese signature image in the pre-trained convolutional neural network model to generate a first feature vector sequence, that is, the first feature The vector sequence is composed of feature vectors corresponding to the feature map. The convolutional neural network model in this application is composed of an input layer, a hidden layer and a classification layer. The input layer is used to input the preprocessed Chinese signature image. The hidden layer removes the fully connected layer and only contains the convolution Layer and pooling layer.

In the specific implementation process, the Chinese handwritten signature needs to be recognized in this application. Since the characters in the handwritten signature are Chinese characters, and Chinese characters are hieroglyphs, China's national standard GB2312-80 for Chinese characters defines 3755 commonly used Chinese characters as one Class-level font library, 3008 Chinese characters are used as the second-level font library. Among all the Chinese characters in the second-level font library, about 2500 Chinese characters are used daily. Therefore, the training selected in this application is in the process of training the convolutional neural network. Text Chinese characters include 3755 commonly used Chinese characters in the first-level font library and 2975 Chinese characters in the second-level font library. Among them, the remaining "丨, 丿, 匚, 冂, 亻, 勹, 亠, 冫, 冖, 讠, 廴 in the second-class font , 凵, 厶, 艹, 彡, 犭, 忄, 帬, 彳, 扌, 囗, 宀, 艶, 囗, 氵, 纟, 屮, 灬, 钅, 礻, 疒, 衤" 33 uncommon fonts are eliminated, so The sample of Chinese characters selected in the process of training the convolutional neural network model in this application is 6,730 Chinese characters.

In an embodiment, as shown in FIG. 4, step S120 includes sub-steps S121, S122, and S123.

S121. Input the preprocessed Chinese signature image to the convolutional layer of the convolutional neural network model for convolution processing to obtain a vector matrix corresponding to the shallow features of the Chinese signature image.

In the process of processing the preprocessed Chinese signature image, the object of the convolution operation is a set of multi-dimensional matrices. At this time, the convolution is actually multiplying different parts of the matrix with the elements at each position of the convolution kernel matrix. Then sum up.

S122. Input the vector matrix corresponding to the shallow features of the Chinese signature image to the pooling layer of the convolutional neural network model for pooling processing to obtain the vector matrix corresponding to the deep features of the Chinese signature image.

Within the convolutional neural network model, pooling is a method of aggregating features of adjacent locations. The pooled features have certain translation and rotation invariance, at the same time it can reduce the number of features and increase the computational efficiency. Commonly used pooling operations generally have two methods: average pooling and maximum pooling, that is, take the maximum value of the corresponding area or average it as the element value after pooling, and finally obtain the vector matrix corresponding to the deep features of the Chinese signature image .

S123. Input the vector matrix corresponding to the deep features of the Chinese signature image to the output layer of the convolutional neural network model for classification processing to form the first feature vector sequence.

In the convolutional neural network model, the classification layer can be regarded as a specific activation function layer, used to classify the information obtained by the previous layer. In this embodiment, the activation function used in the activation function layer is the Sigmoid function, and the Sigmoid function is unlikely to cause the problem of gradient disappearance during the back propagation process, making it easier to train the convolutional neural network model.

For example, the signature in the Chinese signature image in this embodiment is "Ma Dashuai", then the processed Chinese signature image is set to be represented by a 64X96X1 three-dimensional array, and each dimension represents the processed Chinese signature. The height, width, and number of channels of the image. Set the convolutional neural network to consist of 14 layers and divide them into 5 groups. Each of the first four groups consists of a convolutional layer, an activation function layer, and a pooling layer. The last group contains only one convolutional layer and one activation. Function layer. The detailed configuration is shown in Table 1:

Table 1 Configuration of Convolutional Neural Network

S130. Input the first feature vector sequence into a pre-trained bidirectional cyclic neural network model to generate a second feature vector sequence.

The first feature vector sequence is input into a pre-trained bidirectional cyclic neural network model to generate a second feature vector sequence. Specifically, the pre-trained bidirectional cyclic neural network model is composed of two cyclic neural networks with opposite directions, and the two cyclic neural networks with opposite directions respectively receive the first feature vector sequence, so that the two cyclic neural networks are in opposite directions. The cyclic neural network respectively outputs a set of feature vector sequences to obtain two sets of feature vector sequences, and the two sets of feature vector sequences are spliced in a head-to-tail splicing manner to finally obtain the second feature vector sequence.

One of the advantages of the recurrent neural network is that it does not require the position of each element in the sequence target image during training and testing. Because the traditional recurrent neural network has a very good effect in dealing with time series problems, there are still some problems. The more serious one is that the gradient disappears or the gradient explodes easily.

In this embodiment, two reverse long and short-term memory networks receive the feature vector sequence output by the convolutional neural network and obtain an output feature vector sequence respectively. The feature vectors at the corresponding positions of the two output feature vector sequences are spliced to form The second feature vector sequence of the context information in the Chinese signature image is extracted. At the same time, in the process of training the bidirectional cyclic neural network model, the selected Chinese character samples are 6,730 Chinese characters.

For example, the step of calculating the output information of the memory network of a certain second feature vector is divided into five steps, where the second feature vector sequence is composed of multiple second feature vectors. ①Calculate the output information of the forget gate: f(t)=σ(W _f ×h _t-1 +U _f ×X _t +b _f ), where f(t) is the parameter value of the forget gate, 0≤f(t)≤ 1; σ is the activation function calculation symbol, σ can be specifically expressed as f(x)=(e ^x -e ^-x )/(e ^x +e ^-x ), then W _f ×h _t-1 +U _f × The calculation result of X _t + b _f can be calculated as x input activation function σ to obtain f(t); W _f , U _f and b _f are the parameter values of the formula in this cell; h _t-1 is the value of the previous cell Output gate information; X _{t is} the 1×M-dimensional vector input to the current cell in the feature information. If the current cell is the first cell in the long-short-term memory network, h _t-1 is "0". ②Calculate the input gate information: i(t)=σ(W _i ×h _t-1 +U _i ×X _t +b _i ); a(t)=tanh(W _a ×h _t-1 +U _a ×X _t + b _a), where i (t) is a parameter value input gate, 0≤i (t) ≤1; W i, U i, b i, W a, U a and b _a are present in the cells of the formula Parameter value, a(t) is the calculated input gate vector value, and a(t) is a 1×M-dimensional vector. ③Update cell memory information: C _t =C _t-1 ⊙f(t)+i(t)⊙a(t), C is the cell memory information accumulated in each calculation process, C _t is the output of the current cell Cell memory information, C _t-1 is the cell memory information output by the previous cell, ⊙ is the vector operator, and _{the calculation process of C t-1} ⊙f(t) is to divide the value of each dimension in the _{vector C t-1} Multiplying by f(t), the calculated vector dimension is the same as the _{dimension in the vector C t-1.} ④Calculate the output gate information: o(t)=σ(W _o ×h _t-1 +Uo×X _t +b _o ); h _t =o(t)⊙tanh(C _t ), o(t) is the output Gate parameter value, 0≤o(t)≤1; W _o , U _o and b _o are the parameter values of the formula in this cell, h _{t is} the output gate information of the cell, and h _t is a 1×M dimension The second feature vector. ⑤Calculate the output information of the current cell: y(t)=σ(V×h _t +c), V and c are the parameter values of the formula in this cell. Each cell is calculated to obtain an output information, and the output information of N cells can be combined to obtain the output information of the memory network of the second feature vector, and the output information of the memory network of the second feature vector is a 1×N-dimensional second Feature vector.

S140: Join the first feature vector sequence and the second feature vector sequence according to a preset splicing manner to generate a third feature vector sequence.

The first feature vector sequence and the second feature vector sequence are spliced according to a preset splicing manner to generate a third feature vector sequence. After the pre-processed Chinese signature image is extracted from the feature vector sequence of the convolutional neural network model and the feature vector sequence of the bidirectional cyclic neural network model, two sets of feature vector sequences can be obtained, that is, the first feature vector Sequence and the second feature vector sequence. In this embodiment, the first feature vector sequence and the second feature vector sequence are spliced end to end to obtain the third feature vector sequence.

S150. Classify and recognize the third feature vector sequence according to the pre-trained recurrent neural network model, so as to recognize the name in the Chinese signature image.

The third feature vector sequence is classified and recognized according to the pre-trained recurrent neural network model, so as to recognize the name in the Chinese signature image. One of the advantages of the recurrent neural network is that it does not require the position of each element in the sequence target image during training and testing. Specifically, in the process of training the cyclic neural network model, the selected Chinese character samples are 6,730 Chinese characters, and a stop character is also added. The stop character is mainly used to indicate the end of a character sequence, so this application uses The size of the character set of the training text is 6731.

In one embodiment, as shown in FIG. 5, step S150 includes sub-steps S151 to S153.

S151. Input the third feature vector sequence into the recurrent neural network model to generate a hidden state of the recurrent neural unit.

Input the third feature vector sequence into the cyclic neural network model to generate the hidden state of the cyclic neural unit, specifically, input the third feature vector sequence into the cyclic neural unit in the cyclic neural network model , The cyclic neural unit calculates the third feature vector sequence to obtain the hidden state of the cyclic neural unit, wherein the hidden state is used to record that the feature vector sequence is input to the cyclic neural unit at the current moment And output data from the cyclic neural unit.

S152. Receive the hidden state through a preset attention mechanism and search for the feature vector sequence related to the hidden state to obtain the feature vector sequence input at the next moment and input it to the recurrent neural unit to update the Hidden status.

In this embodiment, the hidden state is received through a preset attention mechanism and the feature vector sequence related to the hidden state is searched to obtain the feature vector sequence input at the next moment and input it to the recurrent neural unit To update the hidden state. Specifically, the attention mechanism receives the hidden state of the cyclic neural unit and searches for a feature vector sequence related to the hidden state of the cyclic neural unit, and compares the searched information related to the hidden state of the cyclic neural unit A weighted average calculation is performed on the feature vector sequence to obtain the feature vector sequence input to the recurrent neural unit at the next moment and input it to the recurrent neural unit to update the hidden state. The related formula is as follows:

Among them, c _i represents the feature vector sequence that can be used as the next input, h _j represents the feature vector sequence related to the hidden state, a _ij is the weight corresponding to _{h j} at time i _{, and T x} is the sequence corresponding to the hidden state. The number of state-related feature vector sequences, a _ij can be regarded as the probability of _{being selected by h j , and c i} is the expectation of the feature vector sequence related to the hidden state. The _{calculation formula of a ij} is as follows:

a _ij =σ(s _i-1 , j _j )

Among them, σ is an evaluation function used to estimate the correlation between the feature vector sequence and the hidden state in the recurrent neural network, and s _i-1 is the hidden state of the recurrent neural unit at time i-1.

S153. The classifier receives the hidden state and classifies and recognizes it to generate the name in the Chinese signature image.

The classifier receives the hidden state and classifies and recognizes it to generate the name in the Chinese signature image. Specifically, the classifier uses a Sigmoid function to normalize the hidden state in the recurrent neural unit to predict the name in the Chinese signature image.

In this embodiment, a three-layer BP neural network with a 6731-dimensional vector output is used as the classifier and the hyperbolic tangent function is used as the activation function. The calculation process is as follows:

h _i d _ij =tanh(w ₁₁ ×h _j +w ₁₂ ×s _t-1 +b)

e _ij = h _i d _ij × w ₂₁

Where h _i d _ij is the hidden state of the recurrent neural unit when _{h j is evaluated at time i, e ij} is the score, w ₁₁ and w ₁₂ are the second-layer weights of the BP neural network with three layers, and b is The bias term, w ₂₁ is the weight of the third layer of the BP neural network with three layers.

Since the output of the BP neural network with three layers is a real number, and the sum of the probabilities for selecting each feature vector sequence should be 1, this embodiment uses the Sigmoid function to _{normalize e ij} to obtain a _ij .

S160. Verify the name according to a preset name verification model to obtain a verification result of whether the Chinese signature image passes.

The name is verified according to a preset name verification model to obtain a verification result of whether the Chinese signature image passes. Due to the differences between Chinese characters and English, numbers and other character forms, when Chinese characters are stored in the terminal device, the Chinese characters are converted into corresponding character codes, and the character codes are stored in a binary manner. From the management server To read the corresponding Chinese character in 10, the stored character code needs to be obtained, and the character code is parsed through the corresponding relationship between the character code and the Chinese character to obtain the Chinese character character. The code conversion rule can convert the characters contained in the name to obtain the character code corresponding to each character. The regular expression can be used to verify the converted character code. When a character fails the check, then This information is fed back to the front end and the user is prompted to sign again.

In an embodiment, the name verification model includes code conversion rules and regular expressions. As shown in FIG. 6, step S160 includes sub-steps S161 and S162.

S161. Convert the name into a character code corresponding to the name according to the code conversion rule.

The name is converted into a character code corresponding to the name according to the code conversion rule. Specifically, the encoding conversion rules include the rules for converting each Chinese character, that is, each character corresponds to a character encoding, and the encoding conversion rules are the rules for converting characters using Unicode character set encoding, including UTF-8. There are multiple conversion rules such as encoding method and UTF-16 encoding method. UTF-8 encoding method corresponds to the character encoding of commonly used Chinese characters. UTF-8 is convenient for different computers to use the network to transmit text in different languages and encodings. UTF-16 encoding method Corresponding to the character encoding of Chinese characters other than UTF-8, the character encoding is represented by hexadecimal number.

S162. Verify the character code corresponding to the name with the character code corresponding to the name stored in advance according to the regular expression to obtain a verification result of whether the Chinese signature image passes.

According to the regular expression, the character code corresponding to the name and the character code corresponding to the name stored in advance are checked to obtain a check result of whether the Chinese signature image passes. Regular expression is a kind of logical formula for string manipulation, which is to use some pre-defined specific characters and the combination of these specific characters to form a regular string, and the regular string is used to express one of the string Kind of filtering logic. The regular expression can be used to verify the obtained name, which can be verified by whether the obtained name is consistent with the user information stored in the management server 10.

If the verification result of the name is passed, it is determined that the signature in the Chinese signature image is the user's own signature and the user signature is valid; if the verification result of the name is not passed, the management service terminal feeds back this information To the front end, prompt the user to re-sign.

The embodiment of the present application also provides an apparatus 100 for automatic verification of a handwritten signature based on a neural network model, which is used to execute any embodiment of the aforementioned method for automatic verification of a handwritten signature based on a neural network model. Specifically, please refer to FIG. 7, which is a schematic block diagram of the apparatus 100 for automatic verification of handwritten signatures based on a neural network model provided by an embodiment of the present application. The device can be configured in the management server 10.

As shown in FIG. 7, the device 100 for automatic verification of handwritten signatures based on the neural network model includes an image preprocessing unit 110, a first feature vector sequence generating unit 120, a second feature vector sequence generating unit 130, and a third feature vector sequence. The generation unit 140, the classification recognition unit 150, and the verification unit 160.

The image preprocessing unit 110 is configured to receive a Chinese signature image input by a user, and preprocess the Chinese signature image according to an image preprocessing model to obtain a preprocessed Chinese signature image.

In other embodiments of the invention, as shown in FIG. 8, the image preprocessing unit 110 includes a non-linear regularization unit 111 and a character segment interpolation processing unit 112.

The non-linear regularization unit 111 is configured to perform regularization processing on the Chinese signature image according to the non-linear regularization rule to enlarge or reduce the Chinese signature image.

The character segment interpolation processing unit 112 is configured to perform interpolation processing on the normalized Chinese signature image according to the character segment interpolation processing rule to generate the preprocessed Chinese signature image.

The first feature vector sequence generating unit 120 is configured to place the pre-processed Chinese signature image in a pre-trained convolutional neural network model to generate a first feature vector sequence.

In other embodiments of the invention, as shown in FIG. 9, the first feature vector sequence generating unit 120 includes an image shallow feature generating unit 121, an image deep feature generating unit 122 and a sequence generating unit 123.

The image shallow feature generation unit 121 is configured to input the preprocessed Chinese signature image to the convolutional layer of the convolutional neural network model for convolution processing to obtain the corresponding shallow feature of the Chinese signature image Vector matrix.

The image deep feature generation unit 122 is configured to input the vector matrix corresponding to the shallow features of the Chinese signature image to the pooling layer of the convolutional neural network model for pooling processing to obtain the deep features of the Chinese signature image The corresponding vector matrix.

The sequence generating unit 123 is configured to input the vector matrix corresponding to the deep features of the Chinese signature image to the output layer of the convolutional neural network model for classification processing to form the first feature vector sequence.

The second feature vector sequence generating unit 130 is configured to input the feature vector sequence into a pre-trained bidirectional cyclic neural network model and output it to generate a second feature vector sequence.

The third feature vector sequence generating unit 140 is configured to combine the first feature vector sequence and the second feature vector sequence with the feature vectors at the corresponding positions according to a preset stitching manner to generate a third feature vector sequence.

The classification and recognition unit 150 is configured to classify and recognize the third feature vector sequence according to the pre-trained recurrent neural network model, so as to recognize the name in the Chinese signature image.

In other embodiments of the invention, as shown in FIG. 10, the classification recognition unit 150 includes a hidden state generation unit 151, a hidden state update unit 152 and a name generation unit 153.

The hidden state generating unit 151 is configured to input the third feature vector sequence into the recurrent neural network model to generate the hidden state of the recurrent neural unit.

The hidden state update unit 152 is configured to receive the hidden state through a preset attention mechanism and search for a feature vector sequence related to the hidden state to obtain the feature vector sequence input at the next moment and input it into the loop The neural unit updates the hidden state.

The name generating unit 153 is configured to receive the hidden state through a classifier and classify and recognize it, so as to identify the name in the Chinese signature image.

The verification unit 160 is configured to verify the name according to a preset name verification model to obtain a verification result of whether the Chinese signature image passes.

In other embodiments of the invention, as shown in FIG. 11, the verification unit 160 includes a character code conversion unit 161 and a character code verification unit 162.

The character code conversion unit 161 is configured to convert the name into a character code corresponding to the name according to the code conversion rule.

The character encoding verification unit 162 is configured to verify the character encoding corresponding to the name with the corresponding character encoding of the name stored in advance according to the regular expression to obtain a verification of whether the Chinese signature image passes result.

Please refer to FIG. 12, which is a schematic block diagram of a computer device according to an embodiment of the present application.

Referring to FIG. 12, the computer device 500 includes a processor 502, a memory, and a network interface 505 connected through a system bus 501, where the memory may include a non-volatile storage medium 503 and an internal memory 504.

The non-volatile storage medium 503 can store an operating system 5031 and a computer program 5032. When the computer program 5032 is executed, the processor 502 can execute an automatic verification method of a handwritten signature based on a neural network model. The processor 502 is used to provide calculation and control capabilities, and support the operation of the entire computer device 500. The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute the automatic verification of the handwritten signature based on the neural network model. Methods. The network interface 505 is used for network communication, such as providing data information transmission. Those skilled in the art can understand that the structure shown in FIG. 12 is only a block diagram of part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device 500 to which the solution of the present application is applied. The specific computer device 500 may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

Wherein, the processor 502 is configured to run a computer program 5032 stored in a memory to implement any embodiment of the method for automatic verification of a handwritten signature based on a neural network model.

It should be understood that, in this embodiment of the application, the processor 502 may be a central processing unit (Central Processing Unit, CPU), and the processor 502 may also be other general-purpose processors 502, or digital signal processors 502 (Digital Signal Processors, DSPs). ), Application Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor 502 may be a microprocessor 502 or the processor 502 may also be any conventional processor 502 and the like.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the foregoing embodiments may be implemented by computer programs instructing relevant hardware. The computer program may be stored in a storage medium, and the storage medium may be a computer-readable storage medium. The computer program is executed by at least one processor in the computer system to implement the process steps of the foregoing method embodiment.

Therefore, this application also provides a computer-readable storage medium. The computer-readable storage medium may be non-volatile or volatile. The storage medium stores a computer program that, when executed by a processor, implements any embodiment of the method for automatic verification of a handwritten signature based on a neural network model.

The computer-readable storage medium may be a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a magnetic disk, or an optical disk, and other media that can store program codes.

In the several embodiments provided in this application, it should be understood that the disclosed devices, equipment, and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative, and the division of the units is only a logical function division, and there may be other division methods in actual implementation. Those skilled in the art can clearly understand that, for the convenience and conciseness of description, the specific working processes of the devices, equipment and units described above can refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

The above are only specific implementations of this application, but the protection scope of this application is not limited to this. Anyone familiar with the technical field can easily think of various equivalents within the technical scope disclosed in this application. Modifications or replacements, these modifications or replacements shall be covered within the protection scope of this application. Therefore, the protection scope of this application shall be subject to the protection scope of the claims.

Claims (20)

1. A method for automatic verification of handwritten signatures based on neural network models, including:

   Receiving the Chinese signature image input by the user, and preprocessing the Chinese signature image according to the image preprocessing model to obtain the preprocessed Chinese signature image;

   Placing the pre-processed Chinese signature image in a pre-trained convolutional neural network model to generate a first feature vector sequence;

   Input the first feature vector sequence into a pre-trained bidirectional cyclic neural network model to generate a second feature vector sequence;

   Splicing the first feature vector sequence and the second feature vector sequence according to a preset splicing manner to generate a third feature vector sequence;

   Classifying and recognizing the third feature vector sequence according to the pre-trained recurrent neural network model, so as to recognize the name in the Chinese signature image;

   The name is verified according to a preset name verification model to obtain a verification result of whether the Chinese signature image passes.
2. The method for automatic verification of handwritten signatures based on a neural network model according to claim 1, wherein the image preprocessing model includes non-linear regularization rules and character segment interpolation processing rules, and the image preprocessing is based on The model preprocesses the Chinese signature image, and the steps of obtaining the preprocessed Chinese signature image include:

   Performing regularization processing on the Chinese signature image according to the non-linear regularization rule to enlarge or reduce the Chinese signature image;

   The normalized Chinese signature image is subjected to interpolation processing according to the segmented interpolation processing rule of the character to generate the preprocessed Chinese signature image.
3. The method for automatic verification of handwritten signatures based on a neural network model according to claim 1, wherein the pre-processed Chinese signature image is placed in a pre-trained convolutional neural network model to generate the first A sequence of feature vectors, including:

   Input the preprocessed Chinese signature image to the convolutional layer of the convolutional neural network model for convolution processing to obtain a vector matrix corresponding to the shallow features of the Chinese signature image;

   Inputting the vector matrix corresponding to the shallow features of the Chinese signature image to the pooling layer of the convolutional neural network model for pooling processing to obtain the vector matrix corresponding to the deep features of the Chinese signature image;

   The vector matrix corresponding to the deep features of the Chinese signature image is input to the output layer of the convolutional neural network model for classification processing to form the first feature vector sequence.
4. The method for automatic verification of handwritten signatures based on a neural network model according to claim 1, wherein the first feature vector sequence and the second feature vector sequence are spliced according to a preset splicing manner to generate a first The sequence of three eigenvectors, including:

   The first feature vector sequence and the second feature vector sequence are spliced end to end to obtain a third feature vector sequence.
5. The method for automatic verification of handwritten signatures based on a neural network model according to claim 1, wherein the third feature vector sequence is classified and recognized according to a pre-trained recurrent neural network model to identify all The steps to describe the name in the Chinese signature image include:

   Inputting the third feature vector sequence into the recurrent neural network model to generate a hidden state of the recurrent neural unit;

   Receive the hidden state through a preset attention mechanism and search for the feature vector sequence related to the hidden state to obtain the feature vector sequence input at the next moment and input it to the recurrent neural unit to update the hidden state ；

   The hidden state is received by a classifier and classified and recognized to recognize the name in the Chinese signature image.
6. The method for automatic verification of handwritten signatures based on a neural network model according to claim 5, wherein the receiving the hidden state by a classifier and classifying and recognizing it includes:

   The Sigmoid function is used to normalize the hidden state, and the normalized hidden state is input into a BP neural network with three layers for classification and recognition.
7. The method for automatically verifying handwritten signatures based on a neural network model according to claim 1, wherein the name verification model includes code conversion rules and regular expressions, and the preset name verification model is The step of verifying the name to obtain the verification result of whether the Chinese signature image passes or not includes:

   Converting the name into a character code corresponding to the name according to the code conversion rule;

   According to the regular expression, the character code corresponding to the name and the character code corresponding to the name stored in advance are checked to obtain a check result of whether the Chinese signature image passes.
8. A device for automatically verifying handwritten signatures based on a neural network model, including:

   An image preprocessing unit for receiving a Chinese signature image input by a user, and preprocessing the Chinese signature image according to an image preprocessing model to obtain a preprocessed Chinese signature image;

   A first feature vector sequence generating unit, configured to place the pre-processed Chinese signature image in a pre-trained convolutional neural network model to generate a first feature vector sequence;

   The second feature vector sequence generating unit is configured to input the feature vector sequence into a pre-trained bidirectional cyclic neural network model and output it to generate a second feature vector sequence;

   A third feature vector sequence generating unit, configured to combine the first feature vector sequence and the second feature vector sequence with the feature vectors at the corresponding positions according to a preset splicing manner to generate a third feature vector sequence;

   The classification and recognition unit is configured to classify and recognize the third feature vector sequence according to the pre-trained recurrent neural network model, so as to recognize the name in the Chinese signature image;

   The verification unit is configured to verify the name according to a preset name verification model to obtain a verification result of whether the Chinese signature image passes.
9. A computer device includes a memory, a processor, and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the following steps when executing the computer program:

   Receiving the Chinese signature image input by the user, and preprocessing the Chinese signature image according to the image preprocessing model to obtain the preprocessed Chinese signature image;

   Placing the pre-processed Chinese signature image in a pre-trained convolutional neural network model to generate a first feature vector sequence;

   Input the first feature vector sequence into a pre-trained bidirectional cyclic neural network model to generate a second feature vector sequence;

   Splicing the first feature vector sequence and the second feature vector sequence according to a preset splicing manner to generate a third feature vector sequence;

   Classifying and recognizing the third feature vector sequence according to the pre-trained recurrent neural network model, so as to recognize the name in the Chinese signature image;

   The name is verified according to a preset name verification model to obtain a verification result of whether the Chinese signature image passes.
10. 9. The computer device according to claim 9, wherein the image preprocessing model includes nonlinear regularization rules and character segment interpolation processing rules, and the Chinese signature image is preprocessed according to the image preprocessing model, The steps to obtain the preprocessed Chinese signature image include:

    Performing regularization processing on the Chinese signature image according to the non-linear regularization rule to enlarge or reduce the Chinese signature image;

    The normalized Chinese signature image is subjected to interpolation processing according to the segmented interpolation processing rule of the character to generate the preprocessed Chinese signature image.
11. 9. The computer device according to claim 9, wherein said placing the pre-processed Chinese signature image in a pre-trained convolutional neural network model to generate a first feature vector sequence comprises:

    Input the preprocessed Chinese signature image to the convolutional layer of the convolutional neural network model for convolution processing to obtain a vector matrix corresponding to the shallow features of the Chinese signature image;

    Inputting the vector matrix corresponding to the shallow features of the Chinese signature image to the pooling layer of the convolutional neural network model for pooling processing to obtain the vector matrix corresponding to the deep features of the Chinese signature image;

    The vector matrix corresponding to the deep features of the Chinese signature image is input to the output layer of the convolutional neural network model for classification processing to form the first feature vector sequence.
12. 9. The computer device according to claim 9, wherein the splicing the first feature vector sequence and the second feature vector sequence according to a preset splicing manner to generate a third feature vector sequence comprises:

    The first feature vector sequence and the second feature vector sequence are spliced end to end to obtain a third feature vector sequence.
13. 9. The computer device according to claim 9, wherein the step of classifying and recognizing the third feature vector sequence according to a pre-trained recurrent neural network model to recognize the name in the Chinese signature image comprises:

    Inputting the third feature vector sequence into the recurrent neural network model to generate a hidden state of the recurrent neural unit;

    Receive the hidden state through a preset attention mechanism and search for the feature vector sequence related to the hidden state to obtain the feature vector sequence input at the next moment and input it to the recurrent neural unit to update the hidden state ；

    The hidden state is received by a classifier and classified and recognized to recognize the name in the Chinese signature image.
14. The computer device according to claim 13, wherein said receiving said hidden state by a classifier and classifying and identifying it comprises:

    The Sigmoid function is used to normalize the hidden state, and the normalized hidden state is input into a BP neural network with three layers for classification and recognition.
15. The computer device according to claim 9, wherein the name verification model includes code conversion rules and regular expressions, and the name is verified according to a preset name verification model to obtain the Chinese signature The steps of verifying the result of whether the image passes include:

    Converting the name into a character code corresponding to the name according to the code conversion rule;

    According to the regular expression, the character code corresponding to the name and the character code corresponding to the name stored in advance are checked to obtain a check result of whether the Chinese signature image passes.
16. A computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are executed:

    Receiving the Chinese signature image input by the user, and preprocessing the Chinese signature image according to the image preprocessing model to obtain the preprocessed Chinese signature image;

    Placing the pre-processed Chinese signature image in a pre-trained convolutional neural network model to generate a first feature vector sequence;

    Input the first feature vector sequence into a pre-trained bidirectional cyclic neural network model to generate a second feature vector sequence;

    Splicing the first feature vector sequence and the second feature vector sequence according to a preset splicing manner to generate a third feature vector sequence;

    Classifying and recognizing the third feature vector sequence according to the pre-trained recurrent neural network model, so as to recognize the name in the Chinese signature image;

    The name is verified according to a preset name verification model to obtain a verification result of whether the Chinese signature image passes.
17. The computer-readable storage medium according to claim 16, wherein the image preprocessing model includes nonlinear regularization rules and character segment interpolation processing rules, and the Chinese signature image is processed according to the image preprocessing model. Preprocessing, the steps of obtaining the preprocessed Chinese signature image include:

    Performing regularization processing on the Chinese signature image according to the non-linear regularization rule to enlarge or reduce the Chinese signature image;

    The normalized Chinese signature image is subjected to interpolation processing according to the segmented interpolation processing rule of the character to generate the preprocessed Chinese signature image.
18. 15. The computer-readable storage medium according to claim 16, wherein said placing the pre-processed Chinese signature image in a pre-trained convolutional neural network model to generate a first feature vector sequence comprises:

    Input the preprocessed Chinese signature image to the convolutional layer of the convolutional neural network model for convolution processing to obtain a vector matrix corresponding to the shallow features of the Chinese signature image;

    Inputting the vector matrix corresponding to the shallow features of the Chinese signature image to the pooling layer of the convolutional neural network model for pooling processing to obtain the vector matrix corresponding to the deep features of the Chinese signature image;

    The vector matrix corresponding to the deep features of the Chinese signature image is input to the output layer of the convolutional neural network model for classification processing to form the first feature vector sequence.
19. 15. The computer-readable storage medium according to claim 16, wherein the splicing the first feature vector sequence and the second feature vector sequence according to a preset splicing manner to generate a third feature vector sequence comprises:

    The first feature vector sequence and the second feature vector sequence are spliced end to end to obtain a third feature vector sequence.
20. The computer-readable storage medium according to claim 16, wherein the third feature vector sequence is classified and recognized according to a pre-trained recurrent neural network model to identify the name of the name in the Chinese signature image The steps include:

    Inputting the third feature vector sequence into the recurrent neural network model to generate a hidden state of the recurrent neural unit;

    Receive the hidden state through a preset attention mechanism and search for the feature vector sequence related to the hidden state to obtain the feature vector sequence input at the next moment and input it to the recurrent neural unit to update the hidden state ；

    The hidden state is received by a classifier and classified and recognized to recognize the name in the Chinese signature image.

Images