CN113408373B - Handwriting recognition method, handwriting recognition system, client and server - Google Patents
Handwriting recognition method, handwriting recognition system, client and server Download PDFInfo
- Publication number
- CN113408373B CN113408373B CN202110615845.1A CN202110615845A CN113408373B CN 113408373 B CN113408373 B CN 113408373B CN 202110615845 A CN202110615845 A CN 202110615845A CN 113408373 B CN113408373 B CN 113408373B
- Authority
- CN
- China
- Prior art keywords
- client
- handwriting recognition
- server
- similarity
- recognized
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 64
- 238000000605 extraction Methods 0.000 claims abstract description 57
- 230000015654 memory Effects 0.000 claims description 20
- 238000012549 training Methods 0.000 claims description 11
- 238000004590 computer program Methods 0.000 claims description 7
- 230000006870 function Effects 0.000 description 13
- 230000008569 process Effects 0.000 description 13
- 238000013528 artificial neural network Methods 0.000 description 12
- 238000004891 communication Methods 0.000 description 11
- 238000005516 engineering process Methods 0.000 description 6
- 238000012545 processing Methods 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000003062 neural network model Methods 0.000 description 3
- 230000003068 static effect Effects 0.000 description 3
- 238000013136 deep learning model Methods 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 238000003058 natural language processing Methods 0.000 description 2
- 238000002360 preparation method Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013145 classification model Methods 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000007667 floating Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- KJONHKAYOJNZEC-UHFFFAOYSA-N nitrazepam Chemical compound C12=CC([N+](=O)[O-])=CC=C2NC(=O)CN=C1C1=CC=CC=C1 KJONHKAYOJNZEC-UHFFFAOYSA-N 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
Abstract
The invention relates to a handwriting recognition method, a handwriting recognition system, a client and a server, wherein the method comprises the steps that the client sends a handwriting recognition request to the server; the server receives a handwriting recognition request sent by the client; the server side sends the feature extraction model and the feature value of the target text to the client side; the client receives the feature extraction model and the feature value of the target text; the client receives the handwritten text to be recognized, loads the feature extraction model to obtain the feature value of the handwritten text to be recognized, and matches the feature value of the handwritten text to be recognized with the feature value of the target text to recognize the handwritten text to be recognized. The technical scheme of the invention avoids the problem of data leakage caused by transmitting the client handwriting data to the server side.
Description
Technical Field
The present invention relates generally to the field of artificial intelligence applications. More particularly, the present invention relates to a handwriting recognition method, system, client side and server side.
Background
Currently, deep neural networks are being developed and applied in recent years as a technology in the field of machine learning. Deep learning models can be designed to perform a wide variety of tasks including word recognition, speech recognition, natural language processing, and computer vision processing, among others. The handwriting recognition technology based on the deep neural network has excellent recognition rate and accuracy. However, as the number of layers of the neural network increases, the number of recognizable words increases, and the number of parameters of the neural network model and the corresponding required storage space increases rapidly. The handwriting recognition technology with high storage consumption enables a person skilled in the art to generally deploy handwriting recognition service on a server side, the client side sends handwriting data to the server side, the server side recognizes the handwriting data by using a trained neural network model, and recognition results are returned to the client side. However, this client-server mode has a problem in that if the handwriting data of the client is uploaded to the server, the handwriting data of the client may leak out through the server; thus, this deployment mode risks leakage of customer handwriting data.
Disclosure of Invention
In order to solve at least the above problems, the present invention provides a handwriting recognition method, a system, a client and a server, wherein the server hands over a feature extraction function and a feature matching function related to the characteristics of the handwriting data of a client to the client, and the server cannot touch any handwriting data of a related user, so that the risk of data leakage caused by transmitting the handwriting data of the client to the server is avoided.
In a first aspect, the present invention provides a handwriting recognition method for a client, including: a handwriting recognition request is sent to a server; receiving a feature extraction model and a feature value of target characters returned by a server side; receiving a handwritten character to be recognized, loading the feature extraction model to obtain a feature value of the handwritten character to be recognized, and matching the feature value of the handwritten character to be recognized with the feature value of the target character to recognize the handwritten character to be recognized.
In one embodiment, the matching includes: and calculating the similarity between the characteristic value of the handwritten character to be recognized and the characteristic value of the target character.
In one embodiment, the similarity includes one or more of Euclidean distance similarity, cosine similarity, adjusted cosine similarity, and Pelson correlation coefficient
In one embodiment, further comprising: calculating the similarity between the characteristic value of the handwritten character to be identified and the characteristic values of more than two target characters, selecting the maximum similarity, and comparing the maximum similarity with a threshold value to identify the handwritten character to be identified.
In one embodiment, the receiving the handwritten text to be recognized, loading the feature extraction model, includes: the feature extraction model is loaded once every stroke the user completes.
In a second aspect, the present invention further provides a handwriting recognition method for a server, including: receiving a handwriting recognition request sent by a client; and sending the feature extraction model and the feature value of the target text to the client according to the handwriting recognition request.
In one embodiment, the feature value extraction model selects positive samples and negative samples in a model training stage, and includes: selecting the same characters as positive samples and selecting different characters as negative samples; and/or taking the content obtained by reducing one or two strokes of one word as a negative sample and taking the content obtained by increasing one or two strokes of one word as a negative sample.
In a third aspect, the present invention also provides a client, including: the handwriting recognition system comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the handwriting recognition method for the client in the first aspect and a plurality of embodiments when executing the computer program.
In a fourth aspect, the present invention further provides a server, including: the handwriting recognition method for the server side in the second aspect and the multiple embodiments is realized by the processor when the processor executes the computer program.
In a fifth aspect, the present invention further provides a handwriting recognition system, including the foregoing client and the foregoing server, where the server and the client are communicatively connected.
The invention is completed by handing over the feature extraction function and the feature matching function which relate to the hand-written data features of the clients in the server side to the clients, and the server side does not contact the hand-written data information of any users. On one hand, the method isolates the way of acquiring the user data by the server side, so that the risk of data leakage caused by transmitting the client handwriting data to the server is fundamentally avoided, and the safety of the user privacy data is effectively improved. On the other hand, the client identifies in a mode of loading the feature extraction model and matching the feature value, and the identification accuracy can be ensured; moreover, since the model and the target characteristic value come from the server side, the processing mode does not significantly increase the burden of the client side.
Drawings
The above, as well as additional purposes, features, and advantages of exemplary embodiments of the present invention will become readily apparent from the following detailed description when read in conjunction with the accompanying drawings. In the drawings, embodiments of the invention are illustrated by way of example and not by way of limitation, and like reference numerals refer to similar or corresponding parts and in which:
fig. 1 is a flowchart illustrating a handwriting recognition method for a client according to an embodiment of the present invention;
fig. 2 is a flowchart illustrating a handwriting recognition method for a server according to an embodiment of the present invention;
FIG. 3 is a schematic diagram illustrating a handwriting recognition system workflow in accordance with an embodiment of the invention;
FIG. 4 is a schematic diagram illustrating a workflow of handwriting recognition at a client according to an embodiment of the invention;
Fig. 5 is a schematic diagram showing the structure of a client for handwriting recognition according to an embodiment of the present invention;
fig. 6 is a schematic diagram showing a structure of a server side for handwriting recognition according to an embodiment of the present invention.
Detailed Description
Embodiments will now be described with reference to the accompanying drawings. It will be appreciated that for simplicity and clarity of illustration, where considered appropriate, reference numerals have been repeated among the figures to indicate corresponding or analogous elements. Furthermore, the application has been set forth in numerous specific details in order to provide a thorough understanding of the embodiments described herein. However, it will be understood by those of ordinary skill in the art that the embodiments described herein may be practiced without these specific details. In other instances, well-known methods, procedures, and components have not been described in detail so as not to obscure the embodiments described herein. Moreover, this description should not be taken as limiting the scope of the embodiments described herein.
Currently, deep neural networks are being developed and applied in recent years as a technology in the field of machine learning due to their high robustness and fault tolerance, distributed storage and learning capabilities, and ability to sufficiently approximate complex nonlinear relationships. Deep learning models can be designed to perform a wide variety of tasks, including word recognition, speech recognition, natural language processing (including handwriting recognition), computer vision processing, and the like, with excellent recognition and accuracy. However, as the number of layers of the neural network increases, the number of recognizable words increases, and the number of parameters of the neural network model and the corresponding required storage space increases rapidly. Such an application scenario, where a large amount of parameter data needs to be stored and loaded, has not been suitable for deploying handwriting recognition services in a local web browser.
Taking a single word picture recognition model as an example. If the model supports recognition of 2 ten thousand Chinese characters, 2 ten thousand output nodes are also required for the output of the neural network of the last layer of the model to completely represent the probability of the input picture of the model on each Chinese character classification. If the number of input nodes of the last layer of neural network is N, then N x 20000 x sizeof (float) bytes of memory would be required. Typically for chinese word recognition, N will take a value in the interval 128-1024, i.e. 9.7-78.1 mbytes of memory space will be used for the neural network only for the last layer. The foregoing merely identifies single-word pictures, and when multiple-word pictures or more complex forms are to be identified, the memory space required is more extensive. In order to solve the problem of large storage consumption, users usually deploy the handwriting recognition service on a server side, the client side sends the handwriting data of the users to the server side, and the server side processes the handwriting data of the users according to a handwriting recognition model based on a neural network to obtain a recognition result and returns the recognition result to the client side. The above-mentioned client-server mode has a problem that the client uploads the handwriting data of the user to the server, and the server obtains information (user signature handwriting information, etc.) containing the privacy of the user, so that this mode may cause risk that the server leaks the user data, and affects the security of the privacy information of the user.
Based on the above, the embodiment of the invention provides a handwriting recognition method for a client, a handwriting recognition method for a server, the client, the server and a corresponding handwriting recognition system. And part of functions (which can comprise a feature extraction function and a feature value comparison function) originally deployed in the handwriting recognition model based on the neural network at the server side are handed over to the client side for processing, the server side does not receive handwriting data of the user any more, the risk of leakage of the handwriting data of the user is avoided, and the safety of the user data is effectively ensured.
The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.
Fig. 1 is a flowchart illustrating a handwriting recognition method for a client according to an embodiment of the present invention. As shown in fig. 1, first, at step S101, a client transmits a handwriting recognition request to a server side. Then, at step S102, the feature extraction model returned by the server side and the feature value of the target text are received. Next, at step S103, a handwritten text to be recognized is received. Then, at step S104, a feature extraction model is loaded to obtain a feature value of the handwritten text to be recognized. Finally, at step S105, the feature value of the handwritten character to be recognized and the feature value of the target character are matched to recognize the handwritten character to be recognized. In one implementation scenario, when the feature value matching is performed in the step S105, the similarity between the feature value of the handwritten character to be recognized and the feature value of the target character may be calculated, or the feature value matching may be performed by calculating the feature distance between the two feature values. The feature extraction model received by the client only keeps the extracted feature part when the handwriting recognition model based on the neural network is simplified.
In one embodiment, the foregoing manner of matching the feature value of the handwritten text to be recognized with the feature value of the target text may include a manner of calculating a similarity to match. In one implementation scenario, the similarity may employ one or more of cosine similarity (Cosine Similarity), adjusted cosine similarity (Adjusted Cosine Similarity), pearson correlation coefficient (Pearson Correlation Coefficient), jaccard similarity coefficient (Jaccard Coefficient), euclidean distance (Euclidean Distance), and the like. After the similarity of the two characteristic values is calculated by the method, if the similarity is larger than a similarity threshold, the matching is considered to be successful, and the identification of the handwritten characters to be identified is completed. In one implementation scenario, taking cosine similarity as an example, it may be calculated using the following similarity function SIM (x 1,x2):
Where, taking e=1e-8, x1, x2 can be two eigenvalues. The eigenvalue may be a vector. It will be appreciated that the choice of similarity function described above is merely illustrative and not limiting and may be adapted according to the actual requirements.
The manner in which the feature value matching is performed including calculating the similarity of the feature values is described above, and the processing procedure when a plurality of similarity values are calculated is described next. In an application scene, if the client receives a plurality of characteristic values of the target text, calculating the similarity between the characteristic values of the handwritten text to be recognized and the characteristic values of the target text, selecting the maximum similarity, comparing the maximum similarity with a threshold, and when the maximum similarity is higher than the threshold, the corresponding handwritten text to be recognized passes. The following description will take a "still" word as an example. The client receives the characteristic value of the 'quiet' word of one or more fonts including Song Ti, regular script, clerical script and the like, namely, each font corresponds to one characteristic value, the characteristic value of the 'quiet' word can comprise F Song dynasty style ,F Regular script ,F Song dynasty style and the like, at the moment, the client extracts the characteristic value of one word written by the user, then calculates the similarity between the characteristic value of the word and the characteristic values of the fonts and performs similarity sorting, selects the characteristic value with the largest similarity and compares with a threshold value, and if the threshold value is exceeded, the character recognition is judged to pass.
In one embodiment, the timing of the client loading the aforementioned feature extraction model for feature extraction may be in a variety of ways. Taking recognition of a single-word picture as an example, each time a user performs writing operation, the client loads the feature extraction model once, obtains the feature value of the handwritten word to be recognized at the moment, confirms the similarity between the feature value of the handwritten word to be recognized at the moment and the feature value of the target word, and judges that the recognition is passed if the similarity is higher than a threshold value at the moment, so that the recognition of one word is completed. In other words, in the process of writing a word by the user, the feature extraction model is loaded multiple times, that is, each time a stroke is input (the stroke is not a standard stroke, and is regarded as a stroke from pen down to pen up based on the habit of the user), the feature extraction model needs to be called multiple times. In order to reduce the call frequency of the feature extraction model, in a human implementation scene, the client can also determine that the user has completed a word by enabling the user to manually confirm or by delaying confirmation, loading the feature extraction model at this time, obtaining the feature value of the handwritten word to be recognized at this time, confirming the similarity of the feature value of the handwritten word to be recognized at this time and the feature value of the target word, and if the similarity at this time is higher than a threshold value, judging that the recognition is passed, namely completing the recognition of the word.
The handwriting recognition method for a client of the present invention is exemplarily described above with reference to fig. 1. The overall exemplary description of the functions assumed by the server side in handwriting recognition in the present invention is described below with reference to fig. 2.
Fig. 2 is a flowchart illustrating a server-side operation method according to an embodiment of the present invention. As shown in fig. 2, first, at step S201, a server receives a handwriting recognition request transmitted by a client. The handwriting recognition request can be a command or can contain target text information to be recognized. If the target text information to be identified is contained, the server analyzes the request and invokes the characteristic value of the target text from the database of the server. Next, at S202, the server side transmits the feature extraction model and the feature value of the target text to the client side according to the recognition request. In one scenario, the feature value of the target text sent by the client may include the corresponding target text in the recognition request, or may include other pre-stored text feature values. The following description will take a "still" word as an example. The server side can send the characteristic value of the static character of the Song body, and can also send the characteristic value of various fonts including Song Ti, regular script, clerical script and the like.
In order to make the feature extraction model more accurately recognize characters of different handwriting styles and distinguish similar characters. In one embodiment, the positive and negative samples selected by the feature value extraction model in the model training stage may include selecting, as the positive sample, a word with a lower similarity in the same word, and selecting, as the negative sample, a word with a higher similarity in different words. Taking a training sample of selecting a 'day' word as an example, different fonts of the 'day' word can be selected as positive samples, and 'eye', 'stand', 'old' and 'field' are selected as negative samples, and the selected positive samples and negative samples are utilized to train the feature extraction model so as to obtain a more accurate feature extraction model.
Further, in order to more accurately distinguish between the text that is not written, the text that is written at the place, and the text that has begun to be written with the next text stroke, two sets of negative examples are added during model training in the following manner. Specifically, the content obtained by reducing one or two strokes of one word is used as a negative sample, and the content obtained by increasing one or two strokes of one word and the word itself are used as negative samples. The two groups of negative samples increase the characteristic distance between the written characters and the characters, and can effectively improve the accuracy of the characteristic extraction model. Also taking the training sample of selecting the "day" word as an example, selecting the "mouth", "E" and "two" obtained by reducing one or two strokes of the "day" word as a negative sample, selecting the "field", "old", "denier" and "Zhu" obtained by adding one or two strokes of the "day" word as a negative sample, and training the feature extraction model so that the feature extraction model can accurately extract the feature value of the text.
The above description is made with reference to fig. 1-2 for the method flow of handwriting recognition performed by the client and the server of the present invention, respectively. In view of the above detailed description of the functions executed by the client and the server, when the handwriting recognition method of the present invention is used by a client, the client only needs to load the feature vector corresponding to the target text with the feature extraction model with extremely small calculation amount and parameter amount, and the handwriting collection component of the client is matched to recognize the handwriting text of the user. The server side does not acquire handwriting data of the user any more, and the whole recognition process can be completed at the client side. In order to better understand the principle and process of handwriting recognition by the client, the invention also provides a handwriting recognition method by adopting the handwriting recognition system. An exemplary description of this handwriting recognition method is described below with reference to fig. 3.
Fig. 3 is a flowchart showing the operation principle of the handwriting recognition method according to the embodiment of the present invention.
As shown in fig. 3, in one embodiment, first, at step S301, a client transmits a handwriting recognition request to a server side. In one implementation scenario, the handwriting recognition request may include information of the target text to be recognized. The target text may include one or more text messages or may be a control command to begin recognition. Taking the recognition process of the user signature as an example, the client sends a handwriting recognition request to the server, where the handwriting recognition request may include a to-be-signed kanji string S with a length n to be recognized, and the string S may include S 1、S2、S3、…、Si、…Sn.
Next, after receiving the handwriting recognition request sent by the client, the server sends the feature extraction model and the feature value of the target text to the client according to the handwriting recognition request in step S401. In one implementation scenario, if the recognition request includes the target text information to be recognized, the server may send the feature value and the feature extraction model of the corresponding target text to the client. Correspondingly, if the identification request is only a control command for starting identification, the server side can send the characteristic values of all words in the memory to the client side. In addition, the characteristic value of the target text may include various cases, taking the target text as "safe" as an example, and the characteristic value of the target text may include one or more of characteristic values obtained by extracting the text such as Song Ti, regular script, clerical script, and the like. In an application scenario, the client needs to obtain the feature value corresponding to the computation model (i.e., the feature extraction model, the M web model) and the Chinese character to be signed from the server. After receiving the to-be-signed kanji string S with the length n, the server sends the feature code F 1、F2、F3、…、Fi、…、Fn of the standard kanji (e.g. Song Ti) corresponding to each kanji S 1、S2、S3、…、Si、…Sn therein and the M web model to the client. Taking the recognition process of the user signature as an example, the ith Chinese character S i to be recognized is set as the current target character to be recognized, and the client receives the standard Chinese character characteristic value F i returned by the server side for subsequent characteristic matching.
Then, the client receives the feature extraction model and the feature value of the target text, and at step S302, the client receives the handwritten text to be recognized. In one implementation scenario, a client may obtain handwritten text to be recognized by connecting to a handwriting input device, such as a handwriting pad, drawing pad, or a palm top computer (PDA). The handwriting to be identified, which is obtained by the client through the handwriting input device, may be content obtained by receiving a user writing operation, where the user writing operation may be: 1) Writing a stroke on the canvas of the client (from pen down to pen up is regarded as a stroke); 2) Clicking the empty button of the client clears the strokes on the current canvas.
Then, in step S303, the client loads the feature extraction model to obtain a feature value of the handwritten text to be recognized. Specifically, after receiving the text written by the user, the track data written in the current canvas may be converted into a picture I current, and the client inputs the picture of the handwritten text to be recognized into the feature extraction model M web, and loads the feature extraction model to output the feature value F current corresponding to the handwritten text to be recognized. Wherein I current＝Mweb(Icurrent). Further, the client receives the text written by the user, and the user triggers text recognition once every time when writing one stroke, namely, the M web model is loaded once. The aforementioned feature extraction model is a model trained in the server side (described in detail below).
Again, at step S304, the client calculates the similarity of the feature value of the handwritten text to be recognized and the feature value of the target text. Specifically, the client may perform a similarity operation on the calculated feature value F current and a standard kanji feature value F i corresponding to the current kanji to be written S i to obtain a similarity S, where the similarity may be a cosine similarity, s=sim (F i,Fcurrent).
Finally, in step S305, the client matches the feature value of the handwritten text to be recognized with the feature value of the target text to recognize the handwritten text to be recognized. Specifically, the client may determine whether the calculated similarity is greater than a similarity threshold, and if so, determine that the text recognition is passed. For example, taking T as a similarity threshold, if the similarity S exceeds the threshold T, then the user is considered to write through, the current writing track is recorded as the writing result of the kanji S i, and the process goes to step S506; if S does not exceed the threshold T, it is indicated that the content currently written by the user does not match the target word S i, and possible reasons may include: 1) The user has not written the current word yet; 2) The user writes the content in a sloppy way and the identification fails. Further, the client may also determine whether the feature distance obtained by the calculation is smaller than a threshold value, and if so, determine that the text recognition is passed. In addition, the target text to be recognized may include a plurality of text, after completing one handwriting recognition, at step S306, the client checks whether there is any text that is not recognized, and if so, the processes of steps S302 to S305 are performed again according to the received text to be recognized.
The handwriting recognition method for the client and the server of the present invention is described in detail above with reference to fig. 1-3, and the principles of the method of the present invention are further described below by taking kanji signature recognition as an example. Before describing the method principle of the invention, a feature extraction model trained by a server is described first. In order to provide the client with a feature extraction model M web (hereinafter referred to as M web model) with very small parameters and calculation, the model may be designed by using CNN as a basic unit, so as to extract feature values of characters. The finally designed model architecture can be based on MobileNetV and ResNet, has a small number of parameters and has good feature extraction and generalization capability.
When the feature extraction model is trained, an auxiliary model is constructed by connecting the output of the M web model with a traditional linear character classification model, and the auxiliary model is applied to the feature value of the standard font. And extracting features of the standard fonts (such as Song Ti) by using the trained auxiliary model to serve as character standard features (feature values of target characters) and storing the character standard features at a server side. Training the M web model based on the character standard features can accelerate the convergence rate of the M web model. In the training process of the M web model, characters with lower similarity in the same characters are selected as positive samples, so that the characteristic distance between the characters is smaller; selecting characters with higher similarity in different characters as negative samples, so that the characteristic distance between the characters is larger; through the selection strategies of the positive sample and the negative sample, the model can more accurately recognize characters with different handwriting styles and distinguish similar characters.
In order to further increase the accuracy of the model, besides the above distinction of different handwriting styles, distinction of whether the writing of the characters is completed can be included. The method for distinguishing whether the text is completed or not in the embodiment comprises the following steps: the following two sets of negative samples can be added during the training process. A set of negative examples may be to reduce the content of one or two strokes by one word as a negative example, so that the feature distance between them is larger; another set of negative examples may be to add one letter to the content of one or two strokes as a negative example, such that the feature distance between them is larger. Wherein the increased strokes are from the dataset by random extraction of text strokes. By using the feature extraction model obtained by the added negative sample training, characters which do not finish writing, characters which finish writing and characters which start writing the next character strokes can be distinguished more accurately.
Further, the positive and negative samples specified above can be utilized in the present invention, and the following loss function is used to train M web:
wherein: lambda has a median value of 0 in positive samples and 1 in negative samples; f is the character standard feature of the target character; Is the characteristic value output by the characteristic extraction model,/> I is the picture data of the incoming model.
Further, in order to simplify the volume of the feature extraction model, the M web model uses two bytes to approximate floating point weight data in the representation model through quantization technology to compress the model volume when storing, and the simplified model is integrated at the client.
The above is a description of a feature extraction model trained by a server, and the principles of a method for implementing handwriting recognition by communication between a client and the server according to the present invention are described below with reference to fig. 4 by taking handwriting recognition at the client as an example. The method principle of the invention mainly comprises two groups of processes of an identification preparation stage and an identification stage.
In the recognition preparation stage, the client side needs to acquire the feature value corresponding to the computing model (namely the feature extraction model, abbreviated as M web model) and the Chinese character to be signed from the server side. In the recognition stage, the client receives the handwritten characters of the user, and the user triggers a character recognition method flow when writing one stroke. As shown in fig. 4, taking n handwritten characters to be recognized as an example, n=3 may be taken. The method for enabling the client to start handwriting recognition comprises the following steps of: step S501: setting the ith character to be identified as the current target character to be identified, namely the target character, and receiving the characteristic value corresponding to the target character returned by the server side by the client for subsequent calculation and use. Step S502: the client receives the handwritten text written by the user, namely the handwritten text to be recognized. Step S503: and calculating to obtain the characteristic value of the ith handwritten character to be recognized by using the M web model. Step S504: and carrying out similarity operation on the calculated characteristic value of the ith handwritten character to be recognized and the characteristic value corresponding to the target character to obtain similarity. Step S505: and comparing the calculated similarity with a similarity threshold value to judge whether the identification is passed or not. If the identification is judged to pass, the step is skipped to step S506, and if the identification is judged not to pass, the step is skipped to step S502, and the user is waited for further operation. Step S506: judging whether all the characters are recognized, judging whether the condition is i is an index number corresponding to the last character, if so, ending the flow, otherwise, turning to step S507: increase i by 1. And (3) after the identification of one character is finished, increasing the count value i by 1 until i=3, and judging that the identification of all characters is finished. It will be appreciated that the foregoing process of loop identification is merely exemplary and not limiting and may be adapted according to actual requirements.
The above is a detailed description of the handwriting recognition method of the present invention. When the handwriting recognition method is used, a client only needs to load a characteristic extraction model with small calculated quantity and parameter quantity in the handwriting recognition model based on the neural network and a characteristic vector corresponding to the target text on the client, and the handwriting recognition method is matched with a handwriting acquisition component of the client to recognize the handwriting text. The function related to the client handwriting data in the whole handwriting recognition process is completed at the client, so that the server cannot obtain the client handwriting data, leakage of the client handwriting data is effectively avoided, and safety of user privacy information is improved.
As another aspect of the present invention, an embodiment of the present invention further provides a client for handwriting recognition, as shown in fig. 5, including a processor, a memory, a communication interface, and a communication bus, where the processor, the memory, and the communication interface complete communication with each other through the communication bus, and the processor executes steps of a method implemented by the foregoing client. The method implemented by the client is not described in detail herein since it has been described in the foregoing.
As still another aspect of the present invention, an embodiment of the present invention further provides a server for handwriting recognition as shown in fig. 6, where the server includes a processor, a memory, a communication interface, and a communication bus, where the processor, the memory, and the communication interface complete communication with each other through the communication bus, and the processor executes steps of a method implemented by the server. The method implemented by the server is not described in detail herein since it has been described in the foregoing.
As still another aspect of the present invention, an embodiment of the present invention further provides a handwriting recognition system, including a client as shown in fig. 5 and a server as shown in fig. 6.
In addition, the memories of the server side and the client side of the present invention may include a readable storage medium in which an application program for executing the above-described method is stored. The readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. For example, the computer-readable storage medium may be any suitable magnetic or magneto-optical storage medium, such as, for example, resistance change Memory RRAM (Resistive Random Access Memory), dynamic Random Access Memory DRAM (Dynamic Random Access Memory), static Random Access Memory SRAM (Static Random-Access Memory), enhanced dynamic Random Access Memory EDRAM (ENHANCED DYNAMIC Random Access Memory), high-Bandwidth Memory HBM (High-Bandwidth Memory), hybrid storage cube HMC (Hybrid Memory Cube), or the like, or any other medium that may be used to store the desired information and that may be accessed by an application, a module, or both. Any such computer storage media may be part of, or accessible by, or connectable to, the device. Any of the applications or modules described herein may be implemented using computer-readable/executable instructions that may be stored or otherwise maintained by such computer-readable media.
It should be understood that the terms "first" or "second" and the like in the claims, specification and drawings of the present disclosure are used for distinguishing between different objects and not for describing a particular sequential order. The terms "comprises" and "comprising" when used in the specification and claims of the present disclosure are taken to specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It is also to be understood that the terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in the specification and claims of the present disclosure, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should be further understood that the term "and/or" as used in the present disclosure and claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.
It should also be appreciated that any of the modules, units, components, server-side, computers, terminals, or devices illustrated herein that execute instructions may include or otherwise access a computer-readable medium, such as a storage medium, computer storage medium, or data storage device (removable) and/or non-removable) such as a magnetic disk, optical disk, or magnetic tape. Computer storage media may include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.
Although the embodiments of the present invention are described above, the descriptions are merely examples for facilitating understanding of the present invention, and are not intended to limit the scope and application of the present invention. Any person skilled in the art can make any modification and variation in form and detail without departing from the spirit and scope of the present disclosure, but the scope of the present disclosure is defined by the appended claims.
Claims (8)
1. A handwriting recognition method for a client, comprising:
A handwriting recognition request is sent to a server; the handwriting recognition request contains information of target characters to be recognized; the target characters are characters to be written;
Receiving a feature extraction model returned by a server side and a feature value of the target text;
receiving a handwritten character to be identified which is input by a user according to the target character, loading the feature extraction model to obtain a feature value of the handwritten character to be identified, and matching the feature value of the handwritten character to be identified with the feature value of the target character to identify the handwritten character to be identified;
Receiving the handwritten text to be recognized, loading the feature extraction model, and comprising:
the feature extraction model is loaded once every stroke the user completes.
2. The method of claim 1, wherein the matching comprises:
and calculating the similarity between the characteristic value of the handwritten character to be recognized and the characteristic value of the target character.
3. The method of claim 2, wherein the similarity comprises one or more of euclidean distance similarity, cosine similarity, adjusted cosine similarity, and pearson correlation coefficient.
4. A method according to claim 2 or 3, further comprising:
calculating the similarity between the characteristic value of the handwritten character to be identified and the characteristic values of more than two target characters, selecting the maximum similarity, and comparing the maximum similarity with a threshold value to identify the handwritten character to be identified.
5. A handwriting recognition method for a server side, comprising:
receiving a handwriting recognition request sent by a client; the handwriting recognition request contains information of target characters to be recognized; the target characters are characters to be written;
Transmitting the feature extraction model and the feature value of the target text to a client according to the handwriting recognition request;
The feature value extraction model selects positive samples and negative samples in a model training stage, and comprises the following steps:
Selecting the same characters as positive samples and selecting different characters as negative samples; and/or
And taking the content obtained by reducing one or two strokes of one word as a negative sample, and taking the content obtained by increasing one or two strokes of one word as the negative sample.
6. A client, comprising: memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the handwriting recognition method for a client according to any of claims 1 to 4 when the computer program is executed.
7. A server side, comprising: memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the handwriting recognition method for a server side according to claim 5 when the computer program is executed.
8. A handwriting recognition system comprising a client according to claim 6 and a server according to claim 7, said server being communicatively connected to the client.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110615845.1A CN113408373B (en) | 2021-06-02 | Handwriting recognition method, handwriting recognition system, client and server |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110615845.1A CN113408373B (en) | 2021-06-02 | Handwriting recognition method, handwriting recognition system, client and server |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113408373A CN113408373A (en) | 2021-09-17 |
| CN113408373B true CN113408373B (en) | 2024-06-07 |
Family
ID=
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1255192A2 (en) * | 2001-05-04 | 2002-11-06 | Microsoft Corporation | Web enabled recognition architecture |
| CN102122360A (en) * | 2011-03-01 | 2011-07-13 | 华南理工大学 | Cloud computing-based mobile terminal handwriting identification method |
| CN103473491A (en) * | 2013-09-01 | 2013-12-25 | 西安电子科技大学 | Writing process based mobile terminal user identification system and method |
| CN108229428A (en) * | 2018-01-30 | 2018-06-29 | 上海思愚智能科技有限公司 | A kind of character recognition method, device, server and medium |
| JP2018147312A (en) * | 2017-03-07 | 2018-09-20 | 公立大学法人会津大学 | User authentication system in handwritten characters |
| CN108959664A (en) * | 2018-09-26 | 2018-12-07 | 江苏曲速教育科技有限公司 | Distributed file system based on picture processor |
| CN109086654A (en) * | 2018-06-04 | 2018-12-25 | 平安科技（深圳）有限公司 | Handwriting model training method, text recognition method, device, equipment and medium |
| CN109299663A (en) * | 2018-08-27 | 2019-02-01 | 刘梅英 | Hand-written script recognition methods, system and terminal device |
| CN110033052A (en) * | 2019-04-19 | 2019-07-19 | 济南浪潮高新科技投资发展有限公司 | A kind of the self-training method and self-training platform of AI identification hand-written script |
| CN111832547A (en) * | 2020-06-24 | 2020-10-27 | 平安普惠企业管理有限公司 | Dynamic deployment method and device of character recognition model and computer equipment |
| CN112036323A (en) * | 2020-09-01 | 2020-12-04 | 中国银行股份有限公司 | Signature handwriting identification method, client and server |
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1255192A2 (en) * | 2001-05-04 | 2002-11-06 | Microsoft Corporation | Web enabled recognition architecture |
| CN102122360A (en) * | 2011-03-01 | 2011-07-13 | 华南理工大学 | Cloud computing-based mobile terminal handwriting identification method |
| CN103473491A (en) * | 2013-09-01 | 2013-12-25 | 西安电子科技大学 | Writing process based mobile terminal user identification system and method |
| JP2018147312A (en) * | 2017-03-07 | 2018-09-20 | 公立大学法人会津大学 | User authentication system in handwritten characters |
| CN108229428A (en) * | 2018-01-30 | 2018-06-29 | 上海思愚智能科技有限公司 | A kind of character recognition method, device, server and medium |
| CN109086654A (en) * | 2018-06-04 | 2018-12-25 | 平安科技（深圳）有限公司 | Handwriting model training method, text recognition method, device, equipment and medium |
| CN109299663A (en) * | 2018-08-27 | 2019-02-01 | 刘梅英 | Hand-written script recognition methods, system and terminal device |
| CN108959664A (en) * | 2018-09-26 | 2018-12-07 | 江苏曲速教育科技有限公司 | Distributed file system based on picture processor |
| CN110033052A (en) * | 2019-04-19 | 2019-07-19 | 济南浪潮高新科技投资发展有限公司 | A kind of the self-training method and self-training platform of AI identification hand-written script |
| CN111832547A (en) * | 2020-06-24 | 2020-10-27 | 平安普惠企业管理有限公司 | Dynamic deployment method and device of character recognition model and computer equipment |
| CN112036323A (en) * | 2020-09-01 | 2020-12-04 | 中国银行股份有限公司 | Signature handwriting identification method, client and server |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109086303B (en) | Intelligent conversation method, device and terminal based on machine reading understanding | |
| US20200250226A1 (en) | Similar face retrieval method, device and storage medium | |
| CN110427461B (en) | Intelligent question and answer information processing method, electronic equipment and computer readable storage medium | |
| CN111783902B (en) | Data augmentation, service processing method, device, computer equipment and storage medium | |
| CN110598206A (en) | Text semantic recognition method and device, computer equipment and storage medium | |
| US11010664B2 (en) | Augmenting neural networks with hierarchical external memory | |
| CN112819023A (en) | Sample set acquisition method and device, computer equipment and storage medium | |
| CN113158554B (en) | Model optimization method and device, computer equipment and storage medium | |
| CN113836992A (en) | Method for identifying label, method, device and equipment for training label identification model | |
| CN112231347A (en) | Data processing method and device, computer equipment and storage medium | |
| CN113688955B (en) | Text recognition method, device, equipment and medium | |
| US11810388B1 (en) | Person re-identification method and apparatus based on deep learning network, device, and medium | |
| CN114973229A (en) | Text recognition model training method, text recognition device, text recognition equipment and medium | |
| CN113408373B (en) | Handwriting recognition method, handwriting recognition system, client and server | |
| CN111444906B (en) | Image recognition method and related device based on artificial intelligence | |
| CN113761188A (en) | Text label determination method and device, computer equipment and storage medium | |
| CN112148836A (en) | Multi-modal information processing method, device, equipment and storage medium | |
| CN105975643A (en) | Real-time image retrieval method based on text index | |
| CN113408373A (en) | Handwriting recognition method, system, client and server | |
| KR102435035B1 (en) | The Fake News Video Detection System and Method thereby | |
| KR102348689B1 (en) | Text summarization using sentiment score with sequence-to-sequence | |
| CN113468322A (en) | Method, device, equipment and medium for training and extracting keyword recognition model | |
| Rahul et al. | Deep reader: Information extraction from document images via relation extraction and natural language | |
| CN116645700B (en) | Feature extraction model processing method and device and feature extraction method and device | |
| US20220292812A1 (en) | Zero-shot dynamic embeddings for photo search |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |