CN115311674A

CN115311674A - Handwriting processing method and device, electronic equipment and readable storage medium

Info

Publication number: CN115311674A
Application number: CN202211009903.7A
Authority: CN
Inventors: 杨文凤; 叶虹呐; 白峰; 王石
Original assignee: Boe Digital Technology Co ltd
Current assignee: Boe Digital Technology Co ltd
Priority date: 2022-08-22
Filing date: 2022-08-22
Publication date: 2022-11-08

Abstract

The application provides a handwriting processing method, a handwriting processing device, electronic equipment and a readable storage medium. The handwriting processing method comprises the following steps: receiving handwriting operation input of a target object; predicting a target text corresponding to the handwriting operation input according to a handwriting database corresponding to the target object, wherein the target text is text data in the handwriting database; and generating an output result corresponding to the target text. The embodiment of the application is beneficial to improving the handwriting recognition accuracy and the recognition speed.

Description

Handwriting processing method and device, electronic equipment and readable storage medium

Technical Field

The embodiment of the application relates to the technical field of software, in particular to a handwriting processing method and device, an electronic device and a readable storage medium.

Background

With the development of information technology and electronic technology, the user's demand for handwriting input is also gradually increasing. In the related art, the working principle of the handwriting input method can be understood that a user needs to write characters in a handwriting mode, and further, the corresponding characters are determined by analyzing and comparing the handwritten characters. However, different people have different writing habits, writing fonts and the like have larger differences, and part of the character strokes may be more, and writing requires more visual angles, so in the prior art, the recognition accuracy of the handwriting input is poor and the required time is longer.

Disclosure of Invention

The embodiment of the application provides a handwriting processing method, a handwriting processing device, electronic equipment and a readable storage medium, and aims to solve the problems that the recognition accuracy of handwriting input is poor and the required time is long.

To solve the above problems, the present application is implemented as follows:

in a first aspect, an embodiment of the present application provides a handwriting processing method, including the following steps:

receiving handwriting operation input of a target object;

predicting a target text corresponding to the handwriting operation input according to a handwriting database corresponding to the target object, wherein the target text is text data in the handwriting database;

and generating an output result corresponding to the target text.

In some embodiments, the predicting target text corresponding to the handwriting input from the handwriting database corresponding to the target object includes:

analyzing the handwriting operation input to obtain stroke information;

and taking the text in the handwriting database, the shape matching degree of which with the stroke information is greater than a preset matching threshold value, as a target text.

In some embodiments, before predicting the target text corresponding to the handwriting input from the handwriting database corresponding to the target object, the method further comprises:

acquiring writing data of the target user, wherein the writing data comprises one or more of strokes, characters, letters and symbols;

and establishing a handwriting database corresponding to the target user according to the writing data.

In some embodiments, the establishing a handwriting database corresponding to the target user according to the writing data includes:

extracting feature vectors of an image of the written data, wherein the feature vectors include a query vector, a key vector, and a value vector;

determining the attention value of the feature vector according to the product of the query vector and the key vector;

carrying out normalization processing on the attention value, and taking the product of the attention value after the normalization processing and the value vector as an output vector sequence;

and establishing a handwriting database corresponding to the target user according to the corresponding relation between the output vector sequence and the text data corresponding to the writing data.

In some embodiments, said collecting writing data of said target user comprises:

receiving a data acquisition input;

removing interference points of handwriting generated according to the data acquisition input, wherein the interference points comprise at least one of free points and redundant points, the free points are determined according to the pressure of sampling points of the data acquisition input and/or the relative positions of the sampling points, and the redundant points are determined according to the sampling points with overlapping;

smoothing the handwriting without the interference points;

and taking the smoothed handwriting as the writing data.

In some embodiments, after receiving the handwriting input of the target object, the method further comprises:

under the condition that a target text corresponding to the handwriting operation input does not exist in the handwriting database, identifying text information corresponding to a handwritten text obtained according to the handwriting operation input;

and updating the handwriting database according to the text information and the handwriting text.

In a second aspect, an embodiment of the present application further provides a handwriting processing apparatus, including:

the receiving module is used for receiving the handwriting operation input of the target object;

the prediction module is used for predicting a target text corresponding to the handwriting operation input according to a handwriting database corresponding to the target object, wherein the target text is text data in the handwriting database;

and the generating module is used for generating an output result corresponding to the target text.

In some embodiments, the prediction module comprises:

the analysis submodule is used for analyzing the handwriting operation input to obtain stroke information;

and the confirmation submodule is used for taking the text, which has the shape matching degree with the stroke information and is greater than a preset matching threshold value, in the handwriting database as the target text.

In a third aspect, an embodiment of the present application further provides an electronic device, including: a memory, a processor, and a program stored on the memory and executable on the processor; the processor is configured to read the program in the memory to implement the steps of the method according to the first aspect.

In a fourth aspect, an embodiment of the present application further provides a readable storage medium for storing a program, where the program when executed by a processor implements the steps in the method according to the foregoing first aspect.

In the embodiment of the application, the handwriting operation input is recognized through the handwriting database corresponding to each target object, so that the prediction precision of characters corresponding to the handwriting operation is improved, further, because different users have relatively unique writing habits, in the handwriting input process, the target text is predicted according to the written content and the handwriting database established according to the writing habits of the users, the target text corresponding to the content can be predicted relatively accurately before the content is written, and the efficiency of the handwriting input is improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the description of the embodiments of the present application will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.

Fig. 1 is a schematic flowchart of a handwriting processing method provided in an embodiment of the present application;

FIG. 2A is a schematic diagram of a Chinese character template in an embodiment of the present application;

FIG. 2B is a schematic diagram of Chinese handwriting data collection in an embodiment of the present application;

FIG. 3A is a schematic diagram of an English template in the embodiment of the present application;

FIG. 3B is a diagram illustrating English handwriting data collection according to an embodiment of the present application;

FIG. 4A is a schematic diagram of a symbol template in an embodiment of the present application;

FIG. 4B is a schematic diagram illustrating symbol handwriting data collection in an embodiment of the present application;

FIG. 5A is a schematic diagram of handwriting optimization in the embodiment of the present application;

FIG. 5B is a schematic diagram of handwriting optimization in the embodiment of the present application;

FIG. 5C is a schematic diagram of handwriting optimization in the embodiment of the present application;

FIG. 6 is a schematic diagram of a self-attention model provided by an embodiment of the present application;

FIG. 7 is a schematic diagram of a scenario of a handwriting processing method according to an embodiment of the present application;

fig. 8 is another flowchart illustrating a handwriting processing method according to an embodiment of the present application

Fig. 9 is a schematic structural diagram of a handwriting processing apparatus according to an embodiment of the present application;

fig. 10 is a schematic structural diagram of an electronic device provided in this application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some, but not all, embodiments of the present application. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without making any creative effort belong to the protection scope of the present application.

The terms "first," "second," and the like in the embodiments of the present application are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus. Further, the use of "and/or" in this application means that at least one of the connected objects, e.g., a and/or B and/or C, means that 7 cases are included where a alone, B alone, C alone, and both a and B are present, B and C are present, a and C are present, and a, B, and C are present.

The embodiment of the application provides a handwriting processing method.

The technical scheme of the embodiment is applied to an electronic device, the electronic device supports a handwriting input function, for example, the electronic device may be a smart phone with a touch screen, a tablet computer, an electronic paper reader, or an electronic device supporting handwriting input, such as a notebook computer and a personal computer including a handwriting pad or a touch pad, and the embodiment does not further describe and limit the specific type of the electronic device.

As shown in fig. 1, in one embodiment, the method comprises the steps of:

step 101: receiving handwriting operation input of a target object.

In the technical solution of this embodiment, the target object may execute the handwriting operation input on a touch screen, a handwriting pad, and other devices by means of, but not limited to, finger touch, a stylus pen, and the like.

Step 102: and predicting a target text corresponding to the handwriting operation input according to a handwriting database corresponding to the target object.

After receiving the handwriting input, the prediction is performed by the handwriting database corresponding to the target object in the present embodiment.

In this embodiment, a handwriting database is provided that uniquely corresponds to each target object, and the handwriting database may be understood to include a data set created from the handwriting data of the target user.

In one embodiment, this step 102 is preceded by the step of building a handwriting database.

In an exemplary embodiment, the step of building a handwriting database includes:

As shown in FIG. 2A, in practice, a specific Chinese character stroke, radical and character template can be provided.

In one exemplary embodiment, the Chinese character templates include dots (left stroke), horizontal (vertical), horizontal (horizontal), vertical (horizontal), horizontal falling (horizontal), vertical falling (vertical)

Lifting (12494

Hook (stroke) and the like. 148 common radicals, and a certain number of Chinese characters randomly selected. As shown in fig. 2B, the target object writes the corresponding content according to the provided template.

As shown in fig. 3A, in this embodiment, a template including english letters and english words may be further provided, where in this embodiment, the english template includes upper and lower cases of 26 letters and phrases that are randomly generated or selected. As shown in fig. 3B, the target object may also write according to the template to collect english handwriting data of the target object.

As shown in fig. 4A and 4B, further, a target including a common mathematical symbol may be provided to collect handwriting data of the target object writing the mathematical symbol.

It is understood that corresponding templates can be provided for collecting handwriting data of corresponding contents written by a user according to different handwriting requirements, which may include, for example, different languages such as japanese, korean, arabic, french, etc., or symbols in mathematical fields, symbols in medical fields, symbols in shorthand services, symbols related to civil engineering, etc., and the specific contents and fields are not further limited and described herein.

In some embodiments, handwriting collecting templates for different crowds can be further provided, for example, for students at school, the templates mainly include a chinese template, an english template and a common symbol template, for a specific area or a group using a specific language, a template for a corresponding language can be provided, for a group in a specific technical field, a template for a corresponding symbol can be provided, and the like.

Furthermore, in this embodiment, different writing tools may also be simulated, for example, for a pencil and a ball-point pen, the thickness of the writing handwriting is kept consistent, and for a pen, a writing brush, a mark pen, or other writing tools, corresponding writing simulations such as the thickness of the handwriting, the turning of the pen, or other writing tools may be further provided, which is not further limited and described herein.

After the handwriting data of the user is collected, the handwriting can be further preprocessed, so that the processing precision of the handwriting data is improved.

In one embodiment, the method further comprises, comprising:

receiving data acquisition input;

removing interference points of the handwriting generated according to the data acquisition input;

smoothing the handwriting without the interference points;

and taking the smoothed handwriting as the writing data.

After data acquisition input, removing interference points in sampling points for acquired handwriting data. In this embodiment, the interference point includes at least one of a free point and a redundant point.

It is to be understood that handwriting sampling is actually achieved by touch sampling. Taking a handwriting operation performed by a user on a handwriting board as an exemplary illustration, when a finger of the user touches the handwriting board, the handwriting board performs touch sampling, in the process, the handwriting board collects positions of the finger of the user at a plurality of moments, then a movement track of the finger is generated according to the position change of the finger of the user, and when the handwriting operation is performed, the movement track is a handwriting.

In some embodiments, the sampling points may be determined at certain time intervals, for example, touch sampling may be performed at different time intervals of 0.1 millisecond, 0.2 millisecond, 0.5 millisecond, and the like, so as to obtain the sampling points, and determine the handwriting according to the sampling points.

In this embodiment, the free point is determined according to the pressure of the sampling point and/or the relative position between the sampling points input by data acquisition.

The handwriting track sampling point set Ti = { Pk | k =1,2,3 \8230n), pk (xk, yk, zk, dk), wherein xk and yk are horizontal and vertical coordinates of the sampling point Pk, zk is sampling time, and dk is touch pressure of the sampling point.

In one exemplary embodiment, the freeness point is determined in the following manner. Firstly, filtering pressure values aiming at dk, wherein the pressure value interval is (0, 1), screening out a corresponding continuous or free point set when dk is in the (0, 1) interval, dk =1 and the stay time is 3s, then, continuously processing the point set screened by the pressure values, calculating theta of an angle formed by three points Pk-1, pk and Pk +1, setting an angle threshold alpha, and if theta is less than alpha, judging as a free point and rejecting.

The redundant points in the embodiment are determined according to the sampling points with overlapping, and it can be understood that if two points with different acquisition time sequences but the same coordinates exist, one of the points is used as the redundant point.

After the interference points are removed, smoothing is further performed, as shown in fig. 5A, in the writing process, there may be partial position pixel points of the handwriting missing, resulting in that obvious pixel points appear locally on the handwriting, which is specifically shown as locally zigzag.

As shown in fig. 5B and 5C, the initial handwriting is on the left side, and the optimized handwriting is on the right side, so that after the optimization, the personalized writing feature of the target object is maintained, and the interference is removed, so that the handwriting is more beautiful.

In the embodiment, a corresponding handwriting database is established according to the handwriting data, and in one embodiment, the handwriting database is established according to a self-attention (self-attention) model, so that the number of samples is reduced, meanwhile, through handwriting font synthesis and continuous optimization through a continuous learning model, the variance is reduced, and the established handwriting database can obtain characteristics more fitting to personal handwriting and has followability. Meanwhile, through combination of self-attention and a continuous learning model based on a convolutional network, a font library generation algorithm is merged, character details are highlighted, timeliness is broken through, and personal font style is reserved.

Specifically, the step of establishing the handwriting database comprises:

As shown in fig. 6, each chinese character in the sample represents each input, and each input is mapped to three spaces to obtain three vectors: the query variable q, the key variable k, and the value variable v. Wq, wk, and Wv in fig. 6 are outer linear mapping parameter matrices, respectively.

Next, the product q · k of the query vector q and the key vector k is calculated as the attention size.

Next, normalization processing is performed and a softmax activation function is applied to obtain:

this value is taken to mean a convenience V resulting in the output vector sequence H.

And finally, summing the output vector sequences to obtain an output result Z.

In this way, the present embodiment performs the feature transfer to transfer the features of the input kanji to the target generated text, and obtains the final output result, that is, the target text is a font having personal handwriting features.

Further, in some embodiments, considering that the personal writing habit may change, in this embodiment, the parsed font features may need to be continuously input during the handwriting input, and the target text may be generated, so that the entire model is continuously improved based on the convolutional network, and finally, the actual writing effect of the user is more fitted.

In some embodiments, after step 101, the method further comprises:

It should be understood that, since the content provided by the template is limited and the handwriting data that is not present in the template is predicted by the model, the result of the predictive simulation on the handwriting data may have a deviation, which may result in that the corresponding text information cannot be predicted and simulated correctly.

In this embodiment, the handwriting database is further kept to be continuously updated, and in implementation, the handwriting database can be continuously updated according to the text information which is successfully predicted and the text information which is not successfully predicted, so that the accuracy and reliability of the prediction result of the handwriting input are continuously improved.

In the technical solution of this embodiment, the step 102 includes:

analyzing the handwriting operation input to obtain stroke information;

and taking the text with the shape matching degree with the stroke information larger than a preset matching threshold value in the handwriting database as a target text.

In the technical scheme of this embodiment, in the process of performing handwriting input on the target object, a part that has been written by the user is analyzed, the part that has been written by the user may include one or more strokes, and information of the strokes is matched with a handwriting database to confirm the text matched with the strokes.

It can be understood that, for the target object, the characters written by the target object have certain similarity, and therefore, the characters corresponding to the strokes can be relatively accurately predicted by matching the strokes and the shapes which are written and the handwriting database.

In the related art, the characters need to be recognized after the characters are completely completed, and in the related art, the characters conforming to the stroke order can be screened only according to the stroke order for recognizing the strokes, however, many users write to the habit that there is a difference in actual strokes, for example, when writing "i" characters, the first step is left falling from right to left, whereas in writing habits such as writing, the writing mode of the first stroke of "i" characters is left to right, which may cause the stroke to be recognized as horizontal.

In the embodiment, after the written part of the user is recognized, the shape of the written part is matched with the handwriting database, so that the handwritten characters of the user can be predicted relatively accurately.

Step 103: and generating an output result corresponding to the target text.

In this embodiment, the output result may be generated in different modes.

In one embodiment, the output result is a standard font of the target text, and the target text may be output or saved in a document in a manner of a standard font such as a song style, a regular script, and the like.

In another embodiment, the output result may also be a handwritten font, and for example, the input result may be saved in a notepad or a document file in the manner of the handwritten font.

By providing different output modes, different use requirements can be met, and user experience is improved.

As shown in fig. 7, further, after completing writing a character, the following characters or phrases, etc. may be further predicted in combination with the normal expression habit or the writing habit of the user, so as to improve the handwriting input speed.

For example, in this embodiment, after completing the handwriting of "intelligence", the further predicted phrase may be "intelligence", "wisdom" or "wisdom quotient", and after completing the handwriting of "hand", the further predicted phrase may be "surgery", "handwriting" or "mobile phone".

As shown in fig. 8, the technical solution of this embodiment may be summarized as first collecting handwriting data of a user, then processing and beautifying the handwriting, and establishing a handwriting database of the user.

When the user performs handwriting input, if the user closes the prediction mode, the handwriting of the user is directly output.

And if the user selects the prediction mode, performing handwriting recognition on the handwriting input of the user, matching the handwriting input with a handwriting database, and further outputting a matching result.

It can be understood that, in this process, if the correspondence between the handwriting and the corresponding text already exists in the handwriting database, the handwriting database can be continuously optimized according to the current handwriting input, and if the correspondence does not exist in the handwriting database or the handwriting database cannot be effectively matched with the handwriting, the process of establishing the handwriting database can be referred to, and the continuous learning can be performed according to the recognition matching prediction of the current handwriting on the handwriting database.

The embodiment of the application also provides a handwriting processing device.

As shown in fig. 9, in one embodiment, the handwriting processing apparatus 900 includes:

a receiving module 901, configured to receive a handwriting operation input of a target object;

a predicting module 902, configured to predict, according to a handwriting database corresponding to the target object, a target text corresponding to the handwriting input, where the target text is text data in the handwriting database;

a generating module 903, configured to generate an output result corresponding to the target text.

In some embodiments, the prediction module 802 comprises:

and the confirmation submodule is used for taking the text in the handwriting database, of which the shape matching degree with the stroke information is greater than a preset matching threshold value, as the target text.

In some embodiments, further comprising:

the acquisition module is used for acquiring the writing data of the target user, wherein the writing data comprises one or more of strokes, characters, letters and symbols;

and the database establishing module is used for establishing a handwriting database corresponding to the target user according to the writing data.

In some embodiments, the database building module comprises:

the extraction sub-module is used for extracting a feature vector of the image of the written data, wherein the feature vector comprises a query vector, a key vector and a value vector;

a determination sub-module for determining an attention value of the feature vector based on a product of the query vector and the key vector;

the processing submodule is used for carrying out normalization processing on the attention value and taking the product of the attention value after the normalization processing and the value vector as an output vector sequence;

and the establishing submodule is used for establishing a handwriting database corresponding to the target user according to the corresponding relation between the output vector sequence and the text data corresponding to the writing data.

In some embodiments, the acquisition module comprises:

the input receiving submodule is used for receiving data acquisition input;

the optimization submodule is used for removing interference points of the handwriting generated according to the data acquisition input, wherein the interference points comprise at least one of free points and redundant points, the free points are determined according to the pressure of sampling points of the data acquisition input and/or the relative positions of the sampling points, and the redundant points are determined according to the sampling points with overlapping;

the smoothing sub-module is used for smoothing the handwriting with the interference points removed;

and the writing data confirmation sub-module is used for taking the smoothed handwriting as the writing data.

In some embodiments, further comprising:

the text recognition module is used for recognizing text information corresponding to the handwritten text obtained according to the handwriting operation input under the condition that a target text corresponding to the handwriting operation input does not exist in the handwriting database;

and the database updating module is used for updating the handwriting database according to the text information and the handwriting text.

The handwriting processing apparatus 900 of this embodiment can implement the steps of the handwriting processing method embodiment, and can implement substantially the same technical effects, which are not described herein again.

The embodiment of the application also provides the electronic equipment. Referring to fig. 10, the electronic device may include a processor 1001, a memory 1002, and a program 10021 stored in the memory 1002 and operable on the processor 1001.

When the program 10021 is executed by the processor 1001, any steps of the method embodiments described above can be implemented and achieve the same advantages, which are not described herein again.

Those skilled in the art will appreciate that all or part of the steps of the method according to the above embodiments may be implemented by hardware related to program instructions, and the program may be stored in a readable medium.

The embodiments of the present application further provide a readable storage medium, where a computer program is stored, and when the computer program is executed by a processor, the computer program may implement any step in the foregoing method embodiments, and may achieve the same technical effect, and in order to avoid repetition, details are not repeated here.

The storage medium may be a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk.

It should be noted that the above division of each module is only a division of a logic function, and the actual implementation may be wholly or partially integrated into one physical entity, or may be physically separated. And these modules can be realized in the form of software called by processing element; or can be implemented in the form of hardware; and part of the modules can be realized in the form of calling software by the processing element, and part of the modules can be realized in the form of hardware. For example, the determining module may be a processing element separately set up, or may be integrated into a chip of the apparatus, or may be stored in a memory of the apparatus in the form of program code, and a processing element of the apparatus calls and executes the function of the determining module. Other modules are implemented similarly. In addition, all or part of the modules can be integrated together or can be independently realized. The processing element described herein may be an integrated circuit having signal processing capabilities. In implementation, each step of the above method or each module above may be implemented by an integrated logic circuit of hardware in a processor element or an instruction in the form of software.

For example, the various modules, units, sub-units or sub-modules may be one or more integrated circuits configured to implement the above methods, such as: one or more Application Specific Integrated Circuits (ASICs), or one or more microprocessors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs), among others. For another example, when some of the above modules are implemented in the form of a Processing element scheduler code, the Processing element may be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor that can invoke the program code. As another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

While the foregoing is directed to the preferred embodiment of the present application, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the principles of the disclosure, and it is intended that such changes and modifications be considered as within the scope of the disclosure.

Claims

1. A handwriting processing method, comprising the steps of:

receiving handwriting operation input of a target object;

and generating an output result corresponding to the target text.

2. The method of claim 1, wherein predicting the target text corresponding to the handwriting input from the handwriting database corresponding to the target object comprises:

analyzing the handwriting operation input to obtain stroke information;

3. The method of claim 1, wherein prior to predicting the target text corresponding to the handwriting input from the handwriting database corresponding to the target object, the method further comprises:

4. The method of claim 3, wherein said building a handwriting database corresponding to the target user from the writing data comprises:

5. The method of claim 3, wherein said collecting writing data of said target user comprises:

receiving data acquisition input;

smoothing the handwriting without the interference points;

and taking the smoothed handwriting as the writing data.

6. The method of claim 1, wherein after receiving input of a handwriting operation of a target object, the method further comprises:

7. A handwriting processing apparatus, comprising:

8. The apparatus of claim 7, wherein the prediction module comprises:

9. An electronic device, comprising: a memory, a processor, and a program stored on the memory and executable on the processor; the processor is configured to read a program in the memory to implement the steps in the handwriting processing method according to any one of claims 1 to 6.

10. A readable storage medium storing a program, wherein the program, when executed by a processor, implements the steps in the handwriting processing method according to any one of claims 1 to 6.