WO2022160330A1

WO2022160330A1 - Handwriting recognition method and apparatus

Info

Publication number: WO2022160330A1
Application number: PCT/CN2021/074622
Authority: WO
Inventors: 贾红红; 黄光伟; 张文浩
Original assignee: 京东方科技集团股份有限公司
Priority date: 2021-02-01
Filing date: 2021-02-01
Publication date: 2022-08-04

Abstract

Provided are a handwriting recognition method and apparatus. The handwriting recognition method comprises: detecting information of a plurality of trajectory points corresponding to a handwriting trajectory of a user on a handwriting screen, wherein the plurality of trajectory points comprise a starting trajectory point and the current trajectory point; if the current trajectory point is a termination trajectory point, taking trajectory points between the starting trajectory point and the termination trajectory point as trajectory points to be recognized, wherein if no new trajectory point is detected within a pre-set duration after a pen lifting moment of the current trajectory point, the current trajectory point is the termination trajectory point, and the starting trajectory point is the trajectory point next to the last trajectory point, which was input into a text recognition model last time, or is the first trajectory point of the handwriting screen; recognizing, by using the text recognition model, information of the trajectory point to be recognized, so as to obtain a text recognition result; and displaying the text recognition result in print form. In the embodiments of the present disclosure, a text recognition result of a handwriting trajectory is displayed in real time, such that a user can conveniently check and correct the text recognition result in real time, and the recognition rate can be effectively improved.

Description

Handwriting recognition method and device

technical field

The embodiments of the present disclosure relate to the field of computer technologies, and in particular, to a method and device for handwriting recognition.

Background technique

In the existing handwriting recognition method, generally after the user writes on the electronic whiteboard, the handwriting trajectory is recognized, and the text of the recognition result is stored in the form of a document. If the user wants to check the recognition result, he needs to open the document for browsing in the background. If he wants to confirm whether the recognition is wrong, he needs to compare it with the original handwriting trace word by word. If the recognition is wrong, he needs to edit the document twice. The method is very inconvenient for users to use, and the recognition efficiency is low.

SUMMARY OF THE INVENTION

Embodiments of the present disclosure provide a handwriting recognition method and device, which are used to solve the problem that the user cannot view the handwriting recognition result in real time in the existing handwriting recognition method.

In order to solve the above-mentioned technical problems, the present disclosure is implemented as follows:

In a first aspect, an embodiment of the present disclosure provides a method for handwriting recognition, including:

Detect the information of multiple track points corresponding to the user's handwriting track on the handwriting screen, the information includes coordinates, time, and the user's writing state, and the user's writing state includes: starting a pen, running a pen, and lifting a pen; the multiple track points Including the starting track point and the current track point;

Determine whether the current trajectory point is the termination trajectory point according to the judgment condition, if the current trajectory point satisfies the judgment condition, take the current trajectory point as the termination trajectory point, and take the trajectory point between the start trajectory point and the termination trajectory point as the first trajectory point The track point to be identified, the determination condition is that no new track point is detected within the preset time period after the pen-lifting moment of the current track point, and the starting track point is the last input to the text recognition model last time. The next trajectory point of the trajectory point, or, the first trajectory point of the handwriting screen;

A text recognition model is used to recognize the first track point to be recognized, and a first text recognition result is obtained;

The first text recognition result is displayed in printed form in the first display area of the handwriting screen.

Optionally, a text recognition model is used to recognize the information of the first track point to be recognized, and before obtaining the first text recognition result, it also includes at least one of the following:

Branching the first track point to be identified;

The coordinates of the first track point to be identified are normalized into the same numerical range.

Optionally, branching the first track point to be identified includes:

Obtain the number of X-axis coordinate values of the first to-be-identified trajectory point corresponding to each Y-axis coordinate value of the first to-be-identified trajectory point;

According to the number of the X-axis coordinate values on the Y-axis, the first track point to be identified is divided into rows.

Optionally, use a text recognition model to recognize the information of the first track point to be recognized, and after obtaining the first text recognition result, the method further includes:

Perform semantic recognition on the first text recognition result, and perform semantic correction if the first text recognition result has a semantic error.

Optionally, displaying the first text recognition result in printed form in the first display area includes:

removing the handwriting track corresponding to the first track point to be recognized on the handwriting screen;

determining a first display area according to the coordinates of the first track point to be identified;

The first text recognition result is displayed in printed form in the first display area.

Optionally, determining the first display area according to the coordinates of the first track point to be identified includes:

Obtain the minimum X-axis coordinate, the minimum Y-axis coordinate, the maximum X-axis coordinate and the maximum Y-axis coordinate in the first track point to be identified;

According to the minimum X-axis coordinate, the minimum Y-axis coordinate, the maximum X-axis coordinate and the maximum Y-axis coordinate in the first track point to be identified, a first rectangular frame is determined, and the first rectangular frame is used as the first display area.

judging whether the handwritten trajectory corresponding to the first to-be-recognized trajectory point is inclined;

If the handwritten trajectory corresponding to the first track point to be recognized is inclined, perform tilt correction on the coordinates of the first track point to be recognized to obtain corrected coordinates;

The first display area is determined according to the corrected coordinates.

determining the font size of the first text recognition result according to the size of the first display area;

Determine the word spacing of the first text recognition result according to the size of the font of the first text recognition result;

The first text recognition result is displayed in the first display area with the determined font size and word spacing.

Optionally, after displaying the first text recognition result in printed form, it includes:

Receiving the user's erasing operation on the target text in the first text recognition result, erasing the target text;

Detect the information of the second to-be-recognized track point corresponding to the handwritten track rewritten by the user, the information includes coordinates, time and the user's writing state, and the user's writing state includes: starting the pen, running the pen, and lifting the pen; the second to-be-recognized Track points include start track points and end track points;

Determine whether the current trajectory point is the termination trajectory point of the second to-be-identified trajectory point according to the judgment condition. If the current trajectory point satisfies the judgment condition, the current trajectory point is used as the termination trajectory point of the second to-be-identified trajectory point, and the start The trajectory point between the start trajectory point and the end trajectory point is used as the second trajectory point to be identified, and the judgment condition is that no new trajectory point is detected within the preset time period after the pen-lifting moment of the current trajectory point, and the start trajectory point is not detected. The starting track point is the next track point of the last track point input to the text recognition model last time;

Recognize the second track point to be recognized by using a text recognition model to obtain a second text recognition result;

The second text recognition result is displayed in print in the second display area.

Optionally, displaying the second text recognition result in printed form in the second display area includes:

Obtain the minimum X-axis coordinate, the minimum Y-axis coordinate, the maximum X-axis coordinate and the maximum Y-axis coordinate in the second track point to be identified;

According to the minimum X-axis coordinate, the minimum Y-axis coordinate, the maximum X-axis coordinate and the maximum Y-axis coordinate in the second track point to be identified, a second rectangular frame is determined, and the second rectangular frame is used as the second display area.

Obtain the display information of the first text recognition result, and the display information includes the size and coordinates of the font;

According to the display information, the second display area is determined, and the second text recognition result is displayed in printed form in the second display area, and the size of the font in the second text recognition result is the same as the size of the The fonts in the first text recognition result are of the same size, and the text in the second text recognition result is aligned with the text in the first text recognition result.

In a second aspect, an embodiment of the present disclosure provides a handwriting recognition device, including:

The detection module is used to detect the information of multiple trajectory points corresponding to the handwriting trajectory of the user on the handwriting screen, the information includes coordinates, time and the user's writing state, and the user's writing state includes: starting the pen, running the pen and lifting the pen; The multiple track points include a start track point and a current track point; the second track point to be identified includes a start track point and an end track point;

The judgment module is used to judge whether the current trajectory point is the termination trajectory point according to the judgment condition, if the current trajectory point satisfies the judgment condition, the current trajectory point is used as the termination trajectory point, and the period between the start trajectory point and the termination trajectory point is determined. The track point is used as the first track point to be identified, the determination condition is that no new track point is detected within a preset time period after the pen-lifting moment of the current track point, and the starting track point is the last input to the text Identify the next trajectory point of the last trajectory point of the model, or, the first trajectory point of the handwriting screen;

a recognition module, configured to recognize the first track point to be recognized by using a text recognition model to obtain a first text recognition result;

A display module, configured to display the first text recognition result in printed form in the first display area of the handwriting screen.

In a third aspect, an embodiment of the present disclosure provides an interactive tablet, including a touch module, a display module, a processor, a memory, and a program or instruction stored on the memory and executable on the processor, the program Or, when the instructions are executed by the processor, the steps of the above-mentioned handwriting recognition method of the first aspect are implemented.

In a fourth aspect, an embodiment of the present disclosure provides a readable storage medium, where a program or an instruction is stored on the readable storage medium, and when the program or instruction is executed by a processor, the steps of the handwriting recognition method of the above-mentioned first aspect are implemented .

In the embodiment of the present disclosure, after real-time recognition of the handwritten trajectory, the text recognition result of the handwritten trajectory is displayed in real time in printed form, so that it is convenient for the user to view and correct the text recognition result in real time, which can effectively improve the recognition rate and enhance the communication with the user. interactivity between.

Description of drawings

Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are for purposes of illustrating preferred embodiments only and are not to be considered limiting of the present disclosure. Also, the same components are denoted by the same reference numerals throughout the drawings. In the attached image:

FIG. 1 is a schematic flowchart of a handwriting recognition method according to an embodiment of the disclosure;

2 is a comparison diagram of a handwriting trajectory displayed on a handwriting screen according to an embodiment of the present disclosure and a text recognition result;

3 is a schematic structural diagram of a handwriting recognition device according to an embodiment of the disclosure;

FIG. 4 is a schematic diagram of a composition structure of a handwriting recognition apparatus according to an embodiment of the present disclosure.

Detailed ways

The technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, but not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present disclosure.

Referring to FIG. 1, an embodiment of the present disclosure provides a method for handwriting recognition, including:

Step 11: Detect the information of multiple track points corresponding to the user's handwriting track on the handwriting screen, the information includes coordinates, time and the user's writing state, and the user's writing state includes: starting the pen, running the pen and lifting the pen; Each track point includes the start track point and the current track point;

The handwriting screen can be a handwriting device such as an electronic conference whiteboard, and has a touch module and a display module.

In this embodiment of the present disclosure, the track point may include a track point of one character or multiple characters, and the character may be Chinese, English, numbers, or the like.

In the embodiment of the present disclosure, the upper left corner of the handwriting screen may be used as the origin, the X axis extending from left to right, and the Y axis extending from top to bottom. Alternatively, the lower left corner of the handwriting screen can also be used as the origin, the X axis extending from left to right, and the Y axis extending from bottom to top. The embodiment of the present disclosure does not limit the settings of the coordinate axes.

In the embodiments of the present disclosure, starting a pen refers to the first trajectory point of a stroke, raising a pen refers to the last trajectory point of a stroke, and moving a pen refers to the middle trajectory point of a stroke.

In the embodiment of the present disclosure, optionally, the representation of each track point may be as follows: (x, y, t, flag), where x and y represent the coordinate position of each track point, and t is the writing of the track point Time, flag indicates the user's writing status (starting, running, or lifting).

The current track point refers to the last track point handwritten by the user on the handwriting screen.

Step 12: Determine whether the current trajectory point is the termination trajectory point according to the judgment condition, if the current trajectory point satisfies the judgment condition, take the current trajectory point as the termination trajectory point, and use the trajectory point between the start trajectory point and the end trajectory point As the first track point to be recognized, the determination condition is that no new track point is detected within a preset time period after the pen-lifting time of the current track point, and the starting track point is the last input to the text recognition model The next trajectory point of the last trajectory point of , or, the first trajectory point of the handwriting screen;

The first track point to be recognized includes a start track point and an end track point of the handwritten track.

Step 13: using a text recognition model to recognize the first track point to be recognized to obtain a first text recognition result;

Step 14: Display the first text recognition result in printed form in the first display area of the handwriting screen.

For example, the print type in Chinese adopts Song type, italic type, or bold type, etc., and English and numerals are in Times New Roman, etc., so as to be distinguished from handwritten tracks.

Please refer to FIG. 2 , which is a comparison diagram of a handwriting trajectory displayed on a handwriting screen and a text recognition result according to an embodiment of the disclosure.

In the embodiment of the present disclosure, after the real-time recognition of the handwritten trajectory is performed, the text recognition result of the handwritten trajectory is displayed in real time in printed form, so that it is convenient for the user to view and correct the text recognition result in real time, which can effectively improve the recognition rate and enhance the communication with the user. interactivity between.

Since the text recognition model receives the coordinates of many track points, these track points are arranged in the order of writing time without branching. The text recognition model recognizes the text track points line by line, so firstly, all track points need to be divided into lines (if the user writes multiple lines, the track points need to be divided into lines, if the user writes only one line, it is not branch is required). That is, in the example of the present disclosure, before using the text recognition model to identify and identify the information of the first track point to be identified, the first track point to be identified may also be branched.

In the embodiment of the present disclosure, the projection method may be used to branch the first track point to be identified, that is, branching the first track point to be identified includes:

Step 21: Obtain the number of X-axis coordinate values of the first to-be-identified trajectory point corresponding to each Y-axis coordinate value of the first to-be-identified trajectory point;

Step 22: According to the number of the X-axis coordinate values on the Y-axis, branch the first track point to be identified.

That is, the number of the X-axis coordinate values of all the track points in the first track point to be identified is projected onto the Y-axis. The number will be relatively large. If the handwritten track has branch lines, the number of X-axis coordinate values on the Y-axis in the blank positions in the two lines before and after is small or 0, that is, a trough occurs, and the value of the trough is taken as Branch basis.

However, if the text corresponding to the first track point to be recognized written by the user is inclined, at this time, if the first track point to be recognized is divided into lines according to the above-mentioned projection method, it will be inaccurate. Therefore, optionally, the present disclosure implements In an example, before branching the first track point to be identified, it can also include:

If the handwritten track corresponding to the first track point to be recognized is tilted, the coordinates of the first track point to be recognized are tilted and corrected to obtain corrected coordinates.

Since the coordinate value range of the track point to be recognized may be 0 to several thousand or tens of thousands, in order to facilitate the calculation and eliminate the influence of the order of magnitude, in the embodiment of the present disclosure, optionally, a text recognition model is used for the first to-be-recognized Identifying the information of the track point, and before obtaining the first text recognition result, further includes: normalizing the coordinates of the first track point to be recognized into the same numerical range, for example, normalizing to (0,1).

In the embodiment of the present disclosure, a Seq2Seq network may be used to perform text recognition on the information of the first track point to be identified. Of course, in some other embodiments of the present disclosure, other text recognition models, such as RNN networks, etc., may also be used.

In the embodiment of the present disclosure, optionally, a text recognition model is used to recognize the information of the first track point to be recognized, and after obtaining the first text recognition result, the method further includes: performing semantic correction on the first text recognition result. For example, the semantic correction may include: Chinese semantic correction and/or English semantic correction. According to the semantic information before and after, when there is a letter recognition error in an English word, the wrong letter is corrected.

Because users have a high degree of freedom when writing on the electronic whiteboard, some users like to write on the right half of the electronic whiteboard first, then the left half, and some like to write in the middle area first, and then write the upper or lower half. , and the existing handwriting recognition method is based on the time sequence relationship of the user's writing, and the recognition result is written into the document at one time, and the document is also stored according to the recognition order, without considering the original handwriting track layout space position When the user compares the recognition result with the original handwriting track, if the original handwriting track is written in a random order, the user needs to find the position of the original handwriting track according to the recognition result, which is very inconvenient.

To solve the above problem, in this embodiment of the present disclosure, optionally, displaying the first text recognition result in printed form in the first display area includes:

Step 31: remove the handwriting track corresponding to the first track point to be recognized on the handwriting screen;

Step 32: Determine a first display area according to the coordinates of the first track point to be identified;

Step 33: Display the first text recognition result in printed form in the first display area.

In the embodiment of the present disclosure, after the handwriting track is recognized, the handwriting track on the handwriting screen is erased in real time, and the text recognition result of the handwriting track is displayed in printed form in real time in the area where the handwriting track is located, which further facilitates the user to view and check the handwriting track in real time. Correct text recognition results.

In this embodiment of the present disclosure, optionally, determining the first display area according to the coordinates of the first track point to be identified includes:

Step 41: Obtain the minimum X-axis coordinate, the minimum Y-axis coordinate, the maximum X-axis coordinate and the maximum Y-axis coordinate in the first track point to be identified, namely x _min , y _min , x _max , y _max ;

Step 42: Determine the first rectangular frame according to the minimum X-axis coordinate, the minimum Y-axis coordinate, the maximum X-axis coordinate and the maximum Y-axis coordinate in the first track point to be identified, and use the first rectangular frame as the first display area.

However, if the text corresponding to the first track point to be recognized written by the user is inclined, at this time, if the first display area is determined according to the original coordinates of the first track point to be recognized, the first display area is an inclined rectangular frame, In this embodiment of the present disclosure, optionally, determining the first display area according to the coordinates of the first track point to be identified includes:

Step 51: Determine whether the handwritten trajectory corresponding to the first to-be-recognized trajectory point is inclined;

Step 52: If the handwritten track corresponding to the first track point to be recognized is inclined, perform tilt correction on the coordinates of the first track point to be recognized to obtain corrected coordinates.

Step 53: Determine the first display area according to the corrected coordinates.

In the embodiment of the present disclosure, optionally, the method for performing tilt correction on the coordinates of the first track point to be identified may be: firstly, a rectangular frame may be determined according to the original coordinates of the first track point to be identified, and then a rectangular frame may be determined. A point is used as the rotation center of the rectangle frame to rotate the rectangle frame, and the rotation center can be the center point of the rectangle frame or other points.

According to the corrected coordinates, the first rectangular frame is determined as the first display area in the manner of step 41 and step 42 . That is to say, the minimum X-axis coordinates, the minimum Y-axis coordinates, the maximum X-axis coordinates, and the maximum Y-axis coordinates in steps 41 and 42 are all corrected coordinates.

In this embodiment of the present disclosure, optionally, displaying the first text recognition result in printed form in the first display area includes:

In the embodiment of the present disclosure, optionally, the correspondence between the font and the word spacing may be pre-stored, and after the font size is determined, the word spacing is also determined accordingly. Of course, it is not excluded that the word spacing can be determined according to the size of the first display area. In this embodiment of the present disclosure, optionally, after displaying the first text recognition result in printed form in the area where the handwriting track is located, the method further includes:

Step 51: receiving the user's erasing operation on the target text in the first text recognition result, erasing the target text;

Users can partially or fully erase and rewrite text recognition results.

Step 52: Detect the information of the second to-be-recognized track point corresponding to the handwritten track rewritten by the user, the information includes coordinates, time, and the user's writing state, and the user's writing state includes: starting the pen, running the pen, and lifting the pen; 2. The track points to be identified include the start track point and the end track point;

In this embodiment of the present disclosure, the user can rewrite in the erased area, or can rewrite in other locations.

Step 53: Determine whether the current trajectory point is the termination trajectory point of the second to-be-identified trajectory point according to the judgment condition, and if the current trajectory point satisfies the judgment condition, use the current trajectory point as the termination trajectory point of the second to-be-identified trajectory point , taking the trajectory point during the starting trajectory point and the ending trajectory point as the second trajectory point to be identified, and the determination condition is that no new trajectory point is detected within the preset time period after the pen-lifting moment of the current trajectory point, The starting trajectory point is the next trajectory point of the last trajectory point input to the text recognition model last time;

Step 54: using a text recognition model to recognize the second track point to be recognized to obtain a second text recognition result;

Before identifying the second track point to be identified, the second track point to be identified may also be processed by branching and/or normalization. For the processing method, please refer to the above-mentioned first track point to be identified, which will not be described one by one here. . Similarly, after identifying the second track point to be identified, semantic correction can also be performed.

Step 54: Display the second text recognition result in printed form in the second display area.

For example, Chinese uses Times New Roman, etc. for Chinese, and Times New Roman is used for English and numerals, so as to distinguish them from handwritten tracks. Optionally, it is the same as the printing used for the first text recognition result.

In the embodiment of the present disclosure, after the handwriting track is recognized, the handwriting track on the handwriting screen is erased in real time, and the recognition result of the handwriting track is displayed in print in real time in the area where the handwriting track is located, so as to facilitate the user to view, check, and correct the recognition in real time. As a result, the interactivity with the user is improved.

In the embodiment of the present disclosure, the user is supported to perform multiple erasing and rewriting.

In this embodiment of the present disclosure, optionally, displaying the second text recognition result in printed form in the second display area includes:

Step 61: Obtain the minimum X-axis coordinate, the minimum Y-axis coordinate, the maximum X-axis coordinate and the maximum Y-axis coordinate in the second track point to be identified;

Step 62: Determine a second rectangular frame according to the minimum X-axis coordinate, the minimum Y-axis coordinate, the maximum X-axis coordinate and the maximum Y-axis coordinate in the second track point to be identified, and use the second rectangular frame as the second display area.

Similarly, before the second display area displays the second text recognition result in printed form, it may further include:

Step 71: judging whether the handwritten trajectory corresponding to the second recognition trajectory point is inclined;

Step 72: If the handwritten track corresponding to the second identified track point is inclined, perform tilt correction on the coordinates of the second track point to be identified to obtain corrected coordinates.

Step 73: Determine the second display area according to the corrected coordinates.

For example, according to the corrected coordinates, the second display area is determined according to the above steps 61 and 62 .

Step 81: Acquire display information of the first text recognition result, the display information includes font size and coordinates; Step 82: Determine the second display area according to the display information, and display the second display area in the second display The area displays the second text recognition result in printed form, the size of the font in the second text recognition result is the same as the size of the font in the first text recognition result, and the size of the font in the second text recognition result is the same. The text is aligned with the text in the first text recognition result.

In the embodiment of the present disclosure, optionally, the correspondence between the font and the word spacing may be pre-stored, and after the font size is determined, the word spacing is also determined accordingly. Of course, it is not excluded that the word spacing can be determined according to the size of the second display area.

According to the above display method, the display effect of the text recognition result can be improved.

Referring to FIG. 3, an embodiment of the present disclosure further provides a handwriting recognition device 300, including:

The detection module 301 is used to detect the information of multiple track points corresponding to the handwriting track of the user on the handwriting screen, the information includes coordinates, time and the user's writing state, and the user's writing state includes: starting the pen, running the pen and lifting the pen; The multiple track points include a start track point and a current track point;

The determination module 302 is used to determine whether the current trajectory point is the termination trajectory point according to the determination condition, if the current trajectory point satisfies the determination condition, the current trajectory point is used as the termination trajectory point, and the period between the start trajectory point and the termination trajectory point is determined. The trajectory point is used as the first trajectory point to be identified, and the determination condition is that no new trajectory point is detected within the preset time period after the pen-lifting moment of the current trajectory point, and the starting trajectory point is the last input to the The next trajectory point of the last trajectory point of the text recognition model, or, the first trajectory point of the handwriting screen;

An identification module 303, configured to identify the first track point to be identified by using a text identification model to obtain a first text identification result;

The display module 304 is configured to display the first text recognition result in printed form in the first display area of the handwriting screen.

In the embodiment of the present disclosure, optionally, the handwriting recognition device further includes at least one of the following:

a branching module for branching the first track point to be identified;

The normalization module is used for normalizing the coordinates of the first track point to be recognized to be within the same numerical range.

In the embodiment of the present disclosure, optionally, the branch module is configured to acquire the number of X-axis coordinate values of the first to-be-identified trajectory point corresponding to each Y-axis coordinate value of the first to-be-identified trajectory point number; according to the number of the X-axis coordinate values on the Y-axis, the first track point to be identified is divided into rows.

In the embodiment of the present disclosure, optionally, the handwriting recognition device further includes:

A semantic recognition module, configured to perform semantic recognition on the first text recognition result, and perform semantic correction if the first text recognition result has a semantic error.

In the embodiment of the present disclosure, optionally, the display module is configured to remove the handwriting track corresponding to the first track point to be recognized on the handwriting screen; and determine according to the coordinates of the first track point to be recognized a first display area; displaying the first text recognition result in printed form in the first display area.

In the embodiment of the present disclosure, optionally, the display module is configured to acquire the minimum X-axis coordinate, the minimum Y-axis coordinate, the maximum X-axis coordinate, and the maximum Y-axis coordinate in the first track point to be identified;

In this embodiment of the present disclosure, optionally, the display module is further configured to determine whether the handwritten track corresponding to the first track point to be recognized is inclined; if the handwritten track corresponding to the first track point to be recognized is tilted, Perform tilt correction on the coordinates of the first track point to be identified to obtain corrected coordinates; and determine a first display area according to the corrected coordinates.

In this embodiment of the present disclosure, optionally, the display module is further configured to determine the size of the font of the first text recognition result according to the size of the first display area; The size of the first text recognition result is determined, and the word spacing of the first text recognition result is determined; the first text recognition result is displayed in the first display area with the determined font size and word spacing.

an erasing module for receiving the user's erasing operation on the target text in the first text recognition result, and erasing the target text;

Wherein, the detection module is also used to detect the information of the second track point to be recognized corresponding to the handwritten track rewritten by the user, the information includes coordinates, time and the user's writing state, and the user's writing state includes: starting the pen, running the pen and lift the pen; the second track point to be identified includes a start track point and a termination track point;

The determination module is further configured to determine whether the current track point is the termination track point of the second track point to be identified according to the determination condition, and if the current track point satisfies the determination condition, the current track point is used as the second track point to be identified. The termination trajectory point of the point, the trajectory point during the initial trajectory point and the termination trajectory point is used as the second trajectory point to be identified, and the judgment condition is that the current trajectory point is not detected within a preset time period after the pen-lifting time A new trajectory point, the initial trajectory point is the next trajectory point of the last trajectory point input to the text recognition model last time;

The recognition module is further configured to use a text recognition model to recognize the second track point to be recognized to obtain a second text recognition result;

The display module is further configured to display the second text recognition result in printed form in the second display area.

In the embodiment of the present disclosure, optionally, the display module is further configured to acquire the minimum X-axis coordinate, the minimum Y-axis coordinate, the maximum X-axis coordinate, and the maximum Y-axis coordinate in the second track point to be identified; according to The minimum X-axis coordinate, the minimum Y-axis coordinate, the maximum X-axis coordinate, and the maximum Y-axis coordinate in the second track point to be identified, determine a second rectangular frame, and use the second rectangular frame as the second display area .

In this embodiment of the present disclosure, optionally, the display module is further configured to acquire display information of the first text recognition result, where the display information includes the font size and coordinates; and determine the display information according to the display information. a second display area, and the second text recognition result is displayed in printed form in the second display area, the size of the font in the second text recognition result is the same as the size of the font in the first text recognition result The size is the same, and the text in the second text recognition result is aligned with the text in the first text recognition result.

In the embodiment of the present disclosure, the above-mentioned functional modules may be integrated in one entity device, or may be set in multiple entity devices. For example, the above-mentioned function module used to detect the information of the trajectory point corresponding to the handwriting trajectory of the user on the handwriting screen The detection module, the display module used to display the text recognition results in printed form, can be set on the handwriting screen. The handwriting screen can be called the front end at this time, and is used to use the text recognition model to recognize the information of the track points to be recognized, and obtain The recognition module of the text recognition result can be set on the server. The server can also be called the backend at this time. Please refer to Figure 4. The information of the trajectory points detected by the handwriting screen is sent to the server, and the server recognizes the information of the trajectory points. The text recognition result is obtained and sent to the handwriting screen for display by the handwriting screen.

Embodiments of the present disclosure also provide an interactive tablet, including a touch module, a display module, a processor and a memory, and a program or instruction stored in the memory and executable on the processor, where the program or instruction is executed by the processor Each process of the above-mentioned embodiments of the handwriting recognition method can be realized at the same time, and the same technical effect can be achieved. In order to avoid repetition, details are not repeated here.

Embodiments of the present disclosure further provide a readable storage medium, where a program or an instruction is stored on the readable storage medium, and when the program or instruction is executed by a processor, each process of the above-mentioned embodiments of the handwriting recognition method can be implemented, and the same can be achieved. In order to avoid repetition, the technical effect will not be repeated here.

Wherein, the processor is the processor in the terminal described in the foregoing embodiment. The readable storage medium includes a computer-readable storage medium, such as a computer read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, and the like.

It should be noted that, herein, the terms "comprising", "comprising" or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements, It also includes other elements not expressly listed or inherent to such a process, method, article or apparatus. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in a process, method, article or apparatus that includes the element. Furthermore, it is noted that the scope of the methods and apparatus in the embodiments of the present disclosure is not limited to performing the functions in the order shown or discussed, but may also include performing the functions in a substantially simultaneous manner or in the reverse order, depending on the functions involved To perform functions, for example, the described methods may be performed in an order different from that described, and various steps may also be added, omitted, or combined. Additionally, features described with reference to some examples may be combined in other examples.

From the description of the above embodiments, those skilled in the art can clearly understand that the method of the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is better implementation. Based on this understanding, the technical solutions of the present disclosure essentially or the parts that contribute to the prior art can be embodied in the form of software products, and the computer software products are stored in a storage medium (such as ROM/RAM, magnetic disk, CD-ROM), including several instructions to make a terminal (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) execute the methods described in the various embodiments of the present disclosure.

The embodiments of the present disclosure have been described above in conjunction with the accompanying drawings, but the present disclosure is not limited to the above-mentioned specific embodiments, which are merely illustrative rather than restrictive. Under the inspiration of the present disclosure, without departing from the scope of the present disclosure and the protection scope of the claims, many forms can be made, which all fall within the protection of the present disclosure.

Claims

A method for handwriting recognition, comprising:

Detect the information of multiple track points corresponding to the user's handwriting track on the handwriting screen, the information includes coordinates, time, and the user's writing state, and the user's writing state includes: starting a pen, running a pen, and lifting a pen; the multiple track points Including the starting track point and the current track point;

Determine whether the current trajectory point is the termination trajectory point according to the judgment condition, if the current trajectory point satisfies the judgment condition, take the current trajectory point as the termination trajectory point, and take the trajectory point between the start trajectory point and the termination trajectory point as the first trajectory point The track point to be identified, the determination condition is that no new track point is detected within the preset time period after the pen-lifting moment of the current track point, and the starting track point is the last input to the text recognition model last time. The next trajectory point of the trajectory point, or, the first trajectory point of the handwriting screen;

A text recognition model is used to recognize the first track point to be recognized, and a first text recognition result is obtained;

The first text recognition result is displayed in printed form in the first display area of the handwriting screen.
The method of claim 1, wherein the information of the first track point to be recognized is recognized by a text recognition model, and before obtaining the first text recognition result, the method further comprises at least one of the following:

Branching the first track point to be identified;

The coordinates of the first track point to be identified are normalized into the same numerical range.
The method of claim 2, wherein branching the first track point to be identified comprises:

Obtain the number of X-axis coordinate values of the first to-be-identified trajectory point corresponding to each Y-axis coordinate value of the first to-be-identified trajectory point;

According to the number of the X-axis coordinate values on the Y-axis, the first track point to be identified is divided into rows.
The method according to claim 1, wherein the information of the first track point to be recognized is recognized by a text recognition model, and after obtaining the first text recognition result, the method further comprises:

Perform semantic recognition on the first text recognition result, and perform semantic correction if the first text recognition result has a semantic error.
The method of claim 1, wherein displaying the first text recognition result in printed form in the first display area comprises:

removing the handwriting track corresponding to the first track point to be recognized on the handwriting screen;

determining a first display area according to the coordinates of the first track point to be identified;

The first text recognition result is displayed in printed form in the first display area.
The method according to claim 5, wherein determining the first display area according to the coordinates of the first track point to be identified comprises:

Obtain the minimum X-axis coordinate, the minimum Y-axis coordinate, the maximum X-axis coordinate and the maximum Y-axis coordinate in the first track point to be identified;

According to the minimum X-axis coordinate, the minimum Y-axis coordinate, the maximum X-axis coordinate and the maximum Y-axis coordinate in the first track point to be identified, a first rectangular frame is determined, and the first rectangular frame is used as the first display area.
The method according to claim 5, wherein determining the first display area according to the coordinates of the first track point to be identified comprises:

judging whether the handwritten trajectory corresponding to the first to-be-recognized trajectory point is inclined;

If the handwritten trajectory corresponding to the first track point to be recognized is inclined, perform tilt correction on the coordinates of the first track point to be recognized to obtain corrected coordinates;

The first display area is determined according to the corrected coordinates.
The method according to claim 6 or 7, wherein displaying the first text recognition result in printed form in the first display area comprises:

determining the font size of the first text recognition result according to the size of the first display area;

Determine the word spacing of the first text recognition result according to the size of the font of the first text recognition result;

The first text recognition result is displayed in the first display area with the determined font size and word spacing.
The method of claim 1, wherein after displaying the first text recognition result in printed form, the method comprises:

Receiving the user's erasing operation to the target text in the first text recognition result, erasing the target text;

Detect the information of the second to-be-recognized track point corresponding to the handwritten track rewritten by the user, the information includes coordinates, time, and the user's writing state, and the user's writing state includes: starting the pen, running the pen, and lifting the pen; the second to-be-recognized Track points include start track points and end track points;

Determine whether the current trajectory point is the termination trajectory point of the second to-be-identified trajectory point according to the judgment condition. If the current trajectory point satisfies the judgment condition, the current trajectory point is used as the termination trajectory point of the second to-be-identified trajectory point, and the start The trajectory point between the start trajectory point and the end trajectory point is used as the second to-be-identified trajectory point, and the determination condition is that no new trajectory point is detected within the preset time period after the pen-lifting moment of the current trajectory point, and the start trajectory point is not detected. The starting track point is the next track point of the last track point input to the text recognition model last time;

Recognize the second track point to be recognized by using a text recognition model to obtain a second text recognition result;

The second text recognition result is displayed in print in the second display area.
The method of claim 9, wherein displaying the second text recognition result in printed form in the second display area comprises:

Obtain the minimum X-axis coordinate, the minimum Y-axis coordinate, the maximum X-axis coordinate and the maximum Y-axis coordinate in the second track point to be identified;

According to the minimum X-axis coordinate, the minimum Y-axis coordinate, the maximum X-axis coordinate and the maximum Y-axis coordinate in the second track point to be identified, a second rectangular frame is determined, and the second rectangular frame is used as the second display area.
The method of claim 9, wherein displaying the second text recognition result in printed form in the second display area comprises:

obtaining display information of the target text, where the display information includes font size and coordinates;

According to the display information, the second display area is determined, and the second text recognition result is displayed in printed form in the second display area, and the size of the font in the second text recognition result is the same as the size of the The fonts in the first text recognition result are of the same size, and the text in the second text recognition result is aligned with the text in the first text recognition result.
A handwriting recognition device, comprising:

The detection module is used to detect the information of multiple trajectory points corresponding to the handwriting trajectory of the user on the handwriting screen, the information includes coordinates, time and the user's writing state, and the user's writing state includes: starting the pen, running the pen and lifting the pen; The multiple track points include a start track point and a current track point;

The judgment module is used to judge whether the current trajectory point is the termination trajectory point according to the judgment condition, if the current trajectory point satisfies the judgment condition, the current trajectory point is used as the termination trajectory point, and the period between the start trajectory point and the termination trajectory point is determined. The track point is used as the first track point to be identified, the determination condition is that no new track point is detected within a preset time period after the pen-lifting moment of the current track point, and the starting track point is the last input to the text Identify the next trajectory point of the last trajectory point of the model, or, the first trajectory point of the handwriting screen;

a recognition module, configured to recognize the first track point to be recognized by using a text recognition model to obtain a first text recognition result;

A display module, configured to display the first text recognition result in printed form in the first display area of the handwriting screen.
An interactive tablet, characterized in that it includes a touch module, a display module, a processor, a memory, and a program or instruction stored on the memory and executable on the processor, and the program or instruction is processed by the The steps of implementing the handwriting recognition method according to any one of claims 1-11 when the device is executed.
A readable storage medium, characterized in that a program or an instruction is stored on the readable storage medium, and when the program or instruction is executed by a processor, a method for realizing the handwriting recognition method according to any one of claims 1-11 is realized. step.