CN113298020A - Handwriting track recognition method and device, mobile terminal and storage medium - Google Patents
Handwriting track recognition method and device, mobile terminal and storage medium Download PDFInfo
- Publication number
- CN113298020A CN113298020A CN202110653716.1A CN202110653716A CN113298020A CN 113298020 A CN113298020 A CN 113298020A CN 202110653716 A CN202110653716 A CN 202110653716A CN 113298020 A CN113298020 A CN 113298020A
- Authority
- CN
- China
- Prior art keywords
- point
- coordinate
- video picture
- coordinate point
- preset
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 58
- 238000012545 processing Methods 0.000 claims abstract description 11
- 238000004590 computer program Methods 0.000 claims description 10
- 238000010586 diagram Methods 0.000 description 12
- 230000008569 process Effects 0.000 description 10
- 238000004891 communication Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 5
- 230000009471 action Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 241000203475 Neopanax arboreus Species 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 230000011664 signaling Effects 0.000 description 1
- 230000007480 spreading Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/30—Writer recognition; Reading and verifying signatures
- G06V40/37—Writer recognition; Reading and verifying signatures based only on signature signals such as velocity or pressure, e.g. dynamic signature recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/30—Writer recognition; Reading and verifying signatures
- G06V40/37—Writer recognition; Reading and verifying signatures based only on signature signals such as velocity or pressure, e.g. dynamic signature recognition
- G06V40/394—Matching; Classification
Landscapes
- Engineering & Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention relates to the technical field of image processing, and provides a handwriting track recognition method, a handwriting track recognition device, a mobile terminal and a storage medium. Acquiring a plurality of key points of a hand in a video picture through the acquired video picture; when the video picture comprises a preset writing gesture, determining a target key point from a plurality of key points, wherein the target key point represents a fingertip used for writing; and then according to a preset period, acquiring the position of the target key point in the video picture, namely a coordinate point, and connecting the acquired coordinate points in two adjacent periods to obtain the handwriting track. The handwriting track in the air can be recognized without depending on special equipment, and the convenience of handwriting input is improved.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a handwriting track recognition method and device, a mobile terminal and a storage medium.
Background
In the prior art, a dedicated device is generally required to implement handwriting input, such as a touch screen or a handwriting pad. In addition, in the process of handwriting by using these devices, the user needs to contact the device with the hand of the user, and this contact handwriting mode not only depends on the dedicated device, but also causes inconvenience in the handwriting process, so that the user experience is not good.
Disclosure of Invention
In view of the above, the present invention provides a handwriting track recognition method, device, mobile terminal and storage medium.
In order to achieve the above purpose, the embodiment of the present invention adopts the following technical solutions:
in a first aspect, the present invention provides a handwriting track recognition method, including:
acquiring a plurality of key points of a hand in a video picture according to the acquired video picture;
when the video picture comprises a preset writing gesture, determining a target key point from the plurality of key points; the target key points represent fingertips used for writing;
obtaining coordinate points according to a preset period; the coordinate points represent the positions of the target key points in the video picture;
and connecting the coordinate points obtained in two adjacent periods to obtain the handwriting track.
In an optional embodiment, after the step of obtaining the coordinate points according to the preset period, the method further includes:
according to the obtained coordinate point of the current period, determining whether the first coordinate point is an inflection point; the first coordinate point is a coordinate point obtained in a previous period of the current period;
if so, the first coordinate point is connected with the inflection point determined last time in a straight line mode to obtain the handwriting track.
In an optional embodiment, the step of determining whether the first coordinate point is an inflection point according to the obtained coordinate point of the current cycle includes:
according to the obtained coordinate point of the current period, whether the included angle between the first vector and the second vector is within a preset range is determined; the first vector is a vector taking a second coordinate point as a starting point and the first coordinate point as an end point; the second vector is a vector taking the first coordinate point as a starting point and taking the coordinate point of the current period as an end point; the second coordinate point is a coordinate point obtained at a time interval of one cycle from the current cycle;
if so, the first coordinate point is an inflection point;
and if not, the first coordinate point is not a turning point.
In an alternative embodiment, the method further comprises:
when a preset number of coordinate points are acquired within a preset radius range taking the starting point as the center, stopping acquiring the coordinate points; the starting point is a coordinate point obtained in the first period.
In an alternative embodiment, the method further comprises:
and when the video picture comprises a preset ending gesture, stopping obtaining the coordinate point.
In an optional embodiment, the step of obtaining the coordinate point according to a preset period includes:
acquiring a pending video picture of a current period;
fitting a plurality of key points in the to-be-determined video picture with a preset hand model;
and determining the position of the target key point on the to-be-determined video picture according to the fitting result, and obtaining the coordinate point corresponding to the current period.
In a second aspect, the present invention provides a handwriting track recognition apparatus, comprising:
the acquisition module is used for acquiring a plurality of key points of the hand in the video picture according to the acquired video picture;
the processing module is used for determining a target key point from the plurality of key points when the video picture comprises a preset writing gesture; the target key points represent fingertips used for writing;
obtaining coordinate points according to a preset period; the coordinate points represent the positions of the target key points in the video picture;
and the connecting module is used for connecting the coordinate points obtained in two adjacent periods to obtain the handwriting track.
In an alternative embodiment, the connection module is further configured to:
according to the obtained coordinate point of the current period, determining whether the first coordinate point is an inflection point; the first coordinate point is a coordinate point obtained in a previous period of the current period;
if so, the first coordinate point is connected with the inflection point determined last time in a straight line mode to obtain the handwriting track.
In a third aspect, the present invention provides a mobile terminal, comprising a processor and a memory, wherein the memory stores a computer program, and the processor implements the method according to any one of the foregoing embodiments when executing the computer program.
In a fourth aspect, the present invention provides a storage medium having stored thereon a computer program which, when executed by a processor, implements the method of any of the preceding embodiments.
The embodiment of the invention provides a handwriting track recognition method, a handwriting track recognition device, a mobile terminal and a storage medium. Firstly, acquiring a plurality of key points of a hand in a video picture according to the acquired video picture; when the video picture comprises a preset writing gesture, determining a target key point from a plurality of key points, wherein the target key point represents a fingertip used for writing; and then according to a preset period, acquiring the position of the target key point in the video picture, namely a coordinate point, and connecting the acquired coordinate points in two adjacent periods to obtain the handwriting track. The handwriting track in the air can be recognized without depending on special equipment, so that the convenience of handwriting input is improved.
In order to make the aforementioned and other objects, features and advantages of the present invention comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.
Fig. 1 is a block diagram of a mobile terminal according to an embodiment of the present invention;
FIG. 2 is a flow chart illustrating a handwriting track recognition method according to an embodiment of the present invention;
FIG. 3 is a diagram illustrating an example of a handwriting track recognition method provided by an embodiment of the invention;
FIG. 4 is a flowchart illustrating a handwriting track recognition method according to an embodiment of the present invention;
FIG. 5 is a flowchart illustrating a handwriting track recognition method according to an embodiment of the present invention;
FIG. 6 is a flowchart illustrating a handwriting track recognition method according to an embodiment of the present invention;
FIG. 7 is a flowchart illustrating a handwriting track recognition method according to an embodiment of the present invention;
FIG. 8 is a diagram illustrating another example of a handwriting track recognition method according to an embodiment of the present invention;
fig. 9 is a functional block diagram of a handwriting track recognition apparatus according to an embodiment of the present invention.
Icon: 110-a bus; 120-a processor; 130-a memory; 150-I/O module; 170 — a communication interface; 300-handwriting track recognition means; 310-an acquisition module; 330-a processing module; 350-a connection module; 370-end module.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. The components of embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations.
Thus, the following detailed description of the embodiments of the present invention, presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present invention without making any creative effort, shall fall within the protection scope of the present invention.
It is noted that relational terms such as "first" and "second," and the like, may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
In the process of live video, in order to enhance the interactivity between the anchor and the audience, the live broadcast room usually sets some small games such as you guess me, the anchor draws images, and the audience guesses the content in the images. The anchor may draw the image by means of handwriting input, which generally requires a dedicated device in the prior art, which causes a lot of inconvenience. The embodiment of the invention further provides a handwriting track recognition method, which is independent of special equipment and can recognize the handwriting track of a main broadcast in the air by acquiring the video picture in the live broadcast process. The following describes a handwriting track recognition method according to an embodiment of the present invention in detail.
Fig. 1 is a block diagram of a mobile terminal according to an embodiment of the present invention. The mobile terminal includes a bus 110, a processor 120, a memory 130, an I/O module 150, and a communication interface 170.
The processor 120 may receive commands from the above-described other elements (e.g., the memory 130, the I/O module 150, the communication interface 170, etc.) through the bus 110, may interpret the received commands, and may perform calculations or data processing according to the interpreted commands.
The processor 120 may be an integrated circuit chip having signal processing capabilities. The Processor 120 may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components.
The memory 130 may store commands or data received from the processor 120 or other elements (e.g., the I/O module 150, the communication interface 170, etc.) or commands or data generated by the processor 120 or other elements.
The Memory 130 may be, but is not limited to, a Random Access Memory (RAM), a Read Only Memory (ROM), a Programmable Read Only Memory (PROM), an Erasable Read Only Memory (EPROM), an electrically Erasable Read Only Memory (EEPROM), and the like.
The I/O module 150 may receive commands or data input from a user via input-output means (e.g., a sensor, a keyboard, a touch screen, etc.) and may transmit the received commands or data to the processor 120 or the memory 130 through the bus 110. And for displaying various information (e.g., multimedia data, text data) received, stored, processed from the above-described elements, video, images, data, etc. may be displayed to a user. Optionally, the I/O module 150 may further include a device for capturing images, such as a camera.
Communication interface 170 may be used for communicating signaling or data with other node devices.
It will be appreciated that the configuration shown in fig. 1 is merely a schematic configuration of a mobile terminal, and that a mobile terminal may include more or fewer components than shown in fig. 1, or have a different configuration than shown in fig. 1. The components shown in fig. 1 may be implemented in hardware, software, or a combination thereof.
The mobile terminal provided by the embodiment of the invention can be a smart phone, a personal computer, a tablet computer, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, a Personal Digital Assistant (PDA), and the like. The embodiments of the present invention do not limit this.
It can be understood that the application program is installed in the mobile terminal, for example, the application program for live broadcasting is installed in the mobile terminal of the anchor, and in the running process of the application program, the handwriting track recognition method provided by the embodiment of the present invention can be implemented.
The mobile terminal is taken as an execution subject to execute each step in each method provided by the embodiment of the invention, and achieve the corresponding technical effect.
Referring to fig. 2, fig. 2 is a flowchart illustrating a handwriting track recognition method according to an embodiment of the present invention.
Step S202, acquiring a plurality of key points of the hand in the video picture according to the acquired video picture;
the video pictures are video pictures of the main broadcast in the live broadcasting process.
The key points represent key parts of the hand, such as the palm, joints of the fingers, fingertips, and the like.
Optionally, a video picture of the anchor in the live broadcast process can be acquired in real time through a camera of the mobile terminal, and then a plurality of key points of the hand in the video picture, namely key parts of the hand of the anchor, are obtained based on a gesture recognition algorithm in the application program.
Fig. 3 (a) is an exemplary diagram of a plurality of key points of a hand according to an embodiment of the present invention, which includes 21 key points K0-K20. Each keypoint represents a key part of the hand, from the relative position of which a gesture can be recognized. It should be understood that the exemplary diagram is merely an exemplary illustration of key points of the hand, and may be set according to actual requirements, and the embodiment of the present invention is not limited thereto.
Step S204, when the video picture comprises a preset writing gesture, determining a target key point from a plurality of key points;
wherein the target keypoints represent fingertips used for writing.
Optionally, the application program has a function of displaying a writing prompt, the anchor may make a preset start gesture according to the writing prompt, and when the video picture includes the preset start gesture, the anchor starts to perform handwriting track recognition; and then the anchor can make a preset writing gesture according to the writing prompt, and when the preset writing gesture is included in the video picture, target key points are obtained from the multiple key points, and the target key points are the fingertips of the fingers used for writing.
Optionally, the preset starting gesture may be a five-finger spreading gesture, such as the gesture in fig. 3 (b), and the preset starting gesture may also be set according to actual requirements, which is not limited in the embodiment of the present invention.
The preset writing gesture may be a gesture as shown in fig. 3 (c), and according to the preset writing gesture, distance values between a plurality of key points and a palm key point, that is, K0, may be obtained, and when a maximum distance value is determined among all the distance values, a target key point, that is, key point K8 may be determined, where the target key point represents a fingertip of an index finger used for writing.
Step S206, obtaining coordinate points according to a preset period;
wherein the coordinate point represents the position of the target key point in the video picture.
Optionally, a video picture is acquired once according to a preset period, and the position of the target key point in the video picture is obtained according to the acquisition of the video picture in each period, and the position can be represented by a coordinate point. Alternatively, the coordinate points obtained every cycle may be stored in the trajectory array.
And step S208, connecting the coordinate points obtained in two adjacent periods to obtain the handwriting track.
Optionally, the coordinate points obtained in two adjacent periods are connected by straight lines, so that the handwriting track of the anchor in the air can be obtained.
Based on the design, firstly, a plurality of key points of the hand in the video picture are obtained according to the collected video picture; when the video picture comprises a preset writing gesture, determining a target key point from a plurality of key points, wherein the target key point represents a fingertip used for writing; and then according to a preset period, acquiring the position of the target key point in the video picture, namely a coordinate point, and connecting the acquired coordinate points in two adjacent periods to obtain the handwriting track. Compared with the prior art, the handwriting track in the air can be recognized without depending on special equipment.
It can be understood that a position of a coordinate point may be shifted, which may cause an inaccuracy of an acquired coordinate point, and in order to obtain a more accurate coordinate point, a possible implementation manner provided by the embodiment of the present invention is further provided. Fig. 4 is a flowchart illustrating a handwriting track recognition method according to an embodiment of the present invention. Wherein, step S206 includes the following steps:
step S206-1, acquiring a pending video picture of the current period;
optionally, according to a preset period, a video picture corresponding to the current period is obtained in each period, and the video picture is an undetermined video picture.
Step S206-3, fitting a plurality of key points in the to-be-determined video picture with a preset hand model;
the application program comprises a preset hand model, wherein the preset hand model is a physical hand model obtained by utilizing forward hand dynamics; it is to be understood that the preset hand model is a standard hand model.
Optionally, the to-be-determined video picture obtained in each period is subjected to a gesture recognition algorithm to obtain a plurality of key points of the hand in the to-be-determined video picture, and the plurality of key points are fitted with the preset hand model.
Step S206-5, determining the position of the target key point on the to-be-determined video picture according to the fitting result, and obtaining the coordinate point corresponding to the current period;
optionally, after fitting a plurality of key points of the hand in the to-be-determined video picture with the preset hand model, an optimal fitting result can be obtained, based on the optimal fitting result, the position of the target key point in the to-be-determined video picture can be obtained, and the coordinate point corresponding to the current period can be obtained.
Therefore, the undetermined video picture corresponding to each period is obtained, the plurality of key points in the undetermined video picture are fitted with the preset hand model, and the position of the target key point on the undetermined video picture can be obtained according to the obtained optimal fitting result, so that the accuracy of the coordinate point obtained in each period can be improved, and the obtained handwriting track can be more stable.
Optionally, if the obtained coordinate points are directly connected by straight lines, the handwriting track may not be smooth enough, and the invention provides a possible implementation manner. Fig. 5 is a flowchart illustrating a handwriting track recognition method according to an embodiment of the present invention. Wherein, step S208 includes the following steps:
step S208-1, determining whether the first coordinate point is an inflection point according to the obtained coordinate point of the current period;
the first coordinate point is a coordinate point obtained in a previous period of the current period.
It is understood that if coordinate points obtained in two adjacent cycles are connected by straight lines, each coordinate point has two corresponding straight lines.
The inflection point represents that when an included angle formed by two corresponding straight lines is larger than a certain preset angle value by taking the coordinate point as a center, the turning point between the two straight lines can be considered to be larger, and the coordinate point is regarded as the inflection point.
Optionally, the coordinate points acquired in each period are stored in the trajectory array, and after one coordinate point is acquired in each period, it may be determined whether the coordinate point acquired in a previous period of the current period is an inflection point.
If the first coordinate point is not the inflection point, deleting the first coordinate point in the track array;
if the first coordinate point is an inflection point, step S208-3 is performed.
S208-3, connecting the first coordinate point with the inflection point straight line determined last time to obtain a handwriting track;
optionally, the current inflection point is determined as a first coordinate point, and the first coordinate point is linearly connected with the last inflection point, so that the handwriting track can be obtained.
Regarding the step S208-1, a possible implementation manner provided by the embodiment of the present invention is provided. Step S208-1 includes: according to the obtained coordinate point of the current period, whether the included angle between the first vector and the second vector is within a preset range is determined;
the first vector is a vector taking the second coordinate point as a starting point and the first coordinate point as an end point; the second vector is a vector having the first coordinate point as a starting point and the coordinate point of the current period as an end point.
E.g. TNThe coordinate point obtained for the current period is represented, and the first coordinate point is represented as TN-1(ii) a The second coordinate point is denoted as TN-2Then the first vector is represented as
The second vector is represented as
Optionally, when the number of the coordinate points in the trajectory array is greater than 3, calculating an included angle between the first vector and the second vector after the coordinate points are obtained in each period; and determining whether the included angle is within a preset range.
If the included angle is within a preset range, the first coordinate point is an inflection point; and if the included angle is not in the preset range, the first coordinate point is not the inflection point.
Optionally, the preset range may be 16 to 180 degrees, and may also be set according to actual needs, which is not limited in the embodiment of the present invention.
It can be understood that when the included angle between the first vector and the second vector is 16-180 degrees, the coordinate points obtained in three adjacent periods are not on the same straight line, and then the coordinate point obtained in the previous period of the current period is TN-1Is an inflection point.
When the included angle between the first vector and the second vector is not 16-180 degreesWhen the coordinate obtained in the three adjacent periods is on the same straight line, namely 0-15 degrees, the coordinate point obtained in the previous period of the current period is TN-1Not a point of inflection.
It can be seen that after the coordinate point corresponding to each period is obtained, whether the obtained coordinate point in the previous period of the current period is an inflection point can be determined, and if the obtained coordinate point is the inflection point, the obtained coordinate point can be linearly connected with the inflection point determined last time. By screening out the inflection points and connecting the inflection points, the influence on the handwriting track caused by the fluctuation of the key points can be reduced, the obtained handwriting track can be smoother, and the trend of the track can be better shown.
Optionally, as for the way of ending the recognition of the handwriting track, the embodiment of the present invention provides two possible implementation ways, which will be described below separately.
Fig. 6 is a flowchart illustrating a handwriting track recognition method according to an embodiment of the present invention. After step S208, the following steps may be further included:
step S210A, when a preset number of coordinate points are obtained within a preset radius range taking the starting point as the center, stopping obtaining the coordinate points;
wherein the starting point is a coordinate point obtained in the first period.
Optionally, the coordinate points obtained in each period are stored in the trajectory array, and when a preset number of coordinate points are obtained in consecutive periods within a preset radius range centered on the starting point, it indicates that a plurality of coordinate points exist near the starting point, and it may be considered that the trend of the current handwriting trajectory is continuously close to the starting point, which indicates that the handwriting trajectory is a closed trajectory, and the obtaining of the coordinate points may be stopped, and the recognition of the handwriting trajectory is finished.
Alternatively, the coordinate points within the preset radius range may be coordinate points generated due to hand shaking, and in order to make the obtained handwritten trajectory smoother, the coordinate points within the preset radius range may be deleted from the trajectory array, and the coordinate points within the preset radius range are not connected.
Fig. 7 is a flowchart illustrating another handwriting track recognition method according to an embodiment of the present invention. After step S208, the following steps may be further included:
step S208B, when the video picture includes a preset ending gesture, stopping obtaining the coordinate point;
optionally, when the anchor completes the handwriting input, a preset ending gesture may be made according to the writing prompt, and when the preset ending gesture is included in the video picture, the obtaining of the coordinate point is stopped to end the recognition of the handwriting trajectory.
Alternatively, the preset ending gesture may be a fist-making gesture, such as the gesture shown in fig. 8, and when it is detected that a key point of a fingertip of each finger is close to a key point of a palm from a video picture, it indicates that the preset ending gesture is included in the video picture, and the recognition of the handwriting track is ended. It can be understood that the preset ending gesture can also be set according to actual requirements, and the embodiment of the present invention is not limited.
In order to perform the corresponding steps in the above embodiments and various possible manners, an implementation manner of the handwriting track recognition apparatus is given below. Referring to fig. 9, fig. 9 is a functional block diagram of a handwriting track recognition apparatus 300 according to an embodiment of the present invention. It should be noted that the basic principle and the generated technical effect of the handwriting track recognition apparatus 300 provided in the present embodiment are the same as those of the above embodiments, and for the sake of brief description, no part of the present embodiment is mentioned, and reference may be made to the corresponding contents in the above embodiments. The handwriting trajectory recognition apparatus 300 includes: an acquisition module 310, a processing module 330, and a connection module 350.
The acquisition module 310 is configured to acquire a plurality of key points of a hand in a video picture according to the acquired video picture;
the processing module 330 is configured to determine a target key point from the multiple key points when the video image includes a preset writing gesture; the target key points represent fingertips used for writing;
obtaining coordinate points according to a preset period; the coordinate points represent the positions of the target key points in the video picture;
and a connecting module 350, configured to connect coordinate points obtained in two adjacent periods to obtain a handwriting track.
Optionally, the connection module 350 is further configured to determine whether the first coordinate point is an inflection point according to the obtained coordinate point of the current period; the first coordinate point is a coordinate point obtained in a previous period of the current period;
if so, the first coordinate point is connected with the inflection point straight line determined last time to obtain the handwriting track.
Optionally, the connection module 350 is further configured to determine whether an included angle between the first vector and the second vector is within a preset range according to the obtained coordinate point of the current period; the first vector is a vector taking the second coordinate point as a starting point and the first coordinate point as an end point; the second vector is a vector taking the first coordinate point as a starting point and taking the coordinate point of the current period as an end point; the second coordinate point is a coordinate point obtained at a time interval of one cycle from the current cycle;
if so, the first coordinate point is an inflection point;
and if not, the first coordinate point is not a turning point.
Optionally, the processing module 330 is further configured to obtain a pending video picture of the current period;
fitting a plurality of key points in a to-be-determined video picture with a preset hand model;
and determining the position of the target key point on the picture of the video to be determined according to the fitting result, and obtaining a coordinate point corresponding to the current period.
Optionally, the handwriting trajectory recognition apparatus 300 further includes an end module 370, where the end module 370 is configured to stop obtaining coordinate points when a preset number of coordinate points are obtained within a preset radius range centered on the starting point; the starting point is a coordinate point obtained in the first cycle.
Optionally, the ending module 370 is further configured to stop obtaining the coordinate point when a preset ending gesture is included in the video frame.
The embodiment of the present invention further provides a mobile terminal, which includes a processor 120 and a memory 130, where the memory 130 stores a computer program, and when the processor executes the computer program, the method for recognizing a handwritten trajectory disclosed in the above embodiment is implemented.
The embodiment of the present invention further provides a storage medium, on which a computer program is stored, and the computer program, when executed by the processor 120, implements the handwriting trace recognition method disclosed in the embodiment of the present invention.
In summary, the handwriting track recognition method, the handwriting track recognition device, the mobile terminal and the storage medium provided by the embodiments of the invention. A plurality of key points of the hand in the video picture can be obtained according to the collected video picture; when the video picture comprises a preset writing gesture, determining a target key point from a plurality of key points, wherein the target key point represents a fingertip used for writing; and then according to a preset period, acquiring the position of the target key point in the video picture, namely a coordinate point, and connecting the acquired coordinate points in two adjacent periods to obtain the handwriting track. The handwriting track in the air can be recognized without depending on special equipment, and the effect of improving the convenience of handwriting input is achieved. In addition, a method for realizing the smoothness of the handwriting track can be provided, and the smoothness and the stability of the handwriting track can be ensured.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus and method can be implemented in other ways. The apparatus embodiments described above are merely illustrative, and for example, the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition, the functional modules in the embodiments of the present invention may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A method for handwriting trajectory recognition, the method comprising:
acquiring a plurality of key points of a hand in a video picture according to the acquired video picture;
when the video picture comprises a preset writing gesture, determining a target key point from the plurality of key points; the target key points represent fingertips used for writing;
obtaining coordinate points according to a preset period; the coordinate points represent the positions of the target key points in the video picture;
and connecting the coordinate points obtained in two adjacent periods to obtain the handwriting track.
2. The method of claim 1, wherein after the step of obtaining the coordinate points at a preset period, the method further comprises:
according to the obtained coordinate point of the current period, determining whether the first coordinate point is an inflection point; the first coordinate point is a coordinate point obtained in a previous period of the current period;
if so, the first coordinate point is connected with the inflection point determined last time in a straight line mode to obtain the handwriting track.
3. The method according to claim 2, wherein the step of confirming whether the first coordinate point is an inflection point based on the obtained coordinate points of the current period comprises:
according to the obtained coordinate point of the current period, whether the included angle between the first vector and the second vector is within a preset range is determined; the first vector is a vector taking a second coordinate point as a starting point and the first coordinate point as an end point; the second vector is a vector taking the first coordinate point as a starting point and taking the coordinate point of the current period as an end point; the second coordinate point is a coordinate point obtained at a time interval of one cycle from the current cycle;
if so, the first coordinate point is an inflection point;
and if not, the first coordinate point is not a turning point.
4. The method of claim 1, further comprising:
when a preset number of coordinate points are acquired within a preset radius range taking the starting point as the center, stopping acquiring the coordinate points; the starting point is a coordinate point obtained in the first period.
5. The method of claim 1, further comprising:
and when the video picture comprises a preset ending gesture, stopping obtaining the coordinate point.
6. The method of claim 1, wherein the step of obtaining the coordinate points according to a preset period comprises:
acquiring a pending video picture of a current period;
fitting a plurality of key points in the to-be-determined video picture with a preset hand model;
and determining the position of the target key point on the to-be-determined video picture according to the fitting result, and obtaining the coordinate point corresponding to the current period.
7. A handwritten trajectory recognition device, characterized in that said device comprises:
the acquisition module is used for acquiring a plurality of key points of the hand in the video picture according to the acquired video picture;
the processing module is used for determining a target key point from the plurality of key points when the video picture comprises a preset writing gesture; the target key points represent fingertips used for writing;
obtaining coordinate points according to a preset period; the coordinate points represent the positions of the target key points in the video picture;
and the connecting module is used for connecting the coordinate points obtained in two adjacent periods to obtain the handwriting track.
8. The apparatus of claim 7, wherein the connection module is further configured to:
according to the obtained coordinate point of the current period, determining whether the first coordinate point is an inflection point; the first coordinate point is a coordinate point obtained in a previous period of the current period;
if so, the first coordinate point is connected with the inflection point determined last time in a straight line mode to obtain the handwriting track.
9. A mobile terminal, characterized in that it comprises a processor and a memory, said memory storing a computer program which, when executed by said processor, implements the method of any of claims 1 to 6.
10. A storage medium, characterized in that the storage medium has stored thereon a computer program which, when executed by a processor, implements the method of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110653716.1A CN113298020A (en) | 2021-06-11 | 2021-06-11 | Handwriting track recognition method and device, mobile terminal and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110653716.1A CN113298020A (en) | 2021-06-11 | 2021-06-11 | Handwriting track recognition method and device, mobile terminal and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113298020A true CN113298020A (en) | 2021-08-24 |
Family
ID=77328101
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110653716.1A Pending CN113298020A (en) | 2021-06-11 | 2021-06-11 | Handwriting track recognition method and device, mobile terminal and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113298020A (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101320422A (en) * | 2008-06-06 | 2008-12-10 | 东莞市开心信息技术服务有限公司 | Method and device for judging standardization of cross-connection and cross-separation of handwritten Chinese character strokes |
| CN109344793A (en) * | 2018-10-19 | 2019-02-15 | 北京百度网讯科技有限公司 | Aerial hand-written method, apparatus, equipment and computer readable storage medium for identification |
| CN111950514A (en) * | 2020-08-26 | 2020-11-17 | 重庆邮电大学 | Depth camera-based aerial handwriting recognition system and method |
-
2021
- 2021-06-11 CN CN202110653716.1A patent/CN113298020A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101320422A (en) * | 2008-06-06 | 2008-12-10 | 东莞市开心信息技术服务有限公司 | Method and device for judging standardization of cross-connection and cross-separation of handwritten Chinese character strokes |
| CN109344793A (en) * | 2018-10-19 | 2019-02-15 | 北京百度网讯科技有限公司 | Aerial hand-written method, apparatus, equipment and computer readable storage medium for identification |
| CN111950514A (en) * | 2020-08-26 | 2020-11-17 | 重庆邮电大学 | Depth camera-based aerial handwriting recognition system and method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112926423B (en) | Pinch gesture detection and recognition method, device and system | |
| CN109934065B (en) | Method and device for gesture recognition | |
| CN104239416A (en) | User identification method and system | |
| CN108781252B (en) | Image shooting method and device | |
| US12008167B2 (en) | Action recognition method and device for target object, and electronic apparatus | |
| CN107368820B (en) | Refined gesture recognition method, device and equipment | |
| TW201939260A (en) | Method, apparatus, and terminal for simulating mouse operation by using gesture | |
| CN110599520B (en) | Open field experiment data analysis method, system and terminal equipment | |
| CN112381071A (en) | Behavior analysis method of target in video stream, terminal device and medium | |
| CN103455262A (en) | Pen-based interaction method and system based on mobile computing platform | |
| CN111103981A (en) | Control instruction generation method and device | |
| CN111986229A (en) | Video target detection method, device and computer system | |
| CN111601129B (en) | Control method, control device, terminal and storage medium | |
| KR20210040435A (en) | Gesture recognition method, gesture processing method and device | |
| CN110858291A (en) | Character segmentation method and device | |
| CN113298020A (en) | Handwriting track recognition method and device, mobile terminal and storage medium | |
| CN111382598B (en) | Identification method and device and electronic equipment | |
| EP4250065A1 (en) | Gesture recognition apparatus, head-mounted-type display apparatus, gesture recognition method, program, and computer readable medium | |
| CN110222576B (en) | Boxing action recognition method and device and electronic equipment | |
| CN108874141B (en) | Somatosensory browsing method and device | |
| CN107767712B (en) | Method and device for adjusting matching degree, storage medium and electronic equipment | |
| CN111258413A (en) | Control method and device of virtual object | |
| CN114063772B (en) | Gesture recognition method, device, equipment and medium | |
| CN113703577B (en) | Drawing method, drawing device, computer equipment and storage medium | |
| US8694509B2 (en) | Method and apparatus for managing for handwritten memo data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |